The Bitter Lesson Does Not Apply Here

In natural language processing, the prevailing philosophy is the bitter lesson: simply scaling up unconstrained architectures and feeding them vast data will eventually beat any hand-engineered model. In chemistry, this rule seems to fundamentally break down.

The reason is structural: physical laws are not patterns to be learned from data; they are constraints. A molecule does not change its behaviour depending on what language you use to describe it, or how many other molecules your model has seen. The geometry of space, the symmetries of rotation and translation, the rules of quantum mechanics — these are fixed. Models that are explicitly built to respect these constraints, rather than hoping to infer them from data, consistently outperform those that do not when it comes to complex, unfamiliar chemical scenarios.

This matters beyond scientific accuracy. The energy cost of training a large chemistry model is substantial — comparable to the footprint of transcontinental flights, depending on the hardware and duration. The justification for that cost rests on the assumption that a sufficiently general model will not need to be retrained as new chemical domains come into scope. If that assumption is wrong, the cost is not paid once. It is paid repeatedly.

Scale can help a model interpolate better within the chemistry it has already seen. It cannot, on its own, teach a model to reason about chemistry it has never encountered.

The Extrapolation Wall

If an AI model cannot discover something truly new, what is its value in science? The ultimate goal of chemistry AI is genuine scientific discovery — novel therapeutics, advanced materials, structures that push beyond what currently exists. By definition, this requires out-of-distribution (OOD) generalisation: the model must accurately extrapolate into chemical spaces it has never encountered during training.

This is where current foundation models quietly struggle. When evaluated not on held-out versions of familiar molecules, but on genuinely new chemical compositions assembled from known building blocks, the performance of even the most capable models degrades substantially — often by an order of magnitude or more compared to their in-distribution accuracy. More troublingly, the models that perform best on familiar chemistry are not always the ones that generalise best to new chemistry. Standard benchmarks, which test models on data drawn from the same distribution they were trained on, can paint a misleadingly optimistic picture.

The practical consequence of this generalisation failure is rarely discussed openly. When a model cannot extrapolate to a new chemical domain, practitioners retrain it or fine-tune it on domain-specific data. Each time the scope of the problem shifts — a new class of materials, a new reaction type, an updated reference dataset — the energy cost is paid again. The promise of universality was precisely that this would not be necessary. A model that genuinely generalises across chemical space would need far less of this repeated adaptation, and the cumulative environmental cost of deploying chemistry AI at scale would be meaningfully lower. Closing the OOD gap would not eliminate that cost, but it would reduce it in a way that compounds over time and across the research community.

This gap between in-distribution accuracy and out-of-distribution performance is the central unanswered question in the field. It is also the question our recent work at Cardiff University, Benchmarking Compositional Generalisation for Machine Learning Interatomic Potentials, is designed to probe systematically — not to offer a solution, but to establish clearly where current models succeed, where they fail, and how large the gap actually is.

The Physics Trap: Systematic Softening

Universal machine-learned interatomic potentials — models trained to act as fast surrogates for expensive quantum mechanical calculations across a wide range of materials — represent some of the most impressive engineering in scientific AI. But a specific physical flaw haunts them: systematic softening.

These models are predominantly trained on atomic configurations close to their equilibrium states, where forces are small and well-behaved. When they encounter high-energy environments — distorted structures, defect sites, transition states — they systematically underpredict the forces and the curvature of the energy landscape. The result is an artificially smooth picture of molecular behaviour: the model describes a world that is slightly less tense, slightly less reactive, slightly less physically real than the one it is meant to simulate.

This is not a bug that more data trivially fixes. The distributions of configurations that matter most for reactive chemistry, materials failure, and catalysis are precisely the configurations that are hardest and most expensive to generate, and therefore most underrepresented in training sets. The models are physically flawed at exactly the regimes where accurate physics is most needed.

The response in practice is, again, fine-tuning: taking a universal model and adapting it to a specific high-energy regime with targeted data. This works, to a degree — but it reintroduces the per-domain training cost that universality was meant to eliminate. Each specialised application becomes its own compute and energy expenditure on top of the original training investment. The physics trap is not just a scientific problem; it is a resource problem that compounds with every new deployment context.

So, Do We Actually Need Them?

Given these limitations — poor out-of-distribution generalisation, systematic physical errors, and the failure of naive scaling — one might conclude that chemical foundation models are overhyped. That would be the wrong conclusion.

We need them. We just need to be honest about what they currently are and what they are not.

They are not zero-shot oracles. They are powerful starting points. Fine-tuning on even a small number of task-specific configurations can substantially correct for systematic errors. Better benchmarking — designed to test genuine extrapolation rather than interpolation — can guide the field toward architectures and training strategies that close the OOD gap. Better uncertainty quantification can tell practitioners when to trust a model and when to reach for the quantum chemistry code.

And when integrated thoughtfully into a human-led discovery pipeline, these systems yield real results. There are already drug candidates in clinical trials that would not exist without generative AI tools for chemistry. The technology works — within its limits.

The energy picture deserves an honest summary too. Improving OOD generalisation would reduce the cumulative cost of deploying chemistry AI: fewer retraining cycles, less per-domain fine-tuning, longer useful lifetimes for a given model. That is a meaningful gain, and it is one the field tends to undervalue when discussing the case for better benchmarks and more rigorous evaluation. But it would not eliminate the energy cost of achieving high accuracy in the first place. Training a model that is both physically accurate and genuinely generalisable across chemical space remains an unsolved problem, and any solution to it will itself require substantial compute. The goal of better generalisation is to stop paying the same cost repeatedly — not to make the cost disappear.

The right framing is this: chemical foundation models are engines that drastically accelerate the exploration phase of scientific discovery. They reduce the cost of hypothesis generation, broaden the searchable chemical space, and surface candidates for expert validation. What they are not — and may never be, given the nature of physical law — is a replacement for domain understanding, careful data curation, and rigorous validation.

Scale matters. But in chemistry, physics matters more. And until our models can genuinely compose what they have learned about molecular fragments into predictions about new ones, the universal promise remains exactly that — a promise.

How I Fixed the Packages Problem

That remaining problem — missing packages — was not unique to Fedora. It had been following me across every distribution I had ever used, in different forms. On Debian it was outdated versions. On Arch it was AUR maintenance risk. On CentOS it was the gap between stability and currency. Fedora brought me closer than anything else had, but it did not fully close it. The answer, when it finally arrived, did not come from the distribution at all.

Before getting to that, I want to talk about my relationship with Python — because the solution to the packages problem and the solution to surviving the Python ecosystem turned out to be the same thing.

pixi global install <package_name>

pixi global install zoxide ripgrep fd-find bat eza btm dust ncdu delta hyperfine xh sd helix neovim tmux zellij btop zsh xonsh

The Setup

If you are curious about the tools and how I configure them, this section is for you.

# Modern replacements
alias ls="eza --icons --group-directories-first"
alias tree="eza --icons --tree"
alias treed="eza --icons --tree --level"
alias lsdu='du -a -h --max-depth=1 | sort -hr'
alias du='dust'
alias top='btm'
alias cd='z'
alias grep='rg'
alias vim='nvim'

# Pipe --help and -h output through bat for syntax highlighting
alias -g -- -h='-h 2>&1 | bat --language=help --style=plain'
alias -g -- --help='--help 2>&1 | bat --language=help --style=plain'

git clone --depth 1 https://github.com/AstroNvim/template ~/.config/nvim \
  && rm -rf ~/.config/nvim/.git \
  && nvim

export NODE_SHELL="zsh"

getgnode() {
    local ngpus=1
    local time_limit="02:00:00"
    local partition="workq"  # Adjust for your cluster

    while [[ $# -gt 0 ]]; do
        case "$1" in
            -h|--help)
                echo "Usage: getgnode [OPTIONS]"
                echo "Allocate and attach to an interactive SLURM GPU node."
                echo ""
                echo "Options:"
                echo "  -g, --gpus INT         Number of GPUs to request (default: 1)"
                echo "  -p, --partition NAME   SLURM partition (default: workq)"
                echo "  -t, --time HH:MM:SS    Time limit (default: 02:00:00)"
                echo "  -h, --help             Show this help"
                return 0
                ;;
            -g|--gpus)      ngpus="$2";      shift 2 ;;
            -p|--partition) partition="$2";  shift 2 ;;
            -t|--time)      time_limit="$2"; shift 2 ;;
            *)
                echo "Error: Unknown argument '$1'"
                echo "Type 'getgnode --help' for usage."
                return 1
                ;;
        esac
    done

    salloc --partition="${partition}" \
           --gres=gpu:"${ngpus}" \
           --time="${time_limit}" \
           srun --pty "${NODE_SHELL:-zsh}"
}

fetchgnode() {
    if [[ "$1" == "-h" || "$1" == "--help" || -z "$1" ]]; then
        echo "Usage: fetchgnode <jobid>"
        echo "Attach a new shell to an existing SLURM job."
        return 0
    fi
    srun --jobid="$1" --overlap --pty "${NODE_SHELL:-zsh}"
}

alias sqm="squeue --me"

Where I Ended Up

Looking back, the journey was not really about finding the right distribution. Every distribution I used taught me something — Debian gave me discipline, Arch gave me depth, CentOS gave me an appreciation for stability, Nobara reminded me why community matters. What I was actually searching for, without fully realising it, was a coherent philosophy: a system I could understand completely, trust in production, enjoy on the desktop, and extend without fighting it.

Fedora, combined with KDE, DNF, Flatpak, and pixi, turned out to be that system. The distribution handles the OS. Flatpak handles GUI applications cleanly and keeps them up to date. DNF handles anything system-level that belongs there. And pixi handles everything else — Python environments, binary tools, cross-platform reproducibility — with a single unified interface that works identically on my workstation, my laptop, and a GH200 node on Isambard-AI.

The packages problem that once felt like an unsolvable compromise turned out to have a clean answer. I just had to wait for the Rust ecosystem to mature enough to build it.