TechnicalBeginner4 min read
HPC and AI
Two worlds that became one: what converged between scientific computing and AI, what still sets them apart and why observing a training run is becoming a specialty, with numbers to show it.
For: engineers and SREs · architects · team managers · executives and CIOsPrerequisites: Have read lesson 2 of the course.
Illustrative scenario. The head of a university computing centre has spent fifteen years serving physicists and climate scientists. One Monday, an AI team asks him for 64 GPUs for seven days to train a model. He expects an alien world. Instead he finds his own machines, network, storage and scheduler. What changes is the vocabulary, the indicators to watch and the price of a failure.
Two worlds that became one
Section titled “Two worlds that became one”Until around 2018, the two worlds lived apart. HPC meant clusters of ordinary processors running double-precision simulations and talking over MPI. AI ran on a few GPUs with small datasets. Hardware, tools and conferences were separate.
Today both share the same installation: thousands of GPUs linked by a very fast interconnect, shared parallel storage and scheduler, comparable observability and operations practices. Large-scale AI therefore runs on the same foundation as classic HPC. Mostly the terms and the tracked indicators change.
What still sets AI apart from classic HPC
Section titled “What still sets AI apart from classic HPC”The first difference is numerical precision. A computer stores each number on a certain number of bits (binary digits, 0 or 1): the more bits, the more precise the number. Classic HPC often requires double precision, on 64 bits (written FP64), to keep the simulation stable: the equivalent of counting to the cent. AI tolerates 16 bits, even 8 bits, a bit like rounding to the nearest euro. GPUs then go much faster, but questions arise: is the result precise enough? Can you reproduce exactly the same computation twice?
The second is data scale. Training sets reach hundreds of terabytes. Feeding thousands of GPUs without ever leaving them waiting is a storage and network challenge in its own right.
The third is what you observe. AI training works in steps: at each step, the model reads a batch of examples, measures its error and corrects itself slightly. AI therefore adds its own indicators. The loss curve shows that error, which should fall step after step. Gradient norms tell you how large the corrections are. Step time reveals slowdowns. Per-rank GPU utilization spots the machine that lags behind: a rank is the number given to each process taking part in the computation. Energy per processed token, finally, relates consumption to the amount of text digested (a token is a piece of a word). Observing a training run is becoming a specialty.
What it changes for operations
Section titled “What it changes for operations”Back to the AI team’s request. As an illustration, a 7-day training run on 64 GPUs consumes 64 x 7 x 24 = 10,752 GPU-hours. Multiply this volume by your provider’s hourly price or your internal cost. The same calculation applied to the largest training runs, on thousands of GPUs for weeks, gives hundreds of thousands, even millions of GPU-hours.
A run that long regularly writes checkpoints: full snapshots of its state, like saving a game. If one GPU degrades without anyone noticing and a checkpoint is lost, the run has to restart from the previous one. Depending on how often checkpoints are taken, that can cost hours, even days of computation. At this level of consumption, in my view, observability becomes an economic question as much as a technical one.
Prices change fast and vary by contract: this guide gives none. Use your provider’s dated price list.
What you now know
Section titled “What you now know”You know the five components of a cluster: login node, scheduler, compute nodes, interconnect and parallel storage. You can tell apart the three patterns of parallelism (trivial, tight coupling, data parallelism) and you know why doubling the machines does not halve the time. Finally, you know why large-scale AI rests on the same foundation as classic HPC and why a failure there is so expensive.
Going further
Section titled “Going further”To make these ideas concrete, I recommend reading the first pages of the documentation for Slurm’s sbatch command: you will see what a resource request really looks like. The Top500 ranking of supercomputers gives a sense of how big the largest installations are. If you code a little, try a small MPI or OpenMP program on your workstation: you will not break any record, but you will get a feel for parallelism.
The natural next step is the HPC observability foundations course.
Revised on 2 October 2026: cost of a training run recalculated explicitly and presented as illustrative (64 GPUs for 7 days, or 10,752 GPU-hours). This calculation replaces the mention “hundreds of thousands of euros”; prices removed.