The Three-Character Debugging Session: How "oom, go 20" Unlocked a Flash-Attention Build

In the sprawling, multi-hour conversation that is the subject of this analysis, one message stands out for its stark economy of expression. At message index 56, the user writes exactly three words:

oom, go 20

That is the entire message. No greeting, no explanation, no frustration vented — just a diagnosis and a directive. To the uninitiated, this looks like cryptic shorthand. To anyone who has ever wrestled with building a CUDA extension from source, it reads as a moment of exhausted clarity. This article unpacks the context, reasoning, assumptions, and consequences packed into those three words.

The Context: A War of Attrition Against OOM

To understand message 56, one must understand the battle that preceded it. The session's overarching goal was to set up a full ML environment on a remote Ubuntu 24.04 machine equipped with multiple NVIDIA RTX PRO 6000 Blackwell GPUs. The assistant had successfully installed NVIDIA drivers (590.48.01), CUDA Toolkit 13.1, Python, uv, PyTorch, and most of the standard ML stack. But one package refused to cooperate: flash-attn.

Flash-attention (flash-attn) is a high-performance CUDA library for efficient attention mechanisms in transformer models. Because it contains custom CUDA kernels, it must be compiled from source when no pre-built wheel matches the exact combination of CUDA version and PyTorch version. In this case, PyTorch 2.10.0 had been installed against CUDA 12.8, but the system's primary CUDA toolkit was 13.1 — a mismatch that triggered a source build. The assistant had already resolved the CUDA version conflict by installing a secondary CUDA 12.8 toolkit alongside 13.1, but the build itself kept crashing.

The crash pattern was consistent: the build process would spawn dozens of parallel ptxas (PTX assembler) and nvcc (NVIDIA CUDA compiler) processes, each consuming gigabytes of RAM. When too many ran simultaneously, the system ran out of memory, the kernel's OOM killer would intervene, and the build would fail with an opaque error. The assistant and user had been iterating on the MAX_JOBS environment variable — which controls how many compilation tasks run in parallel — in a desperate search for a value that would fit within the available memory.

The sequence of attempts tells a story of diminishing expectations:

The Message Itself: A Pivot Point

Message 56 arrives immediately after the assistant's attempt with MAX_JOBS=32. In the preceding message ([msg 55]), the assistant had declared "Clean. 425GB free. Now 32 jobs:" and launched the build. The user, presumably watching the build's output or seeing the system become unresponsive, recognized the familiar signs of impending OOM and intervened.

"oom, go 20" is a command, not a suggestion. The user is telling the assistant: this attempt is already failing (or will fail), kill it and retry with 20 parallel jobs. The brevity reflects the urgency — there is no time for pleasantries when a machine is thrashing under memory pressure. It also reflects the rhythm the pair had established: the user would propose a new MAX_JOBS value, the assistant would kill lingering processes and relaunch.

What makes this message remarkable is what it reveals about the user's mental model. The user is reasoning about the relationship between parallelism and memory consumption in real time. They have observed that 32 jobs consume more memory than the 432 GB machine can provide. They are not guessing randomly — they are converging on a value through binary-search-like iteration. The sequence 128 → 64 → 48 → 32 → 20 follows a pattern of aggressive reduction, overshooting, and then a more conservative step. The jump from 32 to 20 (a 37.5% reduction) suggests the user believes the OOM at 32 was not marginal — it was significant enough that a small reduction wouldn't suffice.

Assumptions Embedded in "go 20"

The message makes several implicit assumptions, most of which turned out to be correct:

  1. The OOM is caused by parallel compilation, not a memory leak. The user assumes that reducing parallelism will reduce peak memory usage proportionally. This is generally true for CUDA builds, where each compilation unit is independent and the memory used by ptxas scales linearly with the number of concurrent invocations. However, there is a floor: even a single ptxas process can consume 2-4 GB for large kernel compilations, so the total memory needed at MAX_JOBS=20 could still be 40-80 GB — well within the 425 GB available.
  2. 20 is a safe value. The user has no way of knowing this for certain without trying. They are extrapolating from the failure at 32. If 32 failed, and the memory usage at 32 was close to the OOM threshold, then 20 might succeed. But if the OOM at 32 was caused by a single massive kernel compilation (rather than the aggregate of many small ones), reducing parallelism might not help. The user is betting that the workload is evenly distributed across jobs.
  3. The assistant will understand the shorthand. The user assumes the assistant has been tracking the conversation closely enough to know that "oom" means "the build ran out of memory" and "go 20" means "kill the current build and restart with MAX_JOBS=20." This assumption is validated by the assistant's response ([msg 57]), which immediately checks memory and kills processes.
  4. The system is still responsive enough to receive commands. If the OOM had caused the SSH daemon or the kernel to become unresponsive, the message would have been useless. The user assumes the machine is thrashing but not completely locked.

What the User Needed to Know

To send this message, the user needed a fairly deep understanding of several things:

The Outcome: Success at Last

The assistant's response to "oom, go 20" was methodical. In [msg 57], it checked if the machine was alive, killed orphaned compiler processes, dropped caches, and checked memory — finding 84 GB still in use, confirming the 32-job build had indeed consumed significant memory before being killed. After a more thorough cleanup in [msg 58], memory returned to 6.7 GB used ([msg 59]). Then, in [msg 60], the assistant launched the build with MAX_JOBS=20:

Clean. Going with 20 jobs: ... Built flash-attn==2.8.3 Prepared 1 package in 9m 46s Installed 1 package in 3ms + flash-attn==2.8.3

After nearly ten minutes of compilation, flash-attn was finally installed. The three-word message had been the turning point.

Broader Significance

This message exemplifies a pattern common in infrastructure-heavy ML engineering: the collaborative, iterative debugging of build systems. The assistant brings speed and tirelessness — it can kill processes, check memory, and relaunch builds in seconds. The user brings intuition and pattern recognition — the ability to look at a failing build and know, with reasonable confidence, that 20 jobs will work when 32 did not.

The message also highlights the importance of the human-in-the-loop in AI-assisted coding. An autonomous agent might have continued trying MAX_JOBS=32 indefinitely, or escalated to 64, or given up entirely. The user's intervention — grounded in real-time observation of the system's behavior — provided the crucial insight that the agent could not generate on its own.

In the end, "oom, go 20" is a masterclass in minimal, high-signal communication. It contains a diagnosis (OOM), a target (20 jobs), and an implicit command (kill and retry). In three words and a line break, it encapsulates the entire debugging loop that had consumed the previous hour of conversation. It is the kind of message that only makes sense in context, but within that context, it is perfect.