"OOM again, try 16": Three Words That Reveal the Rhythm of Human-AI Collaboration

In a conversation spanning thousands of messages about training an EAGLE-3 speculative decoding drafter for the Kimi-K2.5 language model, one user message stands out for its extraordinary concision. The message — three words long — reads in its entirety:

OOM again, try 16

This is not a question. It is not a request for clarification. It is a crisp, context-saturated directive issued by a human collaborator to an AI assistant in the midst of a high-stakes GPU memory optimization problem. To understand why this message was written and what it accomplishes, one must reconstruct the dense web of shared context, failed attempts, and mutual understanding that preceded it.

The Binary Search for a Memory Ceiling

The story begins with a performance observation. The assistant had launched EAGLE-3 training on 4 NVIDIA RTX PRO 6000 Blackwell GPUs (each with 96 GB of VRAM) using the speculators library's sequence packing mechanism. Because the model's forward pass expects a batch dimension of exactly 1, the only way to increase GPU utilization was to increase --max-seq-len — the total number of tokens packed into each training step via block-diagonal attention masking ([msg 4261]). At max_seq_len=4096, the GPUs were drawing only 250W out of a 600W power budget, indicating severe underutilization despite showing 100% GPU-Util.

The assistant escalated to max-seq-len=8192, which improved power draw to 412-476W and VRAM usage to 39.8 GB ([msg 4264]). The user immediately flagged that 8192 was too small — many training sequences were themselves close to 8192 tokens, meaning the packing mechanism could fit at most one sample per batch ([msg 4263]). The assistant agreed and jumped to max-seq-len=32768, aiming to pack 4-8 samples per step (<msg id=4265-4266>).

That attempt failed with an out-of-memory (OOM) error. The user reported it in a single word: "OOMed, 24k?" ([msg 4268]), suggesting a retreat to 24576. The assistant killed the process, cleaned the output directory, and relaunched at 24576 ([msg 4270]). A monitoring command with a 180-second sleep was dispatched to check the result ([msg 4271]).

The Message Itself: What "OOM again, try 16" Communicates

Before the assistant's 180-second sleep timer expired, the user already had the answer: 24576 also OOMed. The target message — "OOM again, try 16" — is the user's rapid-fire report and next instruction, issued before the assistant's own monitoring could fire.

The message accomplishes several things simultaneously:

  1. Status report: "OOM again" tells the assistant that the 24576 attempt also failed due to insufficient GPU memory. The user is monitoring the training logs or process state in real time, faster than the assistant's scheduled check.
  2. Corrective directive: "try 16" instructs the assistant to relaunch with --max-seq-len 16384. The user has already performed the mental arithmetic: 32768 OOMed, 24576 OOMed, so the next power-of-two step down is 16384.
  3. Implicit action sequence: The message assumes the assistant will kill the current (already-failed) process, clean the output directory, and relaunch with the new parameter — the same ritual performed after each prior failure.
  4. Trust in shared context: The user does not spell out "max-seq-len=16384" or "please kill and restart the training." The abbreviation "16" is unambiguous only within the tightly scoped conversation about this specific parameter. The user trusts that the assistant will correctly interpret it.

The Reasoning Behind the User's Suggestion

The user's binary search strategy reveals a practical engineering intuition. The known working point was max-seq-len=8192 at 39.8 GB VRAM. The known failure points were 32768 and 24576. The next candidate, 16384, represents a doubling from the working point and a halving from the nearest failure. This is classic binary search applied to resource limits: find the largest value that fits within the 96 GB budget.

However, the relationship between max-seq-len and VRAM consumption is not linear. The packed sequence mechanism uses block-diagonal attention masks with flex_attention, where the computational cost scales quadratically with the length of individual sequences within the pack, not with the total packed length. Additionally, hidden state tensors scale linearly with sequence length, while attention scores scale quadratically. Going from 8192 to 24576 (a 3× increase) would naively suggest ~120 GB for the linear components alone — already exceeding 96 GB — which explains the OOM. The user's instinct to try 16384 next is sound: it represents a 2× increase from the working point, which might keep total VRAM under the 96 GB ceiling.

Assumptions Embedded in the Message

The user makes several assumptions, all of which are validated by the assistant's subsequent actions (<msg id=4273-4274>):

What the Message Reveals About the Collaboration

This three-word message is remarkable because of what it does not contain. There is no greeting, no explanation, no hedging, no question mark. It is pure signal, stripped of all social lubrication. This level of compression is only possible when both parties share an extraordinarily rich context: the parameter being tuned, the history of failed attempts, the standard operating procedure for restarting training, and the mutual understanding that time is valuable.

The user is acting as a rapid feedback loop — a human-in-the-loop monitoring system that can interpret OOM errors and suggest corrective parameters faster than the assistant's own monitoring infrastructure. The assistant, for its part, accepts this direction without resistance, immediately executing the suggested change. The rhythm is almost athletic: the user spots the failure, calls out the next parameter, and the assistant executes the restart. The 180-second sleep timer in the assistant's monitoring command becomes irrelevant because the human feedback cycle is faster.

Input and Output Knowledge

To understand this message, a reader needs to know: that the training uses a --max-seq-len parameter controlling how many tokens are packed per GPU step; that the previous attempts at 32768 and 24576 both failed with OOM errors; that the assistant has a ritual of killing processes, cleaning output directories, and relaunching; and that "16" is shorthand for 16384 within this context.

The output knowledge created by this message is the discovery that max-seq-len=16384 is viable (as confirmed by the subsequent successful launch in [msg 4274]), while 24576 and above are not. This establishes the memory ceiling for this specific model configuration on these GPUs, a piece of systems knowledge that informs future training runs.

Conclusion

"OOM again, try 16" is a masterclass in efficient human-AI communication. In three words, the user reports a failure, diagnoses the cause, proposes a fix, and triggers an execution sequence — all within a shared context so dense that each word carries a paragraph of meaning. The message reveals a collaboration where the human provides rapid, intuitive decision-making and the AI provides reliable execution, with both parties operating at the limit of mutual understanding. It is a small moment in a long conversation, but it captures something essential about how humans and AI systems can work together when trust and context are fully established.