From Bare Metal to Model Serving: The Complete Arc of an ML Infrastructure Build

Introduction

The opencode session spanning messages 0 through 83 is a remarkable case study in the full lifecycle of machine learning infrastructure work. It begins with a twelve-word user request — "Install latest nvidia drivers, cuda 13, python/uv/venv/etc ML tooling" on a remote Ubuntu 24.04 machine — and unfolds into a sprawling, multi-hour odyssey that touches nearly every layer of modern ML deployment: hardware driver installation, CUDA toolkit management, Python environment configuration, compiled CUDA extension debugging, dependency resolution warfare, and finally, model serving with performance optimization. This article synthesizes the entire chunk, tracing the narrative arc from its opening spark to its final pivot into production deployment.

Phase 1: The Foundation — Drivers, CUDA, and Python

The session opens with the user providing an SSH target (10.1.230.175) and a high-level specification of what needs to be built [1]. The assistant's first move is to assess the machine — discovering it runs Ubuntu 24.04.4 LTS, has 128 CPU cores, and is named llm-one — a clear signal of its intended purpose as a large language model server [3]. The machine initially has no NVIDIA drivers installed, and the assistant must build the entire stack from scratch.

The driver installation proceeds through several phases. The assistant adds the NVIDIA CUDA repository, installs the NVIDIA driver package (version 590.48.01), and then discovers something surprising: CUDA Toolkit 13.1 is actually available, contradicting its earlier assumption that "CUDA 13 doesn't exist yet" [7][8]. This moment of discovery — when the assistant's static knowledge is corrected by live system interrogation — is a recurring theme throughout the session. The assistant installs CUDA 13.1, configures environment paths, and verifies that two RTX PRO 6000 Blackwell GPUs are operational [9][12].

The Python environment is set up using uv, a modern Rust-based Python package manager that the user specifically requested [15][16]. The assistant creates a virtual environment at ~/ml-env and installs PyTorch. At this point, the foundation appears solid: NVIDIA drivers, CUDA 13.1, Python 3.12 with uv, and PyTorch with GPU support. The machine is ready for ML tooling.

But the foundation has a hidden crack. PyTorch's official wheels are compiled against CUDA 12.8, not the system's CUDA 13.1. This discrepancy will soon become the source of cascading failures.

Phase 2: The Flash-Attention Crucible

The installation of flash-attn — a CUDA-accelerated attention kernel library — transforms the session from a routine setup into a debugging epic. The first attempt at a bulk package install fails because flash-attn's build system, which uses PyTorch's cpp_extension module, performs a strict CUDA version check [17][19]. The system's CUDA 13.1 does not match PyTorch's CUDA 12.8 base, and the build refuses to proceed.

The assistant installs a secondary CUDA 12.8 toolkit alongside the primary 13.1 installation, setting CUDA_HOME to point at the older version during extension builds [23][24]. This dual-CUDA workaround resolves the version mismatch, but it introduces a new problem: the build process is now pointed at a different CUDA toolkit than the system default, creating a split personality in the environment that will cause confusion later.

With the CUDA version issue resolved, the assistant launches the flash-attn build — and immediately hits a second wall: memory exhaustion. The machine has 128 CPU cores, and flash-attn's build system (using Ninja) spawns up to 128 parallel nvcc and ptxas processes. Each CUDA compiler process consumes gigabytes of RAM, and the system quickly runs out of memory, becoming unresponsive [35][36].

What follows is a remarkable collaborative debugging session between the user and the assistant. The user provides terse, precise guidance — "oom, go 32" [40], "oom, go 20" [56] — while the assistant executes the builds, cleans up after failures, and reports results. The MAX_JOBS parameter is systematically reduced: 128 → 64 → 48 → 32 → 20 [38][39][48][56][60]. Each attempt requires killing lingering compiler processes, dropping kernel caches, and verifying that memory has been fully reclaimed before the next try [53][54][57][58].

The machine is rebooted with expanded RAM — from 288GB to 432GB — yet even then, builds at MAX_JOBS=32 consume 84GB before being killed [57]. Only at MAX_JOBS=20 does the build finally succeed, taking nearly 10 minutes of compilation [60]. The assistant also optimizes the build by setting TORCH_CUDA_ARCH_LIST="10.0", targeting only the Blackwell GPU architecture (sm_100) rather than compiling for multiple architectures, which reduces both build time and memory pressure [55].

This phase is a masterclass in CUDA build memory management. The key insight is that CUDA kernel compilation is not like regular C++ compilation — each ptxas invocation loads the entire PTX code for a kernel variant and performs optimization and register allocation, consuming gigabytes of memory per job. The only reliable way to determine the safe parallelism level is through empirical testing, which is exactly what the assistant conducted [56].

Phase 3: Dependency Hell — When Package Managers Fight Back

The flash-attn build succeeds, and the assistant verifies that PyTorch 2.10.0, flash-attn 2.8.3, and two RTX PRO 6000 GPUs are all operational [61][62]. The remaining ML packages — transformers, vLLM, bitsandbytes, and others — are installed in a separate command [63].

And then everything breaks.

The verification script in message 64 reveals a silent catastrophe: flash-attn's CUDA extension fails to load with an ImportError on the compiled .so file [64]. The cause is insidious. vLLM 0.15.1 has a dependency constraint that requires PyTorch ≤ 2.9.x, and the package resolver silently downgraded PyTorch from 2.10.0 to 2.9.1. Flash-attn's compiled binary was built against PyTorch 2.10's C++ ABI, and the downgrade rendered it incompatible. The Python-level imports all succeed — PyTorch works, CUDA is detected — but the C++ extension cannot load because the symbols it expects from PyTorch's runtime are different [65].

This is a particularly dangerous failure mode because it is silent. The package manager (uv/pip) treats packages as independent units and has no awareness that a compiled CUDA extension is ABI-locked to a specific PyTorch build. The downgrade happens without any warning or error [66][67].

The assistant enters a new phase of debugging: rebuilding flash-attn against the correct PyTorch version. But the dependency resolver keeps fighting back. When the assistant pins PyTorch at 2.10.0 and rebuilds flash-attn, the torchvision package forces a downgrade back to 2.9.1 [71]. Each rebuild takes ~10 minutes. The assistant must uninstall flash-attn, clean cached build artifacts, pin PyTorch, install vLLM first (which sets the PyTorch version), and then rebuild flash-attn against that pinned version [68][69][74].

The final successful build in message 75 is almost anticlimactic — it completes in 4.39 seconds because uv reuses cached build artifacts from a previous compilation [75]. The assistant recognizes the anomaly, questions it, and verifies that the cached binary is compatible with the current PyTorch 2.9.1. The verification command python3 -c "import flash_attn; print(flash_attn.__version__)" returns 2.8.3 — a clean, unambiguous success [76].

The final verified environment includes PyTorch 2.9.1+cu128, flash-attn 2.8.3 (built for Blackwell sm_100), vLLM 0.15.1, transformers 4.57.6, and a dozen other packages — all working together [77][80]. The assistant marks all four initial high-priority todos as completed in message 79, signaling that the infrastructure phase is done [80].

Phase 4: The Pivot to Model Deployment

The user's next message (81) pivots the session sharply: "Added 8 GPUs; Deploy glm-5 nvfp4 — probably requires main/nightly sglang; After running tune the params and allow for more parallel queries; ./ iirc has some load testing tool" [82].

This message transforms the conversation. The machine has been upgraded from 2 to 8 GPUs — a fourfold increase in compute capacity. The focus shifts from building infrastructure to deploying a specific model: GLM-5-NVFP4, a large language model in NVIDIA's 4-bit floating-point quantization format, optimized for Blackwell architecture. The serving framework is SGLang, specifically a nightly build that supports the NVFP4 format [82].

The user's message contains several uncertainties — "probably requires main/nightly sglang" and "iirc has some load testing tool" — that the assistant must resolve through exploration and experimentation. The performance requirements are explicit: tune parameters for parallel query throughput and validate with load testing. This is a production-grade deployment task, not a development experiment.

The empty messages at indexes 82 and 83 represent a moment of transition — the assistant processing the new directive and preparing to execute [83][84]. The session is now in a new phase, with new challenges: downloading a multi-hundred-gigabyte model, configuring SGLang for 8-GPU tensor parallelism, optimizing batch sizes and scheduling policies, and running benchmarks to validate throughput.

Themes and Lessons

Several themes emerge from this complete arc:

The fragility of CUDA extension builds. Building flash-attn from source exposed the full complexity of the CUDA compilation toolchain. Memory management during parallel compilation, ABI compatibility across PyTorch versions, and the interaction between system CUDA and PyTorch's expected CUDA version are all sources of failure that have no simple solution. The only reliable approach is systematic empirical testing — reducing MAX_JOBS, cleaning caches, and verifying at each step.

The silent dependency problem. The most dangerous failures in ML environment management are the ones that don't produce errors during installation. The PyTorch downgrade caused by vLLM's dependency constraints was invisible until runtime. This highlights a fundamental limitation of current package managers: they track Python-level dependencies but have no concept of C++ ABI compatibility between compiled extensions and their host packages.

The collaborative debugging pattern. The user and assistant worked together in a tight feedback loop. The user provided terse, precise guidance ("oom, go 32," "oom, go 20") based on system-level observations, while the assistant executed the builds, cleaned up after failures, and reported results. This pattern — human strategic oversight combined with AI tactical execution — proved highly effective for navigating complex infrastructure problems.

The importance of verification. The assistant's comprehensive verification script — importing every package in the stack and printing version numbers — was the tool that repeatedly revealed hidden problems. Without it, the silent PyTorch downgrade would have gone undetected until runtime, causing far more confusing failures.

The pivot from infrastructure to application. The session's arc — from bare-metal driver installation through CUDA toolkit management, Python environment setup, compiled extension debugging, dependency resolution, and finally model deployment — mirrors the full lifecycle of ML infrastructure work. Each phase builds on the previous, and each phase has its own failure modes and debugging techniques.

Conclusion

The opencode session spanning messages 0 through 83 is a microcosm of modern ML infrastructure work. It demonstrates that setting up an ML environment is never a straightforward installation process — it is an iterative, collaborative journey through multiple layers of software complexity. The assistant's systematic approach to debugging — forming hypotheses, testing them, cleaning up after failures, and verifying results — combined with the user's strategic guidance, produced a working environment that could serve a production model across 8 GPUs. The session stands as a testament to the value of methodical debugging, the importance of comprehensive verification, and the power of human-AI collaboration in tackling complex systems problems.## References

[1] The Seed of a Session: Deconstructing the Opening Message of an ML Environment Setup [3] The First Step: Assessing the Machine Before Building the Stack [7] The Correction: When an AI Assumes Too Much About CUDA Versioning [8] The Verification Checkpoint: Confirming a Successful CUDA and Driver Installation [9] The Quiet Infrastructure: Why Environment Paths Matter as Much as Installation [12] The Moment of Truth: Validating the GPU Stack in an ML Environment Setup [15] The Quiet Gateway: Installing uv as a Pivot Point in ML Environment Setup [16] The Innocent Command That Unleashed Chaos: Installing flash-attn on Ubuntu 24.04 [17] The First Flash-Attention Build Failure: A Turning Point in ML Environment Setup [19] The Flash-Attention Build That Revealed a CUDA Version Chasm [23] The Pivot Point: Installing a Second CUDA Toolkit to Break the Flash-Attention Build Deadlock [35] The Flash-Attn Build That Wasn't: When 128 Cores Meet CUDA's Memory Hunger [36] "OOMing, try 64": A Masterclass in Collaborative Debugging Under Resource Pressure [38] The MAX_JOBS Tango: Finding Memory Limits in Flash-Attention's CUDA Build [39] "Still oom, go 32": The Art of Collaborative Parameter Tuning in ML Infrastructure [40] "Still oom, go 32": The Four Words That Tell a Thousand Stories About Building CUDA Software [48] The 48th Attempt: When Infrastructure Fights Back [53] The Moment Memory Was Reclaimed: A Deep Dive Into Linux Cache Management During ML Build Failures [54] The Quiet Verification: How a Simple free -h Command Carried the Weight of an Entire Build Saga [55] The Flash-Attention Gauntlet: A Case Study in CUDA Build Memory Management [56] The Three-Character Debugging Session: How "oom, go 20" Unlocked a Flash-Attention Build [57] The Cleanup That Finally Worked: A Case Study in ML Build Debugging [60] The 20-Job Threshold: A Case Study in CUDA Build Memory Management [61] The Moment of Truth: Verifying a Hard-Won ML Environment [62] The Verification That Revealed a Missing Piece [63] The Quiet Payoff: Installing the Remaining ML Packages After the Flash-Attn Ordeal [64] The Moment of Discovery: When a Verification Script Reveals a Silent Dependency Conflict [65] The ABI Trap: When vLLM Downgrades PyTorch and Breaks Flash-Attention [66] When the Dependency Resolver Fights Back [67] The Ghost in the Package Manager [68] The Pivot: When Dependency Hell Forces a Strategy Change [69] The Stale .so: A Lesson in Compiled Python Extension ABI Mismatch [71] The Dependency Trap: How One Innocent Package Install Unraveled a Carefully Built ML Environment [74] The Final Rebuild: Resolving a Flash-Attention ABI Mismatch in a Complex ML Environment [75] The Moment of Relief: A Cached Build Succeeds After Hours of Dependency Hell [76] The Moment of Truth: When a Nightmare Build Finally Succeeds [77] The Victory Lap: How a Single Verification Command Captured the Culmination of an ML Environment Setup [80] The Todo That Marked a Milestone: How a Simple Status Update Captured the Completion of a Complex ML Environment Build [81] The Final Summary: Consolidating Knowledge After an ML Environment Setup Ordeal [82] The Pivot: From Infrastructure to Inference [83] The Silence Between Worlds: An Empty Message at the Pivot Point [84] The Silence That Speaks: An Empty Message as a Pivot Point in an ML Deployment Conversation