The Breakthrough: Disabling HMM to Unlock CUDA on Blackwell GPUs in an LXC Container

A Pivotal Moment in a Complex Debugging Odyssey

In any sufficiently complex systems engineering effort, there comes a moment when a stubborn blocker finally yields. Message <msg id=566> captures exactly such a moment in a long-running session to deploy the GLM-5-NVFP4 large language model across eight NVIDIA RTX PRO 6000 Blackwell GPUs. The message is deceptively brief — a triumphant status update, a todo list revision, and a handoff to the next phase — but it represents the culmination of an intricate debugging chain that spanned multiple sessions, involved kernel module parameters, GitHub issue archaeology, and a fundamental understanding of NVIDIA's Heterogeneous Memory Management (HMM) subsystem.

The Message Itself

The assistant writes:

CUDA is fully working in the LXC container! All 8 GPUs detected, matrix multiply runs fine. The BLOCKER is resolved!

This is followed by a todowrite call that updates the task list, marking two items as completed:

The Debugging Chain That Led Here

To understand why this message matters, one must trace the path that led to it. The session had been struggling with a critical failure: cuInit() — the fundamental CUDA runtime initialization call — was returning error code 3 (CUDA_ERROR_NOT_INITIALIZED) on both the Proxmox host and inside an LXC container. Earlier attempts had even seen cuInit() hang indefinitely, making the GPUs completely inaccessible for any compute workload.

The assistant had methodically ruled out several hypotheses:

  1. Missing GSP firmware: The NVIDIA driver 590.48.01 uses a GPU System Processor (GSP) for certain initialization tasks. The assistant checked /lib/firmware/nvidia/590.48.01/ and found only gsp_ga10x.bin and gsp_tu10x.bin — no explicit Blackwell firmware file. However, closer inspection revealed that gsp_ga10x.bin is a container format that actually includes Blackwell ELF sections (kernel_gb20y.elf, kernel_gb20x.elf) inside it. Moreover, nvidia-smi -q confirmed "GSP Firmware Version: 590.48.01" was loaded and running. So firmware was not the issue.
  2. Kernel version mismatch: The Proxmox host was running kernel 6.8.12-9, and a newer 6.8.12-18 was available. But the assistant wisely deferred this upgrade path after discovering the real cause.
  3. CUDA userspace library mismatch: The host had CUDA Toolkit 12.8 installed, while the driver reported "CUDA Version: 13.1." The assistant checked library paths and confirmed the correct libcuda.so.590.48.01 was linked. This was not the problem. The decisive clue came from a web search that landed on NVIDIA's open-gpu-kernel-modules GitHub issue #947, which was a duplicate of issue #797. Both described the exact same symptom: cuInit() returning error 3 with the open kernel module while the proprietary module worked fine. The recommended workaround was to set uvm_disable_hmm=1 as a module parameter for nvidia_uvm. This made immediate sense. The Proxmox host's nvidia-smi output showed "Addressing Mode: HMM" — Heterogeneous Memory Management. HMM is a feature of the NVIDIA open kernel module that allows the GPU to access system memory through the kernel's memory management subsystem, but it requires specific kernel infrastructure (mmu_notifiers, migration primitives) that may not be fully functional on Proxmox VE kernels. Disabling HMM forces the driver to use a more conservative memory management path that works reliably.

The Fix in Action

The assistant executed the fix in message <msg id=560>:

echo 'options nvidia_uvm uvm_disable_hmm=1' > /etc/modprobe.d/nvidia-uvm-hmm.conf

Then stopped the container, unloaded and reloaded the nvidia_uvm kernel module:

rmmod nvidia_uvm && modprobe nvidia_uvm

Verification showed uvm_disable_hmm=Y in the module parameters. A quick test on the host (message <msg id=563>) returned:

cuInit: 0
devices: 8

Then the container was restarted and tested (message <msg id=565>):

CUDA available: True
Device count: 8
  GPU 0: NVIDIA RTX PRO 6000 Blackwell Server Edition
  ...
  GPU 7: NVIDIA RTX PRO 6000 Blackwell Server Edition
Matrix multiply test: OK, shape: torch.Size([1000, 1000])

Message <msg id=566> is the immediate follow-up to that successful test — the assistant's synthesis of what just happened and what comes next.

Reasoning and Decision-Making Visible in the Message

The message reveals several layers of reasoning:

Confidence through verification: The assistant doesn't just declare victory based on the host test. It waited for the container test to succeed before marking the blocker as resolved. The container is the actual deployment target — the LXC environment where the sglang inference server will run. Success on the host was necessary but not sufficient.

Prioritization discipline: The todo list is meticulously maintained. Completed items are marked with "FIXED" and "WORKED!" — the exclamation marks convey genuine relief and satisfaction. The next task ("Verify GPU topology and run quick NCCL/P2P test") is promoted to "in_progress" before the message even finishes, showing that the assistant is already thinking about what the fix enables.

Forward-looking orientation: The message doesn't dwell on the debugging that led here. It acknowledges the blocker is resolved and immediately pivots to what's next. This is characteristic of effective engineering communication: celebrate briefly, then move on.

Assumptions Embedded in the Message

Several assumptions are at play:

  1. The fix is stable: The assistant assumes that disabling HMM is a valid long-term configuration, not just a temporary workaround. This is a reasonable assumption given that it's a documented kernel module parameter and the proprietary driver (which doesn't use HMM) works fine. However, there may be performance implications — HMM can improve GPU memory oversubscription scenarios, and disabling it might limit certain workloads.
  2. The container environment is equivalent to bare metal: The LXC container has direct access to the NVIDIA devices via bind-mounts, and the test confirms CUDA works. But the assistant implicitly assumes that the GPU topology inside the container will match the host's topology. This is validated in the very next message (<msg id=567>) where nvidia-smi topo -m shows NODE/SYS connections at 53 GB/s same-NUMA — a major improvement over the VFIO-constrained KVM VM.
  3. The fix generalizes: The assistant assumes that the uvm_disable_hmm=1 parameter will persist across reboots (it's in /etc/modprobe.d/) and that no other HMM-related issues will surface. This is a safe assumption for a production deployment, but it's worth noting that future kernel or driver upgrades might change the HMM implementation and potentially break things again.

Input Knowledge Required

To fully understand this message, one needs:

Output Knowledge Created

This message creates and confirms:

  1. A validated fix: The uvm_disable_hmm=1 workaround is confirmed to resolve cuInit error 3 on Proxmox VE 6.8.12-9 with NVIDIA driver 590.48.01 and Blackwell RTX PRO 6000 GPUs. This is valuable knowledge for anyone running NVIDIA open kernel modules on Proxmox or similar virtualized environments.
  2. A working deployment baseline: CUDA is confirmed operational in the LXC container with all 8 GPUs accessible. The matrix multiply test validates basic compute functionality. This unblocks the entire downstream workflow: installing sglang, loading the GLM-5-NVFP4 model, and running inference benchmarks.
  3. A confirmed topology hypothesis: The next message (<msg id=567>) validates that the LXC approach provides true bare-metal GPU topology with NODE-level connectivity between GPU pairs on the same NUMA node, avoiding the SYS-level (PCIe root complex) penalties imposed by VFIO passthrough in KVM VMs.
  4. A task transition: The todo list cleanly moves from "fix the blocker" to "characterize the environment." This creates a clear handoff point for both human readers and the AI's own context management.

The Thinking Process Visible in the Message

While message <msg id=566> is short, the thinking process is visible in its structure and timing. The assistant has just received the output of the container CUDA test (<msg id=565>) and immediately synthesizes a response. The thinking is:

  1. Confirm success: The test output shows CUDA available, 8 devices, and a successful matrix multiply. No errors.
  2. Declare the blocker resolved: This is the headline — the thing that has been blocking progress for multiple rounds is finally fixed.
  3. Update the todo system: The todo list is the assistant's persistent memory. By marking items completed and promoting the next task, the assistant ensures continuity across messages.
  4. Proceed immediately: The next message (<msg id=567>) runs nvidia-smi topo -m and enables persistence mode without any gap. There is no hesitation or re-verification — the assistant trusts the test results and moves forward. This pattern — verify, declare, update, proceed — is characteristic of effective autonomous engineering agents. The todo list serves as both memory and commitment device, ensuring that no task is forgotten and that progress is always visible.

Broader Significance

Message <msg id=566> is a microcosm of the entire debugging journey. It illustrates several important principles:

The power of community knowledge: The fix came from a GitHub issue — someone else had encountered and documented this exact problem. Without that issue, the assistant might have spent hours chasing kernel upgrades, firmware versions, or library mismatches. The ability to search, find, and apply community knowledge is perhaps the single most valuable capability in modern systems engineering.

The importance of reading the right signal: The assistant noticed "Addressing Mode: HMM" in nvidia-smi -q output and connected it to the cuInit failure. This is the kind of subtle signal that only makes sense in retrospect but is critical for diagnosis.

The layered nature of debugging: Each hypothesis (firmware, kernel version, library mismatch, HMM) was tested and eliminated in turn. The final fix was not the most obvious candidate — it required understanding the relationship between the open kernel module, the UVM driver, and the kernel's memory management infrastructure.

Containerization as a topology workaround: The LXC approach was chosen specifically to avoid VFIO's PCIe topology limitations. The HMM fix was the last piece needed to make that approach viable. Together, they represent a creative workaround for a hardware limitation (each GPU on its own PCIe root complex preventing native P2P) using software configuration.

Conclusion

Message <msg id=566> is brief — barely a paragraph plus a todo list — but it represents the resolution of a critical blocker that had halted an entire deployment pipeline. The fix itself was elegant: a single kernel module parameter, uvm_disable_hmm=1, that disables a feature incompatible with the Proxmox VE kernel. The message captures the moment when a complex debugging chain converges on a solution, validated through careful testing, and transitions into the next phase of work. It is a testament to the power of systematic debugging, community knowledge, and the discipline of maintaining clear task state across a long-running engineering session.