The Moment CUDA Woke Up: Disabling HMM to Unlock 8 Blackwell GPUs on Proxmox

"CUDA WORKS! cuInit: 0 (success!) and 8 devices detected! The fix was disabling HMM on the nvidia_uvm module."

This single message, message index 564 in the conversation, is the culmination of a long and frustrating debugging session. It is the moment when a system that had been stubbornly refusing to cooperate finally yielded. The message is short — just two sentences of celebration followed by a command to start an LXC container — but it represents hours of investigation, multiple dead ends, and a deep dive into the interaction between NVIDIA's open kernel module, the Proxmox VE kernel, and a feature called Heterogeneous Memory Management (HMM). To understand why this message matters, we need to understand everything that led up to it.

The Problem: CUDA Would Not Initialize

The broader context of this coding session is the deployment of a GLM-5-NVFP4 large language model across 8 NVIDIA RTX PRO 6000 Blackwell GPUs. The user had set up a Proxmox VE hypervisor and was trying to get GPU compute working reliably. After discovering that KVM virtual machines suffered from PCIe P2P (Peer-to-Peer) bottlenecks due to VFIO/IOMMU virtualization overhead, the team switched to an LXC container approach, which provides near-bare-metal GPU access.

But a new blocker emerged: CUDA would not initialize. The cuInit() function — the very first call any CUDA application makes — was returning error code 3 (CUDA_ERROR_NOT_INITIALIZED) on the host machine, and in some cases it was hanging indefinitely, requiring a timeout kill. This was a showstopper. Without CUDA initialization, none of the inference workloads could run.

The Investigation: Tracing the Root Cause

The assistant's investigation in the preceding messages ([msg 546] through [msg 563]) is a textbook example of systematic debugging. The assistant tested multiple hypotheses:

  1. Missing GSP firmware: The NVIDIA GPU System Processor (GSP) requires firmware files. The assistant checked /lib/firmware/nvidia/590.48.01/ and found only gsp_ga10x.bin and gsp_tu10x.bin — no Blackwell-specific firmware. However, further analysis showed that gsp_ga10x.bin actually contains Blackwell ELF sections (kernel_gb20x.elf, kernel_gb20y.elf) inside it, and nvidia-smi confirmed the GSP firmware version was 590.48.01 and running. So firmware was not the issue.
  2. Kernel version mismatch: The PVE kernel was 6.8.12-9, and a newer version 6.8.12-18 was available. The assistant considered upgrading, but this turned out to be a red herring.
  3. CUDA userspace library mismatch: The assistant checked ldconfig and confirmed the correct libcuda.so.1 was linked to the 590.48.01 driver version. Not the issue.
  4. MIG configuration: The assistant noticed MIG minors listing up to gpu14, but correctly identified this as normal template entries, not a problem.
  5. The HMM connection: The breakthrough came when the assistant noticed Addressing Mode: HMM in the nvidia-smi output and found a GitHub issue ([msg 557]) describing the exact same problem: cuInit() returns CUDA_ERROR_NOT_INITIALIZED (3) with the open kernel module. The fix was setting uvm_disable_hmm=1.

The Fix: Disabling Heterogeneous Memory Management

Heterogeneous Memory Management (HMM) is a feature in the NVIDIA open kernel module that allows the GPU to access system memory using the CPU's page tables, enabling seamless memory sharing between CPU and GPU. However, HMM requires specific kernel infrastructure (mmu_notifier, migrate_vma, etc.) that may not be fully functional on all kernels — particularly on the Proxmox VE kernel, which is based on the Ubuntu kernel but with additional patches for virtualization.

The assistant applied the fix in [msg 560] by creating /etc/modprobe.d/nvidia-uvm-hmm.conf with the content options nvidia_uvm uvm_disable_hmm=1. This tells the nvidia_uvm kernel module to disable HMM at load time. After reloading the module (rmmod nvidia_uvm && modprobe nvidia_uvm), the parameter was confirmed active (uvm_disable_hmm=Y), and cuInit() immediately returned success.

Why This Message Was Written

Message 564 serves as the victory announcement. It is written with palpable relief and excitement — the assistant leads with "CUDA WORKS!" in bold, followed by the triumphant output cuInit: 0 and 8 devices detected. The message accomplishes several things:

  1. Confirms the fix worked: The primary purpose is to document that the HMM disable resolved the CUDA initialization failure. This is the output knowledge created by this message — the system now has working CUDA with all 8 GPUs accessible.
  2. Provides a clear summary: The assistant explicitly states "The fix was disabling HMM on the nvidia_uvm module," distilling hours of debugging into one clear sentence. This is valuable for anyone reading the conversation log later.
  3. Moves to the next step: The message immediately transitions to starting the LXC container (pct start 129) to verify CUDA works inside the container as well. This shows the assistant is not resting on the victory but is eager to validate the fix in the actual deployment environment.

Assumptions Made

The assistant made several assumptions in this message and the preceding investigation:

  1. That the fix would work inside the LXC container too: The assistant assumed that since the nvidia_uvm module parameter is set at the host level, the LXC container (which shares the host kernel) would also benefit from the fix. This turned out to be correct, but it was an assumption that needed verification.
  2. That the HMM issue was the sole cause of the CUDA failure: The assistant had previously observed cuInit hanging (exit code 124 from timeout) as well as returning error code 3. The HMM fix addressed the error code 3 case, but the hang might have been a separate issue or a symptom of the same underlying problem manifesting differently under different timing conditions.
  3. That the Proxmox VE kernel was the root cause: The assistant implicitly assumed that the PVE kernel lacked proper HMM support, which is why the fix was needed. This is a reasonable assumption given that the proprietary NVIDIA driver (which doesn't use HMM) reportedly worked fine.

Input Knowledge Required

To understand this message, the reader needs knowledge of:

  1. CUDA initialization flow: Understanding that cuInit() is the entry point for CUDA, and that error code 3 (CUDA_ERROR_NOT_INITIALIZED) means the CUDA driver could not be initialized.
  2. NVIDIA kernel module architecture: The distinction between nvidia (core driver), nvidia_uvm (Unified Virtual Memory), nvidia_modeset, and nvidia_drm. The fix targeted nvidia_uvm specifically.
  3. HMM (Heterogeneous Memory Management): A feature in the open NVIDIA kernel module that enables unified memory access between CPU and GPU. Understanding that HMM requires specific kernel support that may not be available on all kernels.
  4. Proxmox VE and LXC: Understanding that Proxmox VE is a virtualization platform based on Debian/Ubuntu with a custom kernel, and that LXC containers share the host kernel but have their own filesystem namespace.
  5. Module parameters and modprobe: The mechanism for passing parameters to kernel modules via /etc/modprobe.d/ configuration files.

Output Knowledge Created

This message creates several pieces of output knowledge:

  1. The fix is confirmed: uvm_disable_hmm=1 resolves CUDA initialization failure on Proxmox VE with NVIDIA open kernel module 590.48.01 and Blackwell GPUs. This is a concrete, reproducible solution.
  2. All 8 GPUs are accessible: The cuInit: 0 success and 8 devices detected confirms that the system can now use all installed GPUs.
  3. The LXC container can be started: The assistant successfully starts the container, setting up the next phase of testing.
  4. A documented workaround exists: The GitHub issue reference ([msg 557]) provides a link to the upstream bug report, creating a chain of knowledge that future readers can follow.

The Thinking Process

The assistant's thinking process in the messages leading up to 564 is visible and instructive. The assistant:

  1. Formulated multiple hypotheses and tested them systematically, ruling out GSP firmware, kernel version, and library mismatches before landing on HMM.
  2. Used web search effectively to find the exact GitHub issue matching the symptoms. This is a crucial skill — knowing when to stop debugging from first principles and search for known issues.
  3. Interpreted strace output correctly: When strace showed cuInit exiting with code 0 while the Python test showed error code 3, the assistant didn't dismiss the discrepancy but instead ran the test again more carefully, eventually confirming the error code 3 return.
  4. Connected the dots between Addressing Mode: HMM in nvidia-smi and the GitHub issue: This required understanding what HMM is and recognizing that the PVE kernel might not support it properly.
  5. Applied the fix with proper module reloading: The assistant correctly stopped the LXC container first, then unloaded and reloaded the nvidia_uvm module, and verified the parameter was active before testing.

The Broader Significance

This message is a turning point in the session. Before it, the team was stuck — CUDA wouldn't initialize, and none of the inference workloads could run. After it, the path is clear: the LXC container can be started, CUDA works inside it, and the team can proceed to benchmarking and performance tuning.

The fix itself — disabling HMM — is a trade-off. HMM enables more efficient memory sharing between CPU and GPU, and disabling it may impact performance for certain workloads. However, for the immediate goal of getting the GLM-5-NVFP4 model deployed and benchmarked, having working CUDA is far more important than optimal memory management. The team can revisit HMM later if performance tuning reveals it as a bottleneck.

The message also demonstrates the value of persistence in debugging. The assistant went through multiple rounds of testing, searching, and hypothesis refinement before finding the solution. The final fix was simple — a one-line configuration file — but finding it required understanding the complex interaction between the NVIDIA driver, the kernel, and the virtualization platform.

Conclusion

Message 564 is a brief moment of triumph in a long debugging session, but it encapsulates the entire journey: the systematic investigation, the critical insight connecting HMM to the CUDA failure, the application of a known workaround, and the immediate transition to the next phase. It is a reminder that in complex systems engineering, the most elegant solutions often come from understanding the deepest layers of the stack — in this case, the interaction between a kernel module parameter and a GPU driver's memory management strategy. The message is short, but the knowledge it represents is substantial: a documented, reproducible fix for CUDA initialization on Proxmox VE with Blackwell GPUs, and a clear path forward for the deployment.