The Moment CUDA Woke Up: How a Single Kernel Parameter Unblocked Eight Blackwell GPUs

The Message

uvm_disable_hmm=Y (enabled = HMM is disabled). Now let's test cuInit on the host!

>

``bash ssh root@10.1.2.6 'python3 -c " import ctypes lib = ctypes.CDLL(\"libcuda.so.1\") r = lib.cuInit(0) print(\"cuInit:\", r) if r == 0: count = ctypes.c_int() lib.cuDeviceGetCount(ctypes.byref(count)) print(\"devices:\", count.value) else: print(\"CUDA INIT FAILED with code\", r) " 2>&1' cuInit: 0 devices: 8 ``

This message, index 563 in the conversation, is a watershed moment in a grueling debugging session that spanned hours of investigation across a Proxmox virtualized environment. It represents the exact instant when a stubborn, multi-faceted CUDA initialization failure finally yielded to a precise, well-researched fix. The message is deceptively brief — just a status line, a bash command, and its triumphant output — but it encapsulates the resolution of a problem that had blocked all progress on deploying the GLM-5-NVFP4 large language model across eight NVIDIA RTX PRO 6000 Blackwell GPUs.

The Debugging Journey That Led Here

To understand why this message was written, one must appreciate the wall the assistant had been hitting. For several rounds prior ([msg 546] through [msg 562]), every attempt to initialize CUDA had failed. The cuInit() function — the very first call any CUDA program makes — was either returning error code 3 (CUDA_ERROR_NOT_INITIALIZED) or hanging indefinitely until killed by a timeout. This was not a subtle bug; it was a complete blocker. No GPU compute of any kind was possible.

The assistant had methodically worked through a differential diagnosis. Was it a missing GSP (GPU System Processor) firmware? No — nvidia-smi confirmed GSP firmware version 590.48.01 was loaded and running, and the firmware binary actually contained Blackwell (gb20x, gb20y) ELF sections inside its gsp_ga10x.bin container. Was it a kernel version mismatch? The Proxmox host was running PVE kernel 6.8.12-9, and a newer 6.8.12-18 was available but not yet applied. Was it a device file permission issue? The strace output showed all openat() calls to /dev/nvidia* succeeding without errors. Was it a CUDA userspace library mismatch? The system libraries pointed to the correct 590.48.01 versions.

Each hypothesis was tested and eliminated, narrowing the field. The breakthrough came when the assistant searched the NVIDIA open-gpu-kernel-modules issue tracker and found [msg 557] a direct match: Issue #947, "cuInit() returns CUDA_ERROR_NOT_INITIALIZED (3) with open kernel module; proprietary module works," which was closed as a duplicate of Issue #797. The root cause was the NVIDIA open kernel module's Heterogeneous Memory Management (HMM) feature, which was incompatible with certain kernel configurations — precisely the situation on this Proxmox host.

Why This Message Was Written: The Reasoning and Motivation

This message was written to confirm that the fix worked. The assistant had just applied the workaround — setting uvm_disable_hmm=1 as a module parameter for nvidia_uvm ([msg 560]), then reloaded the kernel module ([msg 562]). The status line "uvm_disable_hmm=Y (enabled = HMM is disabled)" is a quick verification that the parameter was accepted by the kernel module. The "Y" output from /sys/module/nvidia_uvm/parameters/uvm_disable_hmm confirmed that HMM was now disabled.

But a parameter file showing "Y" is not proof that CUDA actually works. The assistant needed to run the definitive test: call cuInit() and see if it returns 0 (success) instead of 3 (failure) or hangs. The Python one-liner is the simplest possible CUDA smoke test — import the CUDA driver library via ctypes, initialize the driver, and count the devices. It's the same test that had been failing for rounds.

The motivation was high-stakes. Without CUDA working, the entire deployment of GLM-5-NVFP4 — a massive Mixture-of-Experts model requiring eight GPUs with tensor parallelism — was impossible. The team had already invested enormous effort: setting up the Proxmox environment, installing NVIDIA drivers 590.48.01, configuring LXC containers for bare-metal GPU access, and debugging the VFIO/IOMMU bottlenecks that plagued the KVM VM approach ([msg 546]). This CUDA initialization failure was the last software barrier before they could finally benchmark the model.

How the Decision Was Made

The decision to try uvm_disable_hmm=1 was not arbitrary — it was the result of targeted research. The assistant found the exact bug report on GitHub, read the resolution, and recognized that the symptoms matched perfectly. The Proxmox host's nvidia-smi output showed "Addressing Mode: HMM" ([msg 556]), confirming the driver was using HMM. The PVE kernel 6.8.12-9, being a Proxmox-specific kernel with its own memory management patches, was a likely candidate for HMM incompatibility.

The fix was also minimally invasive. Rather than upgrading the kernel (which carried risk of breaking the Proxmox environment), recompiling the NVIDIA driver, or switching to the proprietary kernel module (which might have licensing implications in a container context), the assistant chose a single module parameter change. This could be applied dynamically by reloading nvidia_uvm without even rebooting the host.

The order of operations was careful: first create the modprobe configuration file (/etc/modprobe.d/nvidia-uvm-hmm.conf), then stop the LXC container that might be holding GPU device references, then rmmod nvidia_uvm followed by modprobe nvidia_uvm to reload with the new parameter. Only then run the test.

Assumptions Made

The message and the actions leading to it rest on several assumptions:

  1. The HMM issue is the sole cause of the cuInit failure. This was a reasonable assumption given the exact symptom match with the GitHub issue, but it wasn't guaranteed. Other factors could have contributed, and the test could have failed for different reasons even after disabling HMM.
  2. The module parameter change is sufficient without a reboot. The assistant assumed that unloading and reloading nvidia_uvm would fully apply the new parameter. This is correct for module parameters loaded at modprobe time, but it required that no other kernel modules or processes held references to nvidia_uvm that would prevent unloading.
  3. The Python ctypes approach is a valid CUDA initialization test. This assumes that the CUDA userspace driver library (libcuda.so.1) is correctly installed and linked, and that calling cuInit(0) with flag 0 is sufficient to initialize the driver. In practice, cuInit(0) is the standard initialization call, so this assumption is sound.
  4. The test environment (SSH to the Proxmox host) has the same CUDA environment as the LXC container. The assistant was testing on the host directly, not inside the container. If the container had additional restrictions (e.g., cgroup device access, AppArmor profiles, or different library paths), the fix might work on the host but fail in the container. This assumption was later validated when the container was also tested successfully.

Mistakes and Incorrect Assumptions Along the Way

The path to this message was not without wrong turns. Earlier in the debugging session, the assistant spent significant time investigating GSP firmware as the culprit ([msg 547], [msg 549]). The hypothesis was that the NVIDIA 590 driver needed Blackwell-specific GSP firmware that wasn't present in the firmware directory. This was a reasonable line of inquiry — the firmware directory only contained gsp_ga10x.bin and gsp_tu10x.bin, and no gsp_gb* files existed. However, strings analysis of the gsp_ga10x.bin file revealed that it actually contained Blackwell firmware internally ([msg 551]), and nvidia-smi confirmed GSP was loaded and functional. This was a dead end.

Another incorrect assumption was that the PVE kernel version (6.8.12-9) was too old for the open kernel module ([msg 549]). The assistant had planned a kernel upgrade as a potential fix. While kernel version could certainly matter for HMM support, the actual fix was a module parameter that worked on the existing kernel, making the upgrade unnecessary for this particular issue.

The assistant also initially misinterpreted the strace output ([msg 552]), which showed cuInit completing successfully in one run but returning error code 3 in another. This intermittent behavior was confusing — sometimes cuInit hung, sometimes it returned quickly with error 3. The inconsistency was likely due to race conditions in the HMM initialization path, where certain kernel memory management operations would block or fail depending on system state.

Input Knowledge Required to Understand This Message

To fully grasp the significance of this message, a reader needs knowledge in several domains:

CUDA Driver Architecture: Understanding that cuInit() is the entry point for all CUDA operations, that it initializes the driver stack and enumerates devices, and that error code 3 (CUDA_ERROR_NOT_INITIALIZED) means the driver failed to establish communication with the kernel module.

NVIDIA Open Kernel Module vs. Proprietary Module: The open kernel module (nvidia-open) is a GPL-licensed alternative to the proprietary driver. It uses different code paths for memory management, including HMM (Heterogeneous Memory Management), which leverages the Linux kernel's mmu_notifier and migrate_vma infrastructure for GPU memory migration.

HMM and Its Requirements: HMM allows the GPU driver to participate in the kernel's unified memory management, enabling features like transparent GPU memory migration. It requires specific kernel support (CONFIG_HMM, CONFIG_MMU_NOTIFIER) and can be incompatible with kernels that have custom memory management patches — like Proxmox VE kernels, which include additional virtualization and container memory management code.

Proxmox VE and LXC: Understanding that Proxmox is a virtualization platform based on KVM and LXC, and that LXC containers share the host kernel but have isolated namespaces. GPU passthrough to LXC requires bind-mounting the NVIDIA device files and having the NVIDIA kernel modules loaded on the host.

Kernel Module Parameters: The mechanism of passing parameters to Linux kernel modules at load time via /etc/modprobe.d/ configuration files, and the ability to dynamically reload modules with rmmod/modprobe.

Output Knowledge Created by This Message

This message created several important pieces of knowledge:

  1. A confirmed fix for CUDA initialization failure with NVIDIA open kernel module on Proxmox: The combination of uvm_disable_hmm=1 and the 590.48.01 driver works on PVE kernel 6.8.12-9 with Blackwell GPUs. This is a specific, actionable recipe that can be applied to similar environments.
  2. Validation of the debugging methodology: The systematic approach — ruling out firmware issues, kernel version, device permissions, and library paths before finding the HMM bug report — was validated as effective. The message demonstrates the value of targeted web search for known issues in open-source projects.
  3. Confirmation of eight-GPU enumeration: The output devices: 8 confirms that all eight NVIDIA RTX PRO 6000 Blackwell GPUs are visible and accessible to CUDA. This was non-trivial — earlier in the session, there were concerns about GPU topology, PCIe enumeration, and MIG configuration that could have reduced the visible device count.
  4. A baseline for further work: With CUDA initialized, the assistant could proceed to launch the SGLang inference server, run benchmarks, and characterize performance. The message marks the transition from environment debugging to application deployment.

The Thinking Process Visible in the Reasoning

The assistant's reasoning, visible in the preceding messages, shows a clear diagnostic pattern. The investigation moved from broad hypotheses to specific ones, using progressively more targeted tools:

Broader Significance

This message is a case study in the challenges of deploying modern GPU computing in virtualized environments. The NVIDIA open kernel module, while offering the benefits of open-source licensing, introduces new compatibility surfaces — like HMM — that can interact poorly with hypervisor kernels. The Proxmox VE kernel, derived from Ubuntu's kernel but with additional patches for container and VM management, is a moving target that driver developers may not fully test against.

The fix itself — disabling HMM — is a pragmatic trade-off. HMM enables features like transparent GPU memory migration that can improve performance and memory utilization in multi-process GPU workloads. Disabling it may limit some of these capabilities, but it makes CUDA work at all, which is the prerequisite for everything else. The assistant implicitly judged that having functional CUDA was more important than having HMM-enabled CUDA, a decision that aligns with the immediate goal of deploying the GLM-5-NVFP4 model.

In the broader narrative of the session, this message is the turning point. After this, the assistant successfully launches the SGLang server, runs benchmarks achieving up to 806 tok/s at 128 concurrency, and confirms that the LXC container provides true bare-metal GPU topology with P2P access. None of that would have been possible without this single, focused message that turned "cuInit: 3" into "cuInit: 0" and "devices: 0" into "devices: 8."