Segment 5

In this sub-session, the critical CUDA initialization failure (error code 3) was fixed by setting `uvm_disable_hmm=1` as a module parameter for `nvidia_uvm`, enabling CUDA compute on both the host and LXC container. With CUDA working, the sglang inference server for the GLM-5-NVFP4 model was launched after upgrading transformers to 5.2.0 and installing ninja-build for FlashInfer JIT compilation. Benchmarks showed throughput of 438 tok/s at 32 concurrency, 757 at 64, and 806 at 128 concurrent requests, though single-stream performance remained around 11 tok/s. The LXC container confirmed true bare-metal GPU topology (NODE/SYS) with P2P access at 53 GB/s same-NUMA, a significant improvement over the VFIO-limited KVM VM. However, further tuning of MoE kernels for Blackwell (SM120) architecture is needed to reach the targets of 1k+ total tok/s and >100 tok/s single-stream.

Disable HMM in nvidia_uvm moduleUpgrade transformers for glm_moe_dsa supportInstall ninja-build for FlashInfer compilationDeploy sglang server and benchmarkVerify P2P GPU topology in LXCIdentify Blackwell kernel optimization needs

The HMM That Almost Stopped Blackwell: From CUDA Blocker to 806 tok/s on Eight RTX PRO 6000 GPUs 3036 words

Chunks