Through the Blackwell Storm: Deploying GLM-5-NVFP4 from NaN Crashes to Production Benchmarking

Introduction

The deployment of large language models on next-generation GPU hardware is rarely a straight line from repository to production. When the model architecture, quantization format, serving framework, and GPU architecture are all simultaneously at the bleeding edge, every launch becomes an expedition into uncharted territory. This article chronicles one such expedition: the deployment of GLM-5-NVFP4—a 4-bit quantized Mixture-of-Experts (MoE) language model with DeepSeek Sparse Attention (DSA)—across eight NVIDIA RTX PRO 6000 Blackwell GPUs using the SGLang inference serving framework.

What began as a straightforward deployment command evolved into a multi-layered odyssey spanning over 150 messages. The assistant navigated a cascade of failures: a missing model architecture in Transformers, a GPU architecture incompatibility requiring a source-code fix, persistent NaN (Not a Number) crashes during decode that resisted multiple configuration changes, and finally the discovery that the underlying virtual machine infrastructure was crippling GPU-to-GPU communication. Each layer peeled back revealed a deeper problem, and each resolution required a different kind of expertise—from dependency management to CUDA kernel debugging to hypervisor configuration.

This article traces the full arc of that journey, from the first optimistic launch command to the final benchmarking results and the infrastructure-level investigation that followed. It examines the key decisions, the false leads, the moments of breakthrough, and the lessons learned for anyone deploying at the frontier of AI infrastructure.

The Setup: Eight Blackwell GPUs and a Plan

The segment opened with a pivotal instruction from the user at <msg id=85>: "Added 8 GPUs; Deploy glm-5 nvfp4." This single sentence transformed the session's trajectory from environment setup to production deployment. The assistant's response at <msg id=86> was a masterclass in structured planning, creating a prioritized todo list that would guide the next hundred messages: verify the GPUs, install SGLang from the main branch, download the model, deploy with tensor parallelism, and tune parameters.

The first verification step at <msg id=87> confirmed that all eight RTX PRO 6000 Blackwell GPUs (96 GB each, compute capability SM120) were visible and operational:

0, NVIDIA RTX PRO 6000 Blackwell Server Edition, 97887 MiB
1, NVIDIA RTX PRO 6000 Blackwell Server Edition, 97887 MiB
... (all 8 GPUs confirmed)

This was not a trivial check—the user had physically added six GPUs mid-session, and the assistant needed to confirm that the NVIDIA driver, PCIe topology, and NCCL communication stack could handle the expanded configuration. The nvidia-smi output showed all eight GPUs with their full 96 GB of memory, confirming the hardware was ready.

The assistant also discovered a local research repository at /home/theuser/glm-kimi-sm120-rtx6000bw/ containing FINDINGS.md, configuration files, and benchmark reports from previous deployments of similar models on the exact same hardware. This repository would prove invaluable in the debugging phase to come.

The First Launch and Its Immediate Failures

With the hardware verified, the assistant proceeded to install SGLang. The initial installation used version 0.5.8.post1 from PyPI, along with flashinfer 0.6.1 for attention kernels. The first launch command at <msg id=109> was carefully constructed, drawing on the HuggingFace model card's recommended parameters: tensor parallelism 8, FP4 quantization via modelopt_fp4, flashinfer attention backends, and aggressive memory reservation (--mem-fraction-static 0.95).

The launch failed immediately—but not with a GPU error. Instead, the server crashed with a KeyError: 'glm_moe_dsa' at <msg id=110>. The installed Transformers library (version 4.57.1) did not recognize the model architecture used by GLM-5. This was the first critical discovery: the GLM-5-NVFP4 model uses a custom architecture (glm_moe_dsa) that was only added to Transformers in version 5.2.0. The assistant upgraded Transformers to 5.2.0 at <msg id=113>, verified the architecture was now recognized, and prepared for a second launch.

The SM120 Correction: Why Main Branch Mattered

Before the second launch could proceed, the user interjected with a crucial observation at <msg id=117>: the installed SGLang version lacked support for the SM120 compute capability of Blackwell GPUs. A specific pull request, #14311, had been merged into SGLang's main branch to fix shared memory block sizes for the RTX PRO 6000's architecture. Without this fix, attention kernels would use Hopper-sized block sizes that exceeded the Blackwell GPU's smaller shared memory capacity (100 KB versus 160+ KB on Hopper), causing crashes or silent correctness issues.

The assistant verified this by inspecting the source code of the installed SGLang at <msg id=120>, confirming that the SM120 fix was absent. The solution required cloning the SGLang repository and building from the main branch at <msg id=123-125>. After installation, the assistant verified the fix by re-inspecting the source code at <msg id=127>, finding the critical lines:

if _is_cuda and CUDA_CAPABILITY[0] == 12:
    # sm120 workstation Blackwell architecture (RTX Pro 6000) has
    # a much smaller shared memory size (100K)

This episode highlighted a fundamental challenge of deploying on new hardware: stable releases of serving frameworks often lack critical patches for the latest GPU architectures, and operators must be prepared to build from source.

The Second Launch: Warnings Before the Storm

The second launch at <msg id=130> used the freshly built main-branch SGLang with Transformers 5.2.0. The server began loading the model—a process that consumed the next several messages as the assistant monitored the download of approximately 250 GB of checkpoint shards across all eight GPUs at <msg id=133-145>.

But the launch log contained two warnings that would prove prophetic. The first warned: "DeepGemm is enabled but the scale_fmt of checkpoint is not ue8m0. This might cause accuracy degradation on Blackwell." The second warned: "Transformers version 5.2.0 is used for model type glm_moe_dsa. If you experience issues related to RoPE parameters, they may be due to incompatibilities." The assistant noted both warnings but could not act on them at this stage—the server was still loading, and the warnings were classified as advisory rather than critical.

The model eventually loaded successfully across all eight GPUs, consuming approximately 63 GB per GPU for weights. KV cache was allocated with 425,664 tokens across 24.37 GB per GPU. CUDA graphs were captured for batch sizes up to 64. The server reported itself as running. Everything looked ready.

The Moment of Truth: First Inference Crash

The assistant sent a simple test query at <msg id=159>: "Hello! What model are you?" with 100 max tokens. The server returned nothing. A health check at <msg id=162> showed "Connection refused"—the server had crashed.

The crash log at <msg id=163> revealed a device-side assert triggered error—a CUDA runtime assertion failure. The assistant's initial grep for error patterns returned cryptic output: a corrupted traceback and a reference to the CUDA documentation. The actual error message was fragmented across multiple log lines, garbled by concurrent GPU threads writing to the same error buffer.

It took several rounds of forensic log analysis to extract the critical clue. At &lt;msg id=185&gt;, the assistant grepped for "Scheduler hit an exception" and found the assertion message: "probability tensor contains either inf, nan or element < 0." The model was producing garbage values during decode. This was not a kernel compatibility issue—it was a numerical stability failure.

The Debugging Spiral: Attention Backends and GEMM Paths

The discovery of NaN values in the probability tensor launched an intensive debugging phase spanning messages 163 through 210. The assistant pursued multiple hypotheses in parallel, each grounded in the warnings from the server logs.

Hypothesis 1: Attention backend incompatibility. The assistant tried switching from flashinfer to triton attention backends at &lt;msg id=170&gt;, but the triton backend crashed with a different assertion about NSA KV cache FP8 incompatibility at &lt;msg id=174&gt;. It then tried flashmla_sparse as the NSA decode backend at &lt;msg id=178&gt;, which appeared to work during warmup but crashed on the first real query at &lt;msg id=182-184&gt;.

Hypothesis 2: DeepGemm scale format mismatch. The warning about DeepGemm being enabled with a non-ue8m0 scale format pointed to a potential numerical corruption in the GEMM (General Matrix Multiply) operations. The assistant forced --fp8-gemm-backend cutlass to bypass DeepGemm at &lt;msg id=187&gt;. The server loaded successfully and warmup passed, but the NaN crash persisted during decode at &lt;msg id=195&gt;.

Hypothesis 3: CUDA graph capture instability. After consulting the local research repository (FINDINGS.md) at &lt;msg id=188-193&gt;, the assistant learned that a previous successful NVFP4 deployment (Kimi K2-Thinking) on the same hardware had used --disable-cuda-graph. The assistant launched a new attempt with this flag at &lt;msg id=204&gt;, along with --kv-cache-dtype auto to avoid forcing FP8 KV cache. The server loaded, but the DeepGemm and Transformers warnings persisted at &lt;msg id=208&gt;.

The Research Pivot: Consulting Local Knowledge

A critical turning point came at &lt;msg id=188&gt;, when the user directed the assistant to "Read research in ./ that run other glm/kimi models." This instruction reframed the debugging effort. Instead of continuing to brute-force configuration combinations, the assistant consulted the local repository at /home/theuser/glm-kimi-sm120-rtx6000bw/ that contained findings from previous deployments of similar models on the exact same hardware.

The FINDINGS.md file at &lt;msg id=191-192&gt; documented a successful deployment of Kimi K2-Thinking NVFP4 on the same eight Blackwell GPUs. The key findings included: the NaN issue was NOT the SM120 shared memory problem (already fixed in the main branch build), previous successful NVFP4 deployments used --disable-cuda-graph, and the DeepGemm + non-ue8m0 scale format warning was a known issue on Blackwell. The repository also contained benchmark data comparing TP8 and TP4 PP2 configurations, showing that TP4 PP2 significantly outperformed TP8 at high concurrency.

This research pivot transformed the debugging approach from trial-and-error to evidence-based reasoning. The assistant now understood that the NaN crash had been encountered before, and that the solution likely involved a combination of --disable-cuda-graph, careful KV cache dtype selection, and possibly bypassing DeepGemm entirely.

Breakthrough: The Working Configuration

After multiple attempts, the assistant finally achieved a stable server configuration. The key was a combination of flags that avoided the known problem areas:

{
  "id": "840bda2ae8ef401c88c2988278cd4e98",
  "object": "chat.completion",
  "created": 1771460620,
  "model": "glm-5",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": null,
      "reasoning_content": "The user is asking a very simple math question..."
    }
  }]
}

The server was alive and producing coherent output. The NaN crash had been defeated.

Benchmarking the Deployment

With a stable server running, the assistant turned to performance benchmarking. Using SGLang's built-in bench_serving module, the assistant measured throughput under various concurrency levels.

The initial baseline at &lt;msg id=225&gt; showed:

The Weight Analysis: Understanding the Model's Memory Footprint

To understand the deployment's memory characteristics and explore alternative configurations, the assistant performed a detailed weight analysis at &lt;msg id=250&gt;. The GLM-5 model is enormous:

The Infrastructure Investigation: Proxmox and P2P

Despite achieving a stable server and measuring its performance, the throughput remained well below targets. The assistant began investigating the underlying infrastructure, suspecting that GPU-to-GPU communication was the bottleneck.

The investigation at &lt;msg id=260-330&gt; revealed a startling discovery: the machine was a Proxmox KVM virtual machine, and P2P (Peer-to-Peer) access was disabled across all GPU pairs. All cross-GPU NCCL transfers were going through host memory with a 13.7µs+ latency floor per small transfer. With 78 layers of all-reduce operations, this latency compounded into a significant throughput penalty.

Further investigation of the Proxmox host configuration revealed several issues:

  1. Missing iommu=pt — The kernel cmdline did not include iommu=pt (passthrough mode), meaning all DMA went through IOMMU translation tables instead of being passed through directly, adding latency to every GPU memory transaction.
  2. Missing pcie=1 — The VM configuration's hostpci entries for GPU passthrough lacked the pcie=1 option, causing GPUs to appear as legacy PCI devices rather than PCIe devices. This prevented the NVIDIA driver from seeing proper PCIe topology and attempting P2P.
  3. i440fx machine type — The VM used the older i440fx chipset which doesn't support PCIe natively, rather than the q35 chipset which provides proper PCIe topology. The assistant documented a remediation plan at &lt;msg id=330&gt;: - Phase 1 (low risk): Add pcie=1 to all GPU passthrough entries in the VM config, then restart the VM - Phase 2 (moderate risk): Add amd_iommu=on iommu=pt to the host kernel cmdline, then reboot the host - Phase 3 (higher risk): Switch to q35 machine type for proper PCIe topology Each phase would improve GPU P2P capabilities, potentially transforming the all-reduce performance and unlocking the full throughput potential of the eight Blackwell GPUs.

Lessons from the Blackwell Gauntlet

This deployment effort reveals several fundamental truths about deploying cutting-edge AI models on new hardware:

1. Stable releases are not enough. The initial SGLang release (0.5.8.post1) lacked critical SM120 support that was only available in the main branch. Operators deploying on new GPU architectures must be prepared to build from source and track nightly development branches.

2. Model architecture support lags behind model releases. The GLM-5-NVFP4 model required Transformers 5.2.0 for its glm_moe_dsa architecture, but this version introduced its own compatibility issues (the RoPE warning). The dependency chain—model → Transformers → serving framework → CUDA kernels → GPU hardware—creates a compatibility matrix where each layer may have gaps.

3. Numerical stability is harder to debug than crashes. A model that loads successfully, captures CUDA graphs, and completes warmup can still produce garbage during decode. The prefill path and decode path use different CUDA kernels, and numerical issues may only manifest in the autoregressive generation loop. This asymmetry makes "it starts" an insufficient test of correctness.

4. Local empirical knowledge is invaluable. The FINDINGS.md repository, documenting previous deployments on the exact same hardware, provided more actionable information than the HuggingFace model card, the SGLang documentation, or GitHub issues. In the frontier of new hardware deployment, the most reliable source of truth is previous experience on the same machine.

5. Warnings are prophecies. The DeepGemm scale format warning and the Transformers RoPE warning, both present from the very first launch, turned out to be the root causes of the NaN crashes. The assistant treated them as advisory, but they were diagnostic gold. In complex systems, warnings should be investigated immediately, not deferred.

6. Infrastructure matters as much as software. The discovery that the VM was crippling GPU P2P communication explained why throughput was stuck at 236 tok/s despite 100% GPU utilization. The most impactful performance optimization was not a kernel parameter or a backend flag—it was fixing the hypervisor configuration to allow proper PCIe passthrough.

Conclusion

The deployment of GLM-5-NVFP4 on eight Blackwell GPUs was a journey through the bleeding edge of AI infrastructure. The assistant navigated hardware verification, source-code builds, dependency conflicts, memory tuning, numerical stability debugging, and hypervisor configuration—each step revealing new dimensions of complexity in the intersection of quantized MoE models, sparse attention mechanisms, and next-generation GPU architecture.

At the end of the segment captured in this article, the server was running stably with a working configuration, producing correct output at approximately 485 total tokens per second. The NaN crash had been resolved through a combination of --disable-cuda-graph, --fp8-gemm-backend cutlass, and trtllm NSA backends. The performance bottleneck had been traced to the Proxmox VM's PCIe passthrough configuration, and a remediation plan was in place.

The session stands as a testament to the challenges of deploying at the frontier, where every component—model architecture, quantization format, serving framework, GPU hardware, and even the virtualization layer—is evolving simultaneously. The only way forward is systematic, iterative debugging informed by local empirical knowledge, with the humility to recognize that the problem may not be where you're looking.