From Garbage Output to Production Service: Debugging, Optimizing, and Deploying GLM-5 on 8× Blackwell GPUs

Introduction

Deploying a cutting-edge large language model on novel hardware is rarely a straightforward process. When the model is GLM-5—a sophisticated Mixture-of-Experts architecture with Multi-head Latent Attention (MLA)—and the hardware is eight NVIDIA RTX PRO 6000 Blackwell GPUs connected over PCIe, the path from a working installation to a production-ready service is paved with subtle bugs, performance bottlenecks, and hard-won optimizations. This article traces that journey through a single opencode session that spanned the complete lifecycle of model deployment: from verifying weight tensor integrity, through diagnosing and fixing silent corruption bugs, to performance tuning and finally creating a systemd service.

The session's arc can be divided into three major phases. First, a meticulous forensic investigation into whether the model's weights were being loaded correctly, culminating in a numerical round-trip verification that proved the kv_b_proj tensor reconstruction was mathematically exact. Second, the identification and fixing of two distinct bugs in vLLM's Triton MLA attention backend and GGUF dequantization shard ordering that were causing the model to produce incoherent text. Third, a performance optimization campaign that doubled throughput from ~20 to ~57 tokens per second, followed by the creation of a systemd service to productionalize the deployment.

Phase I: The Tensor Layout Verification

The session began with a focused sub-investigation triggered by a single, critical question: does the _reassemble_kv_b function in vLLM's patched GGUF loader correctly reconstruct the original HuggingFace kv_b_proj.weight tensor? The GLM-5 model uses MLA, a sophisticated attention mechanism where the key-value projection is decomposed into a low-rank latent space. In the original HuggingFace format, kv_b_proj.weight has shape [28672, 512]—mapping from a latent rank of 512 to a concatenated key-value space of 28672 dimensions (64 heads × (192 nope dimensions + 256 value dimensions)). During GGUF conversion via llama.cpp's convert_hf_to_gguf.py, this single weight matrix is split into two separate tensors: attn_k_b and attn_v_b. Critically, only the key portion is transposed; the value portion is stored as-is. The reconstruction function must exactly reverse this asymmetric transformation.

The assistant's investigation was a masterclass in systematic verification. It began by reading the patched GGUF loader locally and the production version on the remote container simultaneously ([msg 1]). It then queried the actual GGUF binary file to obtain the stored tensor shapes, discovering that attn_k_b.weight had reader-reported shape [192, 512, 64] and attn_v_b.weight had shape [512, 256, 64], both quantized as Q8_0 ([msg 2]). The assistant located the exact conversion code in llama.cpp's script, finding the critical comment: "note: MLA with the absorption optimization, needs these two split and k_b_proj transposed" ([msg 4]).

A pivotal moment came when the assistant dequantized the actual GGUF tensors to verify their shapes ([msg 7]). The dequantized output revealed attn_k_b as (64, 512, 192) and attn_v_b as (64, 256, 512)—confirming that the GGUF file stored the key tensor in its transposed form and the value tensor without transpose. This empirical check was essential because GGUF's dimension conventions are notoriously confusing: the reader reports shapes in an innermost-first convention, while the dequantized output uses standard row-major order.

The assistant then constructed an extensive analytical trace of the full conversion path ([msg 8]), working through each transformation step with careful attention to the dimension ordering. A notable moment of reasoning occurred when the assistant questioned whether the llama.cpp conversion code's use of v_head_dim + qk_nope_head_dim in the reshape was consistent with the subsequent split([qk_nope_head_dim, v_head_dim], dim=1). After cross-referencing the DeepSeek-v2 forward pass code, the assistant confirmed that the original HuggingFace layout has qk_nope before v_head per head group, and the conversion's split order matches this convention.

The culmination of this phase was a numerical round-trip verification using synthetic data ([msg 9], [msg 10], [msg 12]). The assistant created a tensor with monotonically increasing values (using torch.arange), simulated both the conversion and reconstruction paths, and checked for exact equality. The result: torch.equal(original, combined) returned True. The reconstruction was byte-identical to the original. This verification was not merely a shape check—it proved that every single element was in the correct position, ruling out any permutation or transposition error.

Phase II: Diagnosing and Fixing Garbage Output

With the weight loading verified as correct, the investigation turned to the more perplexing problem: why was the model producing incoherent text despite loading successfully? The answer lay in two distinct bugs in vLLM's inference stack.

The first bug was in the Triton MLA attention backend. The custom PyTorch operation used for the MLA attention computation was creating a "phantom tensor"—an output buffer that appeared valid but was disconnected from the actual computation. When the attention kernel wrote its results, the data went to one memory location, but the downstream operations read from a different location, effectively receiving garbage. This is the kind of bug that is notoriously difficult to diagnose because it doesn't produce an error message; it silently corrupts the output.

The second bug was in the GGUF dequantization layer's handling of fused projections. When vLLM loads a GGUF-quantized model with tensor parallelism, it shards the weight matrices across GPUs. The dequantization code for fused projections (where multiple weight matrices are packed into a single tensor) had a shard ordering bug: the shards were being assigned to GPUs in the wrong order, causing each GPU to receive a scrambled portion of the weight. This meant that even though the overall weight reconstruction was correct (as verified in Phase I), the per-GPU shards were corrupted.

Fixing these two bugs restored correct model output. The model began producing coherent, sensible text—a critical milestone that validated the entire deployment effort.

Phase III: Performance Optimization

With correct output achieved, the focus shifted to performance. The initial single-request decode throughput was approximately 20 tokens per second—usable but far from optimal for a production service. The assistant began a systematic optimization campaign, starting with profiling to understand where the time was being spent.

The profiling revealed a startling finding: approximately 42% of the decode time was spent on NCCL allreduce operations. The eight Blackwell GPUs were connected over PCIe rather than NVLink, which meant that the inter-GPU communication was bottlenecked by the relatively slow PCIe bandwidth. Every allreduce operation required synchronizing tensors across all eight GPUs, and the PCIe topology made this disproportionately expensive.

The first optimization was enabling CUDAGraphs. CUDAGraphs allow the GPU to capture and replay entire sequences of CUDA operations, bypassing the CPU launch overhead for each individual kernel. This optimization alone doubled the throughput to approximately 43 tokens per second. The improvement came from batching the kernel launches: instead of the CPU launching each attention kernel, MLP kernel, and allreduce operation one at a time, CUDAGraphs batched them into a single optimized launch sequence.

The second optimization targeted the NCCL communication directly. By tuning the NCCL_PROTO=LL (Low Latency) protocol, the assistant achieved a further boost to approximately 57 tokens per second. The LL protocol uses a different communication algorithm that reduces latency for small message sizes, which is characteristic of the allreduce operations in the decode path.

The assistant explored more advanced optimizations, including custom allreduce implementations and allreduce-RMS normalization fusion. However, these were found to be incompatible with the PCIe-only hardware topology. Custom allreduce algorithms often assume NVLink-connected GPUs with direct peer-to-peer memory access, which was not available in this setup. The allreduce-RMS fusion—combining the allreduce synchronization with the RMS normalization computation—was theoretically promising but required deeper modifications to vLLM's execution engine that were beyond the scope of the session.

The performance optimization effort established a clear hardware bottleneck: the PCIe interconnect between the eight GPUs. With NVLink, the allreduce overhead would be significantly lower, and the throughput ceiling would be much higher. This finding has implications for future hardware procurement decisions—for multi-GPU inference serving, NVLink connectivity is a critical investment.

Phase IV: Productionalization

The final phase of the session was deploying the optimized configuration as a production service. The assistant created a systemd service unit file (vllm-glm5.service) that would start vLLM with the optimal configuration flags, including CUDAGraph enablement and the NCCL protocol setting. The service was configured to restart automatically on failure and to log output to the systemd journal for monitoring.

However, the initial deployment attempt encountered a familiar obstacle: a stale vLLM process from a previous session was still running and holding GPU memory. When the systemd service tried to start a new vLLM instance, it failed because the GPUs were already allocated. This is a classic production deployment issue—clean process management is essential when running GPU-accelerated services, and stale processes can block new deployments.

The solution required manually killing the stale process before starting the service. While this was a straightforward fix, it highlighted the need for more robust process lifecycle management in the deployment pipeline. Future iterations could include pre-start hooks that check for and clean up stale processes, or using container orchestration tools like Docker or Kubernetes that provide better process isolation.

Themes and Lessons

Several overarching themes emerge from this session. First, systematic verification is essential when deploying novel models on novel hardware. The tensor layout investigation—spanning multiple codebases, empirical data inspection, and numerical round-trip testing—demonstrates that correctness cannot be assumed; it must be proven at every level of abstraction.

Second, hardware topology constrains optimization. The PCIe-only interconnect between GPUs was the dominant bottleneck, and no amount of software optimization could fully overcome it. Understanding the hardware's limitations is crucial for setting realistic performance targets and making informed infrastructure decisions.

Third, the debugging depth required for production ML is extraordinary. From phantom tensors in custom PyTorch ops to shard ordering bugs in dequantization code, the bugs that affect model output are subtle and deeply embedded in the software stack. Each bug required tracing through multiple layers of abstraction—from the high-level model architecture down to the CUDA kernel implementations.

Finally, the path from working code to production service is iterative and non-linear. The session moved from weight verification to bug fixing to performance optimization to deployment, but each phase revealed new challenges that required revisiting earlier assumptions. The stale process issue at deployment time, while trivial in isolation, was a reminder that production readiness involves more than just correct model output and fast inference—it requires robust process management, monitoring, and operational discipline.

Conclusion

This opencode session captures the complete lifecycle of deploying a cutting-edge LLM on state-of-the-art hardware. From the meticulous verification of tensor layouts through numerical round-trip testing, to the diagnosis and fixing of two silent corruption bugs, to the performance optimization that doubled throughput, and finally to the creation of a systemd service—each phase built on the previous one, and each revealed new dimensions of complexity.

The final result was a production-ready vLLM service serving the GLM-5 model on eight Blackwell GPUs at approximately 57 tokens per second, with correct output and robust process management. The journey to get there required deep expertise spanning multiple domains: PyTorch tensor operations, GGUF format internals, CUDA kernel debugging, NCCL communication optimization, and Linux service management. It is a testament to the depth of skill required to deploy modern AI systems at scale.