The Final Backend: Testing flashinfer_cutedsl on Blackwell SM120 GPUs

A Pivotal Verification in the MoE Backend Search

In the course of deploying the massive GLM-5-NVFP4 mixture-of-experts model across eight NVIDIA RTX PRO 6000 Blackwell GPUs, an AI assistant found itself navigating a labyrinth of kernel backends, autotuning infrastructure, and GPU architecture differences. Message [msg 299] captures a seemingly mundane moment — a server startup verification — that represents the culmination of an intensive debugging session. After resolving a catastrophic NaN crash during decode, testing multiple MoE runner backends, and discovering that flashinfer's autotuning infrastructure lacked support for the RTX PRO 6000's SM120 architecture, the assistant turned to flashinfer_cutedsl as the final alternative before consolidating results. This message is a verification checkpoint, and its contents reveal much about the state of the system, the assistant's methodology, and the challenges of deploying cutting-edge models on new GPU hardware.

The Message

The subject message consists of a single bash command executed via SSH on the remote server at 10.1.230.175, followed by the output from that command:

[assistant] [bash] ssh 10.1.230.175 "for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40; do if grep -q 'fired up\|Traceback\|RuntimeError\|Error\|assert' ~/sglang-glm5.log 2>/dev/null; then break; fi; sleep 5; done; tail -5 ~/sglang-glm5.log"
[2026-02-19 01:11:14] INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
[2026-02-19 01:11:15] INFO:     127.0.0.1:35534 - "GET /model_info HTTP/1.1" 200 OK
[2026-02-19 01:11:16 TP0] Prefill batch, #new-seq: 1, #new-token: 64, #cached-token: 0, token usage: 0.00, #running-req: 0, #queue-req: 0, input throughput (token/s): 0.00, cuda graph: False
[2026-02-19 01:11:17] INFO:     127.0.0.1:35546 - "POST /generate HTTP/1.1" 200 OK
[2026-02-19 01:11:17] The server is fired up and...

At first glance, this appears to be a routine server health check. But the context transforms it into something far more significant: it is the verification that the flashinfer_cutedsl MoE runner backend successfully launched the GLM-5-NVFP4 model on eight SM120 Blackwell GPUs without crashing.

The Journey to This Point

To understand why this message was written, one must trace the assistant's path through the preceding messages. The session had been a multi-hour effort to deploy GLM-5-NVFP4, a quantized mixture-of-experts model with 256 experts and approximately 453GB of MoE parameters. The deployment target was a Proxmox virtual machine with eight NVIDIA RTX PRO 6000 Blackwell GPUs — a configuration that presented numerous challenges.

The most critical problem had been a NaN crash during decode, which the assistant traced to the NSA (Nested Sliding Window Attention) implementation. By switching to --nsa-decode-backend trtllm and --nsa-prefill-backend trtllm, the assistant achieved stable, coherent output. With the crash resolved, attention turned to performance tuning.

The assistant systematically explored MoE runner backends. The flashinfer_trtllm backend failed because it was SM100-only (designed for datacenter Blackwell GPUs like B200, not the SM120 consumer/workstation variants). The flashinfer_cutlass backend worked but showed GPU utilization at 100% with only 55% power draw, suggesting the forward pass was dominated by many small kernel launches rather than compute-bound operations. The assistant then investigated flashinfer's autotuning infrastructure, discovering that while flashinfer had SM120-specific CUTLASS kernel code (fp4_gemm_cutlass_sm120.cu, gen_cutlass_fused_moe_sm120_module), there was no tuning configuration file for the RTX PRO 6000 Blackwell. The autotuner fell back to default configurations, and crucially, the cutlass_fused_moe path did not use the autotuner at all — it used fixed kernels with tune_max_num_tokens as the only knob.

This realization — that no amount of autotuning would improve the flashinfer_cutlass backend — led the assistant to try flashinfer_cutedsl as the last alternative MoE backend before recording final benchmark results. Message [msg 297] captures this decision explicitly: "Let me quickly test flashinfer_cutedsl (the CuteDSL path) as the last alternative, then record final results."## Why This Message Was Written: Reasoning, Motivation, and Context

The assistant's motivation for writing this message was straightforward: it needed to verify that the newly launched server with the flashinfer_cutedsl backend had started successfully before proceeding to correctness testing and benchmarking. However, the deeper reasoning reflects several important aspects of the assistant's methodology.

First, the assistant was operating under a constraint of systematic completeness. Having tested flashinfer_cutlass and flashinfer_trtllm, and having analyzed why neither was ideal, the assistant wanted to evaluate every available MoE backend option before making a final recommendation. The flashinfer_cutedsl path — which uses NVIDIA's CuteDSL (a domain-specific language for writing CUDA kernels) — represented the last unexplored alternative. The assistant's thinking, visible in [msg 297], was that the GPUs were at 100% utilization but only 55% power, suggesting kernel-launch overhead rather than compute was the bottleneck. A different kernel implementation might have different launch characteristics.

Second, the assistant needed to confirm that the server did not crash during initialization. The previous flashinfer_trtllm attempt had failed with a traceback (see [msg 279]), and the assistant had just killed that server and launched a new one. The polling loop in the bash command — which checks for fired up, Traceback, RuntimeError, Error, or assert — was designed to detect both success and failure conditions, allowing the assistant to react accordingly without waiting the full 40 iterations if an error appeared early.

Third, the timing was important. The assistant was working through a sequence of experiments, and each server restart cost several minutes of startup time (model loading, weight initialization, warmup). By embedding the verification directly in the launch command's follow-up, the assistant minimized idle time between experiments.

How Decisions Were Made

Several implicit decisions are embedded in this message. The choice of polling interval (5 seconds) and maximum iterations (40, totaling 200 seconds) reflects an assumption about the server's startup time. The assistant expected the server to either start successfully or fail within approximately three minutes — a reasonable assumption given the model's size (453GB of MoE parameters across 8 GPUs) and the complexity of the initialization sequence.

The decision to use tail -5 rather than a more comprehensive log dump reflects a pragmatic trade-off. The assistant wanted just enough output to confirm the server was running and had processed its first request (the GET /model_info and POST /generate lines), without being overwhelmed by the verbose debug logging that had plagued earlier attempts (see <msg id=294-295> where HTTP headers made the log nearly unreadable).

The grep pattern itself — fired up\|Traceback\|RuntimeError\|Error\|assert — encodes a decision about what constitutes success or failure. The assistant chose to check for the canonical success message ("fired up") and a set of common error indicators, rather than checking for a specific exit code or process status. This approach is robust to different failure modes but could miss silent failures where the process hangs without producing output.

Assumptions Made by the Assistant

This message rests on several assumptions, some explicit and some implicit:

The server startup would complete within 200 seconds. The assistant assumed that 40 iterations at 5-second intervals would be sufficient. If the server took longer (e.g., due to model download, compilation of CUDA kernels, or memory allocation), the loop would exit without detecting either success or failure, and the tail -5 would show stale or empty output. The assistant would then need to manually investigate.

The flashinfer_cutedsl backend would not cause a crash. Given that flashinfer_cutlass had worked and flashinfer_trtllm had failed, the assistant was testing a hypothesis that CuteDSL might offer different performance characteristics. But there was no guarantee it would work — the SM120 architecture is new, and CuteDSL kernel support might have similar issues to the TRT-LLM path.

The log file would be written to promptly. The assistant relied on grep reading the log file, which depends on the shell's output buffering. The 2>&1 redirect combined with nohup means output is line-buffered or fully buffered depending on the implementation. If the log was buffered, the grep might not see output until the process flushed, potentially causing the loop to miss the "fired up" message.

SSH connectivity would remain stable. The assistant was running commands on a remote server over SSH. Network interruptions or SSH daemon issues could cause the command to fail, which would appear as a timeout or connection refused rather than a server error.

Mistakes and Incorrect Assumptions

While the message itself is correct and the server did start successfully, there are subtle issues worth examining:

The polling loop exits on the first match but doesn't distinguish between success and failure. If grep -q matches Error on line 3 of the log (perhaps a non-fatal warning), the loop breaks immediately and tail -5 shows the last 5 lines, which might not include the actual error context. The assistant would see the error line but might misinterpret its severity. In practice, this didn't happen, but the design is fragile.

The tail -5 command runs unconditionally after the loop. Even if the loop exhausted all 40 iterations without finding any match, the tail -5 would still execute, potentially showing stale output from a previous server instance or an empty log. The assistant would need to manually check whether the output was meaningful.

The assumption that flashinfer_cutedsl was worth testing may have been optimistic. The assistant had already established that the flashinfer_cutlass backend worked and that the performance bottleneck was likely PCIe bandwidth and virtualization overhead rather than kernel efficiency. Testing another kernel backend was unlikely to change the fundamental bottleneck. However, it was a reasonable experiment — sometimes different kernel implementations have dramatically different launch overhead or memory access patterns that can mitigate secondary bottlenecks.

Input Knowledge Required

To fully understand this message, one needs knowledge of:

The SGLang server architecture. The "fired up" message is SGLang's canonical startup confirmation. The log lines showing HTTP requests (GET /model_info, POST /generate) indicate that the server's health check endpoint and a generation request were processed successfully.

The GLM-5-NVFP4 model characteristics. This is a 256-expert MoE model with FP4 quantization, requiring approximately 453GB for the MoE parameters alone. With 8 GPUs each having 96GB of VRAM, the model fits but leaves limited headroom for KV cache and activations.

The Blackwell SM120 architecture. The RTX PRO 6000 Blackwell uses the SM120 compute architecture, which differs from the datacenter Blackwell (SM100) used in B200/GB200. This distinction was critical because flashinfer_trtllm was SM100-only.

The flashinfer library structure. Understanding that flashinfer has multiple MoE backends (cutlass_fused_moe, trtllm, cutedsl) and that the autotuner only applies to the TRT-LLM path is essential for interpreting the assistant's debugging strategy.

The Proxmox virtualization context. The server runs in a KVM/QEMU virtual machine, which affects GPU peer-to-peer communication. This was identified as a major performance bottleneck in later analysis.## Output Knowledge Created

This message produced concrete, actionable knowledge:

The flashinfer_cutedsl backend works on SM120 GPUs. The server started without errors, processed a model info request, and handled a generation request. This was not guaranteed — the flashinfer_trtllm backend had failed on the same hardware. The assistant now had three data points: flashinfer_cutlass (works), flashinfer_trtllm (fails, SM100-only), and flashinfer_cutedsl (works).

The server startup time was approximately 3-4 minutes. The previous server kill (in [msg 297]) and the new launch (in [msg 298]) were followed by this verification. The log timestamps show the server was running by 01:11:14, which is consistent with the expected startup duration.

The server passed its initial health check. The GET /model_info returning 200 OK confirms the model was loaded correctly and the API was responsive. The POST /generate returning 200 OK confirms that at least one generation request completed without crashing — a significant milestone given the NaN crash issues that had plagued earlier attempts.

The CUDA graph optimization was not used. The log line shows "cuda graph: False" for the prefill batch, indicating that CUDA graph capture had not been triggered for this request. This is normal for the first request (CUDA graphs are typically captured during warmup or after observing typical batch sizes), but it's a data point for performance analysis.

The Thinking Process Visible in the Message

While the message itself is a tool call with output, the thinking process that led to it is visible in the surrounding context. The assistant's reasoning chain, reconstructed from [msg 281] through [msg 299], follows this logic:

1. Identify the problem: The flashinfer_trtllm MoE backend crashes on SM120 GPUs because it's designed for SM100 (datacenter Blackwell).
2. Explore alternatives: Test flashinfer_cutlass (works), investigate autotuning infrastructure.
3. Analyze the autotuner: Discover that no tuning config exists for RTX PRO 6000 Blackwell, and that the cutlass_fused_moe path doesn't use the autotuner anyway — it uses fixed kernels.
4. Diagnose the bottleneck: GPU utilization is 100% but power draw is only 55%, suggesting the workload is launch-bound rather than compute-bound. The 78 layers × multiple kernels per layer × 256 expert routing creates many small kernel launches.
5. Test the last alternative: Try flashinfer_cutedsl before concluding the investigation and moving to benchmarking. This is a methodical, scientific approach. The assistant forms hypotheses, tests them, analyzes results, and iterates. The message at [msg 299] is the verification step for hypothesis #5 — "does flashinfer_cutedsl work on SM120?" The assistant also demonstrates an important meta-cognitive skill: knowing when to stop investigating and move on. After the flashinfer_cutedsl test, the assistant does not continue searching for more backends. Instead, it proceeds to benchmark and performance analysis, which is the correct decision given that the fundamental bottleneck (PCIe bandwidth and virtualization overhead) is architectural and unlikely to be solved by kernel selection.

Conclusion

Message [msg 299] appears, on its surface, to be a routine server health check. But within the context of the broader debugging session, it represents a critical verification point — the confirmation that yet another MoE backend (flashinfer_cutedsl) can successfully run the GLM-5-NVFP4 model on eight SM120 Blackwell GPUs. It is the culmination of a systematic search through flashinfer's MoE runner backends, each tested and evaluated against the constraints of the hardware.

The message also reveals the assistant's methodology: methodical, hypothesis-driven, and grounded in empirical verification. Each backend was tested not just for correctness but for performance characteristics, and the assistant was willing to dig deep into the autotuning infrastructure to understand why certain backends performed as they did. When the investigation revealed that no amount of kernel tuning would fix the fundamental bottleneck, the assistant pivoted to the next phase — benchmarking and infrastructure analysis.

In the end, the flashinfer_cutedsl test was a dead end in terms of performance improvement — the assistant would later identify virtualization-induced PCIe latency as the primary bottleneck. But it was a necessary dead end, one that had to be explored to rule out kernel backend selection as a variable. This is the essence of systematic debugging: testing each hypothesis in turn, even when the outcome is predictable, to build a complete picture of the system's behavior.