The Diagnostic Pivot: Tracing NaN Outputs Through Journal Logs on Blackwell GPUs

In the middle of a high-stakes deployment of a 397-billion-parameter MoE model on eight NVIDIA RTX PRO 6000 Blackwell GPUs, a single assistant message stands out as a quiet but critical diagnostic pivot. Message [msg 5848] is deceptively simple — a single journalctl command piped through grep to search for keywords like fp4, nvfp4, gemm.*runner, cutlass, cudnn, and NaN in the systemd service logs. But this seemingly mundane command represents a moment of methodological clarity: after observing garbage output and NaN errors from the freshly deployed Qwen3.5-397B-A17B-NVFP4 model, the assistant stopped guessing and started gathering evidence systematically.

The Context: A Deployment Gone Wrong

The narrative leading to this message is one of rapid progress followed by an unexpected wall. The assistant had just completed a multi-hour effort to deploy the Kimi-K2.5 INT4 model with EAGLE-3 speculative decoding, achieving impressive throughput numbers (759 tok/s at concurrency 30, beating baseline by 10%). With that model hardened into a systemd service, the user pivoted to a newer, more efficient model: nvidia/Qwen3.5-397B-A17B-NVFP4, a 397B-parameter MoE with 512 experts (17B active) quantized to NVFP4 — a format that requires Blackwell GPUs specifically.

The deployment had gone smoothly at first. The model downloaded (223 GB), the latest SGLang main branch was built from source, SM120 patches were applied to the communication libraries, and the systemd service was configured with --attention-backend triton (required for hybrid GDN models with linear attention layers) and --quantization modelopt_fp4. The server came up in about two minutes — much faster than the 547 GB Kimi model.

Then came the first test. The assistant sent a chat completion request asking the model to write a prime-checking Python function. The response was alarming: the reasoning_content field contained nothing but repeated exclamation marks — ! characters stretching to the token limit. A second test confirmed the worst: the response carried "finish_reason": "stop" with "matched_stop": "NaN happened". The model was producing numerical garbage.

The Hypothesis and the Investigation

In the messages immediately preceding [msg 5848], the assistant formed a working hypothesis: the FP4 GEMM (General Matrix Multiply) kernels were incompatible with the SM120 (Blackwell) architecture. The default backend, flashinfer_cutlass, was likely producing incorrect results on the new GPU generation. The assistant began probing the codebase, searching for fp4_gemm_runner_backend definitions and discovering the available choices: auto, flashinfer_cudnn, flashinfer_cutlass, and flashinfer_trtllm (<msg id=5844-5847>).

But at this point, the hypothesis was still just a hypothesis. The assistant had seen NaN outputs and knew the FP4 kernels were involved, but it hadn't confirmed which specific backend was being used or whether the logs contained any additional diagnostic information. This is where [msg 5848] enters the picture.

The Message Itself: A Targeted Log Query

The message executes a single bash command on the remote server:

ssh root@10.1.230.174 'journalctl -u sglang-qwen -o cat --no-pager | grep -i "fp4\|nvfp4\|gemm.*runner\|cutlass\|cudnn\|NaN" | head -15'

This command does three things:

  1. It reads the full journal for the sglang-qwen systemd service
  2. It filters for lines containing FP4-related keywords and NaN errors
  3. It limits output to the first 15 matching lines The output reveals a single matching line — the startup log entry showing the full ServerArgs object. Crucially, this line contains the model path (/data/models/Qwen3.5-397B-A17B-NVFP4) and confirms that the server did start with the expected configuration. But notably, there are no FP4 backend selection messages, no cutlass or cudnn references, and no explicit NaN error messages in the journal. The grep found only the startup line because it contains "NVFP4" in the model path.

Why This Message Matters

The significance of [msg 5848] lies not in its output but in its methodology. It represents a deliberate shift from reactive troubleshooting to systematic diagnosis. The assistant could have simply tried different backend flags blindly — changing --fp4-gemm-runner-backend to flashinfer_cudnn and hoping for the best. Instead, it chose to first gather evidence from the running system's logs.

This decision reveals several important aspects of the assistant's reasoning:

First, the assistant recognized that the NaN problem could have multiple root causes. The FP4 GEMM backend was one candidate, but there could also be issues with the MoE runner backend, the attention backend (already set to triton), or even the model loading process itself. Checking the logs for any explicit error messages or backend selection confirmations would help narrow the field.

Second, the assistant was operating under the assumption that the journal would contain useful diagnostic information. This assumption was partially correct — the startup log confirmed the server arguments — but the absence of FP4-specific log lines was itself informative. It suggested that the backend selection happened silently, without logging which specific kernel implementation was chosen.

Third, the assistant was building a chain of evidence. The NaN outputs from the API ([msg 5843]) were the symptom. The code inspection (<msg id=5844-5847>) revealed the available backends. The journal check ([msg 5848]) confirmed that no explicit errors were being logged about the FP4 kernels. Together, these pieces pointed toward a configuration fix rather than a code bug.

Input Knowledge Required

To understand this message, one needs several layers of context:

GPU architecture knowledge: The SM120 compute capability (Blackwell) is a new NVIDIA architecture that requires specific kernel implementations. FP4 quantization is a Blackwell-native feature, meaning the kernels are relatively new and may not be fully battle-tested.

SGLang architecture knowledge: The fp4_gemm_runner_backend is a server argument that selects which FP4 matrix multiplication kernel to use. The available choices (flashinfer_cutlass, flashinfer_cudnn, flashinfer_trtllm) correspond to different kernel libraries, each with different levels of SM120 support.

Systemd and journalctl knowledge: The -u sglang-qwen flag filters logs to a specific service unit, -o cat removes metadata formatting, and --no-pager ensures the full output is returned without interactive paging.

The deployment history: The model had been downloaded, the service configured, the server started, and initial tests had failed with NaN outputs. Without this context, the journal query would appear to be a random log check.

Output Knowledge Created

The direct output of this message is minimal — a single log line confirming the server arguments. But the indirect output is more significant:

  1. Confirmation that the server started cleanly: No crash, no assertion errors, no explicit FP4 kernel failures. The NaN outputs were happening silently within the model inference, not at the system level.
  2. Verification of the model path and configuration: The log confirms that modelopt_fp4 quantization was correctly loaded and the model path was valid.
  3. Evidence that the FP4 backend selection doesn't log: This is a negative result — the absence of backend-specific log lines means the assistant would need to look elsewhere (code inspection, debug logging) to confirm which kernel was actually being used.
  4. A clean baseline for the next diagnostic step: With the log check done, the assistant could proceed to the next hypothesis test with confidence that the system was otherwise healthy.

What Happened Next

The user, seeing the assistant's diagnostic efforts, provided a crucial piece of external knowledge: a link to catid's GitHub gist documenting the exact SM120 setup for Qwen3.5 NVFP4 ([msg 5850]). The gist revealed the root cause: the default auto MoE runner backend selects a kernel that produces NaN on SM120. The fix was to explicitly pin --moe-runner-backend flashinfer_cutlass and --fp4-gemm-runner-backend flashinfer_cudnn.

This external reference validated the assistant's hypothesis and provided the exact configuration fix needed. The assistant then moved to implement the fix, stopping the server and preparing to update the service configuration ([msg 5854]).

Assumptions and Potential Mistakes

The assistant made several assumptions in this message:

That the journal would contain FP4-specific log lines: This was a reasonable assumption given that SGLang logs many configuration details at startup. However, the FP4 backend selection appears to happen at a lower level (in the kernel loading code) without explicit logging. The grep found only the startup line because it contained "NVFP4" in the model path.

That journalctl output was complete: The --no-pager flag ensures all output is returned, but the head -15 limit could have truncated important information if there were more than 15 matching lines. In practice, there was only one match, so this wasn't an issue.

That the NaN problem was FP4-related: This assumption turned out to be correct, but at the time of the message, it was still a hypothesis. The journal check was designed to either confirm or refute it.

The Broader Significance

Message [msg 5848] exemplifies a pattern that recurs throughout this coding session: the assistant systematically gathering evidence before acting. In a session spanning dozens of messages — installing drivers, resolving build issues, patching source code, configuring services, and benchmarking performance — each diagnostic step builds on the previous one. The journal check may seem trivial in isolation, but it represents a deliberate choice to understand the system's behavior before making changes.

This approach is particularly valuable when dealing with bleeding-edge hardware (Blackwell GPUs) and software (SGLang main branch, nightly PyTorch, NVFP4 quantization). When multiple components could be at fault, narrowing the search space through targeted queries is more efficient than trial-and-error configuration changes. The assistant's decision to check the logs first, rather than blindly flipping backend flags, saved time and reduced the risk of introducing new problems.

The message also highlights the importance of system observability in ML infrastructure. The fact that the FP4 backend selection didn't produce any log output is itself a finding — it means that diagnosing kernel compatibility issues requires either code inspection, debug-level logging, or external references like catid's gist. A more verbose startup sequence could have saved the assistant several diagnostic steps.

Conclusion

Message [msg 5848] is a small but revealing moment in a complex deployment narrative. A single journalctl command, executed with careful keyword selection, represents the assistant's commitment to evidence-based troubleshooting. The output was minimal — just one log line confirming the server configuration — but the process of gathering that output, interpreting it, and using it to guide the next steps demonstrates a methodical approach to diagnosing numerical errors in large language model inference on novel hardware. The NaN outputs that prompted this investigation would ultimately be resolved by explicit backend configuration flags, but only after the assistant had thoroughly mapped the problem space through code inspection, log analysis, and external reference consultation.