The Persistence of Warnings: A Pivotal Diagnostic Check in the GLM-5-NVFP4 Debugging Saga

Introduction

In the complex world of deploying cutting-edge large language models on novel hardware, the difference between success and failure often comes down to correctly interpreting a handful of log lines. Message 208 of this opencode session captures one such diagnostic moment — a seemingly simple grep command that, in its output, reveals the stubborn persistence of two critical warnings that had been the focus of an extended debugging effort. This message is the culmination of a multi-attempt struggle to deploy the GLM-5-NVFP4 model (a quantized mixture-of-experts model) across 8 NVIDIA RTX PRO 6000 Blackwell GPUs using the SGLang inference framework, where the server repeatedly crashed during decode with a device-side assert triggered error caused by NaN/Inf values in the probability tensor.

The Message in Full

The assistant executes the following command:

sleep 10 && ssh 10.1.230.175 'grep -i "nsa.*backend\|kv_cache_dtype\|DeepGemm\|Transformers version" ~/sglang-glm5.log | head -10'

And receives the output:

[2026-02-19 00:19:44] WARNING model_config.py:955: DeepGemm is enabled but the scale_fmt of checkpoint is not ue8m0. This might cause accuracy degradation on Blackwell.
[2026-02-19 00:19:44] WARNING model_config.py:1025: Transformers version 5.2.0 is used for model type glm_moe_dsa. If you experience issues related to RoPE parameters, they may be due to incompatibilities between Transformers >=5.0.0 and some models. You can try downgrading to transformers==4.57.1 as a workaround.

At first glance, this appears to be a routine status check. But in the context of the preceding 25+ messages of debugging, these two lines represent a devastating setback — confirmation that neither of the assistant's latest configuration changes had any effect on the root causes of the NaN crashes.

Context: The Debugging Journey So Far

To understand why this message matters, we must trace the path that led here. The assistant had been attempting to deploy GLM-5-NVFP4, a model using the glm_moe_dsa architecture (a DeepSeek-style sparse attention variant) with FP4 quantization, on 8 Blackwell GPUs. The deployment had been plagued by a persistent crash pattern: the model would load successfully, complete CUDA graph capture and warmup, but then crash on the very first decode step with a device-side assert triggered error. The root cause, as revealed in earlier log inspections (see [msg 185]), was that the "probability tensor contains either inf, nan or element < 0" — the model was producing garbage logits during generation.

The assistant had traced this to two potential sources. First, a warning that "DeepGemm is enabled but the scale_fmt of checkpoint is not ue8m0," which suggested that the DeepGemm library (a high-performance GEMM implementation for Blackwell) was auto-selecting itself but was incompatible with the checkpoint's quantization scale format. Second, a warning that Transformers 5.2.0 might have RoPE parameter incompatibilities with the model.

The assistant had made multiple attempts to fix this. In [msg 187], it tried forcing --fp8-gemm-backend cutlass to bypass DeepGemm. In [msg 204], it added --disable-cuda-graph and removed the explicit --kv-cache-dtype fp8_e4m3 flag, hoping that using default BF16 KV cache would avoid the problematic NSA attention backends. In [msg 207], the most recent attempt, it went further by explicitly setting --nsa-decode-backend trtllm and --nsa-prefill-backend trtllm, along with --kv-cache-dtype auto and --disable-cuda-graph.

Message 208 is the diagnostic check of that latest attempt — the moment of truth.

Why This Message Was Written: The Reasoning and Motivation

The assistant's motivation for issuing this command is deeply strategic. After four consecutive failed server launches, each crashing with the same NaN error, the assistant needed to answer a critical question: Are my configuration changes actually addressing the root causes, or am I fighting the wrong battle?

The grep pattern is carefully chosen. The assistant searches for four specific terms:

  1. nsa.*backend — To verify whether the explicit --nsa-decode-backend trtllm and --nsa-prefill-backend trtllm flags were respected, or whether the DSA model architecture was overriding them with flashmla_kv backends (which had been implicated in the NaN crashes).
  2. kv_cache_dtype — To check whether --kv-cache-dtype auto actually resulted in BF16 KV cache, or whether the model's DSA architecture forced fp8_e4m3 (which in turn forces specific NSA backends).
  3. DeepGemm — To see if the --fp8-gemm-backend cutlass flag successfully suppressed the DeepGemm scale format warning, or if DeepGemm was still being activated somewhere in the pipeline.
  4. Transformers version — To check whether the Transformers 5.2.0 RoPE incompatibility warning was still present, or if downgrading or some other workaround had resolved it. This is not a random log dump. It is a targeted diagnostic designed to test four specific hypotheses about the NaN source, all in one command. The assistant is practicing a form of differential diagnosis: by checking which warnings persist after each configuration change, it can narrow down which component is actually responsible for the numerical instability.

What the Output Reveals: A Double Blow

The output delivers two pieces of bad news, each with distinct implications.

First, the DeepGemm warning persists. Despite setting --fp8-gemm-backend cutlass, the log still reports "DeepGemm is enabled but the scale_fmt of checkpoint is not ue8m0." This is a critical finding because it reveals that the assistant's model of the problem was incomplete. The assistant had assumed that DeepGemm was the FP8 GEMM backend — a component that handles matrix multiplication for linear layers. By setting --fp8-gemm-backend cutlass, it expected DeepGemm to be disabled. But the warning's persistence suggests that DeepGemm is being activated in a different context — perhaps in the attention path, the MoE runner, or some other component that the --fp8-gemm-backend flag does not control. This is a significant insight: the DeepGemm library appears to have a broader scope than just the GEMM backend, and its scale format incompatibility may affect multiple parts of the model's execution.

Second, the Transformers 5.2.0 RoPE warning persists. This confirms that the model's glm_moe_dsa architecture requires Transformers 5.2.0 (or at least, SGLang's model loading path uses it), and the RoPE parameter incompatibility warning is still active. The warning itself is cryptic — it says "If you experience issues related to RoPE parameters, they may be due to incompatibilities between Transformers >=5.0.0 and some models" — but does not specify what those issues might be. The assistant had briefly experimented with downgrading to Transformers 4.57.6 in [msg 201], only to discover that the model repo does not provide custom config code via trust_remote_code; it relies entirely on Transformers 5.2.0+ having the glm_moe_dsa architecture built in. This creates a dependency trap: the model cannot load without Transformers 5.2.0, but Transformers 5.2.0 may itself be the source of the NaN-producing RoPE incompatibility.

Assumptions Made and Lessons Learned

This message reveals several assumptions the assistant was operating under, and the output exposes their flaws.

Assumption 1: --fp8-gemm-backend cutlass would disable DeepGemm. The persistence of the DeepGemm warning disproves this. The warning is generated at model_config.py:955, which is in the model configuration stage, not the runtime GEMM selection. DeepGemm may be enabled based on hardware detection (Blackwell GPUs) rather than the GEMM backend flag. The assistant's mental model of how DeepGemm is controlled was incorrect.

Assumption 2: --kv-cache-dtype auto would avoid fp8_e4m3. The output doesn't directly show the KV cache dtype, but the fact that the NSA backend warnings are absent from the grep output (only two lines matched) suggests that the NSA backend configuration may have been overridden or that the server didn't reach the point of logging them before crashing. The assistant had previously observed in [msg 206] that the DSA model auto-selects kv_cache_dtype=&#39;fp8_e4m3&#39; even when not explicitly passed, so the auto setting may have been overridden.

Assumption 3: The NaN crash has a single root cause. The persistence of both warnings suggests that there may be multiple issues contributing to the numerical instability. The DeepGemm scale format problem could cause accuracy degradation in the GEMM operations, while the RoPE incompatibility could cause incorrect positional encoding in the attention mechanism. Either one alone could produce NaN logits, and they could also interact synergistically.

Input Knowledge Required

To fully understand this message, the reader needs substantial background knowledge:

Output Knowledge Created

This message creates several important pieces of knowledge:

  1. Confirmed persistence of DeepGemm warning: The --fp8-gemm-backend cutlass flag does not suppress the DeepGemm scale format warning. This means DeepGemm is being activated through a different mechanism — possibly hardware detection (SM120 Blackwell GPUs automatically enable DeepGemm features) or through a different configuration path (the MoE runner or attention backend).
  2. Confirmed persistence of RoPE warning: The Transformers 5.2.0 RoPE incompatibility warning is still active. The model cannot be loaded without Transformers 5.2.0 (as verified in [msg 202]), but the warning suggests this version may introduce numerical issues.
  3. Negative evidence: The absence of NSA backend log lines in the grep output (only two lines matched) is itself informative. It suggests either that the NSA backend configuration was overridden silently, or that the server didn't progress far enough in its initialization to log those details before the relevant code path was executed.
  4. The debugging strategy needs to change: The assistant has now exhausted the approach of tweaking backend flags. Both known warnings persist despite multiple configuration attempts. This implicitly points toward the need for a more fundamental fix — perhaps patching the SGLang source code, modifying the model's configuration, or using a different version of the serving framework.

The Thinking Process Visible in the Message

The message itself does not contain explicit reasoning text (the assistant's thinking is not shown), but the reasoning is embedded in the structure of the command. The assistant could have checked the log in many ways — it could have tailed the log, searched for "error" or "exception," or checked the server's HTTP status. Instead, it chose a precise grep for four specific patterns, each corresponding to a hypothesis about the NaN source.

The sleep 10 at the beginning is also significant. It indicates the assistant knows the server takes time to initialize and log its configuration warnings. The assistant is being careful not to check too early and get a false negative.

The choice of head -10 (limiting to 10 lines) suggests the assistant expects only a few matching lines and wants to see them all without being overwhelmed by repeated warnings from multiple tensor-parallel ranks.

The fact that only two lines appear in the output — and that they are the same two warnings the assistant has been fighting for multiple rounds — is a moment of diagnostic clarity. The assistant now knows that its configuration changes are not reaching the root cause. This sets the stage for the next phase of debugging, which will need to be more fundamental: perhaps modifying the SGLang source code, patching the model's configuration, or finding an alternative deployment path.

Conclusion

Message 208 is a deceptively simple diagnostic check that, in its output, delivers a devastating verdict on the assistant's debugging strategy. The persistence of both the DeepGemm scale format warning and the Transformers 5.2.0 RoPE incompatibility warning, despite multiple configuration changes across four server launches, proves that the assistant's interventions were targeting the wrong layer of the system. The DeepGemm warning persists because it is triggered by hardware detection, not the GEMM backend flag. The Transformers warning persists because the model architecture is inseparable from version 5.2.0. This message marks the end of one debugging phase and the beginning of another — one that will require deeper changes to the deployment pipeline itself. In the archaeology of a debugging session, this is the stratum where the easy answers run out and the hard work begins.