The Wave-Quantization Anomaly: Diagnosing Non-Monotonic Throughput Scaling in CUDA-Graph Decode

Introduction

In the high-stakes world of LLM inference optimization on cutting-edge hardware, performance is rarely a straight line. Engineers pore over profiler traces, kernel launch configurations, and occupancy metrics, hunting for the micro-architectural inefficiencies that separate a good deployment from a great one. This article examines a single message from an opencode coding session — message index 13533 — in which an AI assistant, having already root-caused and fixed a persistent high-concurrency corruption bug in a DeepSeek-V4-Flash NVFP4 deployment on Blackwell GPUs, pivots to the next challenge: improving decode throughput scaling from C60 to C90. The message captures a moment of diagnostic insight followed by decisive action: the assistant identifies a wave-quantization anomaly in the attention kernel's CTA (Cooperative Thread Array) grid, hypothesizes a fix via the MMA_BLOCK_H environment variable, and deploys the change to a production server — all within a single reasoning step and a single bash command.

This message is a microcosm of the engineering methodology that characterizes the entire session: evidence-driven reasoning, deep kernel-level understanding, systematic hypothesis testing, and a willingness to act decisively on incomplete but suggestive data. It also reveals the assumptions and blind spots that can lead even a well-informed practitioner down a dead end — the BLOCK_H=16 change ultimately regressed performance across all concurrency levels, as revealed in the subsequent message. Understanding why the assistant made this call, what it got right, and where it went wrong offers a masterclass in GPU inference optimization.

The Broader Context

To appreciate message 13533, one must understand the journey that led to it. The session (segment 72 of a much longer conversation) began with the root-causing and fixing of a bf16 high-concurrency tool-call corruption that had plagued the DeepSeek-V4-Flash NVFP4 deployment for days. The corruption was definitively traced to a multi-stream-overlap race condition: the C4 sparse indexer, running on an alternate CUDA stream under graph capture, produced transient intermediates that aliased with main-stream tensors in the shared captured-graph memory pool. The fix was a single environment variable — SGLANG_OPT_USE_MULTI_STREAM_OVERLAP=0 — requiring no code changes, and was confirmed on a clean git build with 0% corruption across 80-session stress tests.

With the corruption fixed, the user's next priority was clear: improve decode throughput scaling. The assistant had already created a comprehensive project plan based on live bottleneck analysis. The decode was latency/occupancy-bound, with a step time formula of 18ms + 1.05ms/req. The marginal cost of each additional request was dominated by the attention kernel, not the MoE grouped-GEMM as initially suspected. A fresh batch-size-80 profiler trace had revealed that MoE GEMM was essentially flat (15.1ms at bs32 → 15.9ms at bs80, adding only +0.015 ms/req), while attention ballooned from 13.1ms to 51.2ms (+0.79 ms/req) — accounting for virtually the entire marginal cost.

The root cause of this attention scaling was identified as a 1-CTA-per-SM occupancy bottleneck. All dominant kernels were SMEM-capped: attention used 80KB of shared memory, MoE used 89KB, out of approximately 100KB per SM on the Blackwell RTX PRO 6000. This left only 4-12 warps active per SM out of a possible 48, meaning the SMs were "active" but memory-latency-stalled — reflected in the 57% power utilization and 27% DRAM-bound metrics. The assistant had catalogued several concrete, low-risk levers, including wave-aware NSPLIT selection and doubling warps per SM via the MMA_BLOCK_H parameter.

The Subject Message: A Wave-Quantization Anomaly

Message 13533 opens with the assistant examining the baseline benchmark results it had just collected. The baseline was run at the default configuration (BLOCK_H=32, TARGET_CTAS=256, overlap-off) across four concurrency levels: C=48, 64, 80, and 96. The results were:

C=48  n=192 max_tokens=256 | agg=684.7 tok/s | per-req~699.5 tok/s | p50 lat=17.15s
C=64  n=256 max_tokens=256 | agg=719.5 tok/s | per-req~739.6 tok/s | p50 lat=21.71s
C=80  n=320 max_tokens=256 | agg=833.0 tok/s | per-req~857.5 tok/s | p50 lat=23.25s
C=96  n=384 max_tokens=256 | agg=799.3 tok/s | per-req~822.2 tok/s | p50 lat=29.55s

The assistant immediately zeroes in on a striking anomaly: C=96 (799.3 tok/s) is slower than C=80 (833.0 tok/s). This breaks the expected monotonic scaling pattern — adding more concurrent requests should, in a well-behaved system, increase aggregate throughput even if per-request latency rises. But here, throughput actually drops when moving from 80 to 96 concurrent requests. This is a red flag that demands explanation.

The assistant's reasoning reveals a deep understanding of GPU wave quantization. It calculates: at C=96 with the current settings, the attention kernel's grid produces 384 CTAs. With 188 SMs on the RTX PRO 6000 Blackwell, this gives 384 / 188 = 2.04 waves. The first two waves fully occupy all 188 SMs, but the third wave contains only 384 - 376 = 8 CTAs — just 4% utilization of the available SMs. Those 8 CTAs launch, the remaining 180 SMs sit idle, and the kernel pays the full wave-launch overhead for a tiny amount of work.

In contrast, at C=80, the grid produces 320 CTAs, giving 320 / 188 = 1.70 waves. The second wave contains 320 - 188 = 132 CTAs, achieving 70% SM utilization. While not perfect, this is dramatically more efficient than the 4% utilization of the third wave at C=96.

The assistant correctly identifies this as "exactly the kind of wave-quantization waste that the wave-aware NSPLIT adjustment is designed to fix." It then considers the ideal fix: forcing nsplit=1 at C=96 instead of 2 would yield 192 CTAs, which would pack into 1.02 waves — a single nearly-full wave with just 4 CTAs spilling over. But the assistant recognizes that changing the split logic requires code modifications. Instead, it pivots to a different lever that can be applied purely through environment variables.

The Decision: Halving BLOCK_H

The assistant's chosen intervention is to reduce SGLANG_SM120_MMA_BLOCK_H from its default of 32 to 16. The BLOCK_H parameter controls how many query heads each CTA processes. Halving it from 32 to 16 means each CTA handles half as many heads, which halves the shared memory footprint per CTA. This allows the GPU scheduler to place 2 CTAs per SM instead of 1, effectively doubling the warp occupancy from approximately 4 warps/SM to 8 warps/SM.

The reasoning is nuanced. The assistant calculates that at C=96 with BLOCK_H=16, nsplit would drop to 1 (because each CTA does less work, so fewer splits are needed), keeping the CTA count at 384 but now allowing 2 CTAs per SM instead of 1. The key insight is that "halving SMEM per CTA and doubling warp occupancy" should provide better latency hiding — the SMs would have more warps to switch between when waiting for memory operations, reducing the stall cycles that currently dominate the 57% power, 27% DRAM-bound profile.

The assistant also demonstrates good operational hygiene: it backs up the original serve script before modifying it, uses sed to insert the environment variable in a targeted way (after the SGLANG_SM120_MMA_FLASHMLA=1 line), and confirms the change with a grep before restarting the systemd service. The bash command is a single line that chains these operations:

ssh -o ConnectTimeout=15 root@10.1.230.171 "cp /root/serve_dsv4_decode.sh /root/serve_dsv4_decode.sh.mmatune_base.bak; grep -q MMA_BLOCK_H /root/serve_dsv4_decode.sh || sed -i '/SGLANG_SM120_MMA_FLASHMLA=1/a export SGLANG_SM120_MMA_BLOCK_H=16' /root/serve_dsv4_decode.sh; grep -oE 'SGLANG_SM120_MMA_BLOCK_H=[0-9]+' /root/serve_dsv4_decode.sh; systemctl restart sglang-dsv4-decode; echo restart=\$?"

The output confirms success: SGLANG_SM120_MMA_BLOCK_H=16 and restart=0.

Assumptions and Their Consequences

This message is built on several assumptions, some explicit and some implicit:

Assumption 1: Wave quantization is the dominant bottleneck at C=96. The assistant observes the non-monotonic scaling and attributes it primarily to wave inefficiency. This is a reasonable inference — the math checks out (384 CTAs = 2.04 waves with a near-empty third wave) — but it assumes that wave quantization is the main reason C=96 underperforms C=80, rather than a secondary effect or a symptom of a deeper issue.

Assumption 2: Halving BLOCK_H will improve throughput. The assistant hypothesizes that reducing BLOCK_H from 32 to 16 will increase occupancy and thus improve latency hiding. This assumes that the dominant cost is memory latency (which the 57% power, 27% DRAM-bound metrics support) and that more warps per SM will effectively hide that latency. It also assumes that the reduction in per-CTA efficiency (each CTA now does half the work) is more than compensated by the increased occupancy and better wave packing.

Assumption 3: The env-var change is low-risk. The assistant notes that the kernel has a built-in numerical validation check (rel ≤ 6.7e-3) that gates correctness. This assumption proved correct — the BLOCK_H=16 runs completed without errors — but it did not guarantee performance improvement.

Assumption 4: The baseline benchmark is representative. The assistant ran the baseline at a single context configuration (max_tokens=256) with a specific request distribution. This is a reasonable proxy for production load but may not capture all relevant behaviors.

The subsequent message (13534) reveals that these assumptions did not hold in practice. The BLOCK_H=16 benchmark showed:

C=48: 625.4 tok/s (baseline 684.7) — regression of -8.7%
C=64: 680.7 tok/s (baseline 719.5) — regression of -5.4%
C=80: 804.4 tok/s (baseline 833.0) — regression of -3.4%
C=96: 763.4 tok/s (baseline 799.3) — regression of -4.5%

Every concurrency level regressed. The hypothesis was wrong.

Why? The most likely explanation is that halving BLOCK_H increased the total number of CTAs launched per step (from 384 to potentially more, depending on nsplit), which increased the total launch overhead and kernel scheduling cost. While each individual CTA had lower shared memory usage and could theoretically co-reside with another CTA on the same SM, the increased grid size may have exacerbated the wave-quantization problem rather than solving it — more CTAs mean more waves, potentially more fractional waves, and more launch overhead. Additionally, reducing per-CTA work may have increased the relative cost of the "glue" operations (the non-kernel portions of the step) that the assistant had previously identified as a secondary bottleneck.

The Thinking Process: A Window into Expert Debugging

The assistant's reasoning in this message reveals a sophisticated mental model of GPU execution. It doesn't just look at the raw throughput numbers; it reconstructs the underlying CTA grid geometry, calculates wave utilization, and maps the observed anomaly to a specific micro-architectural phenomenon. The chain of reasoning is:

  1. Observe anomaly: C=96 (799.3 tok/s) < C=80 (833.0 tok/s) — non-monotonic scaling.
  2. Hypothesize mechanism: Wave quantization — the CTA count at C=96 produces a highly inefficient wave configuration.
  3. Calculate: 384 CTAs / 188 SMs = 2.04 waves → third wave at 4% utilization.
  4. Identify ideal fix: Force nsplit=1 at C=96 to get 192 CTAs = 1.02 waves.
  5. Pivot to feasible fix: BLOCK_H=16 achieves similar effect through a different mechanism (2 CTAs/SM instead of 1).
  6. Execute: Backup, modify, restart, benchmark. This is textbook diagnostic reasoning: observe → hypothesize → calculate → identify ideal solution → adapt to practical constraints → execute. The assistant also demonstrates awareness of the trade-offs, noting that the "cleaner approach" of adjusting n_hg via BLOCK_H or targeting CTA counts that divide evenly by 188 "requires code changes," while the env-var approach is faster and lower-risk.

Input and Output Knowledge

Input knowledge required to understand this message includes:

Conclusion

Message 13533 captures a pivotal moment in a high-stakes optimization campaign. The assistant, having already demonstrated methodical debugging skills in root-causing a multi-stream-overlap corruption, now turns its analytical lens to throughput scaling. It identifies a genuine anomaly — non-monotonic throughput at high concurrency — and correctly diagnoses the mechanism (wave quantization). It then formulates a hypothesis (BLOCK_H=16 will improve occupancy and throughput) and executes the experiment with operational rigor.

That the hypothesis proved wrong in practice does not diminish the quality of the reasoning. In GPU kernel optimization, hypotheses are cheap and experiments are expensive — the value lies in the systematic elimination of possibilities and the accumulation of evidence. The BLOCK_H=16 regression, documented in the very next message, becomes valuable negative knowledge: it tells the assistant and the user that the occupancy lever, at least in this form, is not the answer. The wave-quantization problem at C=96 remains real, but the solution lies elsewhere — perhaps in wave-aware NSPLIT selection, or in a different kernel configuration entirely.

This message exemplifies the engineering virtue of decisive experimentation. The assistant did not over-analyze or seek perfect information before acting. It formed a clear hypothesis, implemented the change with proper safeguards, and let the benchmark results speak. When the results contradicted the hypothesis, that was progress — another path closed, the search space narrowed, the true solution one step closer.