The Wave-Quantization Fix: How a Single Environment Variable Unlocked 12.8% Decode Throughput on Blackwell

In the relentless pursuit of pushing DeepSeek-V4-Flash decode throughput toward C90 on NVIDIA Blackwell GPUs, a single message ([msg 13537]) captures the moment when a carefully reasoned hypothesis met empirical validation. The assistant had just completed an A/B test comparing two configurations of the sparse attention kernel's split-K decomposition, and the results were unambiguous: setting SGLANG_SM120_MMA_TARGET_CTAS=512 delivered a 12.8% throughput gain at C=64 and a 5.7% gain at C=96, while simultaneously fixing a pathological wave-quantization anomaly where C=96 had been slower than C=80. This message is the hinge point where a week of deep kernel analysis, hypothesis testing, and dead-end exploration crystallized into a concrete, validated optimization—one that required no code changes, only a single environment variable.

The Context: A Week of Occupancy Debugging

To understand the significance of this message, one must appreciate the journey that preceded it. The assistant had been systematically optimizing DeepSeek-V4-Flash on 8× RTX PRO 6000 Blackwell GPUs (sm120 architecture) for days. The core problem was that decode throughput scaled sublinearly with concurrency: the marginal cost per request was 1.05ms, and the goal was to push from C60 to C90 throughput.

Earlier investigation ([msg 13528]) had revised the entire premise of the optimization effort. A fresh profile at batch size 80 revealed that attention was the slope driver, not MoE. The MoE grouped-GEMM kernel was essentially flat (15.1ms at bs32 → 15.9ms at bs80, adding only +0.015 ms/req), while the attention kernel jumped from 13.1ms to 51.2ms (+0.79 ms/req). This meant the entire C60→C90 ceiling was set by the attention kernel's scaling behavior.

The root cause was identified as 1 CTA per SM occupancy. All dominant kernels were shared-memory-capped: attention used 80KB, MoE used 89KB, of ~100KB available per SM. This left only 4-12 warps active per SM out of a possible 48, meaning the SMs were "active" but memory-latency-stalled (57% power-limited, 27% DRAM-bound). The attention kernel specifically used 128-thread blocks with 80KB of shared memory, forcing just 4 warps per SM at 8% occupancy.

The investigation identified two concrete levers: wave-aware NSPLIT selection (adjusting the split count to minimize fractional-wave tails) and doubling warps per SM (by reducing the shared memory staging footprint). The former was essentially free—a config change. The latter required kernel modifications.

The False Start: BLOCK_H=16

Before reaching the TARGET_CTAS optimization, the assistant tried the more ambitious lever first. In message [msg 13533], the assistant set SGLANG_SM120_MMA_BLOCK_H=16, which halves the block height from 32 to 16 heads per CTA. The reasoning was sound: halving SMEM per CTA should allow 2 CTAs per SM, doubling warp occupancy and improving latency hiding.

The result, shown in [msg 13534], was a clear regression: BLOCK_H=16 was worse across the board by -3% to -9%. The assistant correctly diagnosed the failure mode in [msg 13535]: the kernel uses 255 registers per thread × 128 threads = 32,640 registers per CTA. Two CTAs would need 65,280 registers, exceeding the SM's 65,536 register limit. The kernel was register-capped at 1 CTA/SM regardless of SMEM, so halving BLOCK_H just shrank per-CTA work (worse MMA efficiency) without gaining occupancy. This was a dead end.

The Pivot: TARGET_CTAS=512

The assistant pivoted to the wave-fill lever. The key insight was that at the problematic batch sizes (C=64 to C=96), the attention kernel's grid size created fractional-wave tails that wasted SM capacity. At C=96 with the default TARGET_CTAS=256, the grid produced 384 CTAs = 2.04 waves, leaving a near-empty third wave with ~96% of its SMs idle. This explained the pathological behavior where C=96 (799.3 tok/s) was slower than C=80 (833.0 tok/s).

By raising TARGET_CTAS to 512, the split-K decomposition would generate more CTAs, packing the waves more efficiently. The assistant set the environment variable and restarted the decode server in [msg 13535], then waited for it to become ready and ran the benchmark suite in [msg 13536].

The Subject Message: Validation and Commitment

Message [msg 13537] is where the assistant receives and interprets the results of that A/B test. The message opens with the agent's reasoning, which reveals a sophisticated cost-benefit analysis:

"Looking at the performance comparison between the two TARGET_CTAS configurations, I'm seeing significant improvements at certain concurrency levels—particularly a 12.8% gain at C=64 and 5.7% at C=96, which aligns well with the target range the user is aiming for."

The assistant immediately recognizes the strategic importance: the gains are concentrated exactly where they matter most—the C60–C90 range that is the project's primary optimization target.

"What's especially promising is that the scaling is now much more linear and monotonic across the concurrency spectrum (680→812→843→845), and the anomaly where C=96 underperformed C=80 is resolved."

This is the critical observation. The wave-quantization anomaly that had been a persistent frustration—where increasing concurrency actually decreased throughput—is now fixed. The scaling curve is now monotonic and near-linear, which is the hallmark of efficient wave packing.

The assistant then demonstrates a mature engineering judgment:

"Before committing this change, I need to validate correctness since the kernel's split behavior is changing. The split-K reduction with LSE combine should be mathematically exact, but I'll run the corruption reproduction test (60×4) as a gate to confirm there's no degradation."

This is notable because the project had recently suffered from a persistent high-concurrency tool-call corruption bug ([chunk 72.1]), which was ultimately traced to a multi-stream-overlap race condition in the CUDA-graph capture path. The corruption manifested specifically under high concurrency with bf16 index keys—exactly the regime where TARGET_CTAS changes would be active. The assistant is rightly cautious, running a 60-session, 4-round stress test as a correctness gate.

The assistant also considers whether further optimization is possible:

"After that, I should also explore whether pushing TARGET_CTAS even higher (768, 1024) yields further improvements or if 512 is the sweet spot."

The reasoning that follows is a textbook example of marginal-returns analysis:

"Looking at the utilization tradeoffs: at TARGET_CTAS=512, B=64 hits 91% utilization with 4 splits, while B=96 gets 77% with 3 splits. Bumping to 768 actually hurts B=64's utilization (down to 82% with more split overhead) and only marginally helps B=96. The gains at C=96 are smaller than at C=64 anyway, so 512 appears to be the optimal balance without over-engineering."

This analysis correctly identifies that TARGET_CTAS=512 is near the sweet spot: higher values would increase split overhead (each split requires its own attention computation and a reduction step) while providing diminishing wave-fill returns. The assistant has internalized the wave-utilization math well enough to reason about tradeoffs without running every possible configuration.

The Results Table

The message presents a clear comparison table:

| C | baseline | TARGET_CTAS=512 | gain | |---|---|---|---| | 48 | 684.7 | 680.6 | ~0 | | 64 | 719.5 | 811.7 | +12.8% | | 80 | 833.0 | 842.7 | +1.2% | | 96 | 799.3 | 844.6 | +5.7% |

The table tells a compelling story. At C=48, there's no change—the wave-fill is already efficient at low concurrency. At C=64, the improvement is dramatic (+12.8%), suggesting that this batch size was particularly poorly served by the default TARGET_CTAS=256. At C=80, the gain is modest (+1.2%), likely because this batch size already had decent wave utilization. At C=96, the gain is substantial (+5.7%), and crucially, the throughput now exceeds C=80's, restoring monotonic scaling.

The Correctness Gate Results

The bash command and its output confirm both correctness and no regression:

=== correctness gate: corruption repro 60x4 (must be 0%) ===
wall=237.4s counts={"maxrounds": 60}
CORRUPTION sessions: 0/60 = 0%  (leak=0 no_tool=0 error=0 ok-ish[done/maxrounds]=60)

Zero corruption across 60 sessions with 4 rounds each. This is a strong validation that the TARGET_CTAS change does not interact adversely with the CUDA-graph capture path or the multi-stream-overlap mechanism that had caused the earlier corruption bug.

The low-C regression check also passes cleanly:

=== low-C regression check (C=1,8) ===
C=1   n=4   max_tokens=192 | agg=52.9 tok/s | per-req~52.9 tok/s | p50 lat=3.63s | toks=768 errs=0 wall=14.5s
C=8   n=32  max_tokens=192 | agg=279.8 tok/s | per-req~283.8 tok/s | p50 lat=5.31s | toks=6144 errs=0 wall=22.0s

The assistant had expressed concern that low concurrency might regress because nsplit gets capped high at C=1. The results show no pathological behavior—low-C throughput is reasonable, confirming that the change doesn't introduce overhead at the other end of the concurrency spectrum.

Assumptions and Their Validity

The message rests on several assumptions, most of which are validated by the results:

  1. The split-K reduction with LSE combine is mathematically exact. This is a standard property of the flash attention algorithm—the log-sum-exp trick allows exact recombination of partial attention computations. The assumption is well-founded, and the corruption test confirms no numerical issues.
  2. TARGET_CTAS=512 is near-optimal without over-engineering. The assistant's marginal-returns analysis is sound, but it's worth noting that the assistant didn't actually test TARGET_CTAS=768 or 1024. The reasoning is based on utilization math and split-overhead estimates, not empirical data. This is a reasonable engineering shortcut—the gains at C=96 are already smaller than at C=64, suggesting diminishing returns—but it remains an untested assumption.
  3. The corruption gate is sufficient for correctness. The 60×4 stress test is a strong signal but not a proof. The earlier corruption bug was intermittent and depended on specific timing conditions in the CUDA-graph replay. A clean 60-session run is encouraging but doesn't guarantee the change is safe under all conditions.
  4. Low-C performance is acceptable. The assistant only tested C=1 and C=8, not the full range. This is pragmatic—the optimization target is C60–C90, and low-C performance is less critical—but it does leave a gap in the regression coverage.

The Thinking Process

The agent reasoning in this message reveals a disciplined, structured thought process. The assistant:

  1. Interprets the data: Immediately identifies the key patterns—gains at C=64 and C=96, fixed monotonicity.
  2. Assesses strategic fit: Recognizes that the gains align with the project's primary optimization target.
  3. Plans validation: Identifies the correctness risk (changing split behavior) and designs an appropriate test.
  4. Considers further optimization: Evaluates whether higher TARGET_CTAS values would help, using utilization math.
  5. Makes a decision: Concludes that 512 is the sweet spot and commits to the change, gated by the correctness test. This is a model of evidence-driven engineering decision-making. The assistant doesn't just accept the benchmark numbers—it reasons about why they're better, whether they could be even better, and what risks the change introduces.

Input and Output Knowledge

To understand this message, the reader needs:

Broader Significance

This message represents a turning point in the optimization effort. After days of deep kernel analysis, dead-end explorations (BLOCK_H=16, Two-Batch Overlap, various other approaches), and a complex corruption debugging saga, the assistant has found a simple, safe, and effective optimization that directly addresses the project's primary goal. The fix is a single environment variable—no code changes, no kernel rewrites, no complex configuration.

The fact that such a significant gain (12.8% at C=64, 5.7% at C=96) comes from a one-line configuration change is a testament to the value of deep understanding. The assistant didn't stumble on this optimization by accident—it was the direct result of the wave-quantization analysis, the register-pressure diagnosis that ruled out BLOCK_H=16, and the understanding of how TARGET_CTAS interacts with the split-K decomposition. Every dead end was informative, and every hypothesis tested narrowed the search space until the correct lever was found.

The message also demonstrates a mature approach to production engineering: the assistant doesn't just ship the optimization based on benchmark numbers. It validates correctness under stress, checks for regressions at other concurrency levels, and reasons about whether further optimization is worth pursuing. This combination of empirical testing and analytical reasoning is what separates a reliable optimization from a fragile one.

In the broader arc of the project, this message is where the decode throughput optimization effort crosses the threshold from investigation to delivery. The wave-quantization anomaly is fixed, the scaling curve is monotonic, and the C60→C90 target is substantially closer. The assistant has earned the right to commit this change with confidence.