The Commit That Fixed a Wave-Quantization Anomaly: A Deep Dive into GPU Attention Kernel Tuning

Introduction

On the surface, message [msg 13539] appears unremarkable: a single bash command that stages a markdown file and commits it to a Git repository with a descriptive message. The assistant writes:

cd /home/theuser/glm-kimi-sm120-rtx6000bw && git add DSV4_DECODE_PERF_PLAN.md && git commit -q -m "perf(dsv4): #3 attention occupancy — MMA_TARGET_CTAS=512 deployed (+12.8% C64, +5.7% C96, fixes C96<C80 wave-quant anomaly, monotonic scaling); 0% corruption, no low-C regression. BLOCK_H=16 rejected (reg-capped 1 CTA/SM)" && echo committed

But this commit message is a dense artifact of an intense performance debugging session. It encapsulates the culmination of a methodical investigation into why a DeepSeek-V4-Flash (DSV4) inference deployment on Blackwell (sm_120) GPUs exhibited a baffling non-monotonic throughput curve: at concurrency level 96, the system was slower than at concurrency 80. This article unpacks the reasoning, decisions, assumptions, and knowledge embedded in that single message.

The Context: Decode Throughput Scaling on Blackwell GPUs

The assistant had been working for many rounds on optimizing the decode phase of DSV4 inference running on NVIDIA Blackwell RTX PRO 6000 GPUs (sm_120 architecture). The user's goal was to improve decode throughput scaling from C60 to C90—that is, to make the system handle increasing numbers of concurrent requests with smooth, predictable performance gains rather than hitting mysterious plateaus or regressions.

Earlier in the session, the assistant had established a baseline benchmark across concurrency levels 48, 64, 80, and 96. The results revealed a troubling anomaly: C=80 achieved 833.0 tok/s, but C=96 dropped to 799.3 tok/s ([msg 13532]). This violated the expectation that more concurrency should yield higher aggregate throughput, and it pointed to a fundamental inefficiency in how the attention kernel utilized the GPU's Streaming Multiprocessors (SMs).

Diagnosing the Wave-Quantization Problem

The root cause, as the assistant identified through careful analysis of the kernel's grid dimensions, was wave quantization. The attention kernel launches a grid of Cooperative Thread Arrays (CTAs), and the GPU schedules these CTAs onto its SMs in waves. Each Blackwell SM can run one CTA at a time (due to register pressure), and the GPU has 188 SMs. When the total number of CTAs doesn't divide evenly by 188, the final wave is partially filled, wasting compute capacity.

At C=96 with the default configuration (BLOCK_H=32, TARGET_CTAS=256), the kernel launched 384 CTAs. This required 2.04 waves: two full waves of 188 CTAs each, plus a third wave with only 8 CTAs—a mere 4% utilization. Meanwhile, C=80 produced 320 CTAs, fitting into 1.70 waves with the second wave at 70% capacity. The near-empty third wave at C=96 was the direct cause of the performance regression.

This diagnosis was not obvious. It required understanding the GPU architecture (188 SMs on Blackwell), the kernel's grid launch parameters (how BLOCK_H and TARGET_CTAS affect CTA count), and the relationship between batch size, number of attention heads, and the split-K reduction scheme. The assistant had to connect a performance symptom (C96 < C80) to a micro-architectural cause (wave underutilization) through a chain of reasoning about register pressure, shared memory budgets, and arithmetic intensity.

The Rejected Hypothesis: BLOCK_H=16

The assistant's first attempted fix was to reduce MMA_BLOCK_H from 32 to 16 ([msg 13533]). The reasoning was intuitive: halving the block height would halve each CTA's shared memory consumption, potentially allowing two CTAs to run concurrently on the same SM, doubling occupancy and improving latency hiding. This seemed like a clean, environment-variable-tunable change with no code modifications required.

The results were disappointing ([msg 13534]). Across all concurrency levels, BLOCK_H=16 performed worse than the baseline, with degradation ranging from 3% to 9%. The assistant correctly diagnosed the failure: the kernel uses 255 registers per thread, and with 128 threads per CTA, each CTA requires 32,640 registers. Two CTAs would need 65,280 registers, which exceeds the SM's 65,536-register file. The kernel was register-capped at 1 CTA per SM regardless of shared memory, so halving BLOCK_H merely made each CTA do less work (smaller matrix tiles, lower arithmetic intensity) without any occupancy benefit.

This is a valuable lesson in GPU kernel optimization: occupancy improvements must account for all resource limits—registers, shared memory, and warps—not just the one you're trying to relieve. The assumption that SMEM was the binding constraint was incorrect; registers were the true limiter. The assistant documented this dead end in the commit message as "BLOCK_H=16 rejected (reg-capped 1 CTA/SM)," preserving the negative result for future reference.

The Successful Fix: TARGET_CTAS=512

Pivoting from the failed BLOCK_H experiment, the assistant turned to the MMA_TARGET_CTAS parameter ([msg 13535]). This parameter controls the split-K factorization of the attention computation: instead of launching one CTA per attention head per batch element, the kernel can split the work across multiple CTAs that each handle a portion of the reduction, then combine results via a tree reduction with log-sum-exp (LSE) correction.

By increasing TARGET_CTAS from the default of 256 to 512, the assistant changed the wave-fill characteristics. At C=96, the CTA count shifted from 384 to a configuration that packed more efficiently into the available 188 SMs. The results were dramatic ([msg 13536]): C=64 jumped from 719.5 to 811.7 tok/s (+12.8%), C=96 rose from 799.3 to 844.6 tok/s (+5.7%), and the scaling became monotonic across the entire range (680→812→843→845). The C96<C80 anomaly was completely eliminated.

The mechanism here is subtle. Increasing TARGET_CTAS doesn't change the total work—it changes how that work is partitioned into CTAs. More CTAs means more splits, which increases the reduction overhead but also improves wave utilization. The sweet spot is where the marginal benefit of better wave fill exceeds the marginal cost of additional split-K overhead. The assistant's empirical testing found that 512 was the optimal balance, and a quick probe of 768 ([msg 13537]) confirmed that pushing higher would not yield further gains.

Validation: Correctness and Regression Testing

Before committing the change, the assistant ran a rigorous validation suite ([msg 13537]). This was critical because changing the split-K configuration alters the numerical reduction order, which could theoretically introduce floating-point differences or, worse, trigger the persistent bf16 corruption bug that had plagued earlier sessions.

The corruption reproduction test ran 60 parallel sessions with 4 rounds each, checking for tool-call corruption. The result was 0% corruption across all 60 sessions—a clean pass. The assistant also verified that low-concurrency performance (C=1 and C=8) was unchanged, ensuring the optimization didn't regress the single-user case. This thorough validation reflects the operational discipline developed over many rounds of debugging: every performance change must be gated by a correctness check, especially in a system with known numerical sensitivity.

The Commit Message as Knowledge Artifact

The commit message in [msg 13539] is more than a log entry—it's a compressed summary of the entire investigation. It records:

  1. What was done: MMA_TARGET_CTAS=512 deployed
  2. The quantitative impact: +12.8% C64, +5.7% C96
  3. The qualitative fix: fixes C96&lt;C80 wave-quant anomaly, monotonic scaling
  4. Validation results: 0% corruption, no low-C regression
  5. A rejected alternative: BLOCK_H=16 rejected (reg-capped 1 CTA/SM) This structure embodies good engineering practice: it captures not just the winning approach but also the losing one, with the reason for rejection. The parenthetical "(reg-capped 1 CTA/SM)" is a crucial piece of institutional knowledge that could save someone else from repeating the same dead end. The message also references "#3 attention occupancy," indicating this is the third documented iteration in a structured performance optimization campaign. The file being committed, DSV4_DECODE_PERF_PLAN.md, is the living document that tracks the optimization roadmap, hypotheses tested, and results obtained.

Input Knowledge Required

To fully understand this message, one would need:

Output Knowledge Created

This message produces several forms of knowledge:

  1. A documented optimization result: The commit permanently records the TARGET_CTAS=512 configuration as the validated best setting.
  2. A rejected hypothesis: The BLOCK_H=16 experiment is preserved with its root cause (register cap), preventing future re-investigation.
  3. Validation evidence: The 0% corruption result provides confidence that the change is safe.
  4. A reference point for further optimization: Future work on decode throughput can build on this baseline.

Conclusion

Message [msg 13539] appears to be a simple Git commit, but it represents the culmination of a sophisticated debugging and optimization process. The assistant navigated from a puzzling performance anomaly (C96 < C80) through hypothesis formation (wave quantization), an experimental dead end (BLOCK_H=16), a successful fix (TARGET_CTAS=512), and rigorous validation—all within a span of a few messages. The commit message distills this journey into a single line that communicates both the winning configuration and the rejected alternative, embodying the principle that good documentation captures not just what works, but what doesn't and why.