Closing the Book on Register-Reduce: How a Single Edit Documented a Dead End in GPU Kernel Optimization

The Message

## Agent Reasoning
I need to update the plan document to reflect that the register-reducing rewrite approach (#3a) has been refuted by the occupancy lever analysis, so I should mark that option as no longer viable and update the remaining options section accordingly. I'll also need to mark the related todos as complete.
Now update the plan doc to reflect the #3a outcome (the rewrite is no longer the recommended big lever) and finalize tracking:
[edit] /home/theuser/glm-kimi-sm120-rtx6000bw/DSV4_DECODE_PERF_PLAN.md
Edit applied successfully.

At first glance, message [msg 13574] appears unremarkable: a brief note about editing a plan document, a single file edit, and confirmation that it succeeded. Yet this message represents the quiet denouement of a substantial engineering investigation — the formal burial of a promising hypothesis after days of meticulous experimentation. It is the moment when the assistant, having exhausted a line of inquiry through rigorous A/B testing, commits the negative result to the project's permanent record and redirects the team's attention toward more fruitful paths. To understand why this message matters, one must trace the thread of reasoning that led to it, appreciate the experimental rigor that preceded it, and recognize the intellectual discipline required to accept — and document — a dead end.

The Context: A Quest for Throughput on Blackwell GPUs

The message sits within a larger narrative: the deployment and optimization of DeepSeek-V4-Flash, a large language model, on NVIDIA Blackwell (sm_120) GPUs. The system uses prefill-decode (PD) disaggregation, where separate GPU workers handle prompt processing (prefill) and token generation (decode). The decode worker's attention kernel — the flash_mla_sm120_triton kernel — had become the primary bottleneck, limiting throughput to approximately 845 tokens per second at high concurrency.

The assistant had been systematically working through a performance plan, designated DSV4_DECODE_PERF_PLAN.md, which catalogued potential optimization levers. One of the most ambitious entries was item #3a: a "register-reducing rewrite" of the attention kernel. The premise was compelling: the existing kernel used 4 warps (128 threads) per block, and each thread handled 72 accumulator values, creating high register pressure. If the kernel could be rewritten to use 8 warps per block, each thread would handle fewer values, reducing register pressure and potentially allowing more thread blocks to run concurrently on each Streaming Multiprocessor (SM). This increased occupancy could hide memory latency and boost throughput.

This was not a trivial tweak. It was a fundamental restructuring of the kernel's compute pattern. Before committing to such a rewrite, the assistant needed evidence that the premise was sound — that 8 warps per SM would actually improve performance. The experiments that followed would decisively answer that question.

The Experimental Campaign: V1 and V2

The assistant designed two experiments, each testing a different mechanism for increasing occupancy.

V1 (num_warps=8): The first experiment doubled the number of warps from 4 to 8 while keeping the prefetch pipeline at 2 stages. The hypothesis was straightforward: more warps means more threads in flight, better latency hiding, higher occupancy. The result was unambiguous. At low concurrency (C=1), throughput dropped 14–18%. At high concurrency, the numbers were within the noise band but showed no improvement. The 8-warp configuration was rejected everywhere. The root cause was clear in retrospect: the attention kernel uses small [16×16] MMA (matrix multiply-accumulate) tiles. Subdividing such tiny tiles across 8 warps creates excessive overhead from warp scheduling and shared memory partitioning, overwhelming any occupancy benefit. The premise of the register-reducing rewrite — that more warps would hide the gather latency — was empirically refuted.

V2 (num_stages=3): The second experiment took a different approach. Instead of adding warps, it deepened the prefetch pipeline from 2 stages to 3 while keeping the efficient 4-warp configuration. The idea was to improve latency hiding by fetching KV-cache data further ahead, smoothing out the irregular gather pattern that characterizes sparse attention. Shared memory would increase from 60KB to 81KB — still within the 99KB per-SM limit — so the kernel should launch. The results were again negative: C64 showed a 9.9% regression, C96 dropped 2.6%, and low-concurrency cases regressed 5–7%. The deeper pipeline could not be amortized because the loop iterations per block were too short (4–6 iterations at high concurrency). The pipeline filled and drained before delivering benefit, while the extra shared memory consumption added pressure.

Both levers — more warps and deeper prefetch — had failed. The baseline configuration of BLOCK_T=16, num_warps=4, num_stages=2 was already optimal among feasible configurations. The register-reducing rewrite, which had depended on one of these mechanisms for its performance gain, was a dead end.

The Discovery of Benchmark Variance

A critical methodological insight emerged during the V2 post-mortem. When the assistant reverted to baseline and re-ran benchmarks, the C96 numbers varied from 775 to 845 tok/s across runs — a ±5–7% swing. This noise band had important consequences. The apparent 5.7% win from TARGET_CTAS=512 (a separate optimization that increased the number of cooperative thread arrays) sat right at the edge of the noise floor, making it marginal. Conversely, the V1 and V2 regressions were large enough (9.9–18%) to be genuine even accounting for variance. The discovery of this noise band became part of the documented record, serving as a caution for future tuning work: any claimed improvement below ~7% at C96 could be spurious.

The Subject Message: What It Does and Why

This brings us to message [msg 13574]. The assistant has just committed the V1 and V2 results to the A/B test document (DSV4_ATTN_OCCUPANCY_AB.md) with the conclusive finding that both occupancy levers are rejected and the register-reducing rewrite premise is empirically refuted. Now it turns to the master plan document — DSV4_DECODE_PERF_PLAN.md — to update the project's strategic roadmap.

The reasoning in the message is concise but reveals several layers of decision-making:

1. Knowledge integration: The assistant must connect the experimental results from one document to the planning document, ensuring the project's strategic direction reflects empirical reality. The #3a item is no longer "a promising optimization to investigate" but "a refuted approach that should not be pursued."
2. Documentation hygiene: The plan document serves as the authoritative reference for the project's optimization strategy. Leaving a refuted approach listed as a viable option would mislead anyone reading the plan — including the user, future developers, or the assistant itself when it later queries its own context. The update is an act of epistemic hygiene.
3. Task tracking: The message mentions marking "related todos as complete." This reflects a project-management sensibility: experimental work that has been concluded — even with a negative result — should be tracked as done, not left in an ambiguous "in progress" state.
4. Strategic reframing: The assistant explicitly notes that "the rewrite is no longer the recommended big lever." This is a subtle but important shift. Before the experiments, the register-reducing rewrite was considered a potentially major optimization — a "big lever." Now it is downgraded to a non-viable path. The plan document must reflect this changed assessment so that future effort is not misdirected.

Assumptions and Their Fate

Several assumptions underpinned the #3a investigation, and the message implicitly acknowledges their fate:

• Assumption: Occupancy is the primary bottleneck. The experiments tested whether increasing occupancy (through more warps or deeper prefetch) would improve throughput. Both failed, suggesting that the ~845 tok/s ceiling is not an occupancy problem but something more fundamental — likely the gather bandwidth of the sparse attention index or the inherent work pattern of the MLA (Multi-head Latent Attention) kernel. This assumption was refuted.
• Assumption: 8 warps would hide gather latency. This was the core premise of the register-reducing rewrite. The V1 experiment showed that 8 warps actually hurt performance, disproving the premise. The assumption was incorrect because it underestimated the overhead of subdividing small MMA tiles across many warps.
• Assumption: Deeper prefetch would smooth irregular memory access. The V2 experiment showed that the loop structure (4–6 iterations per block at high concurrency) was too short to benefit from a 3-stage pipeline. This assumption was also incorrect for this specific kernel shape.
• Assumption: Benchmark results are stable. The discovery of ±5–7% run-to-run variance challenged the assumption that single-run measurements were reliable. This led to the methodological improvement of running multiple iterations and interpreting results against the noise band.

Input Knowledge Required

To understand this message, a reader needs knowledge of several domains:

• GPU architecture: Understanding what warps, thread blocks, streaming multiprocessors (SMs), shared memory, and register pressure are, and how they interact in GPU kernel execution.
• Triton and CUDA graph capture: The kernel is written in Triton, a Python-based DSL for GPU kernel development. The assistant had previously root-caused a corruption bug related to CUDA graph capture interacting with bf16 index keys — a separate but related thread of work.
• MLA (Multi-head Latent Attention): DeepSeek-V4 uses a custom attention mechanism that compresses the key-value cache into a latent space, then uses sparse indexing to retrieve relevant context. The kernel's performance characteristics are shaped by this architecture.
• The project's optimization history: The message references "the #3a outcome" and "the occupancy lever analysis," which only make sense in the context of the broader performance plan and the preceding experimental campaign (messages [msg 13562] through [msg 13573]).
• PD disaggregation: The system separates prefill and decode onto different GPU workers. The decode worker's attention kernel is the focus of this optimization work.

Output Knowledge Created

The message creates several forms of output knowledge:

1. An updated plan document (DSV4_DECODE_PERF_PLAN.md) that now accurately reflects the viability of optimization approaches. This document serves as the project's strategic memory, preventing future wasted effort on the refuted approach.
2. A documented negative result. In engineering, negative results are as valuable as positive ones — they prevent repetition of the same experiments and sharpen the understanding of the system's behavior. The message ensures this negative result is captured in the project's permanent record.
3. A refined understanding of the kernel's optimization landscape. The message implicitly communicates that the attention kernel is already near-optimal for its current structure, and that further gains must come from different approaches — perhaps algorithmic changes (e.g., different sparse indexing strategies) or system-level optimizations (e.g., the overlap scheduler that the assistant would investigate next).
4. Closure on a line of inquiry. By marking the todos as complete and updating the plan, the message provides psychological and project-management closure. The team (or the user reading the conversation) can see that this path has been thoroughly explored and can move on without lingering doubts.

The Thinking Process: Evidence of Rigor

The reasoning section of the message is brief, but it reveals a disciplined thought process. The assistant does not simply say "update the plan." It articulates why the update is needed: the register-reducing rewrite has been "refuted by the occupancy lever analysis." It specifies what the update entails: marking the option as no longer viable, updating the remaining options section, and marking related todos as complete. It states the strategic consequence: the rewrite is no longer the recommended big lever.

This brevity is itself a sign of maturity. The assistant has already done the hard work of experimentation, analysis, and documentation in the preceding messages. The A/B test document has been updated and committed. The baseline kernel has been restored and verified. The benchmark variance has been characterized. All that remains is to propagate the conclusion to the planning document — a small but essential step that ensures the project's strategic artifacts reflect reality.

The message also demonstrates a key engineering virtue: the willingness to accept and document negative results. It would have been easy to keep #3a on the plan as a "maybe someday" item, preserving the illusion that a breakthrough might still be found. Instead, the assistant makes a clean determination: this path is a dead end, and the plan should say so explicitly. This intellectual honesty is the foundation of effective optimization work.

Conclusion

Message [msg 13574] is a small edit with large significance. It is the moment when a promising hypothesis meets experimental reality and is formally laid to rest. The register-reducing rewrite of the attention kernel — once considered a potential "big lever" for throughput — is definitively refuted by two independent experiments, each testing a different occupancy-increasing mechanism. Both failed. The baseline configuration is already optimal.

The message teaches a valuable lesson about the optimization process: not every promising idea works, and the discipline to document failures is as important as the creativity to generate hypotheses. By updating the plan document, the assistant ensures that this lesson is preserved — not just for itself, but for anyone who reads the project's records. In the high-stakes world of GPU kernel optimization, where a single wrong turn can consume weeks of effort, such documentation is not bureaucracy. It is survival.