Chunk 73.0
In this chunk, the primary task was to stabilize a production PD (prefill-decode) transfer wedge that caused requests to get stuck after decode-only restarts. The achievement was diagnosing the wedge as a degraded NIXL bootstrap state—caused by restarting decode alone against a long-running prefill—and resolving it with a full co-restart of the prefill, decode, and router services. The fix is under verification to ensure it holds under realistic load, with metrics showing zero transfer failures and idle decode GPUs distinguishing this wedge from other failure modes. Operational guidance was established: never restart decode alone; always co-restart the PD pair. Alongside this stability work, the chunk reports the culmination of earlier performance and correctness efforts. The bf16 index-K corruption at high concurrency was definitively root-caused to a multi-stream-overlap race during CUDA-graph capture and fixed by setting `SGLANG_OPT_USE_MULTI_STREAM_OVERLAP=0`, achieving 0% corruption without throughput regression. Decode throughput was further improved by enabling `SGLANG_SM120_MMA_TARGET_CTAS=512`, yielding +12.8% at C64 and +5.7% at C96 while fixing a wave-quant anomaly. Other optimization attempts (overlap-schedule, attention occupancy tuning) were either rejected for correctness risks or showed negative results. The overarching themes are correctness paramount (zero corruption, no silent deadlocks), rigorous evidence-based performance tuning (A/B testing, profiling, careful interpretation of variance), and operational robustness (understanding CUDA-graph capture, stream races, NIXL bootstrap dependencies). The work emphasizes deep technical investigation, clear documentation of findings and procedures, and a disciplined approach to deploying only verified, safe configurations.