Chunk 71.0

## Summary of This Chunk **Core task: diagnosing and resolving two distinct production issues** — a PD deadlock that silently wedged the decode engine under load, and a tool-call leak where DSML markup surfaced as assistant content instead of structured `tool_calls`. The deadlock was traced to a **TP-collective desync in the overlap event loop**: when a mass-abort of in-flight KV transfers (from cancelling a parallel agent) perturbed per-rank scheduling decisions, some ranks entered a collective (all_reduce/broadcast) while others branched to `on_idle`, causing a permanent NCCL/gloo hang that `/health` couldn't detect. The fix (`--disable-overlap-schedule`) was deployed and confirmed effective — all 8 scheduler ranks switched from `event_loop_overlap_disagg_*` to the lockstep `event_loop_normal_disagg_*` path. **The tool-call leak proved independent of the deadlock** (persisted after overlap was disabled). Extensive code tracing showed the serving config is correct (native `encoding_dsv4` path, no chat-template override, `deepseekv4` parser). The leak is a **known DeepSeek-V4 model behavior** where the model intermittently degenerates from DSML tool-call mode into text mode under cumulative context/tool-schema pressure — the sample showed 7 well-formed `read_message` calls then token salad, which the parser correctly couldn't extract as tool_calls. The user firmly rejected the model-deficiency hypothesis, noting the same model works flawlessly from cloud providers at high parallelism, and directed the assistant to re-examine deployment-specific patches versus upstream sglang for residual issues that only manifest under parallel load. **Themes: systematic debugging under production pressure, distinguishing correlated symptoms from shared root causes, and the tension between upstream bugs and deployment-specific regressions.** The assistant's research identified several relevant upstream issues (#26454 overlap desync, #17593 template mismatch, #1244 model degeneration) but the user's constraint — that the tool-call issue is deployment-specific and parallelism-dependent — reframes the investigation toward sglang's internal state management under concurrent request processing, particularly around the detokenizer, parser buffering, and any custom patches (bf16 index keys, SM120 kernels) that might interact with output handling under load.

Two Fronts, One War: Debugging a PD Deadlock and Tool-Call Corruption in Production 2370 words

Message Articles