The DDTree Optimization Campaign: Deploying Kimi K2.6 with Speculative Decoding Across PCIe and NVLink Blackwell GPUs

Introduction

In the span of a single segment of an opencode coding session, an AI assistant and its user navigated one of the most comprehensive inference optimization campaigns documented in this series. The mission: deploy Kimi K2.6—a 590-billion-parameter Mixture-of-Experts model with 384 experts and 61 layers—with DFlash speculative decoding using the DDTree (Draft-Driven Tree) algorithm, across two radically different hardware platforms. The first was an 8× RTX PRO 6000 Blackwell system connected via PCIe (codename CT200), a platform where every cross-GPU communication bears the latency tax of the PCIe bus. The second was an 8× B300 SXM6 system with NVLink interconnect and sm_103 architecture, where GPU-to-GPU communication approaches the speed of local memory access.

What followed was a masterclass in systematic ML engineering: infrastructure recovery from a cascade of CUDA toolkit failures, a parallelism bake-off across four strategies (TP8, PP8, EP8, EP4), the deployment and debugging of DFlash speculative decoding with sliding window attention, the implementation of temperature-based sampling for DDTree, a config sweep that revealed a drafter-bound acceptance ceiling, a breakthrough 2.15× speedup on NVLink hardware, the diagnosis of an architecture-specific CUDA graph bug on sm_103, and finally the synthesis of all findings into a comprehensive report that charts the course for a custom C/C++/CUDA inference stack.

This article traces that arc, examining the reasoning, methodology, and engineering discipline at each stage.

Part I: Infrastructure Recovery — The CUDA Toolkit Cascade

The segment opens with the aftermath of a critical infrastructure failure. The assistant had been attempting to deploy Kimi K2.6 on the Blackwell GPUs using SGLang with FlashInfer for GPU kernel support. The problem was a mismatch between the Python environment (PyTorch 2.11.0 compiled against CUDA 13.0) and the system's CUDA toolkit (version 12.8). When FlashInfer tried to JIT-compile sampling kernels for the new SM120 architecture, it failed because CUDA 12.8 lacked the necessary curand.h headers and other components.

The assistant's initial attempts at a quick fix—symlinking missing headers from CUDA 12.8 into the CUDA 13.0 include directory—were rightly called out by the user as fragile and unprofessional. The user's rebuke reframed the entire approach: install the correct versions of software cleanly. The assistant pivoted from hacking to engineering, installing the full cuda-toolkit-13-0 package via apt, verifying the complete toolkit, and restarting the service. The service came up cleanly in 330 seconds, with JIT compilation now succeeding because the full CUDA 13.0 toolkit provided all necessary headers and libraries.

This infrastructure recovery was the foundation for everything that followed. Without a working CUDA toolkit, no model deployment, no benchmarking, and no optimization could proceed. The episode illustrates a recurring theme: the tension between expedient workarounds and clean engineering, and the importance of user guidance in steering toward the latter.

Part II: The Parallelism Bake-Off — TP8, PP8, EP8, EP4

With the infrastructure stable, the assistant systematically benchmarked four parallelism strategies across the 8× RTX PRO 6000 PCIe-connected system. The results were illuminating and, in some cases, counterintuitive.

TP8 (Tensor Parallelism, degree 8): The baseline achieved approximately 26 tok/s single-request throughput and ~578 tok/s at concurrency 32. With CUDA graphs enabled, single-request throughput reached 98 tok/s, but aggregate throughput plateaued at ~1291 tok/s. The AllReduce bottleneck over PCIe was the dominant limiter—every MoE layer required synchronizing expert outputs across all 8 GPUs over the slow bus.

PP8 (Pipeline Parallelism, degree 8): Despite the theoretical advantage of eliminating MoE AllReduce by assigning contiguous blocks of layers to each GPU, PP8 was disappointing. Single-request throughput was ~24.7 tok/s (slightly worse than TP8), and aggregate throughput at C=4 was only 68 tok/s. The user's follow-up observation identified the root cause: GPU utilization was only 150–200 W out of 600 W capacity, indicating severe pipeline bubbles. Without CUDA graphs and with insufficient concurrency to fill the pipeline, most GPUs were idle most of the time.

EP8 (Expert Parallelism, degree 8): This strategy dramatically improved single-request throughput from 26 to 65 tok/s—a 2.5× improvement over TP8. By assigning different experts to different GPUs, EP eliminated the AllReduce on MoE layers entirely, which was the dominant bottleneck over PCIe. The user's intuition that "expert parallelism avoids PCIe AllReduce bottlenecks" was decisively validated.

EP4 (Expert Parallelism, degree 4 with TP2 groups): At high concurrency, EP4 reached the peak aggregate throughput of ~1531 tok/s, outperforming all other strategies. The hybrid approach (expert parallelism within groups, tensor parallelism within each group) provided the best balance of communication efficiency and compute utilization.

The results told a clear story: for PCIe-bound MoE inference, expert parallelism is the dominant strategy. The AllReduce bottleneck that cripples TP8 is eliminated entirely by EP, while PP's pipeline bubbles create a different set of inefficiencies that are harder to hide without deep pipeline scheduling support.

Part III: DFlash Deployment on PCIe — Sliding Window and the First DDTree Benchmark

With EP8 established as the optimal parallelism strategy, the assistant turned to speculative decoding. The drafter model, SubSir/Kimi-K2.6-DFlash-tmp-long, was a 6.5 GB, 6-layer transformer with block_size=8, trained as a drop-in replacement for the target model's early layers. The assistant enabled sliding window attention for the drafter with --speculative-dflash-draft-window-size 2048, which clamped the draft KV cache to the last 2048 tokens—matching the drafter's training window.

A comprehensive benchmark harness (bench_ddtree_matrix.py) was built to test across context lengths (60, 1024, 4096, 8192) and concurrency levels (1, 8, 32, 64), including a coding correctness evaluation. The first benchmark with budget=8, topk=4, and window=2048 achieved approximately 170 tok/s at C=1 with short context—the best single-stream result yet on the PCIe system.

However, the initial coding evaluation showed 0/5 passes due to a code extraction bug: the thinking model outputs reasoning before a response marker, and the regex was grabbing an incomplete code block from the thinking section. After fixing the extraction to split on response and take the final code block, coding evaluation showed 4/5 passes at 162 tok/s (the single failure was a harness truncation edge case).

The assistant then attempted a full config sweep across budget/topk/window combinations, but all failed to start due to a race condition in the service restart readiness check—the old process answered /v1/models briefly before being killed, while the new process was still loading weights for approximately 6 minutes. This debugging set the stage for the more robust orchestration patterns used later.

Part IV: The DDTree CUDA Graph Fix — Unlocking 109 tok/s on PCIe

A critical breakthrough came when the assistant attempted to deploy TP8+DDTree with CUDA graphs on the PCIe PRO6000 machine. The service started but crashed on the first request with a cryptic error: RuntimeError: The size of tensor a (8) must match the size of tensor b (33) at non-singleton dimension 0.

This error was the key to a fundamental bug in the CUDA graph runner. The DDTree speculative decoding pipeline has two different token counts that matter for graph sizing:

Draft forward: Uses block_size=8 tokens per request (the anchor plus 7 mask tokens)
Target verify forward: Uses budget+1=33 tokens per request (the root plus 32 tree nodes) The CUDA graph runner was using speculative_num_draft_tokens (which equals block_size=8) to size the target verify graph. But DDTree's verify forward actually processes budget+1=33 tokens. The graph was captured with buffers sized for 8 tokens per batch element, but replay tried to copy 33 tokens into those buffers—hence the size mismatch. The fix was elegant and surgical. In cuda_graph_runner.py, the assistant added a conditional: when the speculative algorithm is DDTree and this is the target worker (not the draft worker), set num_tokens_per_bs = speculative_ddtree_budget + 1 instead of speculative_num_draft_tokens. The draft worker continues using block_size. This single 12-line change resolved the crash. After applying the fix and restarting the service, the assistant ran benchmarks. TP8+DDTree (budget=7, topk=1) with CUDA graphs achieved 109.7 tok/s at C=1—beating the TP8 baseline (98 tok/s) by ~1.11× and matching linear DFlash with CUDA graphs (~110 tok/s). The CUDA graph infrastructure was now working correctly with DDTree.

Part V: From Greedy to Sampling — Temperature Support for DDTree

A critical feature was missing from DDTree: temperature-based sampling. DDTree's verifier could only follow the greediest path through the draft tree, always accepting the highest-probability token at each position. This meant it could not support the diversity and creativity that sampling with temperature, top-k, and top-p provides—capabilities essential for production deployments where output variety matters.

The assistant's first breakthrough came when studying the existing codebase. It discovered that SGLang already had a tree_speculative_sampling_target_only kernel used by both the EAGLE speculative decoding system and the linear (non-tree) DFlash path. This kernel takes a tree structure encoded as three index arrays—retrieve_index, retrieve_next_token, and retrieve_next_sibling—along with target probability distributions and candidate token IDs, and performs efficient tree-structured rejection sampling on the GPU.

The key realization was that DDTree already had the tree structure encoded in its parents and child_maps. The assistant just needed to convert from DDTree's representation to the EAGLE-style first-child/next-sibling encoding that the kernel expected. This meant no new CUDA kernel needed to be written—the existing, battle-tested kernel could be reused directly.

But there was a subtlety. The kernel expects siblings to be ordered by descending draft probability, because it tries candidates in order and accepts the first one that passes the rejection test. DDTree's heap-based builder does not guarantee this ordering. The assistant realized it needed to augment the builder to track per-node log weights (node_logws), then use those weights to sort each parent's children in descending probability order when constructing the retrieve buffers. Without this sorting, the rejection sampling kernel would try candidates in the wrong order, potentially accepting a lower-probability token when a higher-probability sibling should have been tried first.

The implementation was decomposed into six independent steps: augmenting the tree builder with node_logws, implementing the retrieve encoding function (compile_ddtree_retrieve), adding retrieve buffers to the verify input dataclass, refactoring the verify method to split greedy and sampling derivation paths from the shared commit phase, implementing the _sample_tree_paths method, and wiring the worker to call compile_ddtree_retrieve and populate the retrieve buffers.

The refactoring that split verification into a derivation phase and a shared commit phase is a textbook application of the separate variation from common structure design pattern. By identifying what changes (the acceptance derivation) and what stays the same (the commit logic), the assistant was able to introduce a new sampling path without touching the delicate KV-cache bookkeeping that had been debugged and validated over many iterations.

With the guard removed, the assistant deployed the temperature-aware DDTree to the CT200 machine and ran a comprehensive test suite. But at budget=16, topk=4, the output was garbled—poems starting with "poemownersWrite" and code prompts producing run-on text. The root cause was a mask indexing mismatch in SGLang's Triton attention backend. The num_draft_tokens variable was hardcoded to block_size (8 in this deployment), but DDTree's verify phase computes draft_token_num = budget + 1, which for budget=16 gives 17 tokens. Every mask index pointer computed with the wrong value pointed to the wrong region of the mask buffer, causing the attention kernel to read garbage mask bits and attend to incorrect KV positions.

The fix was a single conditional assignment at initialization time that sets the target model's num_draft_tokens to ddtree_budget + 1, while the draft worker's backend keeps block_size. This single assignment propagates to every buffer-sizing operation and every index calculation automatically.

With the mask corruption fixed, the assistant ran a systematic parameter sweep across budget and topk values. The results were unambiguous: budget=8 topk=4: 150.2 tok/s at single concurrency—the clear winner. Smaller trees won at C=1 because verification cost scales directly with budget size, and the drafter's block size of 8 already caps effective depth at 7 tokens. Budget=8 created a tree just wide enough (topk=4 candidates per level) to capture diversity without incurring excessive verify overhead.

Part VI: The Config Sweep That Wouldn't Sweep — and the Block Size Ceiling

The assistant had achieved a significant milestone: deploying Kimi K2.6 with DFlash speculative decoding using the DDTree algorithm on the CT200 machine. With the sliding window draft cache enabled, the assistant designed a comprehensive sweep to find optimal parameters across budget (4, 8, 12, 16), top-k (4, 6), and window settings (2048, none).

The sweep driver called a reconfiguration script to stop the SGLang service, modify the systemd unit file with new parameters, restart, and wait for readiness. When executed, every single configuration reported "SKIP (failed to start)" in rapid succession—a complete and baffling failure.

The assistant's initial hypothesis was reasonable: the Kimi K2.6 model is approximately 590 GB distributed across eight GPUs, and when the inference service is stopped, releasing that memory takes time. The sweep script was only waiting 8 seconds between stopping and starting, which seemed far too short. The assistant added a GPU memory release check that polled nvidia-smi until memory was freed, with a 120-second timeout. Yet the failures persisted.

The breakthrough came when the assistant pivoted from inference to direct observation. Running the reconfiguration script with bash -x (verbose trace mode) revealed a critical clue on the very first line: + set -e. This shell directive causes a bash script to exit immediately if any command returns a non-zero exit status. The readiness check used curl to poll the service's HTTP endpoint. During service startup, the port wasn't listening yet, so curl returned a non-zero exit code (connection refused). With set -e active, this single expected transient failure terminated the entire script—not after a timeout, not after a race condition, but instantly on the first poll attempt.

The fix was trivial: remove set -e from the script. After this correction, the sweep ran successfully. The debugging journey—from GPU memory contention hypothesis through service state checks, log inspection, and finally the bash -x trace—is a textbook example of systematic debugging under pressure.

Then came the user's directive to test block size 16. The DFlash drafter was trained with block_size=8, meaning it learned to predict at most 7 tokens ahead (depth = block_size - 1). Setting block_size=16 would require the model to predict tokens at positions 8 through 15—positions it was never trained to handle. The assistant recognized this but ran the experiment anyway. The results confirmed the hypothesis: acceptance rates rose only modestly from ~4.9 to ~5.2. The finding was strategically significant: it identified the drafter—not the tree structure, not the verification kernel, not the parallelism strategy—as the binding constraint on acceptance length. Further gains required retraining the drafter, not further tuning the inference stack.

Part VII: The NVLink Breakthrough — 303 tok/s on B300

The campaign then shifted to a new 8× B300 SXM6 machine with NVLink interconnects, 275 GB of HBM per GPU, and sm_103 architecture. The assistant deployed the K2.6 + DDTree stack by streaming the working CT200 virtual environment and downloading the 590 GB model via aria2 at 575 MiB/s—three times faster than Hugging Face's hf_transfer downloader. After fixing a Triton JIT compilation issue (missing Python.h) and skipping the vision tower warmup (which required unavailable flash_attn.cute), the service launched successfully.

The benchmark results on NVLink were transformative. DDTree with budget=8, NVLS (NVLink SHARP, a hardware-accelerated collective communication primitive), and CUDA graphs achieved 303 tok/s at concurrency 1—a 2.15× speedup over the 133 tok/s autoregressive baseline. At high concurrency (C=128), throughput scaled to 4723 tok/s with 5/5 coding correctness. NVLS alone provided a free ~5% boost by enabling direct GPU-to-GPU communication over NVLink.

The NVLink interconnect fundamentally changed the parallelism calculus. On PCIe, EP8 won because it avoided AllReduce across the slow bus. On NVLink, TP8 became the clear winner because the interconnect was fast enough that tensor parallelism's communication overhead was negligible. This is a critical lesson: parallelism strategy is not a fixed property of a model architecture—it must be reevaluated for each hardware platform.

The power analysis was equally revealing. The GPUs showed 100% utilization but only 360–460 W out of a 1100 W thermal budget—a classic signature of an HBM-bandwidth-bound workload. The INT4 weight-only MoE decode was memory-bound, not compute-bound: the tensor cores were idle, waiting on memory. This meant there was abundant spare compute capacity that larger speculative decoding trees could potentially exploit.

Part VIII: The Budget Ceiling — CUDA Graph Bug on sm_103

The user's sharp observation cut to the heart of the matter: "b8 is really low, will cause us to underutilise compute and best case equal sequential accept len, no? Can we push 16/32/64?" This question set off an intensive investigation into whether larger DDTree budgets could better utilize the B300's spare compute.

The assistant designed a budget sweep experiment, testing budget=16, 32, and 64 with topk=8. But every attempt to scale beyond budget=8 on the B300 failed in progressively more spectacular ways. Budget=16 crashed with cudaErrorIllegalAddress. Budget=32 appeared to run but produced garbled output—acceptance length stuck at exactly 1.0 (meaning no draft tokens were accepted) and the generated code was syntactically invalid, failing all 5 coding evaluations. Budget=64 crashed with CUBLAS errors.

The breakthrough came when the assistant formulated a critical hypothesis: perhaps the failures weren't fundamental kernel bugs at all, but artifacts of CUDA graph capture—a performance optimization that records GPU operations for replay, eliminating CPU launch overhead. By disabling CUDA graphs and testing budget=16 in eager mode, the assistant produced a definitive result: 5/5 coding correctness with acceptance length jumping to 5.3–6.4 tokens per step, compared to budget=8's 4.48. The DDTree logic was sound. The bug was specifically in CUDA graph capture on sm_103 at query lengths greater than 9.

This finding validated the user's hypothesis—larger budgets genuinely improved acceptance rates—while simultaneously revealing a critical infrastructure blocker. The assistant systematically investigated the root cause, examining the cuda_graph_custom_mask buffer sizing in SGLang's Triton attention backend, testing intermediate budgets like budget=12 (which also crashed), and ultimately concluding that the bug was a deep, architecture-specific interaction between Triton-compiled attention kernels and CUDA graph capture on sm_103. The workload was confirmed to be HBM-bandwidth-bound, meaning the compute units were genuinely idle and the extra verification work from larger trees would be nearly free—if the graph bug could be fixed or bypassed.

Part IX: The Strategic Pivot — From SGLang to Custom Inference

The user's response to this impasse was decisive and forward-looking: "Download artifacts from the B300 machine, write a report of ddtree findings so far, next we will most likely try to implement a C/C++/CUDA inference stack to maximally optimise ddtree inference specific to Kimi and the machine we're running on. First we target the pro6000 box."

This marked a fundamental shift in strategy. Instead of continuing to fight SGLang's Python/Triton/CUDA-graph stack—debugging the sm_103 graph bug, patching Triton kernels, and working within the framework's constraints—the team would build something entirely new: a purpose-built inference engine designed for one model (Kimi K2.6) on one hardware configuration (the PRO6000 box first, then B300). The guiding principle was specialization over generality.

The assistant executed the user's instruction with methodical precision. It downloaded all B300 benchmark artifacts (six JSON files capturing autoregressive, b8t4, NVLS, b32, and b12 configurations) and journalctl excerpts showing acceptance metrics and CUDA graph behavior. It then wrote the comprehensive DDTREE_FINDINGS_REPORT.md—a 14,229-byte document covering seven major sections: the model architecture, DDTree algorithm mechanics, three SGLang bug fixes discovered during deployment, cross-platform performance comparisons (PCIe vs NVLink), the HBM-bandwidth-bound bottleneck analysis, prioritized requirements for the custom C/C++/CUDA stack, and open items for future work.

The report identified five ranked optimization priorities for the custom stack:

Own the decode loop — eliminate Python overhead and CUDA graph fragility
Fused Marlin INT4 dequant + MoE GEMM — reduce memory traffic by fusing operations
Custom tree-attention MLA verify kernel — bypass the sm_103 graph instability entirely
GPU-side tree build — move tree construction from CPU heapq to GPU
Fused drafter forward — integrate the drafter into the custom pipeline The concrete performance targets were to beat 150–170 tok/s at C=1 with an acceptance rate of approximately 4.5 tokens per step, with token-exact greedy and distribution-exact temperature correctness.

Part X: The Reproduction Package — Preserving Knowledge

The final phase of the investigation involved preserving all knowledge for the next phase. The user's request transformed the nature of the work: "save all information/artifacts needed to reproduce on a different machine to /data/dflash/." This was not a casual backup request—it was a demand for engineering hygiene, a recognition that the most valuable output of the campaign was not the running service on CT200 but the recipe for recreating it anywhere.

The assistant's response was systematic and thorough. It first gathered environment snapshots from the CT200 machine: OS version (Ubuntu 24.04), NVIDIA driver version, CUDA toolkit version (13.0.88), and a complete pip freeze output. It then generated unified diffs of all five modified SGLang files against clean backups and git history, producing 425 lines of patches across cuda_graph_runner.py, ddtree_utils.py, dflash_info.py, dflash_worker.py, and triton_backend.py.

The directory structure was carefully designed: patches/full/ for complete working files (the source of truth), patches/diffs/ for reviewable diffs, services/ for systemd unit files, bench/ for benchmark scripts and results, env/ for environment snapshots, and drafter/ for model metadata. The assistant wrote three infrastructure scripts: deploy_patches.sh to apply the modified files to a fresh SGLang installation, setup_env.sh to handle CUDA toolkit and Python dependency setup, and REPRODUCE.md as the master guide.

The final verification step was a model of lightweight quality assurance. The assistant made scripts executable, parsed all Python files with ast.parse() to confirm syntactic validity, grepped each diff for characteristic function names to confirm the right changes were captured, verified that the deploy script's path references matched SGLang's actual directory layout, and generated a SHA256 manifest for cryptographic integrity. This multi-layered verification addressed distinct failure modes: syntax errors, missing changes, wrong paths, and silent corruption.

The package at /data/dflash/k26-ddtree-repro/ is the final artifact of this campaign: a complete, verified, and documented recipe for deploying Kimi K2.6 with DDTree speculative decoding on Blackwell GPUs. It contains the distilled essence of every bug fix, every configuration insight, and every benchmark result—ready for the next engineer, the next machine, and the next challenge.

Key Findings and Their Implications

Several findings from this campaign have broad implications for LLM inference optimization:

1. Interconnect topology dictates parallelism strategy. On PCIe, expert parallelism wins because it avoids AllReduce on MoE layers across the slow bus. On NVLink, tensor parallelism is optimal because the interconnect is fast enough that communication overhead is negligible. NVLS provides a free ~5% boost on NVLink.

2. Larger DDTree budgets improve acceptance rates. Budget=16 in eager mode achieved 5.3–6.4 tokens per step versus budget=8's 4.48, confirming that wider trees genuinely help the drafter propose more acceptable tokens. The block_size=8 depth cap (maximum depth 7) means larger budgets add width rather than depth, with diminishing returns.

3. CUDA graph capture is architecture-sensitive. On sm_103, CUDA graphs crash for any DDTree budget > 8 (query length > 9) during the tree-verify path. This manifests as illegal memory accesses at b12/b16 and garbage output at b32. Eager mode works correctly but loses the 3.8× graph speedup.

4. The workload is HBM-bandwidth-bound. 100% GPU utilization but only 33–42% power draw indicates the compute units are idle, waiting on memory. This means speculative decoding's extra compute is nearly free, validating the entire approach.

5. Python overhead is a massive tax. The 3.8× speedup from CUDA graphs over eager mode shows how much overhead the Python serving stack introduces. A C/C++/CUDA implementation that eliminates this overhead could approach graph-captured performance without the fragility.

6. The drafter is the binding constraint on acceptance. The block_size=16 experiment confirmed that a drafter trained at block_size=8 generalizes only modestly beyond depth 7. Further gains in acceptance length require retraining the drafter, not further tuning the inference stack.

Conclusion

The DDTree optimization campaign for Kimi K2.6 represents a comprehensive case study in systematic inference engineering. The assistant navigated infrastructure recovery, parallelism benchmarking, speculative decoding deployment, temperature sampling implementation, config sweep debugging, architecture-specific kernel analysis, and knowledge preservation—all with disciplined methodology: forming hypotheses, designing experiments, isolating variables, and making pragmatic decisions when root causes proved too deep for quick fixes.

The final verdict is both a validation and a challenge. The DDTree algorithm works correctly and delivers real speedups: 2.15× over autoregressive on NVLink hardware, with larger budgets demonstrably improving acceptance rates. But the sm_103 CUDA graph bug blocks the path to full realization, and the Python overhead of the existing serving stack limits throughput. The strategic pivot to a custom C/C++/CUDA inference stack is the natural response to these constraints—a decision to build a precision instrument for one model on one hardware platform, rather than continuing to fight the limitations of a general-purpose framework.

The artifacts preserved in the findings report and the reproduction package ensure that the hard-won knowledge from this campaign will inform the next phase. The path forward is clear: build a custom inference stack that eliminates Python overhead, fuses memory-bound operations, and builds purpose-built kernels for tree-attention verification. The destination—150–170 tok/s on PCIe Blackwell, with acceptance of ~4.5 tokens per step—is worth the journey.

References

[1] "The Optimization Gauntlet: From CUDA Toolkit Recovery to Parallelism Benchmarking and DFlash Deployment for Kimi K2.6" — Chunk 0 of Segment 64, covering infrastructure recovery, parallelism benchmarking, and initial DFlash deployment.

[2] "The DDTree CUDA Graph Fix: Unlocking Speculative Decoding at 109 tok/s on PCIe Blackwell" — Chunk 1 of Segment 64, covering the CUDA graph sizing fix and the 109 tok/s benchmark.

[3] "The NVLink Breakthrough: Deploying DDTree Speculative Decoding on B300 and the Road to a Custom Inference Stack" — Chunk 2 of Segment 64, covering the B300 deployment and 303 tok/s results.

[4] "From Greedy to Sampling: The Architecture of Choice in DDTree Speculative Decoding" — Chunk 3 of Segment 64, covering temperature sampling implementation and the mask corruption fix.

[5] "From Config Sweep to Reproduction Package: The DDTree Optimization Campaign for Kimi K2.6" — Chunk 4 of Segment 64, covering the config sweep debugging, block size analysis, and reproduction package.

[6] "From PCIe to NVLink: The DDTree Deployment Journey Across Two GPU Worlds" — Chunk 5 of Segment 64, covering the cross-platform comparison and strategic pivot to custom inference.