The Data Quality Revolution: How EAGLE-3 Training Was Transformed by Capturing a Model's Own Thinking

Introduction

In the long arc of deploying speculative decoding for the 1-trillion-parameter Kimi-K2.5 model on 8× NVIDIA RTX PRO 6000 Blackwell GPUs, there are moments of quiet triumph—when a pipeline that has been built, debugged, and validated finally runs end-to-end. And then there are moments of strategic redirection—when the user looks at the working pipeline and recognizes that it is solving the wrong problem at the data level. This segment captures precisely such a redirection, and the methodical engineering work required to execute it at scale.

The assistant had just completed an extraordinary engineering feat: an end-to-end EAGLE-3 speculative decoding training pipeline for the 1-trillion-parameter Kimi-K2.5 Mixture-of-Experts model, validated on 10 samples and scaled to 1000. The hidden state extraction ran at 2912 tok/s, the training completed 10 epochs in 27.7 minutes, and the output checkpoint was verified to be vLLM-compatible with weight shapes matching the AQ-MedAI reference model. The session summary in [msg 2839] documented this milestone with precise metrics and a forward plan for a "hero run" on rented B300 hardware.

But the user saw deeper. In [msg 2840], they redirected the entire approach with a concise directive: "Dataset: to capture k2.5 thinking (I think we need to) lets do: from open-perfectblend, on just vllm infer every question, capture thinking and output. Each question should be fed independently to vllm, answer up do 8k tokens." This single message recognized a critical flaw—the training data consisted of hidden states from the model's prefill pass on human-written text, not from the model's own autoregressive reasoning outputs. The draft model needed to learn to predict Kimi-K2.5's actual thinking patterns, not generic conversation continuations.

This segment covers the pivot from pipeline validation to high-quality synthetic data generation: recognizing the fundamental data quality issue, writing the inference script, fixing timeout and field extraction issues, analyzing the economics of cloud vs. local compute, downloading the AQ-MedAI checkpoint for finetuning, updating the training script for finetuning support, launching the large-scale inference run, and discovering new challenges at scale.

The Completed Pipeline: A Foundation Worth Building On

Before the pivot, the assistant had accomplished what many ML engineering teams would consider a significant milestone. The EAGLE-3 training pipeline, built from scratch using the speculators library (v0.3.0), had been validated end-to-end. The pipeline consisted of four stages: data preparation (01_prepare_data.py), hidden state extraction (02_extract_hidden_states.py), vocabulary mapping (03_build_vocab_mapping.py), and training (04_train.py).

The journey to this point had been arduous. The assistant had explored the speculators library's source code to understand the correct API usage—discovering that Eagle3SpeculatorConfig, Eagle3DraftModel, and the built-in Trainer class were the correct interfaces, not the custom training loop initially written. It had monkey-patched the verifier weight extraction to handle Kimi-K2.5's nested configuration structure (where weights are prefixed with language_model.), a structural incompatibility that would have prevented the pipeline from running at all. It had debugged a bf16 dtype mismatch between the model's float32 weights and the hidden states, fixed an unexpected TransformTensors API difference, and set _attn_implementation to "flex_attention" to satisfy EAGLE-3's BlockMask mechanism.

The 10-sample validation run completed in approximately one minute for 3 epochs, confirming the pipeline was mechanically sound. The 1000-sample scale-up then demonstrated production viability: the 547GB model loaded across 8 GPUs in 22.5 minutes, hidden state extraction processed 502,983 tokens in 2.9 minutes at 2912 tok/s, and training ran 10 epochs across 9500 steps in 27.7 minutes at 6 steps/s on a single GPU. The output checkpoint was verified to have the correct flat config format (LlamaForCausalLMEagle3 architecture), matching weight shapes to the AQ-MedAI reference model, and the correct head_dim=128 parameter.

The assistant's session summary in [msg 2839] captured this achievement with characteristic precision: six numbered items tracing the progression from API exploration through implementation through validation through production-scale execution through documentation. The "Next Steps" section presented two paths—a "hero run" on rented B200/B300 hardware (6-18 hours, $180-900) or continued local work with more training data.

The Insight That Changed Everything

The user's message in [msg 2840] cut through this momentum with a crucial realization. The training data used so far—the open-perfectblend dataset—consisted of human-written conversations. The hidden states extracted from Kimi-K2.5 during prefill of these conversations captured how the model processes human text, not how it generates its own reasoning. But the entire purpose of an EAGLE-3 draft model is to predict the verifier model's future tokens during autoregressive generation—specifically, to anticipate the model's chain-of-thought reasoning and final answers.

This distinction is subtle but critical. An EAGLE-3 draft model works by taking the verifier's hidden states from the most recent few layers and using them to predict what tokens the verifier would generate next. If the draft model is trained only on prefill hidden states from human-written text, it learns to predict continuations of human dialogue patterns. But what it really needs to learn is how Kimi-K2.5 itself thinks—the characteristic patterns of its reasoning, the structure of its <think> blocks, the way it transitions from analysis to answer.

The user's phrasing—"I think we need to"—reveals this was a reflective insight, not a command from a pre-existing plan. The parenthetical hesitation signals that the user was reasoning through the problem in real time, arriving at the conclusion that the current approach was fundamentally insufficient.

The assistant's response in [msg 2841] demonstrates a deep understanding of the user's intent: "You're right—we need to capture Kimi-K2.5's actual reasoning outputs (the <think> tokens), not just the prefill hidden states of the raw dataset. The draft model needs to learn to predict the model's thinking patterns, not just generic conversation patterns." The assistant then outlined a clear plan: feed each question to Kimi-K2.5 independently via vLLM inference, capture the full output (thinking + answer, up to 8K tokens), and use these complete conversations as the training data for hidden state extraction.

Building the Synthetic Data Generation Infrastructure

With the strategic direction set, the assistant executed a rapid sequence of preparatory steps. [msg 2842] checked the vLLM systemd service configuration, confirming that the server runs on port 8000 with an OpenAI-compatible API and has the kimi_k2 reasoning parser configured—a custom component that handles the model's special <think> and </think> reasoning blocks. This reconnaissance was essential: without knowing the server's exact configuration, the generation script could target the wrong endpoint or fail to capture reasoning content correctly.

In [msg 2843], the assistant wrote 01b_generate_synthetic.py, a script designed to orchestrate the entire inference campaign. The script loads questions from open-perfectblend, sends each one as an independent inference request to the vLLM server at a concurrency of 128 (later adjusted to 200 per the user's specification in [msg 2849]), requests up to 8,192 completion tokens per question, and captures both the reasoning field and the content field from each response. It then reconstructs the full token sequence with the correct special tokens—token 163606 for <think> and token 163607 for <response>—ensuring that the hidden state extraction step receives the exact same token sequence the model generated.

The script was copied to the remote machine via scp in [msg 2844], and the assistant verified that the openai Python package (version 2.21.0) and requests package (version 2.32.5) were available in [msg 2845] and [msg 2846]. These dependency checks, while seemingly trivial, represent a systematic approach to risk management: the assistant verified every prerequisite before committing to the expensive operations that followed.

The Art of Productive Waiting

With the script deployed and dependencies confirmed, the assistant started the vLLM server in [msg 2850] via systemctl start vllm-kimi-k25-int4. Loading a 540 GB INT4-quantized 1T-parameter model across 8 GPUs takes approximately 22 minutes—a significant fixed cost that cannot be avoided. Rather than idling during this window, the assistant demonstrated a crucial engineering skill: productive waiting.

In [msg 2851], the assistant checked disk usage across the project directories, discovering 143 GB of old training artifacts: 89 GB from the 1000-sample training output, 27 GB from the test run, and 27 GB from the extracted hidden states. In [msg 2852], the assistant deleted all of it—keeping only the 213 MB data_test directory as a "tiny, useful reference"—freeing the disk space needed for the upcoming synthetic data generation run. This was not just housekeeping; it was a deliberate act of closure, marking the transition from the old approach to the new one.

[msg 2853] then initiated a polling loop: sleep 300 && ssh ... curl -s http://localhost:8000/v1/models. The five-minute polling interval was a pragmatic compromise—frequent enough to detect server readiness within one cycle of completion, but not so aggressive as to generate excessive traffic during the 22-minute load window. The dual-signal approach (checking both the HTTP endpoint and GPU memory via nvidia-smi) showed awareness that the API server might report readiness before the model was fully operational, or vice versa.

The Tangent That Refocused Everything

In the middle of this waiting period, the user sent [msg 2854]—a tangent that would reshape the entire data strategy. The user revealed they had their own dataset of actual usage and asked three questions: what format to use, what dataset size to target, and what context length with truncation strategy to employ.

The assistant's response in [msg 2855] is a masterclass in applied ML data engineering. It analyzed two options for using the personal data: re-infer the user's prompts through the vLLM server to capture fresh hidden states (Option A), or tokenize the existing conversations directly and run them through extraction as prefill (Option B). The recommendation leaned toward Option A because "the draft model needs to predict the model's hidden state trajectories, and those are most accurate when the model generates them fresh." This is a distribution alignment argument: if the responses in the user's logs came from a different model version or different sampling parameters, the hidden state distribution would be mismatched.

The assistant also provided a detailed analysis of dataset size, recommending a hybrid approach: mix 500-1000 personal conversations with 5K-10K from open-perfectblend, letting the draft model learn general patterns from the synthetic data while fine-tuning to the user's specific use patterns. On context length, the assistant recommended 8K completion tokens for inference and up to 16K max_seq_len for extraction if disk space allows, with truncation from the end to preserve the beginning of responses where reasoning starts.

The user's response in [msg 2856] resolved the deliberation with characteristic concision: "let's just keep the open mix, makes it easier to publish this model later." This single sentence reveals a sophisticated understanding of the model's lifecycle—choosing open data ensures the resulting draft model can be shared with the community without licensing complications. The user also revealed a critical new resource: "The machine now has /data with 3TB space btw." This single piece of information fundamentally changed the resource calculus, unblocking the scaling plan that had been constrained by the 736 GB available on /root/.

The Hidden State Calculus

With the dataset decision settled and the 3 TB /data volume revealed, the user asked the strategic question in [msg 2856]: "Hero run - disk requirement there? Better to run extraction in place or transfer over 5gbps?" The assistant's response in [msg 2857] is a comprehensive cost-benefit analysis that reveals deep understanding of the pipeline's resource profile.

The analysis begins with the fundamental unit cost: each token of hidden state consumes 57,344 bytes (4 layers × 7168 hidden dim × 2 bytes for bfloat16). From this, the assistant extrapolates to three scenarios: 10K samples at ~4K average sequence length would require ~2.1 TB of disk; 50K samples would require ~10.7 TB; and 100K samples would require ~21 TB. The assistant then catches itself—"Wait — these numbers are huge. Let me reconsider"—and refines the analysis by noting that the average completion length might be much shorter than the full 8K budget.

The transfer tradeoff analysis compares three options: extract on the B300 and transfer the data over 5 Gbps to the local machine for training; do everything on the B300 and transfer only the final checkpoint; or extract on the B300 and train on the B300 in parallel. The assistant's recommendation is Option B—do everything on the B300—based on a cost model: transferring 2+ TB over 5 Gbps would cost more in B300 idle time ($42-$350 at $30-50/hr) than just running training on the B300 itself. The total estimated cost for a production-quality model is $120-250 for 4-5 hours of B300 time.

The key insight is that "10K samples with reasoning might be equivalent in total tokens to 100K short samples." This reframes the entire conversation about dataset size from sample count to token count, recognizing that reasoning-heavy samples carry far more information than the short dataset conversations used by AQ-MedAI.

The Checkpoint Questions: Demanding Evidence

Before committing to the hero run, the user demanded concrete evidence. [msg 2858] asked four critical questions: Do we have timings from local runs? How much faster would B300 8x be? Do we already have accuracy numbers for a test run model? Are we training from scratch or finetuning an existing model?

The assistant's response across [msg 2859] through [msg 2862] is a masterclass in evidence-first reasoning. Rather than guessing or estimating, the assistant pulled actual measured data from the remote machine. [msg 2859] fetched the data_config.json from the test extraction run, revealing that extraction processed 3875 tokens in 1.7 seconds—a throughput of ~2280 tok/s. [msg 2860] grepped the training log for the 1000-sample run, extracting epoch times, step rates, and total training duration. [msg 2861] grepped the extraction log, revealing 1000 samples processed in 173 seconds at 5.8 samples/s and 2912 tok/s.

[msg 2862] synthesized all this data into a comprehensive answer. The local timings established a baseline: model load at 22.5 minutes, extraction at 2912 tok/s, training at 6 steps/s on 1 GPU. The B300 projection estimated 3-5× faster extraction (NVLink eliminating the PCIe AllReduce bottleneck) and 1.5-2× faster training. On accuracy, the assistant made an honest admission: "No, we don't"—the 1000-sample model was never tested, and it "would almost certainly have terrible acceptance rates" because it was trained on raw dataset prefill with short sequences. On training methodology, the assistant revealed that the AQ-MedAI K2 drafter checkpoint is directly compatible with K2.5 and recommended finetuning rather than training from scratch.

The Three Questions That Decided the Hero Run

The user's response in [msg 2863] condensed the entire planning exercise into three questions: "What sample count we want in the hero run? How long will it take on b300 vs our machine? Can we train on 8x gpu too?"

The assistant's response in [msg 2864] is the culmination of the entire segment—a data-driven decision framework that transforms the project from open-ended exploration into a concrete execution plan. The sample count recommendation is 10K-25K, with 10K as the "sweet spot for cost/quality" because it fits within the 3 TB /data volume and the finetuning head start from the AQ-MedAI checkpoint reduces the data needed.

The time estimates break down all three phases across both platforms. For 10K samples on the local machine: inference at ~5.4 hours, extraction at ~3.8 hours (including model load), training at ~2.3 hours—total ~12 hours, cost $0. On the B300: inference at 40-60 minutes, extraction at 45-65 minutes, training at 1.2 hours—total 2-3 hours, cost $60-150.

The answer to the 8-GPU training question is nuanced. Yes, the speculators Trainer supports FSDP2 via torchrun. But training is not the bottleneck—inference and extraction must use all 8 GPUs and run sequentially, so the total pipeline time is dominated by phases that can't be parallelized anyway. The time savings from 8-GPU training (2 hours saved on a 12-hour run) is small compared to inference (5.4h) and extraction (3.8h).

The recommendation is decisive: "Do it locally, overnight." The local machine has 8× RTX PRO 6000 GPUs, 3 TB of free space on /data, and can run the entire pipeline in a single overnight session at zero additional cost. If the resulting draft model achieves a good acceptance rate, it can be deployed immediately. If not, the pipeline is proven and can be rented on a B300 NVL8 for a larger run.

The user's response in [msg 2865]—"Let's do local; Can we do train on 4x / 8x on pcie gpu?"—accepts the plan but probes a specific assumption. By specifying "on pcie gpu," the user demonstrates awareness that the PCIe interconnect topology, already identified as the primary bottleneck during inference, might behave differently for training. This is a moment of productive tension that forces the assistant to refine its analysis for the specific hardware constraints.

Multi-GPU Training on PCIe: The FSDP Analysis

The assistant investigated the FSDP training question in depth across [msg 2866] through [msg 2869]. By inspecting the speculators library's source code, the assistant confirmed that the Trainer uses FSDP2 (fully_shard) for distributed training. But the analysis revealed a critical insight: the draft model is only 2.6B parameters (1.19B trainable), which is small enough to fit easily on a single GPU. FSDP's parameter sharding adds communication overhead (AllGather before forward, ReduceScatter after backward) that likely exceeds the compute savings for such a small model.

The assistant's estimate: 1 GPU achieves 6 steps/s; 8 GPU FSDP might achieve 12-18 steps/s (2-3× speedup), far from the 8× that would be theoretically possible with ideal scaling. The reason is that the model has only a single transformer layer—there's barely anything to shard. The communication overhead of gathering 2.4 GB of parameters over PCIe before each step dominates the compute time.

The assistant noted that Data Distributed Parallel (DDP) would be far more efficient—each GPU processes independent batches and gradients are averaged via AllReduce—but the speculators library only supports FSDP, not DDP. Implementing DDP would require modifying the training script. Given that training is only ~20% of the total pipeline time, the assistant recommended sticking with 1-GPU training and focusing optimization efforts on the inference and extraction phases that dominate the timeline.

Executing the Plan: Download, Launch, Monitor

With the strategic decisions made, the assistant moved into execution mode. [msg 2870] captured the user's final directives: run locally, use /data, target 25K samples, keep training on 1 GPU. The assistant updated its todo list in [msg 2871] and began orchestrating the run.

The first step was verifying that /data was available and had sufficient space. A quick df -h check in [msg 2872] confirmed 2.8 TB free on /dev/rbd0—plenty of room for the hidden states that would be generated later. The vLLM server was checked and confirmed ready in [msg 2873], with the model loaded and responding to API requests.

In parallel with starting the inference run, the assistant also began downloading the AQ-MedAI/Kimi-K2-Instruct-eagle3 checkpoint in [msg 2874]. This was a critical strategic decision: instead of training the draft model from scratch with random weights, the assistant planned to finetune from this existing checkpoint. The AQ-MedAI model was trained for Kimi-K2 (the predecessor to K2.5) with the same architecture, same hidden_size (7168), same auxiliary layers [2, 30, 58], and same weight shapes. Finetuning from this checkpoint would converge faster and likely achieve higher acceptance rates with less data.

The inference run was launched in [msg 2876] with the command:

nohup /root/ml-env/bin/python3 /root/eagle3-train/01b_generate_synthetic.py \
    --model-path /shared/kimi-k2.5-int4 \
    --output-dir /data/eagle3/synth_25k/prepared \
    --dataset mlabonne/open-perfectblend \
    --max-samples 25000 \
    --max-completion-tokens 8192 \
    --max-seq-len 16384 \
    --concurrency 200 \
    --vllm-url http://localhost:8000 \
    --min-completion-tokens 50

The initial monitoring in [msg 2878] showed the GPUs pegged at 100% utilization, with 200 concurrent requests all generating their first completions. The early throughput was low (0.6 req/s) because all 200 requests were still in-flight generating long first responses. The download of the AQ-MedAI checkpoint initially failed because huggingface-cli wasn't in the PATH for the nohup shell—a quick fix in [msg 2879] using the full path /root/ml-env/bin/huggingface-cli resolved this.

Monitoring Progress and Discovering Scale Issues

As the inference run progressed, the assistant monitored its throughput and adjusted expectations. In [msg 2888], the log showed 150/25000 samples completed at 0.9 req/s with an average completion of 648 tokens. The throughput in tokens per second, measured via vLLM's metrics endpoint in [msg 2891], was approximately 1,600 tok/s—close to the previously measured peak of 1,536 tok/s at C=128 concurrency.

The assistant calculated the estimated total time in [msg 2892]: with an average completion of ~650 tokens across 25K samples, that's ~16.25M generation tokens. At 1,600 tok/s, that's approximately 2.8 hours. However, the assistant noted that the average completion might increase as the dataset presented harder questions later, potentially extending the runtime.

The AQ-MedAI checkpoint download completed successfully in [msg 2880], with all four files fetched in 5 seconds. The assistant verified the checkpoint in [msg 2881], listing the weight keys and confirming the midlayer.* naming convention (which would need remapping to layers.0.* for the speculators library's Eagle3DraftModel). The checkpoint was 2.38 GB—a 2.5B-parameter draft model.

With the download complete and inference running, the assistant turned to updating 04_train.py to support finetuning from the AQ-MedAI checkpoint. In [msg 2883], it read the current training script and identified the needed changes: add a --finetune-from argument, load the AQ-MedAI weights into the trainable layers after model creation, and handle the midlayer.* → layers.0.* key remapping. The edits were applied in [msg 2884] through [msg 2886], and the updated script was copied to the remote server in [msg 2887].

The Timeout Problem at Scale

The user's monitoring in [msg 2899] revealed a growing problem. The generation throughput was showing significant variability—dropping from 1,572 tok/s to as low as 183 tok/s at times, with occasional prompt processing spikes. More critically, the inference log showed a pattern of timeout errors:

ERROR sample 17004: Request timed out.
Progress: 2680/25000 (222 errors), 0.5 req/s, avg completion: 1263 tokens, elapsed: 5945s
ERROR sample 7214: Request timed out.
ERROR sample 7216: Request timed out.
Progress: 2690/25000 (224 errors), 0.5 req/s, avg completion: 1262 tokens, elapsed: 5984s

The timeout errors were accumulating—224 errors by the time 2690 samples had been processed. The user's observation was precise: "maybe tune to 128 parallel and increase timeout?" This suggested that the 1800-second timeout, while generous, was still being hit by some requests. The concurrency of 200 might be too aggressive, causing requests to queue up and exceed the timeout window before they even started generating.

This is a classic scaling issue in large-scale inference workloads. At C=200 concurrency, the vLLM server's scheduler has to manage 200 concurrent requests, each potentially generating up to 8K tokens. If the server's generation throughput is ~1,600 tok/s, and each request averages 1,263 tokens of completion, the time to complete 200 requests is approximately 200 × 1,263 / 1,600 = ~158 seconds. But if some requests have much longer completions (approaching the 8K max), they could take 200 × 8,000 / 1,600 = 1,000 seconds—well within the 1800-second timeout but causing other requests to queue behind them.

The 224 errors out of 2690 samples (~8.3% error rate) suggested that a small fraction of requests were either timing out due to the queue depth or failing for other reasons (network issues, server-side errors). The user's suggestion to reduce concurrency to 128 and increase the timeout further was a pragmatic response: lower concurrency reduces queue depth and server load, while a longer timeout ensures that even the longest reasoning chains have time to complete.

The Broader Significance of This Segment

This segment captures a fundamental tension in ML engineering: the difference between a pipeline that works and a pipeline that produces good results. The assistant had built a technically impressive pipeline—validated end-to-end, with verified checkpoint compatibility, running at production-relevant speeds. But the user recognized that the pipeline was only as good as its training data, and the training data was fundamentally misaligned with the inference task.

The pivot from prefill hidden states to model-generated reasoning outputs represents a deep understanding of what makes speculative decoding work. The draft model's job is to predict the verifier's future tokens during autoregressive generation. Training it on prefill hidden states from human-written text is training on the wrong distribution. The model needs to learn from Kimi-K2.5's own reasoning patterns—the characteristic ways it structures its thoughts, the length of its reasoning chains, the vocabulary it uses in its internal monologue.

This insight is particularly important for speculative decoding because the draft model's accuracy directly determines the speedup. If the draft model predicts tokens that the verifier would not have generated, those predictions are rejected and the speculation is wasted. Training on the verifier's actual outputs maximizes the probability that the draft model's predictions will be accepted.

The segment also illustrates several broader lessons for ML engineering at scale:

Data quality trumps pipeline correctness. The original pipeline worked perfectly—it extracted hidden states, trained a model, and produced a valid checkpoint. But the data was fundamentally wrong for the task. The pivot to synthetic reasoning data recognizes that the draft model must learn the verifier's generative distribution, not its prefill distribution.

Empirical grounding beats theoretical reasoning. Throughout this segment, the assistant repeatedly reaches for actual measured data rather than relying on estimates or reasoning from first principles. The timing data from the 1000-sample run, the extraction throughput from the log files, the disk usage from du—every recommendation is anchored in measurement.

Infrastructure awareness is essential. The assistant's ability to reason about PCIe vs. NVLink interconnects, disk space constraints, model loading times, and sequential phase dependencies demonstrates that effective ML engineering requires deep understanding of the hardware and infrastructure, not just the model architecture.

The economics of compute shape the strategy. The decision to run locally overnight rather than renting cloud GPUs is driven by a clear cost-benefit analysis. The assistant recognizes that the pipeline is unproven for quality, so the first real evaluation should happen at zero marginal cost.

Productive waiting is a skill. The 22-minute server load time was not dead time—it was an opportunity to check disk usage, clean up old data, and prepare the environment for the next phase. The assistant's ability to identify and exploit parallel work streams is a crucial engineering competency.

Conclusion

This segment of the conversation captures a fundamental transformation in the EAGLE-3 training strategy. It moves from a working pipeline trained on the wrong data to a new approach that captures the model's own reasoning patterns. The journey involves recognizing the flaw, building the infrastructure, gathering empirical evidence, analyzing tradeoffs, and making a data-driven decision about scale and cost.

The resulting plan—25K synthetic reasoning samples, processed locally overnight, finetuned from the AQ-MedAI checkpoint—represents the convergence of strategic insight, engineering discipline, and economic pragmatism. It is a plan that could only emerge from the kind of systematic, evidence-based reasoning that characterizes this entire segment.

The timeout errors at scale are a reminder that even well-designed systems encounter new failure modes when pushed to production scale. The user's observation—"maybe tune to 128 parallel and increase timeout?"—demonstrates the kind of operational intuition that comes from experience with large-scale inference workloads. The assistant's response to this feedback, and the subsequent tuning of the inference parameters, would determine whether the 25K sample run succeeds or requires another iteration.

In the broader narrative of deploying speculative decoding for a 1T-parameter model, this segment marks the moment when the team stopped asking "does the pipeline work?" and started asking "does the pipeline produce good results?" That shift in focus—from mechanical correctness to data quality—is what separates a proof-of-concept from a production system.