The Build-and-Test Script: Cementing Reproducibility in a CUDA Inference Engine

A Single File That Anchors a Milestone

[assistant] [write] /home/theuser/glm-kimi-sm120-rtx6000bw/kdtree-engine/scripts/build_and_test.sh Wrote file successfully.

At first glance, this message from the opencode session is almost invisible—a bare notification that a file was written, no content shown, no fanfare. The assistant wrote a single shell script called build_and_test.sh into the scripts/ directory of a freshly scaffolded repository. But this seemingly trivial act sits at a critical inflection point in the development of a custom CUDA inference engine. Understanding why this file was written, and what it represents, requires unpacking the dense chain of reasoning and engineering decisions that led to this moment.

The Moment Before: A Hard-Won Validation

The message arrives at the end of a sustained burst of work. In the preceding messages ([msg 11855] through [msg 11869]), the assistant had built, from scratch, a complete GPU tree-building kernel for a speculative decoding technique called DDTree (Draft-Driven Tree). This kernel is the first phase of a larger project to create a native C/C++/CUDA inference engine for the Kimi K2.6 large language model, running on NVIDIA Blackwell RTX PRO 6000 GPUs with CUDA compute capability sm_120.

The tree builder is not a trivial piece of code. It implements a best-first search algorithm that expands a draft tree from log-probability outputs of a language model head. The assistant had to make dozens of micro-decisions: whether to use a custom binary container format (KDTR) or standard NumPy .npy files for test data exchange between Python and C++; how to handle tie-breaking in the heap when cumulative log-weights are equal (a measure-zero case in practice, but one that could break bit-exact reproducibility); what shared memory layout to use for the heap arrays; how to parallelize the visibility matrix construction across threads; and how to ensure that the GPU kernel's arithmetic operations exactly matched the Python reference despite different floating-point accumulation orders.

By message [msg 11868], the payoff had arrived: all eight tree-builder tests passed bit-exact against the golden reference, including edge cases like chain topologies (where topk=1 creates a single linear chain) and underfull trees (where the heap empties before reaching the budget). The assistant's reasoning in [msg 11869] reveals the next thought: "I'll add two more test cases: one with a high batch size of 64 to verify batching works correctly, and another with the maximum budget of 64. After updating the test generator and regenerating the configs, I'll create a build-and-test script for reproducibility, then commit this as the first milestone."

This is the precise motivation for message [msg 11870]. The script is not an afterthought—it is the capstone of a milestone.

Why a Build-and-Test Script?

The assistant's reasoning reveals a clear engineering sensibility. Having just achieved a hard-fought validation—bit-exact GPU kernel output matching a numpy reference across multiple configurations—the natural next step is to lock in that achievement. A build-and-test script serves several critical purposes in this context:

Reproducibility. The script ensures that anyone (including the assistant itself in a future session, or a human collaborator) can reproduce the full build and test cycle with a single command. This is especially important in an environment where the CUDA toolchain, compiler versions, and GPU architecture flags must be set precisely. The script encodes the environment setup (export PATH=/opt/cuda/bin:$PATH), the cmake configuration flags (-DKDTREE_CUDA_ARCH=120), the build invocation, and the test runner call. Without this script, reproducing the test results requires remembering or rediscovering these exact steps.

Automation of a multi-step pipeline. The build-and-test cycle for this project involves: (1) regenerating reference test data by running a Python script that produces .kdtr binary bundles, (2) reconfiguring CMake so it picks up any new test bundles via glob patterns, (3) building the CUDA kernels and test executables, and (4) running ctest. Chaining these steps into a single script eliminates manual errors and reduces friction during iteration.

Milestone demarcation. By creating this script and then running it to verify that all tests pass, the assistant creates a clean checkpoint. The reasoning in [msg 11872] confirms this: "All 11 tests pass (including max budget q_len=65, batch-64, deep-15). The GPU tree builder is validated bit-exact. Let me update the plan status log and commit this milestone." The script is part of the "ship" ritual—it's the tool that proves the milestone is solid before moving on to the next phase.

The Thinking Process: From Validation to Infrastructure

The assistant's reasoning in [msg 11869] is revealing. It begins by acknowledging the success: "The tree-builder tests all pass against the reference implementation, including edge cases like chains and wide budgets, and the GPU version matches bit-exact with SGLang." Then it immediately pivots to hardening: "I'm thinking about adding stress tests with randomized cases and a benchmark to build more confidence, though the current eight configurations already cover the key tree shapes well."

This is a characteristic pattern in the assistant's engineering approach: validate the core functionality, then immediately strengthen the test coverage and create infrastructure for ongoing verification. The assistant decides to add two more test configurations (max budget and high batch) and then—crucially—create the build-and-test script. The order matters: the script is created after the test generator is updated but before the new tests are run. This means the script itself will be used to validate the expanded test suite, serving as both a tool and a proof point.

The assistant also demonstrates awareness of the development workflow's fragility. The CUDA compilation environment is complex: it requires a specific nvcc path (/opt/cuda/bin/nvcc), a specific CUDA architecture flag (sm_120 for Blackwell), and a host compiler compatible with CUDA 13.2. The CMake configuration in [msg 11865] shows that the CUDA compiler was detected as "NVIDIA 13.2.78 with host compiler GNU 15.2.1"—a combination that could easily break with a toolchain update. The script encodes these dependencies, making the build reproducible across sessions and machines.

Assumptions Embedded in the Script

The assistant makes several assumptions when creating this script:

That the build environment is stable. The script hard-codes paths like /opt/cuda/bin and assumes CUDA 13.x with sm_120 support. This is correct for the current machine (an 8× RTX PRO 6000 Blackwell system running Ubuntu 24.04), but would fail on a different GPU generation or CUDA version.

That the test data generation is deterministic. The script regenerates reference data by running gen_tree_build_refs.py. This assumes that the Python script produces identical output across runs given the same random seed (or that any variation is acceptable). The assistant's earlier reasoning shows careful attention to deterministic behavior—using insertion-order tie-breaking and matching float operation order—so this assumption is well-founded.

That CMake's glob patterns will correctly pick up new test bundles. The script re-runs cmake configuration to ensure that any new .kdtr files in the tests/refs/ directory are discovered. This assumes that the CMake file(GLOB ...) directive works correctly and that no stale build artifacts interfere.

That the test exit codes are reliable. The script uses ctest's exit code to determine success or failure. This is standard practice, but it assumes that the test harness correctly reports failures and that no silent failures (e.g., GPU memory errors that don't propagate) occur.

Input Knowledge Required

To understand this message fully, one needs knowledge of:

The broader project context. The kdtree-engine repository is a native C/C++/CUDA implementation of DDTree speculative decoding for the Kimi K2.6 model. The tree builder kernel is the first of three custom kernels (tree build, verify attention, tree accept) that together implement the speculative decoding loop.

The development workflow. The assistant has been working in rounds, writing files, running bash commands, and iterating based on results. The message is a file-write notification from the assistant's tool-use framework.

The CUDA/GPU environment. The project targets NVIDIA Blackwell GPUs (sm_120) with CUDA 13.x. The build system uses CMake with CUDA support, and the test infrastructure uses a custom binary format (KDTR) for cross-language data exchange.

The concept of DDTree. DDTree (Draft-Driven Tree) is a speculative decoding technique where a lightweight drafter model proposes multiple candidate token sequences organized as a tree, and the target model verifies them in parallel. The tree builder kernel constructs this tree from the drafter's log-probability outputs.

Output Knowledge Created

This message creates several forms of knowledge:

A reproducible build-and-test procedure. The script build_and_test.sh encodes the exact steps needed to rebuild and validate the tree builder kernel. This is operational knowledge—it enables anyone with the same environment to verify the implementation.

A milestone checkpoint. The successful execution of this script (shown in [msg 11871], where all 11 tests pass) creates a verified baseline. The assistant can confidently move on to implementing the next kernel (verify attention) knowing that the tree builder is correct and reproducible.

A pattern for future milestones. The assistant has established a workflow: implement a kernel, create a reference implementation in numpy, generate test data, write CUDA code, validate against the reference, harden test coverage, create a build-and-test script, and commit. This pattern will likely be repeated for the verify-attention and tree-accept kernels.

The Broader Significance

In the context of the entire session—which spans environment setup, driver installation, CUDA toolkit configuration, flash-attn build troubleshooting, and multiple model deployments—this message represents a moment of deliberate infrastructure investment. The assistant is not just solving the immediate problem (getting the tree builder to work) but is also building the scaffolding for sustained development. The build-and-test script is a small file, but it embodies a philosophy: that correctness must be reproducible, that automation reduces error, and that milestones should be clearly demarcated before moving forward.

This is the kind of engineering discipline that separates a one-off experiment from a maintainable codebase. The assistant could have simply noted that the tests pass and moved on. Instead, it chose to invest time in creating a script that would make the next iteration—and the next developer or AI agent—more productive. That choice, captured in a single file-write notification, is what makes this message worth examining in depth.