The Build Infrastructure Pivot: Crafting an nvcc-Direct Compilation Pipeline for CUDA Kernel Benchmarking on Blackwell

Introduction

In the sprawling, multi-session effort to deploy and benchmark speculative decoding with Dynamic Dependency Tree (DDTree) inference on 8× NVIDIA RTX PRO 6000 Blackwell GPUs, there comes a moment that seems mundane on the surface: writing a build script. Message [msg 11973] captures one such moment—an assistant reasoning through and executing a critical infrastructure pivot. The target machine, CT200, an LXC container housing a live SGLang inference service for the Kimi K2.6 model, lacks cmake, the build system the entire kdtree-engine repository was designed around. This message documents the assistant's decision to abandon cmake in favor of a direct nvcc compilation pipeline, and the creation of two files: a kernel microbenchmark harness (tests/bench_kernels.cu) and the build script itself (scripts/build_nvcc.sh). While brief in execution, this message encapsulates a deep reasoning process about compilation strategies, cross-platform deployment constraints, and the trade-offs between build-system elegance and practical necessity.

Context: The Road to CT200 Benchmarks

To understand why this message exists, one must trace the arc of the preceding work. The assistant had just completed a major milestone: a fully functional native C/C++/CUDA DDTree inference engine, validated across 31 passing tests on a local RTX 5070 Ti (sm_120 architecture—same as the PRO 6000). The engine included three custom CUDA kernels—a GPU best-first tree builder, a tree-verify MLA-absorb attention kernel, and a greedy tree-accept kernel—all verified bit-exact against numpy references. The user's directive was clear: "proceed with next phases up to getting meaningful benchmarks on CT200."

CT200 is the production environment: an LXC container running on kpro6, a machine with 8× RTX PRO 6000 Blackwell Server Edition GPUs (each with ~98 GiB of memory), hosting a live SGLang 0.5.11 service with the Kimi K2.6 model loaded across all eight GPUs using tensor parallelism. The assistant's reconnaissance of CT200 (in preceding messages) revealed a critical constraint: the container has CUDA 13.0 installed, a Python 3.12 venv with PyTorch 2.11.0+cu130 and SGLang 0.5.11, and the model consuming nearly all GPU memory (~87 GiB per GPU). But crucially, cmake is not available.

This is the problem that message [msg 11973] solves. The entire kdtree-engine repository—its CMakeLists.txt, its test infrastructure, its build-and-test pipeline—was designed around cmake. Without it, the assistant cannot compile the kernels, run unit tests, or execute microbenchmarks on the target hardware. The message represents a deliberate architectural pivot from a cmake-based build to a direct nvcc compilation strategy.

The Reasoning Process: Compilation Strategy Under Constraint

The assistant's reasoning, captured in the "Agent Reasoning" block, reveals a careful deliberation about compilation strategy. The core insight is that the CUDA kernels in the kdtree-engine repository do not have device-side cross-file dependencies—each kernel source file is self-contained at the device level. This means separate compilation is feasible: each .cu file can be compiled into an object file independently, and then linked together at the host level.

The assistant writes:

I'm deciding on the compilation strategy—I'll compile each kernel and engine source file into object files separately, then link them together rather than passing all sources to nvcc each time, since the kernels don't have device-side cross-file dependencies and host linking will resolve the launcher function calls.

This decision is non-trivial. The alternative—passing all source files to a single nvcc invocation—is simpler to script but has significant drawbacks. It recompiles everything every time, even when only one file changes. It produces a single monolithic binary, making it harder to isolate and debug individual components. And for a repository with multiple test targets (31 tests across multiple source files), a monolithic build would be wasteful and slow.

The separate compilation approach, by contrast, is more modular. Each kernel object file can be compiled once and reused across multiple test binaries. The engine object files can be compiled independently and linked into any test that needs them. This mirrors the structure that cmake provided, but implemented through a shell script using nvcc directly.

The assistant also considers the linking step explicitly: "host linking will resolve the launcher function calls." This is a crucial technical detail. CUDA kernel launches in host code are resolved through wrapper functions generated by nvcc. When compiling separately, each object file contains these wrapper functions, and the linker must resolve cross-object references. The assistant's confidence that this works correctly reflects deep knowledge of the CUDA compilation model.

The Two Files: Microbenchmarks and Build Script

The message creates two files, each serving a distinct purpose in the benchmark pipeline.

tests/bench_kernels.cu is a microbenchmark harness. Its purpose is to measure the performance of the three custom CUDA kernels (tree_build, verify_attn, tree_accept) at shapes that match the real Kimi K2.6 model's dimensions. This is the "meaningful benchmark" the user requested—not yet end-to-end throughput on the full model, but kernel-level performance numbers that validate the approach and identify bottlenecks. The benchmark harness is designed to run alongside the live SGLang service, using only the ~10 GiB of free GPU memory per card, without disrupting production inference.

scripts/build_nvcc.sh is the build script itself. It encapsulates the compilation strategy: set CUDA architecture flags (sm_120 for Blackwell), compile each kernel source file into an object file, compile each test source file, and link them together. The script replaces the entire cmake-based build pipeline with a straightforward shell script that can run on any machine with nvcc installed—including CT200.

Assumptions and Their Risks

The assistant makes several assumptions in this message, each carrying implicit risk.

Assumption 1: nvcc is available and functional on CT200. The reconnaissance confirmed CUDA 13.0 is installed, and nvcc --version returned successfully. But the specific compilation flags (architecture sm_120, include paths, library paths) must match the environment exactly. A mismatch could produce silent failures—binaries that compile but crash at runtime.

Assumption 2: Separate compilation works correctly for these kernels. The assistant asserts that "kernels don't have device-side cross-file dependencies." If this assumption is wrong—if, for example, a kernel device function calls another device function defined in a different translation unit—the separate compilation would fail at the link stage with unresolved device symbols. The assistant's confidence is based on knowledge of the codebase, but the risk is real.

Assumption 3: The benchmark harness can run alongside the live service. The free GPU memory (~10 GiB per card) should be sufficient for the benchmark's memory footprint (~80 MB for verify_attn at 4k prefix). But concurrent GPU kernel execution could interfere with the live service's latency, and the benchmark might trigger GPU memory allocation that conflicts with the service's CUDA context.

Assumption 4: The nvcc-direct approach is a temporary workaround, not a permanent divergence. The build script is not intended to replace cmake; it's a deployment-specific adaptation. But maintaining two build systems creates a maintenance burden—changes to source files or compilation flags must be mirrored in both.

Input Knowledge Required

To write this message, the assistant needed deep knowledge across several domains:

Output Knowledge Created

The message produces tangible artifacts and strategic knowledge:

The Broader Significance

Message [msg 11973] sits at the intersection of two competing forces in software engineering: the desire for build-system elegance and the reality of deployment constraints. The kdtree-engine repository was built with cmake—a modern, cross-platform build system that handles dependency resolution, incremental compilation, and test registration. But CT200, the target environment, lacks cmake. The assistant could have attempted to install cmake on CT200, but that introduces its own risks (package availability, dependency conflicts, system modifications to a production container). Instead, the assistant chose to adapt the build to the environment, creating a cmake-free path that uses only the tools already present.

This decision reflects a pragmatic engineering philosophy: the build system serves the code, not the other way around. When the target environment constrains the build system, the build system must adapt. The nvcc-direct script is not as elegant as cmake—it lacks automatic dependency tracking, parallel compilation, and test discovery. But it works on CT200, and it works now. That immediacy is the entire point.

The message also illustrates a broader pattern in the opencode session: the constant tension between the ideal architecture and the practical deployment. The native DDTree engine is the ideal—a fully custom, optimized inference stack. But getting meaningful benchmarks requires running on the real hardware, with the real model, alongside the real service. And that requires build infrastructure that works in that environment. Message [msg 11973] is the bridge between the ideal and the real: a build script that carries the carefully crafted CUDA kernels from the development environment to the production benchmark target.

Conclusion

Message [msg 11973] is, on its surface, a simple act: writing two files. But beneath that surface lies a rich reasoning process about compilation strategy, deployment constraints, and the pragmatics of benchmarking on production hardware. The assistant's decision to adopt separate compilation with direct nvcc invocation, rather than forcing cmake onto CT200 or abandoning the benchmark effort, reflects deep technical judgment and a clear-eyed assessment of trade-offs. The message creates the infrastructure that enables the next phase of the project: measuring whether the custom CUDA kernels actually improve DDTree throughput on the 8× PRO 6000 Blackwell system. Without this message—without the build script and the benchmark harness—the kernels would remain untested on the target hardware, and the "meaningful benchmarks" the user requested would remain out of reach.