The Glob That Opened a Pipeline: Finding the Build System Entry Point for Zero-Copy GPU Proving

Message at a Glance

The message is deceptively simple:

[assistant] [glob] **/build.rs
/tmp/czk/extern/supraseal-c2/build.rs

A single glob command searching for build.rs files across the project tree, returning one result at /tmp/czk/extern/supraseal-c2/build.rs. On its surface, this looks like a mundane file-finding operation — the kind of mechanical step that fills the gaps between moments of genuine engineering insight. But in the context of the larger investigation, this message is a critical hinge point. It marks the moment when the assistant transitions from architectural design into concrete implementation, from reasoning about what to build to discovering where to build it.

The Investigation That Led Here

To understand why this glob was issued, we must reconstruct the chain of reasoning that preceded it. The team had been chasing a GPU underutilization bug in the cuzk proving pipeline for some time. The symptom was stark: the GPU was idle roughly 50% of the time during proof generation, even though there was always work available. After ruling out lock contention and malloc_trim overhead through precise Rust-side instrumentation, the team added timing probes inside the C++ gpu_prove_start function itself. This revealed the true culprit: the Host-to-Device (H2D) transfer of the a/b/c synthesis vectors inside execute_ntts_single was running at 1–4 GB/s instead of the PCIe Gen5 x16 line rate of approximately 50 GB/s.

The root cause traced back to memory allocation. The a/b/c vectors — each holding up to 130 million field elements of 32 bytes, totaling over 12 GiB for a single PoRep partition — were allocated as standard heap memory via Rust's Vec. CUDA cannot perform direct memory access (DMA) from unpinned host memory. Instead, it must stage transfers through a small pinned bounce buffer, copying data in chunks. This staged copy is what limited throughput to 1–4 GB/s. Meanwhile, the SRS points used in MSM operations enjoyed full line-rate transfers because they were allocated with cudaHostAlloc, which registers the memory as DMA-capable.

The chosen solution was Option B: direct synthesis into pinned memory. Rather than allocating a/b/c as ordinary heap vectors and then copying them to pinned staging buffers after synthesis (Option A), the team decided to synthesize the constraint data directly into pre-allocated pinned buffers. This eliminated both the reallocation copies that occur during Vec::push growth phases and the staged H2D transfer. Under heavy memory pressure — with 20+ concurrent synthesis threads consuming over 400 GiB of RAM — even a simple memcpy would be bottlenecked by contested host memory bandwidth. Direct synthesis into pinned memory was the only path to full GPU utilization.

Why This Specific Glob?

With the architectural decision made, the assistant needed to implement the core components. The plan involved three major pieces:

  1. A PinnedPool struct in cuzk-core that manages cudaHostAlloc'd buffers with a free list and integration with the MemoryBudget system.
  2. Modifications to bellperson's ProvingAssignment to accept and safely hold pinned backing buffers.
  3. Wiring the pool through the pipeline so synthesis functions can request pinned buffers before creating assignments. But before any of this could be written, the assistant needed to answer a foundational question: where do the CUDA FFI bindings live? The cudaHostAlloc and cudaFreeHost functions are not available through any existing Rust wrapper in the project — a grep for cudaHostAlloc|cudaFreeHost|cudaMallocHost|cuda_host returned no results (see [msg 3071]). The assistant had to add these bindings from scratch. The supraseal-c2 crate was the natural home for these bindings. It already contained the CUDA error type via sppark::cuda_error!() and wrapped the SRS structure with its CUDA-managed pointers. But the assistant needed to understand how the crate's CUDA code was compiled — specifically, whether there was a build.rs that invoked nvcc to compile .cu source files. If the existing CUDA kernels were compiled through a build script, then adding new CUDA functions would follow the same pattern. If the crate relied on pre-built libraries or a different mechanism, the approach would differ. The glob **/build.rs was the most direct way to answer this question. By searching recursively for all build.rs files, the assistant could locate the build system entry points for every crate in the workspace. The single result at /tmp/czk/extern/supraseal-c2/build.rs confirmed that the crate had its own build script — and reading it (in the following message, [msg 3073]) would reveal the exact compilation command, the CUDA source file being compiled, and the pattern for adding new CUDA functions.

The Thinking Process Revealed

This message exposes a specific mode of engineering reasoning: implementation archaeology. The assistant had a clear design in mind — a pinned memory pool backed by cudaHostAlloc — but the exact mechanism for adding those CUDA bindings depended on the project's existing build conventions. Rather than guessing or making assumptions about the build system, the assistant traced the dependency chain backward: from the need for CUDA host allocation functions, to the supraseal-c2 crate as the binding layer, to its build.rs as the compilation entry point.

This is characteristic of working with large, multi-crate Rust projects that wrap foreign code. The build system is not always obvious from the source code alone — a .cu file might be compiled by a build.rs, by a separate Makefile, by CMake integration, or by a custom build tool. The glob was the fastest way to disambiguate these possibilities without reading through directory listings or configuration files.

The assistant also made a subtle but important assumption: that the existing CUDA code in supraseal-c2 was compiled through a build.rs script rather than being pre-compiled or linked as an external library. This assumption was validated by the glob result, which confirmed the existence of a build script. Had the glob returned no results, the assistant would have needed to explore alternative compilation mechanisms — perhaps a Cargo manifest with links directives, or a Makefile invoked by a custom build step.

Input Knowledge Required

Understanding this message requires familiarity with several layers of the project architecture:

Output Knowledge Created

The glob produced one critical piece of information: the path /tmp/czk/extern/supraseal-c2/build.rs. This told the assistant:

  1. That supraseal-c2 has its own build script, confirming the crate compiles native CUDA code at build time.
  2. The exact location to read for understanding the compilation command, the source file being compiled (cuda/groth16_cuda.cu), and the flags being passed to nvcc.
  3. That adding new CUDA functions would follow the existing pattern: write a .cu file, add it to the build.rs compilation command, and expose the FFI functions through Rust extern "C" declarations. This single path was the key that unlocked the implementation phase. In the messages immediately following, the assistant would read the build.rs contents, understand the compilation pattern, and proceed to implement the PinnedPool struct, the PinnedBacking mechanism in bellperson, and the release_abc method — all of which depended on having a place to add the cudaHostAlloc and cudaFreeHost FFI bindings.

The Broader Significance

In the narrative arc of the segment, this glob message is the bridge between design and implementation. The preceding messages ([msg 3070] and [msg 3071]) contained the extended reasoning about Option B, the ManuallyDrop approach, the budget integration strategy, and the grep that confirmed no existing CUDA host allocation bindings existed. The following messages ([msg 3073] onward) would read the build script, write the PinnedPool, modify ProvingAssignment, and validate the changes with cargo check.

The glob itself is almost invisible — a single line in a conversation spanning thousands of messages. But it represents a fundamental engineering discipline: never assume you know the build system. The assistant could have guessed at the compilation pattern, or started writing CUDA bindings in a random location, or wasted time exploring non-existent build configurations. Instead, it asked the file system directly, letting the project's own structure guide the implementation. This is the difference between reasoning about code and actually building it — the moment when abstract design meets concrete reality, mediated by a simple recursive search.