Probing GPU Compatibility Boundaries: The Blackwell Compatibility Check in cuzk's Phase 0 Validation

Introduction

In the course of validating the Phase 0 scaffold of the cuzk proving daemon—a pipelined SNARK proving engine for Filecoin proof generation—the assistant encountered a critical question: would the RTX 5070 Ti GPU, based on NVIDIA's Blackwell architecture (compute capability 12.0), be compatible with the existing CUDA-based proving pipeline? Message [msg 203] captures a single, seemingly narrow action: a grep command searching for architecture-specific patterns in the rust-gpu-tools library's CUDA utilities. Yet this message sits at the intersection of hardware compatibility assessment, build system architecture, and the practical realities of running production cryptographic workloads on cutting-edge GPU hardware. Understanding why this grep was necessary, what assumptions it tested, and what knowledge it produced reveals the meticulous investigative process behind deploying a zero-knowledge proof system on modern hardware.

The Message in Full

The assistant executed the following command:

grep -n "compute_cap\|min_cc\|sm_\|unique_id\|pci_id\|Framework\|fatbin" ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rust-gpu-tools-0.7.2/src/cuda/utils.rs 2>/dev/null | head -20

The output returned lines from the source file showing functions for get_pci_id, get_compute_capability, and device enumeration logic that uses compute capability and PCI ID information.

Context and Motivation: Why This Message Was Written

To appreciate why this grep was necessary, one must understand the chain of reasoning that led to it. The session had been building toward the first real end-to-end validation of the cuzk proving pipeline. The assistant had already verified that the workspace compiled cleanly, that the 45 GiB SRS parameters were available at /data/zk/params/, and that the test data (a 51 MB C1 output JSON) was present. The critical remaining question was whether the GPU—an RTX 5070 Ti with compute capability 12.0—would actually work.

The proving pipeline has two possible CUDA backends. The first is supraseal-c2, a specialized Groth16 prover that uses sppark for CUDA kernel compilation. The second is the bellperson native prover, which uses rust-gpu-tools and ec-gpu-gen for GPU acceleration. The assistant had already examined the supraseal-c2 build system ([msg 200]) and found that it delegates to nvcc via sppark::build::ccmd(), which compiles CUDA source files at build time. Since nvcc on CUDA 13.1 can target Blackwell GPUs through JIT compilation, the supraseal path seemed likely to work.

However, the bellperson path was more concerning. Bellperson uses rust-gpu-tools, which in turn uses rustacuda to interact with CUDA devices. The critical question was whether rust-gpu-tools performs any runtime filtering based on compute capability that might reject a Blackwell GPU. This is where message [msg 203] comes in.

The Investigative Thread

The assistant had been systematically working through the dependency chain. In [msg 199], it discovered the GPU's compute capability was 12.0 (Blackwell). In [msg 200], it examined the supraseal-c2 and bellperson build scripts. In [msg 201], it looked at sppark's build system to see if it filtered architectures. In [msg 202], it concluded that sppark uses cc::Build with .cuda(true) which lets nvcc choose the default architecture, and then turned to investigate rust-gpu-tools for device filtering.

Message [msg 203] is the direct follow-up to that investigation. The assistant is searching for specific patterns that would indicate whether rust-gpu-tools performs any compatibility checks that might block Blackwell GPUs. The patterns searched for are telling:

Assumptions Made

The assistant operated under several assumptions in this message. First, it assumed that rust-gpu-tools was the component most likely to cause compatibility issues, since it sits between the application and the CUDA runtime and might perform its own device filtering. Second, it assumed that searching for these specific patterns in the utils.rs file would be sufficient to determine whether Blackwell support existed—an assumption that proved correct, as the grep output showed the relevant functions. Third, it assumed that the rust-gpu-tools version 0.7.2 was the version actually being used by the dependency chain, which it had confirmed through earlier file system searches.

A more subtle assumption was that the grep output would be interpretable without reading the full source context. The assistant relied on the function names and line numbers to infer behavior: get_compute_capability retrieves the compute capability, and the device enumeration logic at line 79 calls this function and then calls get_pci_id. This suggests that devices are identified and potentially filtered by their compute capability and PCI location.

Potential Mistakes and Incorrect Assumptions

The most significant potential mistake in this message is the assumption that finding no explicit min_cc or architecture filtering in utils.rs would mean Blackwell is supported. In reality, compatibility could fail at multiple other layers: the CUDA driver itself (version 590.48.01, CUDA 13.1), the rustacuda library, the precompiled CUDA kernels in ec-gpu-gen, or even the nvcc compiler used during build. The grep only checked one file in one library.

Additionally, the assistant did not check whether rust-gpu-tools uses a device whitelist or blacklist elsewhere—for instance, in device.rs (which appeared in the grep results for a broader search in [msg 202]) or in the CUDA module's mod.rs. The grep was limited to utils.rs, which contains utility functions but may not contain the actual device selection logic.

Another subtle issue: the assistant was searching for sm_ as a pattern, which would match architecture-specific code like sm_120 or sm_89. However, in rust-gpu-tools, architecture-specific code might be expressed differently—for example, as compute capability tuples like (12, 0) rather than sm_120. The grep for compute_cap would catch the tuple representation, but the assistant didn't verify that the comparison logic would accept compute capability 12.0.

Input Knowledge Required

To understand this message, one needs substantial context about the cuzk project and its dependencies. The reader must know that rust-gpu-tools is a Rust library that abstracts GPU device management across CUDA and OpenCL backends, and that it is used by ec-gpu-gen and bellperson for GPU-accelerated cryptographic operations. One must understand that compute capability is NVIDIA's versioning scheme for GPU architectures (e.g., 8.9 for Ada Lovelace, 9.0 for Hopper, 12.0 for Blackwell), and that precompiled CUDA kernels (fatbins) are architecture-specific. One must also know the broader context: that the cuzk daemon needs to run Groth16 proofs on this GPU, and that the proving pipeline involves both CPU synthesis and GPU computation phases.

The reader also needs to understand the build system architecture: that supraseal-c2 uses sppark which compiles CUDA at build time (making it more architecture-flexible), while bellperson uses rust-gpu-tools which may use precompiled kernels (making it architecture-sensitive). This distinction explains why the assistant was more concerned about the bellperson path than the supraseal path.

Output Knowledge Created

This message produced specific, actionable knowledge: the rust-gpu-tools CUDA utilities do contain compute capability detection logic and PCI ID-based device identification. The grep output showed that get_compute_capability is called during device enumeration (line 79), and that devices are matched by PCI ID. This means the library is aware of compute capability, but the grep did not reveal any explicit rejection of high compute capabilities.

More importantly, this message contributed to the broader decision-making process. The assistant was weighing two CUDA backends: cuda-supraseal (using supraseal-c2) and cuda (using bellperson native). The grep results, combined with the earlier analysis of sppark's build system, suggested that the supraseal path was safer for Blackwell because it compiles CUDA kernels at build time rather than relying on precompiled fatbins. This influenced the assistant's subsequent decision to build with --features cuda-supraseal (see [msg 204]), which ultimately succeeded.

The Thinking Process Visible in the Message

The message reveals a systematic, hypothesis-driven investigation style. The assistant is not randomly grepping; it is following a logical chain: "sppark doesn't filter architectures → let me check rust-gpu-tools which might." The choice of search patterns is itself revealing—each pattern corresponds to a specific hypothesis about how compatibility might be enforced. compute_cap and min_cc test for version-based filtering. sm_ tests for architecture-specific code. fatbin tests for precompiled kernel dependencies. pci_id tests for device identification logic.

The head -20 limit shows the assistant is doing a quick reconnaissance, not an exhaustive audit. It wants to know "is there anything here that would obviously block Blackwell?" rather than "exactly how does device selection work?" This is appropriate for the Phase 0 validation context—the goal is to get a real proof running, not to fully understand the GPU compatibility matrix.

Broader Significance

This message exemplifies a recurring pattern in systems engineering: the need to validate hardware compatibility when deploying production software on new hardware. The cuzk daemon is being developed on an RTX 5070 Ti, which at the time of writing was a very recent GPU (Blackwell architecture launched in early 2025). The proving pipeline's dependencies were written for older architectures (Ampere, Ada Lovelace, Hopper). The assistant's investigation bridged this gap by tracing through the dependency chain to identify potential failure points.

The message also illustrates the importance of understanding build-time vs. runtime architecture dependencies. sppark compiles CUDA at build time, making it inherently more portable across GPU generations. rust-gpu-tools may use precompiled kernels, making it architecture-sensitive. This distinction is crucial for anyone deploying cryptographic workloads on modern GPUs.

Conclusion

Message [msg 203] is a small but critical piece of a larger validation effort. It represents the moment when the assistant moved from theoretical compatibility analysis to concrete code inspection, searching for the specific mechanisms that might prevent a Blackwell GPU from running Filecoin proofs. The grep command, while simple, was the culmination of a chain of reasoning that touched on build systems, CUDA compilation models, and the dependency graph of a complex cryptographic proving pipeline. The knowledge produced—that rust-gpu-tools does perform compute capability detection but does not obviously reject Blackwell—directly informed the decision to proceed with the cuda-supraseal backend, which ultimately succeeded in producing the first real Groth16 proof through the cuzk daemon.