The Pivotal Read: How Examining the Standard Prover Path Revealed the Root Cause of a 196-Input Mismatch

Introduction

In the course of debugging a crash in the CuZK zero-knowledge proving engine, a single file-read operation marked the turning point between chasing red herrings and identifying the true root cause. Message 171 in this opencode session is deceptively simple: it is a tool call that reads a portion of /tmp/czk/extern/bellperson/src/groth16/prover/supraseal.rs. Yet this read operation was the culmination of an increasingly focused investigation, and its result — the discovery that the standard prover uses ProvingAssignment rather than WitnessCS — shattered a key assumption and led directly to the fix.

The Message

The subject message is a single tool invocation:

[assistant] [read] /tmp/czk/extern/bellperson/src/groth16/prover/supraseal.rs
<path>/tmp/czk/extern/bellperson/src/groth16/prover/supraseal.rs</path>
<type>file</type>
<content>320:         Instant,
321:         std::vec::Vec<ProvingAssignment<Scalar>>,
322:         std::vec::Vec<std::sync::Arc<std::vec::Vec<Scalar>>>,
323:         std::vec::Vec<std::sync::Arc<std::vec::Vec<Scalar>>>,
324:     ),
325:     SynthesisError,
326: >
327: where
328:     Scalar: PrimeField,
329:     C: Circuit<Scalar> + Send,
330: {
331:     let mut pro...
</content>

The file content is truncated at line 331, showing only the tail end of a function signature. But the critical detail leaps out: std::vec::Vec<ProvingAssignment<Scalar>> at line 321. This single type name — ProvingAssignment — is the key that unlocks the entire debugging puzzle.

The Context: A 196-Input Mystery

To understand why this read was so consequential, we must trace the investigation that led to it. The session's overarching goal was implementing Pre-Compiled Constraint Evaluator (PCE) extraction for all proof types in the CuZK proving engine. PCE extraction works by running circuit synthesis through a RecordingCS (Recording Constraint System) that records the structure of the circuit rather than computing actual witness values. This recorded structure can then be reused across proofs, avoiding repeated synthesis.

The developer had already implemented PCE extraction for WinningPoSt and WindowPoSt proof types, and had added a partitioned pipeline for SnapDeals. However, enabling PCE for WindowPoSt caused a crash. The C++ GPU code asserted that points_a.size() == p.inp_assignment_size + p.a_aux_popcount, meaning the dimensions of the pre-compiled circuit did not match the witness produced during actual proving.

The numbers told a stark story. The standard prover path produced num_inputs = 25840, while the PCE extraction path produced num_inputs = 26036. The difference was exactly 196 — the number of parallel chunks used in the synthesize_extendable function. Each chunk appeared to contribute one extra input in the PCE path.

The Reasoning: Why This Message Was Written

The assistant had already made significant progress. In [msg 164], it identified the mismatch and noted that the PCE had 196 more inputs and 196 more constraints than the witness. In [msg 165], it traced the synthesize_extendable flow and hypothesized that the issue might be in how the parent constraint system is created. In [msg 167], it walked through both the WitnessCS and RecordingCS flows in detail, concluding that both should produce the same num_inputs count given the same extend() logic.

But there was a nagging inconsistency. The assistant had been assuming that the standard prover used WitnessCS as its constraint system type. After all, the comment in the code said "assignments are produced by WitnessCS." Yet the numbers didn't add up. The assistant's own trace of WitnessCS::new() showed it pre-allocated the ONE input at index 0, and WitnessCS::extend() skipped index 0 when merging children. The RecordingCS::extend() also skipped one input per child via self.num_inputs += other.num_inputs - 1. So both should produce the same result — unless the parent constraint system was created differently.

This is the moment captured in [msg 168] through [msg 170]. The assistant started questioning its assumptions: "The parent CS is NOT created via CS::new(). The parent is the CS passed into synthesize(). For WitnessCS, the parent is created by the bellperson prover. For RecordingCS, the parent is created by extract_precompiled_circuit which calls RecordingCS::new()."

The assistant then searched for how the prover actually creates its constraint system, finding references to WitnessCS in comments but no direct instantiation. This prompted the read of supraseal.rs — the file that implements the GPU proving path — to see exactly what type the prover uses.## The Broken Assumption

The critical assumption that this message shattered was that the standard prover path used WitnessCS as its constraint system type. This assumption was natural — the codebase had comments stating that "assignments are produced by WitnessCS," and the developer's earlier work on making RecordingCS extensible had explicitly modeled it after WitnessCS to ensure compatibility. The comment in RecordingCS::new() even read: "Pre-allocate the 'ONE' input variable at index 0, matching WitnessCS and ProvingAssignment conventions."

But this comment contained a subtle error. While WitnessCS::new() did pre-allocate ONE, ProvingAssignment::new() did not. The assistant's read of supraseal.rs revealed that the standard prover used ProvingAssignment, not WitnessCS. And as the assistant would discover in the following messages ([msg 172] through [msg 175]), ProvingAssignment::new() started with empty input_assignment vectors — zero inputs, not one.

This difference cascaded through the synthesize_extendable pipeline. When the prover created child constraint systems via CS::new() and then called alloc_input("temp ONE"), the behavior diverged:

• For ProvingAssignment: new() produced 0 inputs, alloc_input added ONE at index 0 → child had 1 input before sector synthesis
• For RecordingCS: new() produced 1 input (pre-allocated ONE), alloc_input added another ONE at index 1 → child had 2 inputs before sector synthesis When extend() was called, both skipped index 0. For ProvingAssignment, this skipped the actual ONE (which was already present in the parent), so only sector-specific inputs were appended. For RecordingCS, this skipped the pre-allocated ONE at index 0, but the "temp ONE" at index 1 survived as an extra input. With 196 parallel chunks, this produced exactly 196 extra inputs — the exact mismatch observed in the crash logs.

The Thinking Process Visible in the Reasoning

The assistant's reasoning in the messages leading up to [msg 171] reveals a methodical debugging process. In [msg 167], the assistant manually traced through both the WitnessCS and RecordingCS flows, step by step, computing the expected num_inputs after each operation. This is classic rubber-duck debugging — walking through code paths with explicit state tracking.

When the math didn't add up — both flows should have produced the same count, yet the logs showed a 196-input difference — the assistant didn't dismiss the discrepancy. Instead, it doubled down on the investigation. It checked whether extract_precompiled_circuit added extra input constraints (it didn't). It re-read the extend() implementations to verify they were structurally equivalent. It searched for how the prover created its constraint system, initially finding only comments referencing WitnessCS.

The grep in [msg 169] and [msg 170] is particularly telling. The assistant searched for WitnessCS::new and WitnessCS::< and found nothing — the prover never directly instantiates WitnessCS. This negative result was the clue that something was off. If the prover doesn't use WitnessCS, what does it use? The answer was in supraseal.rs, which the assistant had already seen referenced but hadn't examined closely.

Input Knowledge Required

To fully understand this message and its significance, one needs several pieces of background knowledge:

1. Constraint System (CS) types in bellperson: The codebase has multiple implementations of the ConstraintSystem trait — WitnessCS (for witness generation), ProvingAssignment (for the standard prover), and RecordingCS (for PCE extraction). Each has slightly different initialization and behavior.
2. The ONE input convention: R1CS constraint systems reserve input index 0 for the constant value ONE. This is a standard convention in Groth16 proving systems. The question is when and how this ONE input is allocated — whether it's pre-allocated in new() or explicitly allocated by the caller before synthesis.
3. The synthesize_extendable pattern: This is a parallel synthesis strategy where a circuit is split into chunks, each synthesized independently in child CS instances, then merged via extend(). The extend() method must correctly handle the ONE input to avoid double-counting.
4. PCE extraction: The Pre-Compiled Circuit Evaluator records circuit structure by running synthesis through a RecordingCS, then serializing the resulting constraint system for reuse. The recorded structure must exactly match what the prover would produce, or the GPU code will crash on dimension mismatches.

Output Knowledge Created

This message created critical knowledge that directly led to the fix:

1. The standard prover uses ProvingAssignment, not WitnessCS: This was the key realization that broke the incorrect assumption.
2. ProvingAssignment::new() starts with zero inputs: Unlike WitnessCS::new() which pre-allocates ONE at index 0, ProvingAssignment starts empty. The ONE is allocated explicitly by the prover code before calling circuit.synthesize().
3. The fix direction: RecordingCS::new() should NOT pre-allocate ONE. It should start with num_inputs = 0, matching ProvingAssignment::new(). The caller (extract_precompiled_circuit) should explicitly allocate ONE, just like the prover does. This knowledge was immediately actionable. In the messages following [msg 171], the assistant confirmed that ProvingAssignment was extensible and had the same extend() logic, then traced the exact initialization difference. The fix was implemented in the next round: RecordingCS::new() was changed to start with zero inputs, and extract_precompiled_circuit was updated to explicitly allocate the ONE input before synthesis, mirroring the prover's pattern exactly.

The Broader Significance

This debugging episode illustrates a fundamental principle of systems integration: when two paths through the same code produce different results, the root cause is often not in the logic of the shared code, but in the initialization or setup that precedes it. The extend() methods were identical in their arithmetic — both skipped one input per child. But the starting state of the parent and children differed, and that difference propagated through the arithmetic to produce the 196-input gap.

The read of supraseal.rs in [msg 171] was the moment when the investigation shifted from "why do these two CS types behave differently?" to "what type does the prover actually use?" — a question that seems obvious in retrospect but required peeling back layers of assumption to reach. It is a testament to the value of reading the actual code rather than relying on comments or assumptions about what the code does.