The Pivotal Decision: Replacing Bincode with a Raw Binary Format for 25.7 GiB of Pre-Compiled Circuit Data

Introduction

In the course of optimizing the SUPRASEAL_C2 Groth16 proof generation pipeline for Filecoin PoRep, a seemingly routine implementation task—persisting a 25.7 GiB Pre-Compiled Circuit Evaluator (PCE) to disk—unexpectedly revealed a critical performance bottleneck. Message [msg 1639] captures the precise moment when the assistant recognized that the chosen serialization library, bincode, was introducing a 10–16× slowdown, and made the strategic decision to replace it with a raw binary format. This message is a masterclass in performance debugging: it demonstrates how measuring the right thing (load time vs. extraction time), understanding the memory layout of the underlying types, and doing a quick back-of-the-envelope calculation can converge on a decisive optimization before writing a single line of new code.

The Context: Why PCE Disk Persistence Mattered

The PCE (Pre-Compiled Circuit Evaluator) is a key innovation in the cuzk proving engine. It pre-computes the R1CS constraint matrices (A, B, C) in CSR (Compressed Sparse Row) format—three sparse matrices with a combined 722 million non-zero entries—so that synthesis of a proof no longer requires re-traversing the circuit constraint by constraint. Instead, synthesis becomes a simple matrix-vector multiplication (MatVec). The PCE extraction itself takes about 47 seconds and occupies roughly 25.7 GiB of memory.

The natural next step was to persist this PCE to disk so that it could be loaded at daemon startup rather than re-extracted on every launch. The assistant had already implemented a first version using bincode, a popular Rust serialization library, and wired it into the benchmark tool (cuzk-bench) with a --save-pce flag. The initial implementation used a 32-byte header (magic, version, circuit dimensions) followed by bincode-serialized PreCompiledCircuit<Scalar>.

The Discovery: A 49.9-Second Load That Defeated Its Purpose

When the assistant ran the benchmark in [msg 1637], the results were sobering:

| Operation | Time | Throughput | |-----------|------|------------| | Save to tmpfs | 16.8s | 1.6 GB/s | | Save to NVMe | 30.6s | 0.9 GB/s | | Load from tmpfs | 49.9s | 0.6 GB/s |

The load time of 49.9 seconds was actually slower than the 47-second extraction time. This meant that loading the PCE from disk was strictly worse than re-extracting it from scratch. The entire purpose of disk persistence—to avoid the extraction cost on subsequent launches—was defeated.

The assistant correctly identified the root cause: "bincode deserialization of 722M scalars involves per-element parsing. The bottleneck is bincode's deserialize, not I/O." Bincode, like most serde-based serializers, processes each element individually. For 722 million Scalar values (each 32 bytes), this means 722 million calls into the deserialization machinery, each performing bounds checks, type dispatch, and field-by-field decoding. The I/O subsystem was delivering data at ~1.6 GB/s, but bincode could only consume it at 0.6 GB/s.

The Decision: Raw Binary Format

Message [msg 1639] opens with the assistant's conclusion:

Bincode is 10-16x slower than necessary because it deserializes each scalar individually. Let me implement a raw binary format that writes/reads bulk byte slices.

This is the core decision of the message. The assistant then articulates the key technical insight that makes the raw format viable:

The key insight: blstrs::Scalar is #[repr(transparent)] over blst_fr which is [u64; 4] = 32 bytes. We can safely reinterpret &[Scalar] as &[u8] and vice versa.

This insight is crucial. The repr(transparent) attribute guarantees that Scalar has the exact same memory layout as its inner type blst_fr, which is a fixed-size array of four u64 values. This means a contiguous Vec<Scalar> is laid out in memory as a flat sequence of 32-byte blocks, with no padding, no indirection, and no per-element metadata. A &[Scalar] can be safely cast to &[u8] of length len * 32 using a simple pointer reinterpretation.

The same logic applies to Vec<u32> (used for column indices and row pointers) and Vec<u64> (used for density bitmaps). All are flat arrays of fixed-size primitives that can be written and read as bulk byte dumps.

The Reasoning Process

The assistant's thinking in this message is exemplary. Let's trace the logic:

  1. Measure first: The benchmark data from [msg 1637] provides concrete numbers. The 49.9s load time is not just slow—it's quantitatively worse than the alternative (re-extraction at 47s).
  2. Identify the bottleneck: The assistant distinguishes between I/O throughput and deserialization throughput. The save operation achieves 1.6 GB/s (close to tmpfs sequential write speed), but the load operation achieves only 0.6 GB/s. Since both operations touch the same data, the difference must be in the read path—specifically, in bincode's deserialization.
  3. Calculate the theoretical optimum: In [msg 1638], the assistant runs a quick Python calculation: 25.7 GiB at ~8 GB/s (DDR5 write bandwidth) would be ~3.2s; at ~5 GB/s (NVMe sequential read) would be ~5.1s. This establishes that 3–5 seconds is the physically achievable target, compared to the current 49.9s.
  4. Verify type safety: Before committing to the raw format, the assistant checks the memory layout of Scalar. The repr(transparent) attribute and the fixed [u64; 4] representation guarantee that byte reinterpretation is safe and well-defined.
  5. Read the existing code: The assistant reads disk.rs to understand the current format structure and plan the replacement. This is a deliberate step—not jumping straight into implementation, but first understanding what needs to change.

Assumptions and Their Validity

The message makes several assumptions, all of which proved correct:

Assumption 1: Bincode's per-element overhead is the bottleneck. This was confirmed by the throughput disparity between save (1.6 GB/s) and load (0.6 GB/s). Save also uses bincode (serialization), but serialization is cheaper than deserialization because it doesn't involve allocation and bounds-checked construction of each element.

Assumption 2: Scalar can be safely byte-reinterpreted. The #[repr(transparent)] attribute guarantees ABI compatibility with the inner type. However, the assistant also implicitly assumes that the byte representation is stable across Rust compiler versions and platform architectures. For blstrs::Scalar (a BLS12-381 field element), this is a safe assumption—the representation is defined by the BLS standard, not by the compiler.

Assumption 3: A raw binary format would achieve near-memcpy speeds. This was validated in [msg 1648], where the raw format achieved 9.2s load from tmpfs (3.0 GB/s) and 7.7s save to tmpfs (3.6 GB/s). The 9.2s load is still above the theoretical 3.2s memcpy limit because it includes Vec allocation time (25.7 GiB of heap allocation is not free) and the overhead of reading the header and performing length-prefix validation.

Input Knowledge Required

To fully understand this message, one needs:

Output Knowledge Created

This message creates several important outputs:

  1. The decision to implement a raw binary format: This is the strategic choice that drives the subsequent implementation in messages [msg 1640] through [msg 1642].
  2. The performance target: The assistant now knows that ~3–5 seconds is the physically achievable load time, providing a clear goal for the implementation.
  3. The type-safety argument: The insight about repr(transparent) and safe byte reinterpretation becomes the foundation for the raw format implementation.
  4. A diagnosis that generalizes: The lesson that bincode is inappropriate for bulk numeric data at this scale is a reusable insight. Any future serialization of large vectors of primitive types in the cuzk project should use a raw format.

The Result: 5.4× Speedup

The implementation that followed this decision (messages [msg 1640]–[msg 1642]) replaced the bincode-based save/load with a raw binary format. The format uses a 32-byte header followed by length-prefixed raw byte dumps of each vector: u64 length + raw bytes for each of row_ptrs, cols, vals in each CSR matrix (A, B, C), plus the density bitmaps.

The results, reported in [msg 1648], were dramatic:

| Operation | Bincode | Raw Format | Speedup | |-----------|---------|------------|---------| | Save (tmpfs) | 16.8s | 7.7s | 2.2× | | Load (tmpfs) | 49.9s | 9.2s | 5.4× | | Save (NVMe) | 30.6s | 22.3s | 1.4× |

The load time dropped from 49.9s to 9.2s—a 5.4× improvement. More importantly, the 9.2s load was now significantly faster than the 47s extraction, making disk persistence genuinely useful. The PCE could be loaded in under 10 seconds at daemon startup, eliminating the first-proof penalty entirely.

Broader Significance

This message is significant beyond its immediate technical achievement. It represents a shift in the assistant's approach to optimization: from optimizing the computational path (synthesis speed, GPU utilization) to optimizing the data path (I/O and serialization). The PCE disk persistence is the bridge between the "cold start" world (where every daemon launch requires re-extraction) and the "warm start" world (where PCE data is loaded in seconds). This enables the Persistent Prover Daemon architecture described in the optimization proposals, where the daemon maintains a hot PCE cache across proof requests.

The message also illustrates a general principle of performance engineering: measure before optimizing, and compare against the physical limit. The assistant didn't just notice that 49.9s was slow—it calculated that 3–5s was the theoretical minimum, establishing that a 10–16× improvement was possible. This kind of back-of-the-envelope calculation is what separates targeted optimization from guesswork.

Conclusion

Message [msg 1639] is a turning point in the cuzk proving engine optimization effort. A routine implementation task—persisting PCE data to disk—revealed an unexpected bottleneck, and the assistant's response was to diagnose the root cause, calculate the theoretical optimum, verify the type-safety of the proposed solution, and commit to a fundamentally different approach. The resulting 5.4× speedup transformed PCE disk loading from a liability (slower than re-extraction) into a genuine asset (9.2s vs. 47s), enabling the daemon integration that would follow. It is a textbook example of how deep understanding of the underlying data representation can unlock order-of-magnitude improvements in systems that handle large-scale numerical data.