The JSON Serialization Gap: Diagnosing Cross-Language Proof Failures in CuZK

Introduction

In the course of debugging a persistent failure where PSProve PoRep proofs generated through the CuZK proving engine failed verification, the assistant embarked on a systematic investigation of the JSON serialization boundary between Rust and Go. The core hypothesis was that a JSON format mismatch between Rust's SealCommitPhase1Output struct and Go's Commit1OutRaw struct was causing the Rust-side serde_json::from_slice deserialization to produce incorrect or corrupted data, leading to proof verification failures. This article traces that investigation across multiple source files, language boundaries, and serialization frameworks.

The Two Paths: Working vs Failing

The investigation was motivated by a clear dichotomy in the system's behavior. In the working path, raw Rust-serialized SealCommitPhase1Output JSON (produced by GeneratePoRepVanillaProof) was base64-encoded, wrapped in a C1OutputWrapper envelope, and sent directly to the CuZK Rust server, which successfully deserialized it with serde_json::from_slice. In the failing path, the same raw Rust JSON was first deserialized into Go's Commit1OutRaw struct via json.Unmarshal, then later re-serialized with json.Marshal before being wrapped and sent to CuZK. The Go round-trip introduced a subtle transformation that caused the Rust deserializer to produce incorrect results, ultimately causing ffi.VerifySeal to fail.

The user's request to the assistant was precise: examine the Go type definitions in /tmp/czk/lib/proof/porep_vproof_types.go and /tmp/czk/lib/proof/merkle.go, locate the corresponding Rust type definitions in the cargo registry cache, and produce a detailed comparison of every serialization difference between the two formats (see [msg 0]).

Tracing the Rust Source

The assistant began by locating the Rust source files ([msg 1] through [msg 5]). The workspace used filecoin-proofs-api version 19.0.0, which was found in the cargo registry at /home/theuser/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/filecoin-proofs-api-19.0.0/. From there, the assistant traced the type chain: SealCommitPhase1Output (defined in src/seal.rs), RegisteredSealProof enum (in src/registry.rs), VanillaSealProof enum (also in seal.rs), and the domain types PoseidonDomain and Sha256Domain (in the filecoin-hashers crate at version 14.0.0). The Merkle proof types were found in storage-proofs-core-19.0.0/src/merkle/proof.rs, and the stacked proof types (LabelingProof, EncodingProof, Column, ColumnProof, ReplicaColumnProof) were found in storage-proofs-porep-19.0.0/src/stacked/vanilla/.

This tracing exercise itself was a significant achievement ([msg 6] through [msg 10]). The assistant had to navigate a complex dependency graph spanning five Rust crates, each with multiple versions cached in the registry, and map each Rust type to its Go counterpart. The grep tool was used extensively to locate type definitions, serde attributes, and custom serialization implementations.

Systematic Type-by-Type Comparison

The assistant's analysis spanned dozens of messages ([msg 11] through [msg 41]), each building on the previous findings. The key comparisons are documented across multiple articles in this segment: the serde derive revelation [20], the PhantomData serialization trap [22], the serde enum trap [23], and the bincode discovery [18].

RegisteredSealProof / registered_proof

The registered_proof field in Rust is a RegisteredSealProof enum with #[derive(Serialize, Deserialize)]. Serde's default enum serialization produces a JSON string containing the variant name, such as "StackedDrg32GiBV1_1". In Go, the field is StringRegisteredProofType which is simply a string type alias. Both serialize as JSON strings with identical values. No mismatch.

VanillaSealProof / vanilla_proofs

This field presented a more nuanced comparison. In Rust, VanillaSealProof is an enum with variants like StackedDrg32GiBV1(Vec<Vec<RawVanillaSealProof<SectorShape32GiB>>>). Serde's externally-tagged enum serialization produces a JSON object with a single key matching the variant name: {"StackedDrg32GiBV1": [[...]]}. Critically, the variant name is always the *V1 form (e.g., StackedDrg32GiBV1), never the *V1_1 or *Feat_SyntheticPoRep form that appears in registered_proof. This is because the from_raw constructor maps any sector size to the corresponding base V1 variant regardless of the proof features enabled.

In Go, the field is map[StringRegisteredProofType][][]VanillaStackedProof. The Go map faithfully preserves whatever key string it receives during deserialization and writes it back unchanged during re-serialization. Since the Rust JSON always uses the *V1 key form, Go preserves it through the round-trip. No mismatch.

Commitment, Ticket, and Domain Types

The byte-array types — Commitment ([u8; 32] in Rust, [32]byte in Go), Ticket (same), PoseidonDomain (newtype over [u8; 32]), and Sha256Domain (newtype over [u8; 32]) — all serialize as JSON arrays of 32 integers: [1, 2, 3, ..., 32]. This is the default behavior for both serde_json (for [u8; 32] and newtype structs wrapping it) and Go's encoding/json (for [32]byte). The Go types PoseidonDomain and Sha256Domain have custom MarshalJSON methods that explicitly call json.Marshal([32]byte(p)), confirming the integer-array format.

A critical detail is that neither PoseidonDomain nor Sha256Domain has a custom UnmarshalJSON method. This means Go relies on the default deserialization of [32]byte from a JSON integer array, which works correctly. However, it also means the Go types are fragile: if any upstream change altered the Rust serialization format (e.g., to base64 strings), Go deserialization would break silently. No mismatch in current format, but a fragility concern.

MerkleProof / ProofData Enum

The Rust ProofData enum has three variants: Single, Sub, and Top. Serde's externally-tagged serialization produces {"Single": {...}}, {"Sub": {...}}, or {"Top": {...}}. In Go, this is modeled as a struct with three optional pointer fields, each with json:"...,omitempty":

type ProofData[H HasherDomain] struct {
    Single *SingleProof[H] `json:"Single,omitempty"`
    Sub    *SubProof[H]    `json:"Sub,omitempty"`
    Top    *TopProof[H]    `json:"Top,omitempty"`
}

This struct correctly emulates the Rust enum: only the non-nil variant appears in the JSON output. No mismatch.

PhantomData / _h Fields

Several Rust types use PhantomData<H> for generic type parameters. The handling of these fields during serialization differs by type:

The testMarshal Flag: A Red Herring

During the investigation, the assistant discovered a testMarshal flag in merkle.go that controls the serialization behavior of PoseidonDomain and Sha256Domain:

var testMarshal = false

func (p PoseidonDomain) MarshalJSON() ([]byte, error) {
    if testMarshal {
        return json.Marshal(p[:])  // p[:] is []byte -> base64 string!
    }
    return json.Marshal([32]byte(p))  // [32]byte -> integer array
}

When testMarshal = true, the domain types serialize as base64 strings (because p[:] is []byte). When false (production), they serialize as integer arrays matching Rust. This flag is used in post_vproof_test.go for testing PoSt proof serialization but is set to false in production. While not the bug itself, this flag reveals that the developers were aware of potential format differences and had experimented with alternative serialization strategies.

The Round-Trip Test Gap

The existing round-trip test in porep_vproof_test.go unmarshals Rust JSON into Commit1OutRaw, re-marshals with Go, and compares the two using cmp.Diff on deserialized map[string]interface{} representations. However, this test only covers 2 KiB sectors (StackedDrg2KiBV1_1), which use SectorShape2KiB = LCTree<..., U8, U0, U0>. This shape produces only ProofData::Single variants. For larger sector sizes like 32 GiB (SectorShape32GiB = LCTree<..., U8, U8, U2>), the ProofData::Top variant is used, which has additional sub_proof and top_proof fields with more complex nested structures. The round-trip test does not cover this case, leaving a gap in test coverage for the exact proof types that are failing in production.

The Bincode Decode Test

The assistant also examined porep_vproof_bin_test.go, which decodes a Commit1OutRaw from bincode format, marshals it to JSON with Go, and passes it to Rust's SealCommitPhase2 via the FFI (cgo.SealCommitPhase2). This test presumably passes (it is guarded by EXPENSIVE_TESTS), demonstrating that Go-produced JSON IS compatible with the Rust SealCommitPhase2 function when called through the FFI. However, the CuZK path uses a different Rust entry point — serde_json::from_slice in the CuZK server — which may have different deserialization behavior or error handling.

Conclusion: Structural Compatibility, but Subtle Risks Remain

After exhaustive analysis of all type definitions, the assistant concluded that the JSON type definitions are structurally compatible between Rust and Go. Field names match (all use snake_case via json tags), value types produce the same JSON representations, and nested structures are correctly mapped. The assistant identified several potential risk areas:

  1. No UnmarshalJSON for domain types — while Go's default deserialization works for integer arrays, the lack of custom deserialization makes the Go types fragile against format changes.
  2. The testMarshal flag — while not the bug, it indicates awareness of format fragility and could accidentally be left enabled in production.
  3. Round-trip test coverage gap — only 2 KiB sectors are tested, leaving larger sector sizes (which use different ProofData variants) untested for JSON round-trip compatibility.
  4. Two Go round-trips in the failing path — the proof data goes through json.Unmarshal then json.Marshal twice (once for proofshare wrapping, once for the inner JSON), potentially amplifying any subtle data corruption. The investigation ultimately ruled out a simple type-definition mismatch as the root cause, narrowing the search to more subtle issues such as numeric precision loss during intermediate map[string]interface{} deserialization, data corruption during proofshare transmission, or differences in how the CuZK Rust server handles the Go-produced JSON compared to the FFI path. This systematic approach — tracing every type across language boundaries, verifying each serialization behavior, and identifying gaps in test coverage — exemplifies the methodical debugging required for cross-language distributed systems.## References [1] "The JSON Serialization Gap: Diagnosing Cross-Language Proof Failures in CuZK" — Article analyzing the user's initial investigation request at [msg 0]. [2] "Tracing the Rust Type Definitions: A Pivotal Discovery in the CuZK PoRep JSON Serialization Investigation" — Article on locating Rust type definitions at [msg 5]. [3] "The Hunt for Rust Source: Tracing a JSON Serialization Mismatch Across Language Boundaries" — Article on the Rust source location effort at [msg 2]. [4] "Tracing the Serde Serialization of PoseidonDomain and Sha256Domain: A Critical Link in the Rust-Go JSON Format Investigation" — Article on domain type serialization at [msg 10]. [5] "The Serde Derive Revelation: Tracing a JSON Serialization Mismatch Between Rust and Go in Filecoin Proofs" — Article on serde derive behavior at [msg 11]. [6] "The PhantomData Revelation: Tracing JSON Serialization Mismatches Across Rust and Go in Filecoin Proof Systems" — Article on PhantomData handling at [msg 32]. [7] "The Serde Enum Trap: Tracing a JSON Serialization Mismatch Between Rust and Go in Filecoin Proofs" — Article on enum serialization at [msg 28]. [8] "The Bincode Discovery: A Pivot Point in the PoRep Serialization Investigation" — Article on the bincode decode test at [msg 37]. [9] "Tracing the Failing Path: A Single Read That Revealed the Data Flow Bug" — Article on the failing code path at [msg 16]. [10] "The Critical Clue: Reading ProofData in the Serialization Mismatch Investigation" — Article on ProofData analysis at [msg 19].