The Research-Driven Approach: Mapping the Filecoin Proofs API Surface for Phase 1 Integration

Introduction

In the development of complex distributed systems, the most critical decisions are often made not during coding, but during the research that precedes it. Message 296 of the cuzk proving engine conversation captures this exact moment: the assistant, having just completed the protobuf and type system updates for Phase 1, pauses to investigate the precise API surface of the filecoin-proofs-api library before writing the actual proving function implementations. This message is a research probe — a deliberate, structured inquiry into the import paths, function signatures, and type discriminants needed to wire up WinningPoSt, WindowPoSt, and SnapDeals proof generation in the cuzk daemon.

The message itself is deceptively simple:

Now let me check what types we need to import from filecoin-proofs-api for the PoSt and update functions: [task] {"description":"Find PoSt/Update API imports","prompt":"I need to find the exact import paths for types and functions we need from filecoin-proofs-api version 19.0.0. Search in ~/.cargo/registry/src/ for the filecoin-proofs-api-19.0.0 crate.\n\nI need to know:\n> 1. The pub use re-exports in `...\n> task_id: ses_3939d341fffeqdmAohEkW8qQv3

Beneath this straightforward surface lies a wealth of engineering reasoning, architectural decision-making, and domain knowledge that merits deep examination.

Why This Message Was Written: The Motivation and Context

To understand why this message exists, we must reconstruct the assistant's position in the workflow at this moment. The cuzk proving engine project was in the midst of Phase 1 implementation, which called for expanding the daemon from a PoRep-only proof server to one supporting all four Filecoin proof types: PoRep (already implemented), WinningPoSt, WindowPoSt, and SnapDeals. The assistant had already completed several preparatory steps:

Reading every source file in the extern/cuzk/ workspace (msg 286) to understand the existing architecture.
Exploring the filecoin-proofs-api signatures (msg 287) to get a broad overview of the available functions.
Studying Curio's Go FFI layer (msg 288) to understand how vanilla proofs are serialized for each proof type in the upstream system.
Updating the protobuf definition (msg 292) to add repeated bytes vanilla_proofs for multi-proof support.
Updating types.rs (msg 294) to add the vanilla_proofs field and rename SnapDeals commitment fields. At this point, the assistant was ready to write the actual proving functions in prover.rs. But instead of diving directly into implementation, it launched this research task. The motivation is clear: the cost of an incorrect import or wrong function signature is high. A single type mismatch, a wrong enum discriminant, or a misnamed function would cause a compilation failure — or worse, a runtime panic in the production proving pipeline. The assistant was practicing defensive engineering: verify the API surface before writing code that depends on it. This is particularly important because the filecoin-proofs-api crate is a complex, multi-layered library. It re-exports types from filecoin_proofs_v1, which in turn depends on filecoin-proofs. The function signatures involve deeply nested types like SectorUpdateProofInputs, PoStConfig, and RegisteredUpdateProof — each with their own serialization requirements and FFI compatibility concerns. Getting the import paths wrong would not just be a compile error; it could silently produce incorrect proofs or corrupt the daemon's state.

The Research Task Structure: A Model of Systematic Inquiry

The task prompt embedded in this message is notable for its precision. The assistant specifies exactly what it needs:

The pub use re-exports in lib.rs — understanding the public API surface is essential because filecoin-proofs-api acts as a facade over multiple underlying crates. The assistant needs to know which types are publicly accessible and under which module paths.
Function signatures for the three proving functions — generate_winning_post_with_vanilla, generate_single_window_post_with_vanilla, and generate_empty_sector_update_proof_with_vanilla. Each of these functions has a distinct parameter list involving sector IDs, challenge seeds, prover IDs, commitment arrays, and vanilla proof vectors. The assistant needs the exact type signatures to construct the correct calls.
The RegisteredUpdateProof enum and its discriminant values — this is a critical detail for SnapDeals proof generation. The RegisteredUpdateProof enum maps proof types to numeric identifiers used in the C FFI layer. The assistant needs to know the exact numeric values (0, 1, 2, 3, 4) and whether they are assigned via #[repr(u64)] or through default Rust enum discriminant ordering.
How to extract bytes from EmptySectorUpdateProof — the SnapDeals proof function returns an EmptySectorUpdateProof struct, which wraps a Vec<u8>. The assistant needs to know how to serialize this back over the gRPC response. The task result reveals a particularly subtle finding: RegisteredUpdateProof uses self as u64 in its porep_id() method, meaning it derives discriminant values from default Rust enum ordering rather than an explicit #[repr(u64)] attribute. This is the kind of detail that could easily be missed in a cursory reading of the documentation, but it has real consequences for FFI compatibility. If the assistant had assumed explicit #[repr(u64)] annotations, it might have written code that silently produced wrong proof type identifiers.

Assumptions and Their Validation

Every research task rests on assumptions, and this message is no exception. The assistant makes several implicit assumptions:

Assumption 1: The API surface is discoverable through source code inspection. The assistant assumes that reading the filecoin-proofs-api source files in the Cargo registry will reveal the complete public API. This is a reasonable assumption for an open-source Rust crate, but it depends on the crate being installed in the local registry cache. The task searches ~/.cargo/registry/src/ for the specific version 19.0.0.

Assumption 2: The function signatures are stable and match the version in use. The assistant assumes that the filecoin-proofs-api version 19.0.0 installed in the registry matches the version that Curio's build system actually uses. This is validated by the fact that the workspace already compiles against this version, but it's worth noting that API drift between versions could introduce subtle incompatibilities.

Assumption 3: The RegisteredUpdateProof enum's discriminant values are discoverable through static analysis. The assistant correctly assumes that examining the porep_id() method implementation will reveal the numeric values. The task result confirms this, showing that the enum uses self as u64 which derives values from the default ordering.

Assumption 4: The serialization format for vanilla proofs in the gRPC layer matches what filecoin-proofs-api expects. This assumption was validated in the earlier research (msg 288) where the assistant studied Curio's Go FFI layer. The Go code serializes each sector's vanilla proof as a byte slice and passes them as a vector, which matches the repeated bytes protobuf field added in msg 292.

One potential incorrect assumption is worth examining: the assistant assumes that the RegisteredUpdateProof enum values (0, 1, 2, 3, 4) map directly to the C FFI enum used in filecoin-proofs. The task result notes that there is no #[repr(u64)] or From<u64> implementation, which means the Rust enum's discriminant values are used directly. However, the C FFI side might use a different numbering scheme. The assistant would need to verify this by examining the C header files or the FFI bridge code — a detail that may be deferred to a later validation step.

Input Knowledge Required

To fully understand this message, one needs knowledge spanning several domains:

Rust crate architecture: Understanding how filecoin-proofs-api re-exports types from filecoin_proofs_v1 and filecoin-proofs requires familiarity with Rust's module system and the common pattern of facade crates in large projects. The assistant needs to know that pub use statements in lib.rs define the public API surface, and that types may be re-exported under different paths than their original module locations.

Filecoin proof types: The message assumes knowledge of the four Filecoin proof types — PoRep (Proof of Replication), WinningPoSt (Proof of Spacetime for winning blocks), WindowPoSt (Proof of Spacetime for windowed sectors), and SnapDeals (empty sector update proofs). Each has different proving requirements: PoSt proofs involve multiple vanilla proofs (one per sector), while SnapDeals requires three commitment CIDs (commD, commR, commRLast).

FFI and enum representation: The assistant's concern about RegisteredUpdateProof discriminant values reflects an understanding of how Rust enums are represented in C FFI boundaries. Without explicit #[repr] annotations, Rust enums use a compiler-chosen layout that may not be compatible with C's expectations. The self as u64 cast in porep_id() is a deliberate choice to produce a stable numeric value, but it relies on the default discriminant ordering being consistent across compilations.

Protobuf and gRPC serialization: The assistant's earlier work on the protobuf definition (msg 292) shows an understanding of how repeated bytes maps to Vec<Vec<u8>> in Rust, and how this serialization format must align with what the upstream Go FFI layer produces.

Output Knowledge Created

This message produces several valuable pieces of knowledge:

Explicit API surface documentation: The task result provides a complete reference of the public types and functions available in filecoin-proofs-api 19.0.0, including the exact module paths for each type. This serves as a living documentation artifact that the assistant can reference while writing the proving functions.

Validation of enum discriminant assumptions: The finding that RegisteredUpdateProof uses self as u64 with default discriminant ordering (0, 1, 2, 3, 4) confirms that the numeric values match the expected C FFI enum. This is critical for SnapDeals proof generation, where the RegisteredUpdateProof value is passed through the FFI boundary.

Confirmation of serialization patterns: The task result confirms that EmptySectorUpdateProof wraps a Vec<u8> and can be extracted via its inner field, validating the approach of serializing the proof bytes directly into the gRPC response.

Risk identification: The absence of #[repr(u64)] or From<u64> on RegisteredUpdateProof flags a potential risk area. If the C FFI side expects a different numeric mapping, the proof type identifier could be silently wrong. This risk is now documented and can be addressed in testing.

The Thinking Process: Methodical and Defensive

The thinking process visible in this message reveals a methodical, research-driven approach to software engineering. The assistant does not assume it knows the API surface from prior context or documentation. Instead, it explicitly verifies every detail before writing code that depends on it.

The sequence of research tasks is revealing: msg 286 reads the entire cuzk workspace, msg 287 explores the filecoin-proofs-api signatures broadly, msg 288 studies the Curio Go FFI layer, and msg 296 drills into the specific import paths and enum discriminants. Each task narrows the focus from the general to the specific, building a pyramid of validated knowledge before any code is written.

This approach reflects an understanding that in distributed proving systems, the cost of a bug is not just a failed compilation — it's a corrupted proof, a stalled mining operation, or a lost block reward. The assistant is treating the codebase with the respect it deserves, recognizing that the FFI boundary between Rust, C++, and CUDA is where the most insidious bugs live.

The message also demonstrates a key principle of effective AI-assisted development: decompose the problem into verifiable units. Rather than attempting to write all three proving functions in one shot and hoping they compile, the assistant breaks the work into: (1) understand the API surface, (2) update the protobuf, (3) update the types, (4) write the proving functions, (5) update the scheduler, (6) compile and test. Each step produces a verifiable artifact — a task result, a compiled binary, a passing test — that validates the assumptions made in the previous step.

Conclusion

Message 296 is a small but crucial node in the larger network of the cuzk proving engine implementation. It represents the moment when research transitions into implementation, when the abstract knowledge of API documentation becomes concrete, validated understanding. The assistant's decision to pause and verify the import paths and enum discriminants before writing code is not hesitation — it is the hallmark of a disciplined engineer who understands that in complex systems, the most expensive bugs are the ones that pass compilation and fail silently in production.

This message teaches us that effective software development is as much about asking the right questions as it is about writing the right code. By systematically mapping the API surface, validating assumptions about enum representations, and documenting the findings for future reference, the assistant builds a foundation of verified knowledge that makes the subsequent implementation not just faster, but safer. In the world of Filecoin proof generation, where a single wrong byte can invalidate an entire proof, this research-first approach is not a luxury — it is a necessity.