Orchestrating Systematic Test Coverage: How Subagents Mapped a Distributed Storage Codebase

Introduction

In the middle of a high-velocity development session on a distributed Filecoin storage gateway, a single message stands out as a turning point from reactive firefighting to proactive quality assurance. After successfully deploying a fallback provider mechanism to fix a production outage where CIDgravity's API returned zero storage providers, the assistant received a simple instruction from the user: "Start subagents to investigate state of unit and integration tests, come up with a plan to cover all subsystems."

The response—message index 2418 in this conversation—is not a typical developer reply. It is an orchestrated, multi-agent investigation that produces three comprehensive reports spanning the entire Go codebase. This article examines that message in depth: why it was written, how the decisions embedded within it were made, what assumptions underpinned the analysis, and what knowledge was both consumed and produced. By dissecting this single interaction, we gain insight into how modern AI-assisted development handles systematic quality improvement at scale.

The Subject Message: A Multi-Agent Investigation

The message begins with a straightforward declaration:

"I'll launch subagents to investigate the current state of unit and integration tests across the codebase."

What follows is not a single analysis but three parallel investigations, each dispatched as a structured task to a subagent. The tasks are:

  1. Analyze existing unit tests — Find all *_test.go files, summarize what each covers, identify untested subsystems, and estimate coverage percentages.
  2. Analyze integration tests — Look for integration test files, Docker Compose test infrastructure, end-to-end S3 tests, Ansible playbook testing, and CI/CD configuration.
  3. Map all subsystems/packages — Create a comprehensive map of every package, its purpose, key functions, and critical paths that need testing. Each task is followed by the compiled report from that subagent. Together, they form a complete picture of the codebase's testing posture.

Why This Message Was Written: The Context and Motivation

To understand why this message exists, we must look at what happened immediately before it. The session had been deep in production debugging. The CIDgravity "Get Best Available Providers" (GBAP) API had returned NO_PROVIDERS_AVAILABLE, causing deal-making to stall entirely. The assistant diagnosed the issue, implemented a configurable fallback provider mechanism (RIBS_DEAL_FALLBACK_PROVIDERS), deployed it to the QA cluster, and verified that three of four fallback providers immediately accepted deals. The fix was committed as ba62e5b.

With the production fire extinguished, the user pivoted to a strategic concern: test coverage. The instruction to "come up with a plan to cover all subsystems" reveals a recognition that the codebase had grown complex—spanning deal-making, S3 proxying, caching, garbage collection, wallet management, and more—without commensurate test coverage. The user wanted a systematic understanding of what existed and what was missing before diving into writing tests.

The assistant's response reflects this strategic shift. Rather than guessing or providing a superficial overview, it launches a thorough, data-driven investigation. The message is the embodiment of "measure before you improve."

How Decisions Were Made: The Architecture of the Investigation

The most striking feature of this message is its use of subagents. The assistant does not attempt to manually grep through the codebase, read every test file, and compile a report. Instead, it decomposes the problem into three well-defined tasks and dispatches each to an autonomous subagent.

Task Decomposition

The decision to split the investigation into three parallel streams is itself a design choice. Each task has a distinct focus:

The Subagent Task Format

Each task is specified as a JSON object with description and prompt fields. The prompts are detailed and prescriptive. For example, the unit test analysis task instructs the subagent to:

  1. Find all existing test files and list them with locations
  2. Summarize what each covers
  3. Identify untested subsystems
  4. Estimate coverage percentages The integration test task similarly enumerates specific things to look for: Docker Compose files, testcontainers usage, S3 endpoint tests, Ansible testing, and CI/CD workflows. The subsystem mapping task asks for package paths, key functions, and critical testing paths. This structured task design reveals a key assumption: that subagents can reliably execute file system exploration and code analysis without hallucinating or missing important details. The assistant trusts the subagent architecture to produce accurate, comprehensive results.

Assumptions Embedded in the Message

Every analysis rests on assumptions, and this message is no exception.

Assumption 1: The Codebase Follows Go Conventions

The subagents are instructed to find *_test.go files, which is the standard Go test file naming convention. The assistant assumes that all tests in the project follow this pattern. If any tests were named differently (e.g., *_test.go is enforced by Go tooling, so this is a safe assumption), they would be missed.

Assumption 2: Test Files Are Co-located with Source Files

The investigation looks for test files within each package directory. This is the standard Go convention, but the assistant does not explicitly verify that no centralized test directory exists. The assumption holds in this codebase.

Assumption 3: Coverage Can Be Estimated from File Presence Alone

The unit test report estimates coverage percentages based on which packages have test files and which do not. This is a coarse metric—a package might have a test file that only covers 5% of its code, or conversely, a single test file might comprehensively test multiple packages. The assistant acknowledges this limitation implicitly by also requesting a subsystem map, which provides structural context.

Assumption 4: The Subagent Architecture Is Reliable

This is perhaps the most significant assumption. The assistant launches three subagents and trusts their outputs without cross-validation. In a production setting, one might want to verify the findings manually, but in the context of AI-assisted development, this level of trust is necessary for velocity.

Input Knowledge Required to Understand This Message

A reader needs substantial context to fully grasp what this message accomplishes.

Knowledge of the Codebase Structure

The message references packages like rbdeal/, rbcache/, configuration/, s3gate/, wallet/, and retrieval/. Understanding the reports requires knowing that rbdeal handles Filecoin deal-making, rbcache implements multi-tier caching, s3gate provides an S3-compatible API frontend, and so on. Without this context, the coverage map is just a list of directory names.

Knowledge of Go Testing Conventions

The message uses Go-specific terminology: *_test.go, testify/suite, testcontainers, gomock. A reader unfamiliar with Go testing idioms would struggle to interpret the findings.

Knowledge of the Project's Architecture

The reports reference "RIBS" (the storage node), "CIDgravity" (the deal-making API), "YugabyteDB" (the distributed SQL database), and "Lotus" (the Filecoin node). These are domain-specific concepts from the Filecoin ecosystem. The message assumes the reader (or the user) is already familiar with these components.

Knowledge of Prior Session Context

The message builds on the fallback provider work that preceded it. The user's request to investigate tests came immediately after committing the fallback fix. Understanding why the user asked for this investigation—and why the assistant responded with such thoroughness—requires knowing that the codebase had just undergone a production crisis that was resolved with a configuration-driven workaround.

Output Knowledge Created by This Message

The message produces three distinct knowledge artifacts.

Artifact 1: The Unit Test Coverage Report

This report lists every test file in the codebase, organized by package. It identifies which subsystems have tests (rbcache, rbdeal metrics, configuration loading) and which do not (s3gate handlers, wallet key management, retrieval HTTP client, deal repair logic). It estimates coverage percentages, giving the user a quantitative baseline.

Key finding: The codebase had approximately 164 tests across 8 test files, but large swaths of critical business logic were untested.

Artifact 2: The Integration Test Infrastructure Report

This report maps the end-to-end testing capabilities: Docker Compose files for local clusters, testcontainers-based integration tests for YugabyteDB, Ansible playbook testing with Docker, and GitHub Actions CI/CD workflows. It reveals that while unit tests existed, integration testing was sparse and focused on database connectivity rather than S3 protocol correctness or deal flow validation.

Artifact 3: The Comprehensive Subsystem Map

This is the most valuable output. It enumerates every package, its purpose, key types and functions, and critical testing paths. It covers:

The Thinking Process: What the Message Reveals About Reasoning

Although the message does not contain explicit "thinking" tags (like some other messages in this conversation), the structure of the response reveals a clear reasoning process.

Step 1: Decompose the Problem

The user asked for an investigation of test state and a plan to cover all subsystems. The assistant recognized that this required three distinct analyses: what tests exist (unit), how they run (integration), and what needs testing (subsystem map). Rather than attempting a single monolithic analysis, it decomposed the problem into parallel, focused investigations.

Step 2: Design Effective Prompts

Each subagent prompt is carefully crafted to elicit specific, actionable information. The prompts include explicit instructions to list files, summarize coverage, identify gaps, and estimate percentages. This reflects an understanding that vague prompts produce vague results.

Step 3: Trust but Verify (Implicitly)

The assistant does not cross-validate the subagent outputs, but it does structure the tasks so that they overlap in useful ways. The subsystem map, for example, can be cross-referenced against the unit test report to identify packages that appear in the map but have no corresponding test file. This implicit verification is built into the task design.

Step 4: Compile and Present

The final message compiles the three reports into a coherent narrative. Each report is presented with a clear heading and formatted for readability (tables, bullet points, code blocks). The assistant acts as an editor, curating the subagent outputs into a unified deliverable.

Mistakes and Incorrect Assumptions

No analysis is perfect, and this message contains several limitations worth noting.

Limitation 1: Coverage Estimation Is Coarse

The unit test report estimates coverage as "Low," "Medium," or "High" based on the presence of test files and rough function counts. This is a heuristic, not a measurement. A package with a "High" coverage label might still have critical untested paths. The assistant does not run go test -cover to obtain actual line coverage percentages.

Limitation 2: No Dynamic Analysis

The investigation is entirely static—it looks at file names, reads source code, and infers coverage from structure. It does not run the tests to see which pass, which fail, or which are flaky. It does not check for test dependencies, mocking strategies, or test isolation.

Limitation 3: Integration Test Depth

The integration test report identifies infrastructure (Docker Compose, testcontainers) but does not evaluate the quality or completeness of the integration tests themselves. A test that connects to a database and runs a query is counted as an integration test, even if it only covers a single happy path.

Limitation 4: No Prioritization

The subsystem map lists everything that needs testing but does not prioritize. In a real testing initiative, some subsystems are more critical than others—deal-making logic is more important than a helper function in configuration parsing. The message leaves prioritization as an exercise for the reader (or for a subsequent message).

Conclusion: A Blueprint for Test-Driven Quality

Message 2418 is more than a status report—it is a blueprint. By systematically mapping the codebase, cataloging existing tests, and identifying gaps, the assistant created the foundation for a comprehensive test-writing campaign. The 164 new tests that were subsequently implemented (committed as 5344f33) are a direct outcome of this analysis.

The message exemplifies a mature approach to software quality: measure before you improve, decompose complex problems into parallel investigations, and trust your tools to handle the grunt work while you focus on synthesis and strategy. For any developer or team looking to systematically improve test coverage, this message offers a template worth studying.