The Moment of Discovery: Uncovering a Hardcoded Web UI in a Distributed S3 Architecture

Introduction

In the midst of debugging a complex distributed S3 storage cluster, a single message from an AI assistant represents a pivotal moment of discovery—a point where assumptions collapse and reality asserts itself. Message 231 in this coding session captures the instant when the assistant, after being challenged by the user, digs into the source code and finds that the web UI server for the Kuri storage nodes is hardcoded to port 9010, not configurable via command-line flags as previously assumed. This discovery unravels a chain of incorrect configurations and forces a fundamental rethinking of how the test cluster's services interact.

The Message

The message reads in full:

## Agent Reasoning Found it! The web UI is hardcoded to start on port 9010 in kuboribs.go line 164. Let me check that file to see the full context. [read] /home/theuser/gw/integrations/kuri/ribsplugin/kuboribs.go <file> 00156| ri.Lc.Append(fx.Hook{ 00157| OnStop: func(ctx context.Context) error { 00158| return r.Close() 00159| }, 00160| }) 00161| 00162| if ri.H != nil || true { 00163| go func() { 00164| if err := web.Serve(context.TODO(), ":9010", r); err != nil { 00165| panic("ribsweb serve failed") 00166| } 00167| }() 00168| _, _ = fmt.Fprintf(os.Stderr, "RIBSWeb at http://%s\n", "127.0.0.1:9010") 00169| } 00170| 00171| return r, nil 00172| } 00173| 00174| func ribsBlocks...

The tone is one of triumph—"Found it!"—but the content reveals a deeper story about the dangers of assumption-driven development.

Context: The Port Conflict Problem

To understand why this message matters, we need to trace the conversation that led to it. The assistant had been building a Docker Compose-based test cluster for a horizontally scalable S3 architecture. The architecture involved three layers: stateless S3 frontend proxies, Kuri storage backend nodes, and a shared YugabyteDB database. The cluster included a monitoring web UI exposed on port 9010.

The user raised a critical question in message 202: "Is the :9010 cluster mon not going to conflict with :9010 on individual kuri nodes?" This was an astute observation—if every Kuri node also tries to bind to port 9010 for its own web UI, there would be a port conflict when running under Docker Compose, even if the ports weren't all mapped to the host.

The assistant's initial response (message 203-212) was to "fix" this by adding --webui=:0 to the Kuri node commands in docker-compose.yml, assuming this flag would disable the web UI on individual nodes. The assistant also added --s3-api flags, assuming those were real command-line parameters.

But the user wasn't satisfied. In message 213, they pushed back: "are you sure --s3-api and --webui are real flags on kuri daemon?" This question set off a chain of investigation that would reveal the truth.

The Investigation

What follows is a beautiful example of forensic software archaeology. The assistant begins searching for how the kuri daemon actually works:

1. Searching for command definitions (messages 214-223): The assistant searches for kuri daemon, kuri init, and looks for Cobra/flag definitions. The search leads to main.go in the kuri command directory, which reveals that kuri is actually a Kubo (IPFS) wrapper—it uses the standard IPFS command structure, not custom flags.
2. Checking the main docker-compose.yml (message 224): The assistant looks at the project's main docker-compose.yml to see how ports are actually configured. The command is simply ./kuri init ; ./kuri daemon—no --s3-api or --webui flags.
3. Finding the S3 API configuration (message 225): A grep for S3API reveals that the S3 API port is configured via the environment variable RIBS_S3API_BINDADDR (default :8078), not a command-line flag.
4. Searching for web UI configuration (messages 226-230): The assistant searches for how the web UI is started. This leads to ribsweb.go and eventually to kuboribs.go, where the web UI server is started.

The Discovery

Message 231 is the payoff. The assistant reads kuboribs.go and finds line 164:

if err := web.Serve(context.TODO(), ":9010", r); err != nil {
    panic("ribsweb serve failed")
}

The web UI is hardcoded to port 9010. There is no environment variable, no configuration option, no command-line flag—just a literal string ":9010" passed directly to web.Serve. The --webui=:0 fix the assistant had applied was completely ineffective because the flag doesn't exist.

Why This Matters

This message is significant for several reasons:

1. The Assumption Trap

The assistant made a classic developer mistake: assuming that because a feature exists (a web UI), it must be configurable via command-line flags. This assumption felt natural—many services use --port or --bind flags. But the Kuri daemon is a Kubo wrapper, and its configuration is handled through environment variables and the Kubo plugin system, not custom CLI flags. The assistant projected a familiar interface pattern onto an unfamiliar system.

2. The Value of Skepticism

The user's simple question—"are you sure those are real flags?"—demonstrates the value of healthy skepticism in debugging. Rather than accepting the assistant's fix, the user challenged the underlying assumption. This is a textbook example of the "rubber duck debugging" principle: sometimes the most valuable contribution is asking "are you sure?"

3. The Architecture Implication

The hardcoded port 9010 means every Kuri node in the cluster will attempt to start a web UI on port 9010. In Docker Compose, this isn't a direct conflict (each container has its own network namespace), but it means:

• The cluster monitoring UI (also on 9010) can't simply be mapped to host port 9010 without conflicting with individual node web UIs if those are also exposed.
• If you want to access individual node web UIs for debugging, you need to map them to different host ports.
• The web UI cannot be disabled through configuration—it's always started (note the || true condition on line 162).

4. The Thinking Process

The assistant's reasoning in this message is worth examining. The phrase "Found it!" conveys the excitement of discovery after a search. The assistant then immediately reads the file to get full context—a good practice of verifying the finding before acting on it. The message doesn't jump to conclusions about what this means for the docker-compose configuration; it simply presents the evidence.

Input Knowledge Required

To fully understand this message, one needs:

• Knowledge of the project structure: Understanding that kuri is a Kubo wrapper, that it uses the IPFS plugin system, and that kuboribs.go is where the RIBS plugin is initialized.
• Understanding of Docker networking: Knowing that each container has its own network namespace, so port 9010 in one container doesn't conflict with port 9010 in another unless ports are mapped to the host.
• Familiarity with Go patterns: Recognizing fx.Hook, fx.Invoke, and the Uber FX dependency injection framework used in the codebase.
• Context of the debugging session: Knowing that the assistant had been building a test cluster and had already applied a fix based on incorrect assumptions.

Output Knowledge Created

This message creates several important pieces of knowledge:

1. The web UI port is hardcoded: It cannot be changed via environment variables or command-line flags without modifying the source code.
2. The previous fix was ineffective: The --webui=:0 flag added to docker-compose.yml does nothing because the kuri daemon doesn't recognize it.
3. A new approach is needed: The assistant must now find another way to handle the port situation—perhaps by not exposing individual node web UIs, or by modifying the source code to make the port configurable.
4. The S3 API is also environment-variable-driven: The RIBS_S3API_BINDADDR variable controls the S3 API port, not --s3-api.

Mistakes and Incorrect Assumptions

The most significant mistake was the assumption about command-line flags. The assistant had written --webui=:0 and --s3-api=:8078 in the docker-compose.yml without verifying that these flags existed. This is a form of "cargo cult programming"—applying a pattern that works in other contexts without understanding whether it applies here.

A secondary issue is that the assistant didn't check the source code before applying the fix. The user's challenge was necessary to trigger the investigation. In a well-functioning development process, verifying configuration options against the actual codebase should precede configuration changes.

Conclusion

Message 231 is a microcosm of the debugging process itself: assumption, challenge, investigation, discovery. It captures the moment when the ground shifts beneath a developer's feet—when something you thought was true turns out to be false, and you have to rebuild your mental model from the evidence. The hardcoded ":9010" on line 164 of kuboribs.go is a small detail, but its discovery ripples through the entire test cluster configuration, forcing a redesign of how the cluster's services are exposed and monitored. It's a reminder that in complex systems, the most dangerous assumptions are the ones you don't know you're making.