The Art of the Single-Line Edit: Adding an Eviction Callback to an ARC Cache

The Message

[assistant] ## Agent Reasoning
Let me add a SetEvictionCallback method after the NewARCCache function.
[edit] /home/theuser/gw/rbcache/arc.go
Edit applied successfully.

Introduction

At first glance, the message above appears to be one of the most mundane moments in any software engineering session: a developer decides to add a method to a struct and applies the edit. It is a single line of reasoning followed by a tool invocation that reports success. Yet this message sits at a critical inflection point in a much larger implementation effort—the wiring of an L1-to-L2 cache promotion chain for a distributed storage system's retrieval provider. Understanding why this particular method addition matters, what assumptions it carries, and what knowledge it both requires and produces reveals the hidden complexity behind even the simplest-seeming edits in a complex distributed systems codebase.

This article examines message index 2579 from a coding session focused on closing critical implementation gaps in the Filecoin Gateway (FGW) project, a horizontally scalable S3-compatible storage system built on top of YugabyteDB and IPFS technologies.

Context: The Cache Hierarchy Problem

The message occurs during a sustained effort to implement what the session's analyzer summary calls "pragmatic gap-filling"—fixing blocking issues without over-engineering. The assistant had already completed a proper Prefetcher.Fetch() implementation in retr_provider.go and had just marked that task as done in its todo list. The next item, marked "in_progress," was "Add L1→L2 cache promotion callback."

The retrieval provider in this system uses a multi-tier cache architecture. L1 is an in-memory Adaptive Replacement Cache (ARC) that stores recently fetched block data for fast access. L2 is an SSD-backed cache that provides a larger, slower storage tier. When the L1 cache reaches its capacity limit and must evict entries, those entries contain valuable block data that could still be useful—just not frequently enough to justify keeping them in expensive memory. The obvious optimization is to promote evicted L1 entries into the L2 cache rather than discarding them entirely.

This is the classic "cache hierarchy" pattern used in CPU architectures (L1→L2→L3→RAM→disk) and in storage systems like Redis with its memory-to-disk persistence. The implementation gap was that the ARC cache had no mechanism whatsoever for notifying external consumers when entries were evicted. The evictFromT1() and evictFromT2() methods simply removed entries from the T1/T2 lists, added ghost entries to the B1/B2 ghost lists (used for ARC's adaptive algorithm), and updated internal counters. Any data that was evicted was gone forever from the system's perspective.

Why This Message Was Written

The reasoning behind this message is straightforward but sits atop a pyramid of prior decisions. The assistant had already:

  1. Identified the gap: The todo list explicitly called out "Add L1→L2 cache promotion callback" as a high-priority item. This came from a comprehensive analysis (referenced in the chunk summary as "the earlier comprehensive analysis") that identified critical implementation gaps across the codebase.
  2. Chosen the ARC cache as the target: The retrieval provider supports two L1 cache policies—ARC and legacy LRU. The assistant had already decided to modify the ARC implementation because it is the primary/default policy. The legacy LRU path would not benefit from this change.
  3. Determined the mechanism: Rather than building a complex event system or observer pattern, the assistant opted for a simple callback function stored as a field on the ARCCache struct. This is a Go-idiomatic approach: a function value (closure) that gets invoked when evictions occur.
  4. Planned the wiring: The callback would be set after both L1 and L2 caches are initialized, because the callback needs to reference the L2 cache's Put method. This ordering constraint is critical—you cannot wire the callback until the L2 cache exists. The specific message—"Let me add a SetEvictionCallback method after the NewARCCache function"—represents the decision to implement this mechanism. The assistant had already read the arc.go file (message 2573), examined the eviction methods (messages 2574-2575), and applied a preliminary edit to add the callback field to the struct (message 2576). Now it was adding the public API method that external code would use to register the callback.

The Thinking Process

The assistant's reasoning, visible in the "Agent Reasoning" block, reveals a methodical, code-reading-driven approach. The chain of thought proceeds as follows:

Step 1: Locate the eviction points. The assistant greps for "evict|Evict" in arc.go and finds the evictFromT1() and evictFromT2() methods, plus the ghost list tracking that already exists. This confirms where data leaves the cache.

Step 2: Read the eviction methods. By reading evictFromT1() (message 2575), the assistant sees the exact sequence: get the back element of T1, remove it from the list, delete from the key map, decrement the size counter, and add a ghost entry to B1. This is where the callback invocation must be inserted.

Step 3: Add the callback field. In message 2576, the assistant adds an evictionCallback field to the ARCCache struct. The type of this field is a function that takes a key and value—func(K, V)—which matches the signature needed for L2 promotion.

Step 4: Add the SetEvictionCallback method. This is message 2579, the target of our analysis. The method provides a public, thread-safe way to set the callback after construction. The assistant places it "after the NewARCCache function" in the source file, following Go conventions where constructors are followed by public API methods.

Step 5: Modify the eviction methods. In messages 2580-2581, the assistant updates evictFromT1() and evictFromT2() to call c.evictionCallback(entry.key, entry.value) when the callback is non-nil.

Step 6: Wire it up in the retrieval provider. In messages 2582-2585, the assistant reads the L1 cache initialization code in retr_provider.go, identifies where the L2 cache is created, and adds the callback wiring after both caches exist. The callback is a lambda that calls l2Cache.Put(key, value).

Assumptions Made

Several assumptions underpin this seemingly simple edit:

Assumption 1: The callback should be optional. The assistant chose to make the callback field nil by default and check for nil before invocation. This assumes that not all consumers of ARCCache need eviction notifications. This is a safe, backward-compatible design choice.

Assumption 2: The callback signature matches the L2 cache's Put method. The ARC cache stores [K, V] pairs. The L2 cache (also a generic cache) likely has a Put(K, V) method with the same key/value types. The assistant assumes the types are compatible without checking the L2 cache's interface explicitly in this message—though earlier reading confirmed the pattern.

Assumption 3: No locking is needed for the callback invocation. The ARC cache already uses a mutex (c.mu) for thread safety. The eviction methods are called while holding this lock. The assistant assumes that invoking the callback within the locked context is safe—that the callback (which calls l2Cache.Put) won't deadlock or cause excessive lock contention. This is a reasonable assumption for a cache-to-cache promotion where the L2 cache has its own locking.

Assumption 4: The callback should be set after construction, not during. By adding a SetEvictionCallback method rather than a constructor parameter, the assistant assumes that the callback may not be known at construction time, or that it depends on objects (like the L2 cache) that are created later. This matches the actual initialization order in retr_provider.go.

Assumption 5: The ARC cache's eviction path is the only path where data leaves L1. This is correct for the ARC implementation—data is only removed from T1/T2 via the eviction methods. But it's worth noting that there's no explicit "remove" or "invalidate" API on the ARC cache that might also need to trigger the callback. The assistant implicitly assumes that explicit removals (if they existed) should not trigger promotion, only evictions due to capacity pressure.

Input Knowledge Required

To understand and evaluate this message, a reader needs:

  1. Go language proficiency: Understanding of generics (ARCCache[K, V]), function types as fields, and the concept of closures/lambdas for callbacks.
  2. ARC cache algorithm knowledge: Understanding that ARC maintains four lists (T1, T2, B1, B2) and that eviction from T1/T2 is the mechanism by which the cache enforces its capacity limit. Ghost entries (B1/B2) track recently evicted items for adaptation but do not store data.
  3. Distributed storage architecture awareness: Understanding of multi-tier caching (memory → SSD → network) and why promoting evicted entries to a slower tier is beneficial. The L1 cache holds hot data in RAM; the L2 cache holds warm data on SSD; the network retrieval is the cold path.
  4. The FGW project's codebase structure: Knowledge that rbcache/arc.go contains the ARC implementation, that rbdeal/retr_provider.go contains the retrieval provider that owns both L1 and L2 caches, and that the initialization order matters.
  5. The session's task prioritization: Understanding that this work is part of closing "critical implementation gaps" identified by a prior analysis, and that the assistant is working through a todo list in priority order.

Output Knowledge Created

This message and its surrounding edits produce several forms of knowledge:

  1. A new public API: The SetEvictionCallback method on ARCCache becomes part of the cache's interface. Future developers can use it to hook into eviction events for logging, metrics, or data migration.
  2. A cache promotion pipeline: The wiring in retr_provider.go creates a data flow where L1 evictions automatically feed into L2. This closes a performance gap where previously evicted L1 data was lost and had to be re-fetched from the network.
  3. A pattern for extension: The callback mechanism demonstrates how to extend the ARC cache without modifying its core algorithm. This pattern can be reused for other hooks (e.g., pre-fetch triggers, metrics collection, audit logging).
  4. Documentation of design intent: The code now explicitly documents that L1 evictions should promote to L2. This intent was previously implicit or absent.
  5. A testable interface: With the callback in place, tests can verify that evictions trigger the expected behavior, enabling better coverage of the cache hierarchy.

Potential Mistakes and Considerations

While the implementation is sound, several considerations merit examination:

The callback is invoked under lock. The ARC cache's mutex is held during eviction. If the L2 cache's Put method is slow (e.g., due to SSD write latency or internal locking), it could delay the ARC eviction path and increase contention. A production system might want to invoke the callback asynchronously or with a timeout.

No error handling. The callback signature func(K, V) returns nothing. If the L2 cache's Put fails (e.g., disk full, write error), the error is silently swallowed. The data is lost from both L1 and L2. A more robust design might log errors or implement a retry mechanism.

No ordering guarantee. The ARC algorithm evicts from T1 or T2 based on its adaptive policy. The callback receives evicted items in eviction order, which is not necessarily the order of least-recent-use or least-frequent-use. The L2 cache receives items in whatever order ARC decides to evict them.

The legacy LRU path is untouched. If the retrieval provider uses the legacy LRU cache (the else branch in the initialization code), evictions from L1 will not promote to L2. This creates a behavioral difference between the two cache policies that may be surprising.

No eviction from L2. The implementation only handles L1→L2 promotion. There is no corresponding mechanism for L2 evictions (e.g., promoting to a network-level cache or triggering re-fetch). The cache hierarchy is incomplete at the bottom.

Conclusion

Message 2579 is a study in how the most significant architectural changes often begin with the simplest-seeming edits. A single line of reasoning—"Let me add a SetEvictionCallback method"—encapsulates hours of prior analysis, dozens of file reads, and a clear understanding of cache hierarchy design. The assistant's methodical approach, reading code before editing and reasoning about ordering constraints, demonstrates the discipline required to modify complex distributed systems safely.

The L1→L2 cache promotion callback may appear to be a small feature, but it closes a critical gap in the data lifecycle. Without it, every L1 eviction was a data loss event that forced a costly network re-fetch. With it, the cache hierarchy operates as a true multi-tier system, keeping warm data accessible on SSD even after it falls out of the hot memory cache. This is the essence of pragmatic gap-filling: identifying the missing link in an existing architecture and adding just enough code to complete the chain, without over-engineering or premature optimization.