The Eviction Callback: A Pivot Point in Multi-Tier Cache Architecture

The Message

The subject of this article is a single message from an AI-assisted coding session, index 2577, in which the assistant writes:

## Agent Reasoning Now I need to add a method to set the eviction callback and modify the evictFromT1 and evictFromT2 methods to call it. [read] /home/theuser/gw/rbcache/arc.go <file> 00101| Name: name + "_misses_total", 00102| Help: "Total cache misses", 00103| }), 00104| evictions: promauto.NewCounter(prometheus.CounterOpts{ 00105| Namespace: "fgw", 00106| Subsystem: "cache", 00107| Name: name + "_evictions_total", 00108| Help: "Total cache evictions", 00109| }), 00110| size: promauto.NewGauge(prometheus.GaugeOpts{ 00111| Namespace: "fgw", 00112| Subsystem: "cache", 00113| Name: name + "_size_bytes", 00114| ...

On its surface, this message is unremarkable: a brief reasoning note followed by a file read. Yet it sits at a critical juncture in the implementation of a multi-tier caching system for a distributed Filecoin gateway. Understanding why this message was written, what assumptions it encodes, and what knowledge it both consumes and produces reveals the intricate decision-making process behind building production-grade infrastructure software.

Context: The Gap Analysis and the Todo List

To understand this message, one must first understand the situation that produced it. The coding session had been running for hours across dozens of messages, working on a horizontally scalable S3-compatible storage system built on top of Filecoin and YugabyteDB. A comprehensive gap analysis had identified several critical unimplemented features, and the assistant was systematically working through them using a prioritized todo list.

At the moment of this message, the todo list showed:

Item 1: Implement RefCounter integration with S3 object operations (pending)
Item 2: Fix Prefetcher Fetch() implementation (completed)
Item 3: Add L1→L2 cache promotion callback (in progress)
Item 4: Wire AccessTracker to retrieval flow (pending)
And several more items The assistant had just completed the Prefetcher Fetch() implementation (item 2) and had moved on to item 3: adding a callback that promotes cache entries from the L1 (in-memory ARC cache) to the L2 (SSD-based persistent cache) when they are evicted. This is a classic multi-tier caching strategy: the fastest, most expensive cache (L1) holds recently accessed data; when data is evicted from L1, rather than being discarded entirely, it is promoted to a slower but larger cache (L2) so that future accesses can still be served without going to the network.

What Had Already Happened

In the immediately preceding message (index 2576), the assistant had taken the first step: it added an evictionCallback field to the ARCCache struct. The assistant's reasoning was clear: "I need to add an eviction callback to the ARCCache. Let me modify it to support an optional callback that's called when items are evicted from T1 or T2." It had read the ARC implementation, identified the evictFromT1() and evictFromT2() methods as the points where eviction occurs, and applied an edit to add the callback field to the struct.

But adding a field is not enough. The field needs:

A method to set it (a setter)
Actual invocation at the eviction points This is precisely what message 2577 is about. The assistant recognizes that it has completed only half the work and articulates the next two steps: "add a method to set the eviction callback" and "modify the evictFromT1 and evictFromT2 methods to call it."

Why This Message Was Written: The Reasoning and Motivation

The assistant wrote this message for several interconnected reasons, each revealing a different facet of the software engineering process.

First, the message serves as a planning checkpoint. The assistant had just applied an edit (adding the field) and needed to verify its work and plan the next steps. By reading the file again and explicitly stating what remains to be done, the assistant creates a clear mental model of the task structure. This is a form of "thinking out loud" that helps maintain focus and catch missed steps.

Second, the message reflects a decomposition of a complex change into atomic steps. The implementation of an eviction callback involves three distinct changes:

Add the callback field to the struct (done in msg 2576)
Add a public method to set the callback (planned in this message)
Modify the eviction methods to invoke the callback (planned in this message) This decomposition is characteristic of systematic software engineering: each step is independently verifiable, and the ordering respects dependencies (you cannot invoke a callback that hasn't been set yet). Third, the message reveals the assistant's awareness of the ARC cache's internal architecture. The ARC (Adaptive Replacement Cache) algorithm maintains four lists: T1 (recently accessed), T2 (frequently accessed), B1 (ghosts of T1 evictions), and B2 (ghosts of T2 evictions). Eviction happens in two methods: evictFromT1() and evictFromT2(). By naming these specific methods, the assistant demonstrates an understanding that both eviction paths must be modified, not just one.

How Decisions Were Made

Several design decisions are implicit in this message and the surrounding context.

Decision 1: Use a callback rather than direct coupling. The assistant chose to implement the L1→L2 promotion as an eviction callback on the ARC cache rather than hard-coding the promotion logic inside the cache itself. This is a sound architectural decision: it keeps the cache implementation generic (it doesn't need to know about L2 caches or SSDs) while allowing the promotion behavior to be configured externally. The callback pattern is a classic application of the Hollywood Principle: "Don't call us, we'll call you."

Decision 2: Wire the callback in the retrieval provider, not in the cache constructor. In subsequent messages (2582-2585), the assistant wires the callback in retr_provider.go by calling SetEvictionCallback on the L1 cache after both L1 and L2 caches are initialized. This is a deliberate choice: the callback needs a reference to the L2 cache, which may not exist when the L1 cache is constructed. By wiring after initialization, the assistant ensures the L2 cache is available.

Decision 3: Promote evicted items to L2 via l2Cache.Put(key, value). The callback lambda captures the L2 cache and calls its Put method. This is straightforward but has implications: it means every L1 eviction triggers an L2 write, which could be expensive if L1 is highly volatile. The assistant does not add any filtering or batching—every evicted item is promoted unconditionally. This is a reasonable starting point, but it assumes that L2 has sufficient capacity and write throughput to absorb L1 evictions.

Assumptions Made

The assistant makes several assumptions in this message and the surrounding implementation:

Assumption 1: The eviction callback will be set before any evictions occur. The ARC cache starts empty, so no evictions happen during initialization. But if the callback is set after some items have been inserted and evicted, those earlier evictions would be missed. The assistant assumes that the callback is set once during initialization and never changed.

Assumption 2: The callback is optional. The evictionCallback field is presumably nil by default, and the eviction methods check for nil before calling. This is a safe assumption—it means existing code that doesn't set a callback continues to work unchanged.

Assumption 3: The callback signature matches func(key K, value V). The assistant later uses rp.l2Cache.Put(string(key), value) inside the callback, implying the callback receives the key and value of the evicted entry. This assumes that the ARC cache's internal entry representation exposes both key and value at eviction time, which is true for the existing arcEntry struct.

Assumption 4: The L2 cache's Put method is synchronous and non-blocking. The assistant does not add any error handling or retry logic around the callback invocation. This assumes that L2 writes are fast enough that they won't block L1 eviction processing, and that failures can be silently ignored.

Input Knowledge Required

To understand this message, a reader needs knowledge spanning several domains:

ARC cache algorithm knowledge. The reader must understand that ARC maintains four lists (T1, T2, B1, B2) and that eviction happens from T1 and T2 when the cache exceeds its target size. The references to evictFromT1 and evictFromT2 are meaningless without this context.

Go generics knowledge. The ARCCache[K, V] is a generic type. The eviction callback must be generic over the same type parameters. The reader must understand how Go generics work to appreciate why the callback field and setter method are declared with type parameters.

Multi-tier caching architecture. The concept of L1 (in-memory, fast, small) and L2 (SSD, slower, larger) caches with promotion on eviction is a standard pattern in systems that need to balance latency and hit rate. The reader must understand why promoting evicted items to L2 is beneficial: it preserves cache warmth across restarts and provides a second chance for items that were evicted due to capacity rather than disuse.

The project's cache hierarchy. The specific implementation uses rbcache.ARCCache for L1 and rbcache.SSDCache for L2. The reader must know that SSDCache.Put(key string, data []byte) is the method for inserting into L2, and that it returns no error.

Output Knowledge Created

This message, combined with the subsequent edits, creates several pieces of output knowledge:

The SetEvictionCallback method. A new public method on ARCCache that allows external code to register a callback function. This is a reusable extension point that other parts of the system could use for logging, metrics, or other eviction-triggered behaviors.

The modified eviction methods. evictFromT1() and evictFromT2() now call the callback (if set) before or after removing the entry from the cache lists. The exact ordering matters: calling the callback before removal means the callback can still access the entry's data; calling after means the data is already gone. The assistant's subsequent edits call the callback after removal, which means the callback receives a copy of the key and value that were stored in the entry.

The wiring in retr_provider.go. The lambda func(key mhStr, value []byte) { rp.l2Cache.Put(string(key), value) } connects L1 evictions to L2 promotions. This is the concrete manifestation of the multi-tier caching strategy.

A pattern for future callback-based extensions. By establishing the eviction callback pattern, the assistant creates a precedent for other hooks in the cache system, such as insertion callbacks, hit callbacks, or miss callbacks.

The Thinking Process Visible in Reasoning

The assistant's reasoning in this message is concise but reveals a structured thought process:

State recognition: "Now I need to add a method to set the eviction callback and modify the evictFromT1 and evictFromT2 methods to call it." This acknowledges that the previous edit (adding the field) was only the first step.
Verification through re-reading: The assistant reads the file again, even though it was just modified. This is a form of verification: checking that the edit was applied correctly and seeing the current state before making further changes.
Explicit enumeration of remaining work: By naming both the setter method and the modification of eviction methods, the assistant creates a mini-plan with two clear tasks. This prevents the common mistake of adding a field but forgetting to wire it up.
Focus on the eviction methods specifically: The assistant names evictFromT1 and evictFromT2 rather than saying "the eviction logic." This precision indicates that the assistant has already identified the exact locations in the code that need modification. The reasoning is notably free of hesitation or uncertainty. The assistant does not ask "should I use a callback or a different pattern?" or "is this the right approach?"—those decisions were made earlier. This message is purely about execution of a predetermined plan.

Broader Significance

This message, though brief, exemplifies several principles of effective software engineering in AI-assisted coding:

Incremental verification. The assistant reads the file after each edit to confirm the state is as expected. This is analogous to compiling after each change in traditional development.

Task decomposition. Complex changes are broken into atomic steps: add field, add setter, modify callers. Each step is independently testable.

Architectural awareness. The assistant understands the ARC cache's internal structure well enough to identify the exact methods that need modification, without having to re-read the entire implementation each time.

Pragmatic scope. The callback is simple—no error handling, no filtering, no batching. This is appropriate for a first implementation. The assistant avoids over-engineering while still delivering the required functionality.

In the broader narrative of the coding session, this message represents the moment when the multi-tier caching system transitions from a design concept to a working implementation. The L1→L2 promotion callback is the glue that binds the two cache tiers together, and getting it right is essential for the system's performance characteristics. The assistant's methodical approach—field, setter, invocation, wiring—ensures that each piece is correctly implemented before moving to the next.

The message also illustrates a key dynamic of AI-assisted development: the assistant serves as both architect and implementer, making design decisions (callback pattern) while also executing the detailed code changes. The reasoning note is the visible trace of this dual role, showing the assistant's awareness of what has been done and what remains.