The Eviction Callback: Bridging Memory and SSD in a Distributed Cache Hierarchy

The Message

[assistant] ## Agent Reasoning I need to add an eviction callback to the ARCCache. Let me modify it to support an optional callback that's called when items are evicted from T1 or T2. [edit] /home/theuser/gw/rbcache/arc.go Edit applied successfully.

Introduction

At first glance, this message appears to be one of the most mundane moments in a coding session: a developer stating an intention and executing an edit. But in the context of a complex distributed storage system, this single message represents a critical architectural inflection point. The assistant is not merely adding a callback to a cache—it is bridging two fundamentally different tiers of a storage hierarchy, transforming a simple eviction policy into a promotion pipeline that determines how data flows between memory, SSD, and ultimately the network. This message, index 2576 in a long conversation, captures the precise moment when the L1→L2 cache promotion mechanism was conceived and implemented.

Context: The Data Lifecycle Problem

To understand why this message was written, one must understand the system under construction. The Filecoin Gateway (FGW) is a horizontally scalable distributed S3 storage system built on a multi-tier architecture. At the retrieval layer, blocks (chunks of content-addressed data identified by multihashes) must be fetched from storage providers, cached for performance, and served to clients. The caching architecture follows a classic hierarchy: an L1 in-memory cache using the Adaptive Replacement Cache (ARC) algorithm, an L2 persistent SSD cache, and finally HTTP network retrieval as the fallback.

The problem is fundamental to any multi-tier cache: what happens when an item is evicted from the fast, expensive L1 cache? In a naive implementation, evicted items are simply discarded. But in a well-designed system, eviction from L1 should trigger promotion to L2—the data isn't worthless just because it no longer fits in memory; it still has value and should be preserved on a slower but larger tier. The system was missing this promotion path. Items evicted from the ARC-based L1 cache were simply lost, forcing subsequent requests to go all the way to the network to re-fetch them.

This gap was identified by an earlier comprehensive analysis of critical implementation gaps. The assistant had already fixed the Prefetcher Fetch() method (implementing proper CID-based retrieval using the cache hierarchy) and was now moving to the next item on the priority list: wiring the L1→L2 cache promotion callback.

The Decision: Callback Over Hardcoding

The assistant's reasoning reveals a deliberate architectural choice. Rather than hardcoding the promotion logic directly into the ARC cache's eviction methods—which would couple the generic cache implementation to the specific promotion behavior—the assistant chose to add an optional callback mechanism. This is visible in the phrasing: "add an eviction callback to the ARCCache" and "support an optional callback that's called when items are evicted."

This decision embodies several design principles:

  1. Separation of concerns: The ARC cache remains a generic data structure. It doesn't need to know about L2 caches, SSD storage, or promotion policies. It simply provides a hook that callers can use.
  2. Optionality: The callback is optional. The ARC cache works perfectly without it, maintaining backward compatibility with any existing usage.
  3. Composability: The callback pattern allows the promotion logic to be wired up at the point of cache creation, where both the L1 and L2 cache instances are available. The alternative would have been to embed the promotion logic inside the ARC cache itself, perhaps by passing a promotion function at construction time. The assistant's approach of adding a SetEvictionCallback method after construction is more flexible—it allows the callback to be set after both caches are initialized, which is exactly what the subsequent wiring requires.

The Implementation Path

While the message itself only shows the reasoning and the edit command, the surrounding messages reveal the full implementation trajectory. The assistant had previously read the ARC cache code to understand the eviction paths (evictFromT1 and evictFromT2), identified that eviction happens in two places (T1 for recently accessed items and T2 for frequently accessed items), and determined that both paths needed to invoke the callback.

The implementation involved:

  1. Adding an evictionCallback field to the ARCCache struct
  2. Creating a SetEvictionCallback() method to set the callback after construction
  3. Modifying evictFromT1() to call the callback with the evicted key and value
  4. Modifying evictFromT2() similarly
  5. Wiring the callback in retr_provider.go by calling SetEvictionCallback on the L1 cache after initialization, providing a lambda that calls l2Cache.Put(key, value)

Assumptions Made

The assistant made several assumptions in this message and the surrounding implementation:

  1. That the ARC cache's eviction paths are the correct interception points: The ARC algorithm maintains four lists (T1, T2, B1, B2). Items are evicted from T1 and T2 when the cache exceeds its size limit. The assistant assumed that intercepting these two eviction paths covers all cases where data leaves the L1 cache. This is correct for the standard ARC algorithm.
  2. That the callback should receive the key and value: The assistant assumed that the L2 cache needs both the key (multihash string) and the value (block data) to store the promoted item. This is reasonable but assumes the L2 cache's Put method accepts these exact types.
  3. That the callback can be set after construction: The SetEvictionCallback method allows the callback to be set after the cache is created. This assumes that no evictions happen between construction and callback registration, which is safe because the cache starts empty.
  4. That the L2 cache is available when the callback is set: The wiring in retr_provider.go initializes both L1 and L2 caches before setting the callback. This ordering is critical and the assistant had to reorganize the initialization code to ensure this.

Mistakes and Corrections

The immediate subsequent messages reveal a mistake in the initial wiring. When the assistant first wrote the callback lambda in retr_provider.go, it used rp.l2Cache.Put(string(key), value) as a value expression, but the LSP diagnostics flagged an error: "rp.l2Cache.Put(string(key), value) (no value) used as value." The SSDCache.Put method returns nothing (void), so it cannot be used as a value in a context where a value is expected.

The assistant corrected this in message 2588 by reading the SSDCache.Put signature (which confirmed it returns nothing) and fixing the callback. This is a classic type-mismatch error that occurs when composing unfamiliar APIs. The mistake was caught by the LSP before compilation, demonstrating the value of real-time static analysis.

A more subtle assumption that could be questioned is whether promoting every evicted item to L2 is the optimal strategy. The ARC algorithm is designed to be scan-resistant and adaptive; evicted items are already deemed less valuable than retained ones. Promoting all evictions to L2 could fill the SSD cache with items that have low reuse probability. However, for this system's use case—where refetching from the network is expensive—the conservative approach of promoting everything is reasonable.

Input Knowledge Required

To understand this message, one needs:

  1. ARC cache algorithm: Knowledge of Adaptive Replacement Cache, its four lists (T1, T2, B1, B2), and how eviction works. The assistant references "evicted from T1 or T2," which are the two real-data lists in ARC.
  2. The project's cache hierarchy: Understanding that L1 is an in-memory ARC cache, L2 is an SSD-based persistent cache, and HTTP retrieval is the final fallback. This hierarchy is defined in retr_provider.go and configured via the cacheCfg structure.
  3. Go generics: The ARCCache is a generic type ARCCache[K, V] with type parameters for keys and values. The callback must respect these type parameters.
  4. The existing codebase structure: Knowledge of where arc.go lives in the rbcache package, how retr_provider.go initializes caches, and how the ribs system connects storage and retrieval.

Output Knowledge Created

This message and its surrounding implementation produced:

  1. A reusable eviction callback mechanism in the ARC cache that can be used for any purpose beyond L2 promotion—metrics collection, logging, ghost list management, etc.
  2. The L1→L2 promotion pipeline that ensures data evicted from memory is preserved on SSD, reducing network retrieval costs for items that have moderate reuse probability.
  3. A pattern for future cache tiering: The callback mechanism establishes a template for adding additional cache tiers or alternative eviction handlers without modifying the core ARC implementation.

The Thinking Process

The assistant's reasoning in this message is concise but reveals a structured thought process:

  1. Goal identification: "I need to add an eviction callback to the ARCCache." The assistant has a clear objective from the todo list.
  2. Design decision: "Let me modify it to support an optional callback." The word "optional" is key—it indicates a conscious choice to make the feature non-intrusive.
  3. Scope definition: "called when items are evicted from T1 or T2." The assistant has already determined the exact interception points by reading the ARC code in the previous message (2575), where it examined the evictFromT1 and evictFromT2 methods.
  4. Action: The edit command is executed with confidence, suggesting the assistant had already formed a mental model of the changes needed. The brevity of the reasoning belies the depth of understanding required. The assistant didn't need to re-examine the ARC algorithm or debate alternatives because it had already done that work in the preceding messages. The reasoning here is the culmination of a chain: read the ARC code, understand the eviction paths, decide on a callback approach, and implement.

Conclusion

Message 2576 is a study in how complex architectural decisions are encoded in seemingly simple developer actions. The addition of an eviction callback to an ARC cache is not merely a coding task—it is the resolution of a fundamental tension in multi-tier storage systems between memory speed and SSD capacity. By choosing a callback-based approach, the assistant preserved the generality of the ARC implementation while enabling a critical data lifecycle feature. The mistake that followed (the void-value error) and its quick correction demonstrate the iterative nature of systems programming, where even well-reasoned designs encounter type-level friction. This message, sandwiched between the completion of the Prefetcher and the wiring of the AccessTracker, represents the connective tissue that makes a distributed cache hierarchy function as a coherent whole rather than a collection of isolated tiers.