Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Follow-up Tree: Redis Caching

Overview

“Add a cache to make reads faster” is the most prescribed and least examined sentence in systems work, and it escalates into an interview ladder because the sentence silently bundles at least five assumptions — locality exists, the workload is read-heavy, staleness is tolerable, coherence is cheap, and Redis itself will not become the bottleneck. Each follow-up removes one assumption and asks what survives. This page walks the ladder rung by rung: what each follow-up tests, how to reason, and the answer with its edge cases. The assumption audit lives in Caching Assumptions, the misconception in Cache Always Improves, and the mechanism-deep comparison of caching against recomputation and distribution in Caching and Distribution — this chain stays on the Redis-specific escalations, with the repo’s Redis internals in Redis Patterns and Internals and the operational playbook in Cache Patterns.

The Opening Question

“We need reads to be faster — add a cache.”

The expected core answer names the pattern before the product: cache-aside (lazy loading). On read: GET the key from Redis; on hit, serve from memory; on miss, read the source of truth, SET the key back with a TTL, and serve. On write: update the source of truth and delete the cache key (why delete and not update is rung 1’s question). State the contract honestly: after warm-up, reads are served from RAM in well under a millisecond instead of paying a database round trip, and the cache is disposable — a cold or dead Redis is a slow system, not a broken one, which is precisely why it can sit on the read path.

Why this answer is correct: naming the pattern and its disposability predicts every follow-up. Disposability implies misses are normal (stampede, rung 3); application-side population implies races (rung 1); a bounded store implies eviction policy (rung 2); a separate store implies consistency is your code’s job, not a transaction’s (rung 6); and “faster” was never the same claim as “correct” — the reason the reflex needs a boundary (Cache Always Improves).

Follow-up 1: The Cache-Aside Race — What Happens Between Read, Write, and Delete?

What it tests: whether cache-aside is a flow you can race in your head — the difference between drawing the pattern and having operated it.

How to reason: the pattern is application logic with no transaction around it. Walk the interleavings of two actors — a reader refilling on miss and a writer committing — and find the one that poisons the cache.

Answer. The classic race: a reader misses, reads the database and gets the old value V1; a concurrent writer commits V2 and deletes the key; the reader then SETs V1 into the cache — the cache now serves a stale mapping until the TTL expires, past the moment the write acknowledged. Interleaved:

Reader:  GET  k        → miss
Writer:  UPDATE db k=V2
Writer:  DEL  k
Reader:  GET db k      → V1 (read before commit landed in its snapshot)
Reader:  SET  k=V1     → stale until TTL

Note the uncomfortable truth this exposes: the database’s isolation guarantees end at its own boundary — the cache is populated by application code that read outside the write’s protection, so no isolation level rescues you. The defenses, in order of deployment frequency: TTL as the universal backstop — staleness is bounded by construction, so the design question becomes “what staleness bound does this data class tolerate?”; versioned keys (k:v7, the version bumped on write) so a stale refiller populates a key nobody reads, at the cost of a version lookup per access; delayed double delete — delete, wait longer than the refill window, delete again — which shrinks the poisoned window to the delay but adds an asynchronous step; and write-through for the small hot set of truly freshness-sensitive values, paying the write-path cost to keep cache and source in one motion. Why delete rather than update on write: an update from the write path can race the same way in mirror image (write path reads its own half-committed state, or two writers update out of order), costs more (computing the cached representation on every write, including values never read again), and deletion is idempotent — the lazy refill is the single writer, which is what makes the race enumerable in the first place.

Follow-up 2: TTLs and Eviction — What Does Redis Do When Memory Runs Out?

What it tests: whether the policy knobs are mechanisms you can predict or labels you recognize — TTL semantics, maxmemory, and the eviction zoo.

How to reason: Redis holds a bounded amount of RAM, and both expiry and eviction are approximations with budgets. For each policy, ask: when does it run, what does it sample, and what does it do to a key you still need?

Answer. TTL is not a deadline. Redis removes expired keys two ways: passively, when a key is accessed and found expired, and actively, a background cycle that runs about ten times per second, samples a small batch of keys from the expiring set, evicts the expired ones, and repeats the cycle when too large a fraction was expired (SET with EX/PX attaches the TTL atomically at creation — setting the value and TTL in two commands is the immortal-key bug the rate limiter chain documents). Consequence: an expired but never-sampled, never-accessed key can outlive its TTL under memory pressure — TTL bounds staleness, not footprint. Eviction is what happens when maxmemory is reached; the policy decides who dies: noeviction (default — writes against full memory error, which turns a memory problem into an application outage), allkeys-lru / allkeys-lfu, volatile-lru / volatile-lfu / volatile-ttl / volatile-random (candidates drawn only from keys with a TTL), and allkeys-random. The two details that separate users from operators: Redis’s LRU is approximate — it samples a configurable number of keys and evicts the least-recently-used among the sample rather than maintaining a global LRU list, trading precision for constant overhead (tunable via maxmemory-samples); and volatile-* silently degrades to noeviction when no keys have TTLs — mix durable keys into a cache instance, reach maxmemory, and writes start failing for reasons no one connected to the policy. The operational rule the rung ends on: one Redis process, one policy, one class of keys — cache keys and durable keys never share an eviction policy, so they should not share an instance (separate processes or at minimum separate clusters), and the eviction policy is a declared contract about which keys may vanish (Redis Patterns and Internals walks the engine side).

Follow-up 3: The Cache Stampede — and the Math That Sizes Its Mitigation

What it tests: the signature production failure of caching: whether you can reason about what happens when many concurrent requests miss at once — and price the mitigations.

How to reason: a stampede is a synchronized miss. Ask what synchronizes the misses (a shared expiry, a mass invalidation, a cold start), how many requests are in flight at that instant, and what the source can actually absorb.

Answer. The mechanism: a hot key expires, and every request in flight during the refill window misses and hits the database simultaneously. The arithmetic that makes it scary: the database might absorb 5K queries per second while the key serves 1M requests per second — the cache is the only thing standing between the two, and expiry is the cache blinking. One unmitigated hot-key expiry can be a full outage; mass events (a fleet of keys loaded with the same TTL, a manual flush, a Redis restart) synchronize the same failure across the keyspace. The mitigations, each with its price: per-key refill locks — the first miss takes a short-lived lock (SET lock:key ... NX PX 100), refills, releases; other misses wait or serve a cached-adjacent fallback. Price: added latency for waiters and a failure mode when the lock holder dies (the short lock TTL bounds it). Request coalescing (single-flight) — collapse concurrent misses in-process first, so one pod generates one refill instead of fifty; cross-pod, the per-key lock does the same job at fleet level. Probabilistic early refresh — the design popularized by Facebook’s memcached layer (Nishtala et al., “Scaling Memcache at Facebook”, NSDI 2013): each reader refreshes the value slightly before expiry with a probability that grows as the remaining TTL shrinks and as the recompute cost grows — hot, expensive keys refresh early and stagger; cold, cheap keys ride to expiry and nobody cares. Price: slightly more refreshes than strictly needed, purchased against eliminating the synchronized cliff. Stale-while-revalidate — serve the expired value immediately (staleness is usually milliseconds here) while one worker refreshes; the DB never sees the herd. Jittered TTLs — bulk-loaded keys given identical TTLs are a self-inflicted stampede schedule; add random spread. And the forgotten sibling of the stampede is negative caching: keys that do not exist produce misses that cannot be fixed by refilling, so absent-key lookups hammer the source forever unless absence itself is cached (short-TTL tombstones) — the same logic the URL shortener chain applies to dead links.

Follow-up 4: Invalidation — Why It Is the Hard Part and What Actually Ships

What it tests: whether you can turn “cache invalidation is hard” from an aphorism into a strategy selection with failure modes.

How to reason: invalidation answers “when does a cached value stop being trustworthy?” — and the strategies form a spectrum from “never trust it beyond T” (TTL-only) to “know exactly which write kills it” (event-driven). Each rung of the spectrum trades invalidation bugs against staleness.

Answer. The spectrum, in increasing sophistication: TTL-only — no invalidation logic at all; staleness is bounded by the TTL and invalidation bugs are impossible by construction, because there is no invalidation code to have bugs. Right when a minutes-scale staleness bound is acceptable; wrong the moment it is not. Explicit delete on write — the write path deletes the keys it knows it affects; cheap and immediate, and its failure mode is coverage: the third code path that writes the same table and forgets the delete is how stale-cache bugs ship, which is why this strategy begs to be centralized (one function owns “write X” including its invalidations) rather than sprinkled. Versioned keys / namespacing — never delete, rotate: bump a version prefix and old generations expire by TTL; wholesale invalidation becomes O(1) and forgetting nothing is structural (nothing to forget). Log-derived invalidation (CDC) — a tailer consumes the database’s own change log (binlog/WAL — Debezium-style change data capture) and drives cache updates from the same stream the replicas consume; coverage bugs become impossible because invalidation is no longer hand-written code, at the price of pipeline complexity and a replication lag that becomes a staleness bound. Pub/sub fan-out — publish “key X changed” to subscribers that drop local copies; viable for in-process L1s, but Redis pub/sub is fire-and-forget with no delivery guarantee, so it can optimize invalidation and must never be correctness. The dependency-graph trap gives the aphorism its teeth: invalidating one key is mechanical; invalidating a derived aggregate (the dashboard computed from twelve cached primitives) means walking a dependency graph, and that way lies the “naming things” level of difficulty — the reason the deep-dive page frames invalidation as the real cost center of the cache-vs-recompute decision (Caching and Distribution). The senior framing to close on: choose the smallest invalidation surface you can prove correct, and let the TTL backstop cover everything you cannot.

Follow-up 5: Hot Keys and Sharding — When One Key Is One Core

What it tests: Redis’s execution model and its scale-out story — whether you know what vertical and horizontal scaling can and cannot do about concentration.

How to reason: Redis executes commands on one thread, so throughput per key is a single core. Scaling adds cores by adding nodes — but a key lives on exactly one node. Ask what happens when the workload’s entropy collapses onto few keys.

Answer. The execution model first: commands run single-threaded (threaded I/O since Redis 6 moves network traffic off the main thread; command execution remains serialized), so a hot key — one celebrity, one viral link, one misconfigured client polling in a loop — caps at roughly one core of command throughput no matter how large the cluster behind it. The scale-out mechanisms: Redis Cluster shards the keyspace into 16,384 hash slots, each key mapped by CRC16(key) mod 16384, with hash tags ({user1000}:profile) forcing co-location when multi-key operations need it — the full topology and failover story is Redis Cluster Architecture; older deployments do client-side sharding over a consistent-hashing ring, which is the same math the hash-table chain climbs to (Hash Tables, Consistent Hashing). Neither fixes a hot key, because both route by key — and the hot key has one route. The mitigations: replicate the hot key to N nodes and let readers pick randomly (spreads reads, multiplies writes N-way — fine for a read-hot mapping, wrong for a counter); split the hot key into sharded sub-keys (page:views:0..N, aggregate on read) — the standard treatment for hot counters, walked in Hot Keys and Sharded Counters; an in-process L1 in front of Redis — a per-pod cache with a seconds-scale TTL collapses thousands of reads per second into a few Redis gets, at the price of N pods × staleness windows and a per-pod stampede risk that coalescing (rung 3) also fixes; and the design-level answer — the key is hot because it is the wrong key: a per-item counter on a viral item wants to be a sharded counter or a batched pipeline, not a faster lookup. One adjacent landmine to name: big keys — a 10 MB value or a ten-million-element set blocks the single thread during DEL (hence UNLINK / lazy freeing), so key shape is part of the same conversation.

Follow-up 6: Consistency Trade-offs — What Does “Cached” Mean to the Reader?

What it tests: whether staleness is an engineering quantity you can bound and sell, or a shrug — and whether you know when the cache is the wrong layer entirely.

How to reason: cache-aside gives eventual consistency with a computable bound: TTL plus invalidation propagation delay. Derive the bound per data class, then ask which access patterns violate it at the human level.

Answer. The honest model: with invalidation working, staleness is the propagation delay (milliseconds inside one region); with invalidation missed, staleness is the TTL. So “the cache is eventually consistent” is an incomplete sentence — the complete one is “stale at most T seconds, assuming the invalidation paths we tested; T is 60 for prices and 1 for session state,” which is a product decision per data class, made once and audited forever. The access patterns that break it are the rung’s substance: read-your-writes — the user edits their profile, reloads, and sees the old value from the cache; the fixes in ascending cost are a short TTL small enough for UX tolerance (cheap, approximate), per-user versioned keys bumped on that user’s writes (precise for the common case), or write-through for user-owned data (paying write-path cost where freshness is the product). Cross-key consistency — a page reads two cached values (inventory and price) and can see generations from different moments; the database’s serializable transaction does not extend across your cache, so multi-value invariants need either co-versioning (one version number covering the group) or reading both from the source under one transaction when the business rule actually requires agreement. When the answer is “do not cache” — values where staleness is incorrectness (rate-limit counters that must be exact, stock that must not oversell, the coordination data that decides who holds a lock) belong in the store that owns the invariant, not in a TTL’d copy; that boundary and its per-operation consistency table are exactly what Caching and Distribution and Consistency formalize. The one-sentence version for the interviewer: caching trades a quantified staleness bound for latency, and every column of that trade is a decision, not a default (Caching Assumptions audits the five assumptions the opening sentence bundled).

Follow-up 7: When Redis Is the Wrong Answer

What it tests: tool-boundary judgment — the rung that separates “knows how to use Redis” from “knows when not to.”

How to reason: each wrong answer has a specific reason: a cost model it violates, a guarantee it cannot give, or a layer that already does the job. Name the reason, not just the alternative.

Answer. Four regimes where the reflex misleads. Dataset beyond RAM: Redis lives in memory; a working set of 200 GB against 64 GB nodes turns every miss into a trip plus the cache overhead — the cost model inverts, and the fix is either sharding until it fits or accepting that the database’s own buffer pool is the right cache, because it caches pages where access already happens (Buffer Pool owns that mechanism). Durability as a requirement: Redis persistence exists — RDB snapshots and AOF with fsync policies — but a cache configured as source of truth has inverted its own design contract; if the data matters when the process dies, it belongs in a store whose durability story is the product (Redis Persistence details the windows). Query richness: a key-value GET/SET surface with no secondary indexes or ad-hoc queries — the moment the access pattern wants “all users where…”, the cache becomes a hand-rolled query engine over denormalized copies, which is a database with worse tools. Strongly consistent coordination: locks, leader election, exact quotas — a TTL’d, asynchronously-expiring store is the wrong substrate for invariants that must hold now (the rate limiter chain shows the narrow slice where single-command atomicity is enough). And the quiet one: the wrong layer entirely — an in-process cache (no network hop), an HTTP/CDN cache (before the request reaches you), or the database’s own cache may beat Redis at zero added failure modes; “add Redis” presumes the missing speed is between your app and the database, which is a measurement question (Cache Always Improves prices the cases where the cache nets negative).

How to Approach Unfamiliar Variations

The transferable template for cache follow-ups:

  1. Name which assumption the variant removes. “Just cache it” assumes locality, read-heaviness, cheap coherence, tolerated staleness, and stable hit rates; every twist — flat keyspaces, write-heavy data, coherence requirements, expiration storms — deletes one clause. The fix is answering the clause, not defending the reflex (Caching Assumptions).
  2. Answer for the miss path, not the hit path. Hits are the demo; misses (cold start, expiry, eviction, dead Redis, absent keys) are the design. A cache design that has not said what a miss costs and what a synchronized miss costs is a diagram, not a design.
  3. Locate the atomicity. Cache-aside is application logic; the only atomicity Redis guarantees is within one command or script. Anything that must not interleave — refill locks, counters, rate-limit decisions — lives in one command or Lua, or it does not live anywhere.
  4. Quantify staleness before defending it. “Eventually consistent” is not an answer; “stale at most T = TTL + propagation, T is 60 s for this class” is. Every data class gets its own T, and read-your-writes exceptions get named explicitly.
  5. Ask whether Redis is the layer at all. In-process, CDN, HTTP cache, buffer pool, or the database itself may own the problem with fewer moving parts — the strongest cache answer is sometimes a place you did not put a cache.

Interview Questions

Q: Cache hit rate is 95%, but database CPU pegs every time a flash sale starts. Hypothesis? A: A synchronized miss: the sale’s hot keys expire (or are bulk-loaded with aligned TTLs) and the herd lands on the database at the moment of peak interest — the 95% average hides a 0% hit rate for the refill window. Mitigations in order: jittered TTLs so expiries de-synchronize, stale-while-revalidate so the herd is served stale during refresh, per-key refill locks with in-process coalescing, and probabilistic early refresh for the keys whose recompute cost makes the stampede dangerous. Negative caching too, if the sale features lookups for items that do not exist.

Q: Why delete cache keys on write instead of updating them? A: Three reasons and one caveat. Deletion is idempotent — repeated writes collapse safely; it avoids computing the cached representation for values that may never be read; and it makes the lazy refill the single cache writer, which is what makes the refill race enumerable and fixable. The caveat is the mirror race: a slow reader can repopulate a deleted key with a pre-write value, so the TTL backstop (or versioned keys) is not optional — deletion alone is not a correctness argument.

Q: Your team runs volatile-lru and keys start failing writes at maxmemory — but the cache looks warm. What happened? A: No keys carry TTLs, so the volatile candidate set is empty and the policy degrades to noeviction: memory is full, nothing is evictable, and writes error. The policy was chosen for a keyspace that no longer matches the data in it — the fix is choosing the policy against the actual keys (allkeys-lru for a pure cache) and separating durable keys from cache keys so no future migration silently flips the semantics again.

Q: After “cache everything” shipped, users complain that edits do not appear when they reload. Fix ladder? A: Read-your-writes is broken by the cache’s staleness bound. Ascending cost: shrink the TTL for user-owned data until the window passes user tolerance; bump a per-user version on their writes and fold it into that user’s keys, so their own reads always see their own generation; write through to the cache on the update path for exactly the fields users read back immediately. Do not fix it by invalidating aggressively across the board — that trades a bounded, human-visible problem for unbounded invalidation coverage bugs.

Q: Why does adding Redis Cluster nodes not fix a hot key, and what does? A: Cluster shards by key: routing is CRC16(key) mod slots, so one key routes to one node forever, and that node’s single execution thread is the ceiling. What fixes it is reducing the concentration: replicate the key to N nodes for read fan-out (if reads dominate), split it into sharded sub-keys aggregated on read (if it is a counter), or put a per-pod L1 with coalescing in front (if the access is repetitive). The design-level question is why one key is hot — often the counter should be a pipeline, not a lookup.

Key Takeaways

  • Cache-aside is application logic, not a transaction: the stale-refill race exists, TTL is the universal backstop, and versioned keys or write-through are the cures where the bound is not enough.
  • TTL bounds staleness, not footprint; eviction bounds footprint, and Redis’s LRU/LFU are sampled approximations whose volatile-* variants silently become noeviction when no keys carry TTLs.
  • The stampede is the failure that gives caching its reputation: synchronize misses (shared expiry, cold start) and the source sees the herd; locks, coalescing, probabilistic early refresh, and stale-while-revalidate each buy back the cliff at a stated price.
  • Invalidation strategies form a spectrum from TTL-only (no bugs because no code) to log-derived (no coverage bugs because no hand-written invalidation); choose the smallest surface you can prove.
  • Hot keys are a single-core problem that sharding cannot route around — replicate, split, or L1 them, and ask why one key is hot in the first place.
  • Caching trades a quantified staleness bound for latency; when the bound is a product requirement (read-your-writes, oversell prevention), the cache is either carefully versioned or the wrong layer.

Cross-References