Caching and Redis

TTL expiry is the simplest thing that works: entries die after a set time, and you accept staleness up to that time in exchange for writing no invalidation logic at all. The reason it survives contact with production is that it is self-healing. Any bug, any missed path, any inconsistency corrects itself within one TTL, which is a property none of the cleverer schemes have.

Explicit invalidation deletes the key when the underlying data changes. It is fresher and it is fragile, because every write path must know every key derived from that data. Add a new page that composes the same record into a different key, forget the delete, and you have a permanently stale entry that no amount of waiting will fix, discovered eventually by a customer who says the price is wrong.

Versioned keys sidestep deletion entirely. Put a version or an updated-at timestamp in the key, bump it on write, and old entries become unreachable rather than wrong. Invalidation becomes one write to one value instead of a fan-out of deletes, and the stale entries age out on their own. This is usually the best of the three, and the cost is that the old entries occupy memory until they expire, which is a memory problem rather than a correctness one.

Whichever you choose, jitter the TTLs. Entries created together with identical lifetimes expire together, and a cache populated at deploy time will empty itself in one synchronised moment some hours later, which arrives as an unexplained database spike at an odd hour. A random spread of ten to twenty per cent is enough to turn the cliff into a slope.

Distributed invalidation is where this gets genuinely hard, because deletes have to reach every node in every region, and a delete that is lost in transit leaves one region serving stale data indefinitely with nothing to detect it. The pragmatic answer most large systems reach is to stop trying: short TTLs plus versioned keys, so correctness never depends on a message arriving. There is one more failure worth naming: the stale set. A client reads a value, is slow, and writes what it read into the cache after a concurrent update has already invalidated it, so the cache now holds an old value with a fresh lifetime. Facebook's leases exist partly for this, since a lease token issued at read time can be invalidated by an intervening write, letting the cache reject the late set.

Redis stores structures, not just strings, and treating it as a string cache is the most common way to leave performance on the table. Sorted sets maintain order by score and answer rank and range queries directly, which is a leaderboard or a time-ordered feed without the application fetching everything to sort it. Hashes let you update one field of an object without rewriting the whole record. Sets handle membership and deduplication. Streams add consumer groups and acknowledgements, which is real message processing rather than a list pretending to be a queue.

The operational model matters as much as the data model, and the key fact is that Redis executes commands on a single thread. Redis 6 added threaded I/O for reading and writing sockets, but the commands themselves still run one at a time, which has two consequences. One slow command blocks everything, so a KEYS scan over a large keyspace or an O(n) operation on a million-element set is an outage rather than a slow query. And a single hot key is bounded by one core, no matter how many nodes you add.

Memory is managed by an eviction policy you choose, and the default is the one that surprises people: noeviction, which starts returning errors on writes when maxmemory is reached rather than making room. For a cache you almost certainly want allkeys-lru or allkeys-lfu. Both are approximations, sampling a handful of keys and evicting the worst rather than maintaining a true ordering, because exact LRU across millions of keys costs more than it saves. LFU, added in Redis 4.0, is the better choice when a small set is genuinely hot, since one burst of scanning traffic cannot evict everything the way it can with LRU.

Expiry and eviction are different mechanisms and get confused constantly. Expiry removes a key because its TTL passed; it is checked lazily on access and sampled by a background cycle, so an expired key can occupy memory for a while after it logically died. Eviction removes a key because you are out of memory, regardless of TTL. A cache filling up with keys that have no TTL at all will evict things you wanted while holding things you did not.

Clustering shards the keyspace across 16,384 hash slots assigned to nodes, and the constraint to design around is that a multi-key operation only works when the keys live in the same slot. Hash tags, the braces in user:{42}:profile, force related keys together for exactly this reason. Get that wrong and transactions and Lua scripts that worked on a single node start failing in the cluster with a cross-slot error, usually the week after launch.

Finally, persistence. Redis offers RDB snapshots and an append-only file, and both are useful, and neither turns a cache into a database. RDB loses everything since the last snapshot; AOF with the default fsync policy loses up to a second. Both are reasonable for a cache that would rather restart warm than cold, and neither is a promise you should make to a user about their data.

A stampede happens when a popular key expires and every concurrent request misses at once, so all of them hit the database together to compute the same value. The load is proportional to concurrency rather than to traffic, which is why it appears suddenly at a scale that was fine yesterday. The standard fix is a short lock: one request wins the right to recompute while the others wait briefly or serve the stale value they can still see. Facebook's version is a lease. On a miss the cache hands one client a token granting permission to recompute, and tells everyone else to wait or use stale data. It is the same shape as a lock, expressed as something the cache issues, which also lets the cache reject a set whose lease was invalidated by an intervening write. Probabilistic early expiry is the other approach: as an entry approaches its TTL, each reader has a small and rising chance of refreshing it early, so the recomputation happens before the expiry rather than at it, spread across readers.

An avalanche is the same problem multiplied: many keys given identical TTLs expire simultaneously, usually because the cache was warmed in one pass at deploy time. The database sees the entire working set arrive as cold reads in a few seconds. Jitter fixes it, and it is worth adding at the moment you write the TTL rather than after the first incident.

A hot key is one entry so popular that a single node becomes the bottleneck, and it is not solved by adding nodes, because the key still hashes to one of them. Redis executing commands on a single thread makes it a single-core limit. The two fixes are replication of that key under several suffixed names with clients choosing at random, and a small in-process cache in front of the shared one, holding the top few keys for a second or two. The second is the more effective and the more dangerous, since every process now has its own slightly different copy.

Cache penetration is the quieter cousin: requests for keys that do not exist anywhere, so nothing is ever cached and every one becomes a database read. It appears naturally with user-supplied identifiers and it is the standard shape of a cheap denial of service. The fixes are negative caching, storing a short-lived marker meaning this does not exist, and a Bloom filter holding the set of ids that do exist, which answers definitely not present in memory.

The pattern behind all four is synchronisation. Requests miss together, keys expire together, traffic concentrates on one key, or absent keys share a path with no memory. Each fix is a deliberate desynchronisation: a lock so one goes first, jitter so they separate, replication so they spread, a marker so the second one is cheap. When you can name which of the four you are looking at, the fix is usually already obvious.

The cheapest cache is the one you do not operate. HTTP has caching built into it, and a correctly labelled response can be held by the browser, by any proxy in the path and by a CDN with a few hundred points of presence, none of which you pay for or run. The whole mechanism is a handful of headers, which is why getting them wrong is both easy and expensive.

Cache-Control carries the instructions. max-age is how long any cache may reuse the response; s-maxage overrides it for shared caches only, which is how you say five seconds at the CDN and none in the browser. private means only the browser may store it, which is what you want for anything user-specific. no-store means keep no copy at all, and it is what people mean when they wrongly write no-cache, which actually means store it but revalidate before reuse.

Validators handle the revalidation. The server sends an ETag or a Last-Modified, the client sends it back as If-None-Match or If-Modified-Since, and the server answers 304 Not Modified with no body when nothing changed. The saving is the payload, and on APIs it can be most of the traffic: GitHub does not count a conditional request that returns 304 against your rate limit, which turns polite polling into something the platform actively encourages.

Two extensions from RFC 5861 are worth more than they cost. stale-while-revalidate lets a cache serve a slightly stale copy immediately and refresh in the background, so the person waiting never pays for the refresh. stale-if-error lets it serve stale content when the origin is failing, which converts an origin outage into slightly old pages for anyone whose request the edge can answer.

The cache key is the part that bites. By default it is the URL, and Vary adds request headers to it, so Vary: Accept-Encoding is fine and Vary: User-Agent shatters your hit ratio into thousands of fragments, one per browser string. Query parameters count too, which is why an analytics parameter appended to a shared link produces a fresh miss for every recipient of that link. Normalising the key is often the single largest hit-ratio improvement available.

Purging is where people reach first and should reach last. Content-addressed URLs, the hashed filenames every bundler emits, make purging unnecessary: a new build is a new URL, so the old one can be cached forever and the new one is never stale. That works precisely because the URL identifies the bytes. Applying the same immutable policy to a URL whose content can change is how a site serves a year-old page to everyone who visited during a bad deploy, and no purge fixes the copies already held in browsers.

Caching adds a second source of truth and a new class of bug, and it earns that when three things are true: reads dominate writes, the data tolerates some staleness, and recomputing it is genuinely expensive. Miss any one of them and you have added complexity for nothing, or worse, for a correctness problem that shows up as a customer complaint rather than as an alert.

Data that changes on nearly every read gains nothing. You pay the write cost, then miss anyway, and the cache becomes a tax on the read path plus an extra system to operate. Per-user data with no reuse is usually the same story: a cache entry read once before it expires has cost more than it saved, and a million of them evict the entries that were actually working.

Anything where stale means wrong deserves a different answer entirely. A permission set cached for five minutes means access revoked five minutes ago still works, and that is a security failure rather than a latency tradeoff. If you must cache it, cache it for seconds, with an explicit revocation path, and write the staleness into the contract so it is a known property rather than an accident.

The arithmetic of hit ratios is worth internalising, because it is not linear and people reason about it as if it were. At a 1ms hit and a 50ms miss, a 95% hit ratio gives an average of about 3.5ms, and 99% gives about 1.5ms. That looks like a modest difference until you look at the origin instead of the average: going from 99% to 98% doubles the load reaching your database, and from 99% to 95% multiplies it by five. Cache work should be judged by what it does to the miss rate, not by what it does to the mean.

It follows that a cache is not a fix for a slow query, it is a way to run a slow query less often. The queries still run, on every miss, on every eviction, and on every cold start, which is exactly the moment you are least able to absorb them: a restart after an incident empties the cache and sends the full read load at a database that has just recovered. If the origin cannot survive its own traffic without the cache, the cache has become a load-bearing part of the system, and it should be designed as one rather than described as an optimisation. The honest question, then, is not what to cache but what staleness is acceptable for, and for how long. Answer it per dataset, write the answer down next to the TTL, and the caching design follows. If the answer is none, the fix is a faster query, a better index, or a different shape of data, and no amount of caching will substitute for it.