Caching: where, what, invalidation
Most reads ask for something that was asked for a moment ago. Keep the answer close with a deadline on it, and the database only sees the misses; the deadline is how stale you have agreed to be.
Problems worked on this page, and more to practise
What is the difference between cache-aside, read-through, and write-through?
They differ in who talks to the database. In cache-aside the application checks the cache, reads the database itself on a miss, and fills the cache. In read-through the cache does that fetch for you, so the application only ever calls get. Write-through is about writes: the write goes to the database and then to the cache in the same call, so the next read sees it. Most real designs are cache-aside or read-through for reads, and either write-through or delete-on-write for writes.
How do I pick a TTL?
Start from staleness, not speed: ask how long a changed value may be served old, and make that the TTL for any change the cache is not told about. Then check the cost. Every key misses at least once per TTL, so the reads reaching the database are at least the number of hot keys divided by the TTL. If that is too many, send writes through the cache so the TTL only has to cover the writes you do not control, and lengthen it.
Why does going from a 90% to a 99% hit rate matter so much?
Because the database sees the misses, not the hits. At 90% it serves one read in ten; at 99% it serves one in a hundred, which is ten times less load for a nine-point change. That is why the hit rate is the number to say out loud, and why a small cache that holds the hot set can be worth more than a fast one that does not.
What is a cache stampede and how do you stop one?
A hot key expires, and every request that arrives before the refill lands misses and goes to the database, all for the same row. At thousands of reads a second and a slow query that is hundreds of identical queries at once. The fixes all make sure one caller refreshes: coalesce requests behind a single loader, serve the stale value while one caller refreshes it, or refresh early before the deadline. Put a lease on whoever holds the refresh so a crashed loader cannot block the key forever.
When should I not put a cache in front of something?
When a read must be exactly current, as with a balance you are about to debit, or when a write must not be lost, as with a ledger entry. A cache is a copy that is allowed to be behind, and a write-back cache acknowledges writes the database has not seen yet. For money, keep reads and writes on the system of record and scale it another way.