Caching Layers: Where to Cache and What Can Go Wrong
A request can be cached in the browser, at a CDN, in your app's memory, in Redis, and in the database's own buffer pool. Each layer cuts latency and load, and each one adds a copy that can go stale, leak between users, or refuse to be invalidated. Here's what each layer is good for, how their staleness adds up, and how to invalidate a stack of caches in the right order.
The Price That Wouldn't Update
Marketing drops a product's price from $49 to $39 for a sale. The database has the new price within a second. Ten minutes later, half the customers still see $49. Support escalates, an engineer flushes Redis, and the price is still wrong for some people.
There were four caches between the database and the customer: an in-memory cache on each of the ten app servers (60 seconds), Redis (5 minutes), the CDN (5 minutes), and the browser (max-age=300). Flushing Redis didn't clear the CDN, the CDN didn't clear the app servers, and nothing could clear the copies already sitting in customers' browsers. Worse, after the Redis flush, CDN misses kept landing on app servers that still held the old price in memory, so the CDN cached $49 all over again.
Each cache was a reasonable decision on its own. Stacked together, they made the data up to 16 minutes stale, with no single place to fix it.
The Layers a Request Passes Through
| Layer | Who shares it | Good for | Main risk |
|---|---|---|---|
| Browser (HTTP cache) | Who shares itOne user | Good forStatic assets, the user's own data on repeat visits | Main riskYou can't purge it |
| CDN / reverse proxy | Who shares itEveryone | Good forPublic pages, assets, API responses that don't vary per user | Main riskServing one user's response to another |
| In-process memory | Who shares itOne app instance | Good forSmall, hot, rarely changing data: config, feature flags, lookup tables | Main riskInstances disagree; memory per instance |
| Distributed cache (Redis, Memcached) | Who shares itAll app instances | Good forExpensive query results, sessions, computed objects | Main riskAnother network hop and another system to run |
| Database buffer pool | Who shares itAll queries | Good forRecently read pages, automatically | Main riskNothing to manage, but limited by RAM |
That last row is easy to forget. The database already caches hot data pages in memory. A query that reads a few pages by primary key may already take well under a millisecond, and putting Redis in front of it adds a network hop, an invalidation problem, and very little speed. Caching pays off most for results that are expensive to compute, not merely frequently read.
Two Things That Multiply and Add Up
Hit rates multiply. Each layer only sees what the layers in front of it missed. A CDN with a 90% hit rate in front of Redis with a 90% hit rate means the database sees 1% of requests. That's why stacking caches is so effective at reducing load, and why the inner layers often look like they have poor hit rates: the easy hits were already taken.
Staleness adds up. When a layer misses, it refills from the next layer down, and that copy may already be old. If Redis cached a value 4 minutes 59 seconds ago, and the CDN fetches it just then and caches it for 5 more minutes, the CDN is serving data nearly 10 minutes old. In the worst case, a stack's maximum staleness is the sum of its TTLs, not the largest one.
The simulator stacks four caching layers in front of a database at 10,000 requests per second. Set a TTL for each layer and watch load, latency, and staleness.
Each layer only sees the requests the layer above it missed, so hit rates multiply. Staleness adds up: a layer can refill from a lower layer's copy just before that copy expires, so the worst case is the sum of the TTLs. Hit rates follow a simple warm-up curve and are illustrative.
Things to try:
- Only Redis, 60 seconds. A big drop in database load and one place to invalidate. For a lot of systems, this is enough.
- Every layer at 300 seconds. The database barely sees traffic, and the worst-case staleness is 20 minutes, with four places to clear. One of them you can't clear at all.
- Turn on "personalized." With the CDN enabled, this is the setup behind a whole class of incidents where users see each other's account pages.
- In-process at 300 seconds. Each of the ten instances caches separately, so it takes longer to warm up, and after a change, different instances serve different values until each one expires.
What Goes Wrong at Each Layer
The browser: you can't take it back
Once a response is cached in a browser with Cache-Control: max-age=600, it will be used for 10 minutes without asking you. There's no purge API. For static assets the standard solution is versioned URLs: put a content hash in the filename (app.3f9a2c.js), cache it for a year with immutable, and change the URL when the content changes. For HTML and API responses, use short lifetimes or no-cache, which lets the browser store the response but requires it to check with the server (cheaply, using an ETag and a 304 Not Modified) before each use.
💡 no-cache doesn't mean don't cache
no-cache means "store it, but revalidate before using it." no-store means "don't store it at all." For sensitive responses like account pages, no-store is the one you want.
The CDN: shared means shared
A CDN or reverse proxy serves the same cached response to everyone whose request maps to the same cache key, and by default the key is basically the URL. If /account returns a user's details and the response is cacheable, the next person to request /account gets them.
The protections, roughly in order of reliability:
Cache-Control: privateon any per-user response. Shared caches must not store it; the browser still can.- Not caching authenticated requests by default at the CDN, and opting specific routes in.
- Being careful with
Vary.Vary: Cookietechnically makes the cache key include the cookie, but cookies usually include analytics and session IDs, so it either stores a separate copy per user (useless) or the CDN ignores it (dangerous). Don't rely on it for privacy.
Use s-maxage to give shared caches a different lifetime from browsers. A common pattern for public pages is a short browser max-age (so users pick up changes quickly) with a longer s-maxage at the CDN, which you can purge.
In-process caches: every instance has its own opinion
An in-memory cache inside each app instance is the fastest cache there is, with no network hop. But with ten instances there are ten independent copies. After a change, some instances serve the new value and some the old, and a user whose requests are load-balanced across instances can see the price flip back and forth. Invalidating means telling every instance, usually through a pub/sub message, and any instance that misses the message stays stale until its TTL. Memory is also multiplied: a 2 GB cache on 10 instances is 20 GB.
Keep in-process caches for small, hot, rarely changing data where brief disagreement between instances is harmless: configuration, feature flag rules, country lists.
Distributed caches: the keys are the bugs
Redis or Memcached gives all instances one shared copy, which fixes the disagreement problem. The common bugs here are about cache keys. A key has to include everything the cached value depends on. Forget the locale and French users get English text. Forget the currency and prices show in the wrong currency. Forget the user or tenant ID on a per-user value, and you've rebuilt the CDN leak inside your own infrastructure.
Two other traps:
- Caching failures. If the database times out and the code caches the empty result or the error, that failure is now served for the full TTL. Cache errors deliberately, with a short TTL, or not at all.
- Negative caching done badly. Caching "this product doesn't exist" saves load from repeated lookups, but if the product is created a second later, it stays invisible until the negative entry expires. Give negative entries a much shorter TTL than positive ones.
When many requests miss at the same moment, such as a popular key expiring, they can all hit the database together. That's covered in The Thundering Herd Problem.
Invalidating a Stack of Caches
When the data changes, invalidate from the inside out: the layer closest to the database first, then each layer further out.
Write to the database
The source of truth changes first.
Clear the shared application cache
Delete (don't overwrite) the Redis key. Deleting means the next read rebuilds it from the database.
Clear in-process copies
Publish an invalidation message that every instance listens for, and keep in-process TTLs short so missed messages heal quickly.
Purge the CDN
Purge the affected URLs or surrogate keys (tags). Many CDNs let you tag responses, for example with every product ID on the page, so one purge clears every page showing that product.
Accept what's left in browsers
Whatever browsers already hold lives until its max-age. That's why the browser TTL for changeable data should be short.
Going outside-in is the mistake in the opening story: purge the CDN first, and the next CDN miss refills it from Redis, which is still stale. Even inside-out, there's a race. A read that started before your write can put the old value back into Redis after you deleted it. Short TTLs bound how long that lasts, and for data where it matters, a second delete a few seconds after the first catches most cases. (Caching Strategies covers the read and write patterns these layers use.)
Choosing Where to Cache
Cache as close to the user as the data's rules allow, and no closer. For each piece of data, ask three questions: is it the same for everyone, how stale can it be, and can you purge it if you need to?
Is the response the same for every user?
And measure: track the hit rate, latency, and memory of each layer separately. A layer with a 5% hit rate is costing you a network hop and an invalidation path for almost no benefit.
Key Takeaways
- Each layer serves a different scope: the browser is per user, the CDN is shared by everyone, in-process caches are per instance, Redis is shared by instances.
- Hit rates multiply, staleness adds up. Stacked TTLs make the worst-case age the sum of every layer's TTL.
- The database already caches. Put caches in front of expensive work, not every fast query.
- Browsers can't be purged. Use versioned URLs for assets and short lifetimes or
no-cachefor changeable responses. - Shared caches leak per-user data. Mark per-user responses
privateorno-store, and put every dependency (user, tenant, locale, currency) in server-side cache keys. - Don't cache failures for long, and keep negative-cache TTLs short.
- Invalidate inside-out: database, Redis, in-process, CDN. Purging outer layers first lets them refill from stale inner ones.
References
- RFC 9111: HTTP Caching, the definitions of
max-age,s-maxage,private,no-cache, andno-store - MDN: HTTP caching
- RFC 5861: HTTP Cache-Control Extensions for Stale Content, for
stale-while-revalidate - Facebook, Scaling Memcache at Facebook (NSDI 2013), on invalidation, leases, and stale sets at scale