What is caching?

You learned yesterday that an index makes reads faster by trading write speed and storage. Today: a cache makes reads faster too — but it trades something different: staleness and memory. If you can explain the difference between an index and a cache, you’ve got a system-design answer that most candidates can’t articulate.

The idea in one sentence

A cache is a temporary storage layer that keeps recently or frequently accessed data closer to the consumer, so you don’t have to fetch it from the slow source every time.

That’s it. Every cache — browser cache, CDN, Redis, CPU cache — is the same idea at a different scale.

The worked example: three layers

You visit a website. Where does the HTML come from? It might be cached at three different places:

Each layer is roughly 10× slower than the one before it. A cache hit at any layer means you skip all the slower layers behind it.

Cache hit vs cache miss

A cache hit is when the data is in the cache — you serve it immediately, no database query. A cache miss is when it’s not there — you fetch from the database, store a copy in the cache, and return it. The next request for the same data? Now it’s a hit.

The metric that matters: your hit rate — the percentage of requests served from cache. High hit rate = the cache is doing its job. Low hit rate = you’re paying cache overhead (the check-then-fetch) without the benefit.

The tradeoffs

Caches aren’t free. A strong answer names the costs:

• Stale data — the cache may serve outdated information if the source changed and the cache wasn’t updated.
• Invalidation — keeping the cache fresh is the hard problem. Strategies: TTL (time-to-live, auto-expire), write-through (update cache on every write), cache-aside (lazy load on miss).
• Memory cost — caches live in RAM, which is finite and expensive. More cache = more memory = more money.
• Thundering herd — when a popular cache entry expires, all requests hit the database at once. The cure (caching) creates the disease (cache stampede).

Cache vs index (from Day 1)

Both make reads faster. But they trade completely different things:

• An index lives inside the database. Always consistent. Costs write speed + storage.
• A cache lives outside the database. Can be stale. Costs consistency + memory.

An index trades write speed for read speed. A cache trades consistency for read speed. That distinction is what most candidates miss in interviews.

Check your understanding