Caching
Some answers are slow to figure out — they live in a faraway database, or take real work to compute. Caching is one of the oldest tricks in computing: the first time you work out an answer, keep a copy somewhere super close and fast. The next time someone asks the same question, hand them the copy instead of doing all that slow work again. That's it. The whole topic is 'remember the answer so you don't redo it.'
The killer idea: a hit vs. a miss
When a request comes in, you check the cache FIRST. Two things can happen. A cache HIT means the answer is already sitting in the cache. You hand it straight back — fast, done, the slow database was never even bothered. A cache MISS means the cache doesn't have it. Now you have to do the slow work (go ask the database), get the answer, give it to the user — AND drop a copy into the cache on the way out, so the very next person who asks the same thing gets a hit. So a cache fills itself up as it's used: the first asker pays the slow price (a miss), and everyone after them rides for free (hits). The more hits you get, the faster your whole system feels.
Why it makes things fast: it's about latency
Latency is how long ONE answer takes to come back. A cache attacks latency directly. Reading something out of a cache (which usually lives in fast memory, right next to the server) might take a fraction of a millisecond. Going all the way to a database — maybe on another machine, reading from disk — can take ten, a hundred, even more times longer. So a cache hit isn't 'a little faster.' It's often the difference between an instant answer and a noticeable wait. And because each hit skips the slow database entirely, the database also gets asked far fewer questions — which means it can serve far more people before it's overwhelmed. One trick, two wins: snappier for each user, and more users served overall.
Catch #1: the copy can go stale
Here's the price of keeping a copy. A copy is frozen at the moment you made it — but the real answer back in the database can change afterward. Now your cache is holding an OLD answer. That's called staleness: the cached copy and the truth have drifted apart. Example: you cache a product's price as $10. Tomorrow the shop changes it to $8 in the database. But the cache still cheerfully hands out $10 to everyone, because nobody told it to update. The fix is usually to give each cached copy an expiry timer — a TTL, 'time to live.' Maybe you say 'this price is only trusted for 60 seconds.' After that the copy is thrown away, the next request is a miss, and a fresh copy is fetched. TTL is you admitting 'a slightly old answer is fine for a little while, but not forever.'
Catch #2: the cache is small, so you must throw things out
A cache is fast precisely because it's small and close — it's not a second database, it can't hold everything. So it fills up, and then to add a new copy you must throw an old one away. Deciding WHAT to throw out is called eviction. The most popular rule is plain common sense: throw out whatever hasn't been used in the longest time. That's LRU — 'least recently used.' The thinking is, if nobody's asked for it in ages, they probably won't ask soon, so it's the safest thing to drop. It's like clearing out your fridge: the stuff you haven't touched in weeks goes first; the things you grab every day stay. Between TTL (toss it when it's too OLD) and LRU (toss it when it's the most NEGLECTED and we need room), the cache stays small, fresh-ish, and fast.
It's a tradeoff
| Option | 👍 Pro | 👎 Con |
|---|---|---|
| Use a cache | Repeated questions get instant answers, and the slow database is bothered far less — so it can serve many more users. | The copy can go stale (show old data), and you have to decide what to evict when the cache fills up. More moving parts to reason about. |
| No cache (always ask the source) | Always perfectly fresh — you never serve an out-of-date answer, and there's nothing extra to manage. | Every request pays the full slow cost, and the database takes ALL the load — it'll buckle under a crowd much sooner. |
Questions you might have
▸Isn't a cache just a second database? What's the difference?
A cache is small, fast, and temporary; a database is big, slower, and the permanent home of the truth. The cache only holds COPIES of recently-wanted answers and is allowed to forget them. If the cache vanished, you'd lose nothing real — every answer can be rebuilt from the database. That's the key difference: the database is the truth, the cache is just a handy shortcut.
▸If the cache can show old data, isn't that dangerous? Why use it at all?
Because for tons of things, a slightly-old answer is totally fine. Does it matter if a blog's view count says 1,000 when it's really 1,003? No. We use TTLs to control how stale we'll tolerate — short timers for things that must be fresh, long timers for things that barely change. You cache where 'a bit old' is acceptable, and skip the cache where it isn't (like your bank balance).
▸What's the difference between TTL and LRU? They both throw things out.
They answer different questions. TTL throws a copy out because it's gotten too OLD to trust (the timer ran out). LRU throws a copy out because the cache is FULL and you need room, so you drop whatever's been ignored the longest. One is about freshness, the other is about space. A cache often uses both at once.
▸Why is the cache so much faster than the database?
Two reasons: it's usually kept in super-fast memory (RAM) rather than on a slower disk, and it's right next to the server instead of across a network on another machine. Both 'closer' and 'faster storage' mean a cache hit comes back in a flash, while a database trip is a longer journey.
▸What happens the very first time, when the cache is empty?
That first request is a miss — there's nothing cached yet, so you pay the full slow cost and ask the database. But on the way back you save a copy, so the SECOND person asking the same thing gets a fast hit. A cache 'warms up' as it's used; it's the repeated questions it pays off on.
Best read after: What is a server?, Latency vs. throughput