TOPIC 08 — SCALE
Capacity Planning
Reliability isn't only about bugs — plenty of systems fail simply because more users showed up than the system had room for. Capacity planning is how you make growth boring.
Two kinds of demand
Organic growth is the smooth curve: more users, more usage per user, heavier features. It's forecastable from trend data — project it forward with seasonality, and add error bars that widen with the horizon.
Inorganic demand is the step function: a product launch, a marketing campaign, a celebrity tweet, Black Friday. It doesn't appear in trend data, which is why capacity planning is a conversation with the business, not just a spreadsheet. The launch your marketing team is planning matters more than last month's curve.
Capacity isn't instant, even in the cloud: quota increases take days, GPU capacity takes months, database resharding takes quarters. A perfect forecast that arrives inside the lead time is a well-documented outage.
N+2: the redundancy arithmetic
If N is the capacity needed to serve peak load, Google's rule of thumb is to provision N+2: enough to keep serving peak when one unit is down for planned maintenance and another fails unexpectedly at the same time. With only N+1, every planned upgrade is a bet that nothing breaks while you do it.
| Setup | Survives | The catch |
|---|---|---|
| N | Nothing | Any failure at peak = user-visible outage |
| N+1 | One failure | Maintenance windows leave you at N — praying |
| N+2 | Maintenance + a failure | Costs more; this is what your error budget is buying |
The same logic applies across regions: if you serve from three regions and one goes dark, the surviving two must absorb its traffic. That means each region runs well below its ceiling in normal times — “full utilisation” and “survives a region failure” are mutually exclusive goals, and pretending otherwise is how global outages happen.
Load testing: measuring the truth
- Test the system, not a server. Real capacity is set by the weakest dependency — often the database, a lock, or a third-party API, not your stateless web tier.
- Use production-shaped traffic. Uniform synthetic requests lie: real traffic has hot keys, fat payloads, and cache-hostile patterns.
- Find the knee, not just the ceiling. Latency degrades gradually, then suddenly. The knee of that curve is your practical capacity; the crash point is trivia.
- Retest after change. Capacity numbers rot with every release. Continuous load testing in staging beats an annual ritual.
When demand wins anyway: overload protection
Some day, traffic will exceed capacity regardless of planning. The design question is whether the system degrades or collapses:
- Load shedding — reject excess requests early and cheaply (HTTP 429) instead of queueing everything into mutual timeout. Serving 80% of users well beats serving 100% of users nothing.
- Graceful degradation — drop expensive features first: static fallbacks, cached results, disabled recommendations. The checkout works even when the personalisation doesn't.
- Retry discipline — clients must back off exponentially with jitter, and cap retries. Naive retries triple the load precisely when you can least afford it (the “retry storm”).
- Circuit breakers — when a dependency is drowning, stop calling it for a while. Recovery needs slack, and hammering a struggling service denies it that slack.
One node dies → its load moves to the survivors → the busiest survivor tips over → repeat, faster each round. Autoscaling can't outrun it (new capacity arrives in minutes; the cascade takes seconds). The defences are headroom, load shedding, and circuit breakers — designed in beforehand.
Common pitfalls
- Planning to averages. Capacity is a peak-load problem; your average Tuesday is irrelevant to Black Friday.
- Assuming the cloud is infinite. Quotas, instance-type shortages, and single-zone dependencies all bite at the worst moment.
- Autoscaling as the whole strategy. It handles gentle slopes, not step functions or cascades — and it happily scales your bill during a retry storm.
- Forgetting the dependencies. Your service scaled beautifully; the payment provider you call did not.