Reliable, scalable,
maintainable.

That shift has a consequence the chapter draws out carefully. No single tool covers the whole job any more, so we assemble systems out of parts: a database for the source of truth, a cache for the expensive reads, a search index for queries the database is bad at, a queue for work that should not block the request, a stream or batch processor for everything derived.

The moment you put an API in front of that assembly, something changes that is easy to miss. You have not integrated tools — you have built a new data system, and its guarantees are now yours. The cache does not promise it agrees with the database. The search index does not promise it is current. Whatever consistency your users experience is a property you designed, deliberately or not.

Which makes the first task descriptive rather than architectural. You have to say what "load" means here, as a number: requests per second, ratio of reads to writes, number of simultaneously active users, hit rate on the cache. The chapter calls these load parameters, and choosing the wrong one is how teams end up optimising something that was never the bottleneck.

The chapter then spends its best pages on a single example, and it is worth following closely because the punchline is not the architecture — it is how the right load parameter was found. The figures come from a November 2012 Twitter talk that Kleppmann cites.

Describing performance once load is defined

Once load is described, you can ask what happens when it grows, and the chapter phrases that as two concrete questions. If load increases and resources stay the same, how does performance change? And if load increases, how much do resources have to grow to keep performance where it was?

Both questions need a number for "performance". Batch systems care about throughput: records processed per unit of time. Online systems care about response time: what the user waits through.

And response time is not a number — it is a distribution. The same endpoint, called repeatedly with identical arguments, returns a spread of times: a context switch, a garbage collection pause, a lost packet and retransmission, a cold page. Which is why the chapter is blunt about averages: the mean does not tell you how many users actually waited that long. Percentiles do.

Percentiles are also how performance becomes a contract. Service level objectives and agreements are usually written in them: for example, the service counts as up if the median response time is under 200 ms and p99 is under 1 s. Two practical traps follow. First, percentiles cannot be averaged: to combine data from several machines or time windows, add up the histograms and compute the percentile from the result. Second, a load-testing client must keep sending requests without waiting for the previous response, or it will hide the very queues it is supposed to measure.

Why tails get worse in distributed systems

Two mechanisms make this sharper than it first looks. The first is head-of-line blocking: a server handles a limited number of requests in parallel, so a few slow ones delay everything queued behind them. Requests that are fast to process still arrive late. This is why the measurement has to be taken on the client side, where the queueing is visible.

The second is tail latency amplification, and it is the one that surprises people. If a single user request fans out into several backend calls, the user waits for the slowest of them. Rare slowness stops being rare as fan-out grows.

Coping with load

Only now, with load and performance described, does the chapter allow the architectural question, and it deflates it on purpose. Scaling up (a bigger machine) and scaling out (more machines) are not a religious choice. Real systems use both, because a few powerful machines are often simpler and cheaper than a large fleet of small virtual ones. The same pragmatism applies to elasticity: adding resources automatically helps when load is unpredictable, while scaling by hand is simpler and has fewer operational surprises.

Distributing stateless services is largely mechanical. Distributing stateful data systems is where the difficulty concentrates, and that is why the common advice has long been to keep the database on a single node until cost or availability forces otherwise. Part II of the book is about what happens after that point. The closing warning is the one worth memorising: there is no generic scaling architecture. A system that handles 100,000 requests per second of 1 kB each looks nothing like one that handles 3 requests per minute of 2 GB each, even though both move the same amount of data. The architecture follows from the load parameters, and those are specific to your problem.

The distinction inside simplicity is the one that earns its keep in code review. Essential complexity comes from the problem: tax rules really are that complicated. Accidental complexity comes from how we built it: tangled dependencies, inconsistent naming, special cases papering over a leaky abstraction. Only the second kind is negotiable, and separating them turns "this is too complex" from an opinion into a claim you can check.