✏️ Explanatory Question
Little's Law: L = A x W, or concurrency = arrival rate x time in system.
It holds for any stable system, regardless of the arrival distribution - which makes it a very cheap sanity check.
Sizing a thread pool: at 1,000 QPS with 200 ms average latency, concurrency is 1,000 x 0.2 = 200 concurrent requests. A 50 thread pool will queue and latency will climb; you need about 200 threads, or async I/O.
Sizing connection pools: 500 QPS with 20 ms database time needs 500 x 0.02 = 10 connections. Provisioning 200 wastes database memory.
Diagnosing: if latency rises while throughput is flat, requests are queueing - concurrency has hit a limit somewhere.
Pair it with queueing theory: as utilisation approaches 100%, waiting time grows as 1/(1-utilisation). At 80% utilisation, wait is about 4x service time; at 90%, about 9x; at 95%, about 19x. This is why you target roughly 60-70% utilisation rather than running servers flat out.