CAP in context
CAP in Context
Understand what the CAP theorem actually proves, why the popular "choose two" framing is misleading, and how to reason about consistency and availability as a spectrum rather than a binary choice.
Prerequisites
Recommended Knowledge
- Partial failure and network uncertainty
- Replication and replica lag
- Quorum reads and writes
- Leader-follower and leaderless topologies
- Transactions and isolation levels
- Timeouts and failure detection
- Caching and staleness
- Idempotency and conflict handling
What CAP Actually States
The CAP theorem is a formal impossibility result. It states that a distributed data store cannot simultaneously guarantee consistency, availability, and partition tolerance. The precision of its definitions matters more than the acronym.
| Property | Formal Meaning | Common Misreading |
|---|---|---|
| Consistency | Every read returns the most recent write or an error | Confused with the C in ACID |
| Availability | Every request to a non-failing node receives a response | Confused with uptime or reliability |
| Partition tolerance | The system operates despite dropped messages between nodes | Treated as an optional design choice |
Why "Choose Two" Is Wrong
The popular summary suggests three symmetric options: CA, CP, or AP. This framing is misleading because partition tolerance is not something a designer elects to have.
Simple Analogy
Two branch offices lose their phone line. Each can either stop processing transactions until contact resumes, or continue independently and reconcile discrepancies later. There is no third option where the line stays connected by policy.
CP and AP in Practice
CP: Consistency Preferred
- Minority side rejects requests
- Only a quorum may accept writes
- Clients see errors, never stale data
- No conflicts to reconcile afterwards
- Suits financial and inventory state
AP: Availability Preferred
- Every reachable node responds
- Writes accepted on both sides
- Clients may see stale values
- Conflicts resolved after healing
- Suits content, sessions, and telemetry
| Dimension | CP Behavior | AP Behavior |
|---|---|---|
| Minority partition | Rejects operations | Continues serving |
| Write acceptance | Quorum required | Any reachable replica |
| Read result | Current or error | Possibly stale |
| Conflict possibility | Prevented | Expected and resolved |
| Client complexity | Must handle unavailability | Must handle staleness |
| Recovery work | Replicas catch up | Divergence must be merged |
Quorum Mechanics
Quorums are the practical mechanism by which systems implement the CP side of the trade-off. Requiring a majority ensures two disjoint groups cannot both proceed.
With \(N\) replicas, \(W\) required for a write and \(R\) for a read, this condition guarantees the read and write sets overlap on at least one replica holding the latest value.
| Configuration | Overlap Guaranteed | Characteristic |
|---|---|---|
| N=3, W=2, R=2 | Yes | Balanced, tolerates one failure |
| N=3, W=3, R=1 | Yes | Fast reads, writes fail on any loss |
| N=3, W=1, R=3 | Yes | Fast writes, reads fail on any loss |
| N=3, W=1, R=1 | No | Highly available, stale reads possible |
| N=5, W=3, R=3 | Yes | Tolerates two failures |
function evaluateQuorum(config) {
const { replicas, writeQuorum, readQuorum } = config;
const strictlyConsistent =
readQuorum + writeQuorum > replicas;
const writeFaultTolerance = replicas - writeQuorum;
const readFaultTolerance = replicas - readQuorum;
return {
strictlyConsistent: strictlyConsistent,
survivesWriteFailures: writeFaultTolerance,
survivesReadFailures: readFaultTolerance,
classification: strictlyConsistent ? "CP-leaning" : "AP-leaning"
};
}
Common Misinterpretations
| Claim | Why It Is Wrong |
|---|---|
| "Relational databases are CA" | A single node is not a distributed system; replicated ones face the same trade-off |
| "NoSQL means AP" | Many NoSQL stores offer tunable or strongly consistent modes |
| "AP systems are always available" | They remain available during partitions, not during overload or node loss |
| "CP systems are unavailable" | They are unavailable only to the minority side during a partition |
| "The choice is made once per system" | Different operations within one system can sit at different points |
| "CAP governs normal operation" | It constrains behavior only while a partition is active |
The Latency Trade-off CAP Omits
CAP's greatest practical limitation is that partitions are rare, yet the consistency decision affects every request. A system choosing strong consistency pays coordination latency continuously, not only during failures.
| Deployment | Coordination Cost | Practical Effect |
|---|---|---|
| Single rack | Sub-millisecond | Negligible |
| Multiple availability zones | Low single-digit milliseconds | Usually acceptable |
| Cross-region | Tens to hundreds of milliseconds | Materially affects user experience |
| Global quorum | Bounded by the furthest member | Often prohibitive for interactive writes |
Applying the Trade-off Per Operation
Treating CAP as a system-wide decision is the most costly mistake. Within a single application, different operations warrant different positions.
| Operation | Preference | Reasoning |
|---|---|---|
| Payment authorization | Consistency | Double charging is unacceptable |
| Inventory decrement | Consistency | Overselling has real cost |
| Account balance display | Consistency | Stale balances mislead decisions |
| Unique username claim | Consistency | Duplicates cannot be merged cleanly |
| Product catalogue read | Availability | Slightly stale descriptions are harmless |
| Recommendation feed | Availability | Approximate results are acceptable |
| View counters | Availability | Eventual accuracy is sufficient |
| Session state | Availability | Losing availability blocks all use |
| Audit record | Consistency | Loss undermines the entire purpose |
Expressing the Choice in Code
const CONSISTENCY_POLICY = {
"payment.authorize": { level: "linearizable", onPartition: "reject" },
"inventory.reserve": { level: "linearizable", onPartition: "reject" },
"username.claim": { level: "linearizable", onPartition: "reject" },
"catalog.read": { level: "eventual", onPartition: "serve-stale" },
"recommendations.get": { level: "eventual", onPartition: "serve-fallback" },
"profile.read": { level: "read-your-writes", onPartition: "serve-stale" }
};
async function executeOperation(operationName, request) {
const policy = CONSISTENCY_POLICY[operationName];
if (!policy) {
throw new Error(`No consistency policy for ${operationName}`);
}
const clusterState = await clusterMonitor.currentState();
if (clusterState.partitioned && !clusterState.hasQuorum) {
switch (policy.onPartition) {
case "reject":
throw new UnavailableError(
"Quorum unavailable for a strongly consistent operation"
);
case "serve-stale":
return await readLocalReplica(request, { stale: true });
case "serve-fallback":
return buildFallbackResponse(request);
}
}
return await executeWithLevel(policy.level, request);
}
Communicating Staleness to Clients
An AP system that serves stale data without indicating it forces every consumer to guess. Surfacing the degraded state allows sensible client decisions.
{
"data": {
"accountId": "acct-4821",
"balance": 152400,
"currency": "INR"
},
"consistency": {
"level": "eventual",
"asOf": "2026-09-24T05:11:00Z",
"stalenessSeconds": 47,
"degraded": true,
"reason": "quorum unavailable, served from local replica"
}
}
What Clients Should Be Told
- Which consistency level was actually applied
- The timestamp the data reflects
- Whether the system is currently degraded
- Whether writes are being accepted
- Whether the value may be superseded later
Life After a Partition Heals
Choosing availability defers work rather than eliminating it. When connectivity returns, divergent state must be reconciled.
| Resolution Strategy | Mechanism | Limitation |
|---|---|---|
| Last writer wins | Highest timestamp prevails | Silently discards a valid write |
| Version vectors | Detects genuine concurrency | Requires application-level merging |
| Conflict-free data types | Merge is mathematically defined | Only suits certain data shapes |
| Application merge | Domain rules decide the outcome | Logic must exist for every conflict type |
| Manual reconciliation | A human resolves the divergence | Does not scale |
| Compensating action | Correct the effect after the fact | Some effects cannot be undone |
A Worked Example
Consider an e-commerce platform deployed across two regions with a partition between them.
Browsing and Search
Availability preferred. Both regions serve catalogue data from local replicas. Slightly outdated descriptions are acceptable, and blocking browsing would eliminate all revenue.
Cart Contents
Availability preferred. Carts are user-scoped, so concurrent modification across regions is rare and additive merging handles most conflicts.
Inventory Reservation
Consistency preferred. Only the quorum-holding region may reserve stock. The minority region rejects reservations rather than overselling.
Payment Processing
Consistency preferred. Payments require quorum and idempotency keys. Rejection is the correct outcome when coordination is impossible.
Order History
Availability preferred for reads. Displaying a slightly incomplete history is preferable to an error page, provided the staleness is disclosed.
Inventory and Payments CP Store
Observing the Trade-off
Signals Worth Tracking
- Quorum availability per partition group
- Rejected writes attributed to lost quorum
- Replica lag distribution
- Stale reads served and their age
- Conflicts detected during reconciliation
- Conflicts requiring manual intervention
- Leader election frequency
- Cross-region round-trip latency
- Time spent in a partitioned state
- Divergence duration after healing
Common Design Mistakes
Weak Reasoning
- Describing a system as simply CP or AP
- Believing CA is an achievable option
- Confusing CAP consistency with ACID consistency
- Ignoring the latency cost of coordination
- Accepting library defaults without examination
- Choosing AP without conflict resolution logic
- Relying on last writer wins for meaningful data
- Serving stale data without disclosing it
- Never testing partition behaviour
Strong Reasoning
- Assigns a policy per operation
- Treats partitions as inevitable
- Uses precise consistency terminology
- Accounts for coordination latency
- Configures quorums deliberately
- Implements explicit conflict resolution
- Preserves concurrent writes for merging
- Surfaces staleness in responses
- Rehearses partition scenarios
System Design Interview Discussion
| Question | What Your Answer Should Cover |
|---|---|
| Is this system CP or AP? | Per-operation analysis, not a single label |
| Why is CA not available? | Partitions are imposed, not chosen |
| What does consistency mean here? | Linearizability, distinct from ACID |
| What happens during a partition? | Behaviour of majority and minority sides |
| How are quorums configured? | The overlap condition and fault tolerance |
| How are conflicts resolved? | Concrete strategy, not last writer wins |
| What does consistency cost normally? | Coordination latency outside partitions |
| Do clients know data is stale? | Staleness metadata in responses |
Design Checklist
Production Checklist
- Document a consistency policy per operation
- Identify operations that genuinely need linearizability
- Separate strongly consistent data from tolerant data
- Configure quorums against the overlap condition
- Verify minority-side behaviour during a partition
- Prevent split-brain writes through quorum
- Implement conflict resolution before choosing AP
- Preserve concurrent versions rather than discarding
- Avoid last writer wins for meaningful values
- Return staleness metadata to clients
- Measure coordination latency in normal operation
- Monitor quorum health and conflict rate
- Alert on prolonged partitioned state
- Test partition scenarios deliberately
- Document expected behaviour for each failure mode
Knowledge Check
Why is "choose two" misleading?
Partition tolerance is not elective. Networks partition regardless of design intent, so the real choice is only between consistency and availability during a partition.
What does consistency mean in CAP?
Linearizability, where the system behaves as if a single copy exists and every operation takes effect instantaneously. It differs from the C in ACID.
What does the quorum condition guarantee?
That read and write sets overlap on at least one replica, so a read always encounters a node holding the most recent acknowledged write.
Why is last writer wins risky?
Both concurrent writes represent genuine intent, and keeping only the later timestamp discards one silently, made worse by clock skew between nodes.
What does CAP fail to address?
The latency cost of coordination during normal operation, which applies to every request rather than only during rare partitions.
Summary
CAP states that a distributed data store cannot simultaneously guarantee linearizable consistency, availability on every non-failing node, and tolerance of network partitions. Its definitions are narrow and precise, and the consistency it refers to is far stronger than that of ACID.
The popular "choose two" framing misleads because partition tolerance is imposed by physical reality rather than selected. The genuine decision is how the system behaves during a partition: refuse requests on the minority side to preserve correctness, or continue serving and accept divergence.
Quorum configuration demonstrates that CP and AP are endpoints of a tunable continuum rather than fixed categories. The same datastore can be positioned differently by adjusting read and write quorum sizes against the overlap condition.
CAP's practical shortcoming is that it addresses only partitions, which are rare, while consistency decisions impose coordination latency continuously. Most importantly, the trade-off belongs at the operation level: payments and inventory warrant consistency, while catalogues, feeds, and sessions warrant availability.
Key Takeaway
Stop labelling systems and start classifying operations. Partitions are inevitable, so decide in advance what each operation does when quorum is lost, configure quorums deliberately, implement real conflict resolution before choosing availability, and disclose staleness rather than letting clients assume correctness.