System Design · Lesson 3

CAP Theorem Explained

Learn what consistency, availability, and partition tolerance mean during failures, why pick-two is misleading, and how PACELC improves the model.

Quick answer

The CAP theorem says that when a network partition prevents parts of a distributed system from communicating, the system cannot guarantee both linearizable consistency and availability for every request. It must sometimes reject or delay an operation to preserve one-copy behavior, or accept an operation that may temporarily disagree with another partition.

CAP is not a general instruction to “pick two” forever. Partition tolerance is a failure condition a distributed system must plan for, not a feature that can usually be switched off. The useful design question is: during a partition, which operations must remain consistent, and which may remain available with reconciliation later?

A partition timeline

Consider two regions that both hold an account record:

12:00:00  Region A and Region B agree: status = active
12:00:01  The inter-region link fails
12:00:02  A client sends "suspend account" to Region A
12:00:03  Another client reads the account from Region B

Region B cannot know whether Region A accepted a newer value. A consistent design may refuse the read or require a quorum that cannot currently be reached. An available design may answer active, accepting that the response can be stale. No algorithm can create the missing communication.

After the link recovers, an available design must exchange versions, resolve conflicts, and converge. That is where version vectors, last-write-wins rules, conflict-free replicated data types, domain-specific merge logic, or a durable event log may enter the design. “Available during the partition” does not mean “free of cleanup afterward.”

Precise definitions

CAP consistency means a read behaves as though there were one up-to-date copy of the data. The formal result is commonly expressed with linearizability: once a write completes, later operations must observe that write in a single real-time order.

CAP availability means every request received by a non-failing node eventually gets a non-error response. A timeout, deliberate rejection, or indefinite wait sacrifices availability under this definition. The response does not have to contain the latest value, which is why an AP design can remain responsive with stale state.

Partition tolerance means the system continues operating according to its chosen rules even when the network drops or delays an arbitrary number of messages between groups of nodes. A process crash, a severed link, a routing problem, and extreme delay can be indistinguishable long enough to force the same decision.

These definitions are stronger and narrower than casual product language. A service with 99.99% uptime is not automatically “available” in the CAP sense. A database with ACID transactions is not automatically “consistent” across every distributed read.

Why “pick two” is misleading

The popular triangle suggests a designer can select CA and ignore partitions. That can describe a single-node database or a system that stops when communication is lost, but a multi-node system cannot guarantee that the network will never partition. When a real partition happens, a supposed CA system still has to choose behavior.

CAP also applies specifically while communication is broken. During normal operation, a system can provide both consistent and available responses. Even during a partition, the choice need not be global: an application can preserve consistency for balances and permissions while allowing stale catalog reads or queued profile updates.

The choice can differ by operation. A product page may read from a local replica, while checkout requires a primary or quorum. A “like” counter may accept concurrent increments and merge them, while a username claim must have one winner. Treating an entire database brand as simply CP or AP hides the configuration, topology, quorum, and request-level behavior that actually determine the result.

Failure detection adds another nuance. A node cannot instantly distinguish a broken peer from a slow network, so timeouts encode an operational judgment. A shorter timeout may restore responsiveness sooner but trigger more false failovers; a longer timeout may preserve stability while requests wait. CAP proves the boundary, while engineering still chooses the detection and recovery policy.

CP decisions during a partition

A consistency-first operation rejects or delays work when it cannot establish the required ordering. Quorum systems commonly require overlapping read and write sets. Leader-based systems may allow only the side containing the valid leader to accept writes; the other side returns an error or becomes read-only.

This is appropriate when accepting conflicting state would violate an invariant that is difficult or impossible to repair:

  • Spending the same balance twice.
  • Granting an exclusive lease to two owners.
  • Reusing a one-time security credential.
  • Confirming inventory that does not exist.
  • Applying an irreversible workflow transition out of order.

Consistency-first does not remove every failure. Clients can still time out after the server committed, so retry safety and idempotency remain necessary. It also does not guarantee business correctness; the transaction and domain rules must still encode the right invariant.

AP decisions during a partition

An availability-first operation accepts reads or writes on both sides, then reconciles. This fits data whose temporary divergence is tolerable and whose conflicts have a clear merge rule:

  • Shopping-cart additions that can be unioned.
  • Telemetry events that can be appended.
  • Cached product descriptions that may be briefly stale.
  • Social reactions modeled as commutative counters.
  • Draft profile edits that can surface a conflict later.

The system needs a convergence contract. Last-write-wins is simple but can discard a valid update and relies on a trustworthy ordering rule. A CRDT can merge specific data types, but it does not automatically encode every business invariant. Manual resolution may be correct for a valuable record but expensive for users and operators.

Eventual consistency describes convergence after updates stop and communication resumes. It is not a maximum-lag guarantee. Production teams should define a bounded objective—such as “99.9% of replicas apply writes within five seconds”—and monitor the tail, not merely assume “eventual” is good enough.

CAP consistency versus ACID consistency

The C in ACID means a transaction moves the database from one valid state to another, assuming the declared constraints and application logic are correct. The C in CAP concerns the observable ordering of operations across distributed copies. They answer different questions.

An ACID database can expose an asynchronously replicated read replica whose results are not linearizable. Conversely, a distributed key-value operation can be linearizable without enforcing a relational foreign key or a business rule. When reviewing architecture, name the actual guarantee: serializable transactions, read committed isolation, monotonic reads, read-your-writes, linearizable register behavior, or eventual convergence.

PACELC improves the model

Daniel Abadi’s PACELC formulation extends the decision:

If there is a Partition (P), choose Availability (A) or Consistency (C);
Else (E), choose lower Latency (L) or Consistency (C).

The “else” clause matters because partitions are rare, while cross-region latency is paid on ordinary requests. A synchronous cross-region quorum can provide stronger ordering but adds network round trips. A local read is faster but may be stale. The everyday latency-versus-consistency choice often shapes the user experience and cost more than the emergency partition choice.

PACELC is still a model, not a product scorecard. A system may let clients select quorum levels, session guarantees, or consistency modes. Geography, replication protocol, failure detector, conflict policy, and durability settings all affect observed behavior.

Product examples, with qualifications

PostgreSQL streaming replication normally sends write-ahead log records from a primary to standby servers. A hot standby can serve reads, but asynchronous replication permits stale results and can lose unapplied changes if promoted. Synchronous replication can strengthen durability for acknowledged commits but increases latency and can block commits when required standbys are unavailable. These are PostgreSQL configuration choices, not a permanent “CAP label.”

Amazon Dynamo-style systems popularized tunable quorums and conflict reconciliation, while systems such as etcd prioritize a consensus-backed, linearizable control plane and may reject operations without quorum. Managed products evolve, so verify the exact API, consistency setting, region topology, and failure semantics in the current documentation before relying on a marketing category.

Common mistakes

The first mistake is saying CAP means “consistency, availability, performance.” Partition tolerance is about lost communication, not general speed.

The second is treating consistency as a boolean. Applications often need a small set of strong invariants and can accept weaker guarantees elsewhere.

The third is calling any stale cache an AP system. CAP concerns replicated operations under partition; ordinary cache expiration may be a simpler freshness problem.

The fourth is claiming replication is a backup. Replicas quickly reproduce accidental deletes and corruption. Recovery requires independent backups and tested restore procedures.

The fifth is ignoring the client session. Even an eventually consistent system can sometimes provide read-your-writes by pinning a user to a leader, carrying a version token, or waiting until a replica reaches a required log position.

Practical checklist

  • Identify which data crosses machines or regions.
  • Write each business invariant in plain language.
  • Decide behavior when quorum or the leader is unreachable.
  • Classify operations separately instead of labeling the whole service.
  • Define acceptable stale-read and convergence windows.
  • Choose a deterministic conflict policy for accepted concurrent writes.
  • Provide idempotency for ambiguous write outcomes.
  • Measure replica lag, quorum failures, rejected writes, and reconciliation errors.
  • Test real network loss and recovery, not only process crashes.
  • Document client-visible responses during degraded operation.
  • Revisit the normal-operation latency trade-off using PACELC.
  • Keep independent, restorable backups.

Frequently asked questions

Does CAP say a distributed database can have only two properties?

No. When the network is healthy, it can be both consistent and available. During a partition, it cannot guarantee both for every operation under the formal definitions.

Can I avoid partition tolerance?

A single process can avoid distributed coordination, but it becomes one failure domain. Once independent nodes communicate over a network, lost or delayed messages must be handled somehow.

Is eventual consistency always AP?

No. Eventual consistency describes convergence, while availability describes whether every non-failing node answers during a partition. A system can converge eventually yet reject writes on a minority partition.

Are quorum reads always strongly consistent?

Not automatically. The quorum sizes, version selection, sloppy quorum behavior, concurrent writes, and read-repair rules matter. Use the product’s documented guarantee rather than the arithmetic alone.

Should money systems always be CP?

Critical balance invariants usually need strong coordination, but the surrounding system can mix guarantees. Statements, notifications, analytics, and search may update asynchronously while the ledger remains authoritative.

Sources and further reading

Continue with Eventual Consistency Explained for convergence patterns, Database Sharding Explained for partitioned data ownership, and Database Read Replicas Explained for stale-read controls. Review Database Transactions Explained to separate CAP consistency from ACID transaction guarantees.

Knowledge check

Check your understanding

Answer both questions correctly to mark this lesson as mastered. You can retry without penalty.

1. Which description best matches CAP Theorem Explained?

2. Which statement matches the lesson's quick answer?