Under the Hood
Consistency

CAP, PACELC, and what consistency actually costs

The most-quoted and most-misunderstood result in distributed systems. CAP does not say 'pick two of three' — it says something far narrower and far more useful: when the network partitions, you must choose between staying consistent and staying available, and you cannot have both. This lesson states the theorem correctly, kills the 'two of three' meme, extends it with PACELC (the part that matters on the 99.9% of days there is no partition — the everyday trade of latency against consistency), and shows why a single-node system like Fable's sidesteps the whole thing by not being distributed at all.

CAP, PACELC, and what consistency actually costs

The moment your data lives on more than one machine — replicas, shards, a multi-region cluster — you inherit a constraint that no amount of engineering removes. It's usually stated as the CAP theorem, and it's usually stated wrong. Getting it right is the foundation of the whole consistency stage, because almost every distributed-database decision is a specific answer to the question CAP actually poses.

The theorem, stated correctly

CAP names three properties of a distributed system:

  • Consistency (C) — every read sees the most recent write. (This is a specific, strong meaning — linearizability — and note it is not the "C" in ACID, which is a different idea. Same letter, unrelated concept. The overloading is a genuine source of confusion.)
  • Availability (A) — every request to a non-failed node gets a (non-error) response, even if it might be stale.
  • Partition tolerance (P) — the system keeps working when the network partitions: messages between nodes are dropped or delayed, splitting the cluster into groups that can't talk to each other.

The popular framing — "pick two of the three" — is misleading to the point of being wrong. Here's why: in any real distributed system, partitions happen. Networks drop packets, links fail, switches reboot. You do not get to choose partition tolerance; the network chooses for you. So P is not optional — it's a fact of life. That collapses the "choose two" into a much sharper statement:

When a partition occurs, you must choose between Consistency and Availability. You cannot have both — during the partition.

That's the whole theorem. It's not a menu of three; it's a single forced choice that only fires during a partition, between the two things you'd actually like to keep.

Why the choice is forced

The intuition is a two-second thought experiment. Two nodes, normally in sync, and the link between them dies — a partition. A write arrives at node A. Now A has exactly two options:

  • Accept the write and answer the client (stay available) — but node B, cut off, can't learn about it, so a read on B returns stale data. You've sacrificed consistency.
  • Refuse the write (or refuse to serve reads it can't confirm are current) until the partition heals and it can sync with B (stay consistent) — but now it's returning errors to clients. You've sacrificed availability.

There is no third option. A cannot both acknowledge a write B doesn't know about and guarantee B's reads are current, while they can't communicate. This is why CAP is a theorem and not a guideline — the trade is logically unavoidable, not an engineering weakness.

CP vs AP: two honest choices

So real distributed systems are classified by which they give up during a partition:

  • CP (consistent, not available under partition) — refuse to serve requests that could violate consistency rather than serve stale or divergent data. A partitioned minority of nodes goes read-only or errors out. You reach for CP when wrong data is worse than no data — a bank balance, an inventory count, a distributed lock. (Consensus systems like ZooKeeper/etcd, and single-leader databases with synchronous replication, sit here.)
  • AP (available, not consistent under partition) — keep answering on every node even when partitioned, accept that copies temporarily diverge, and reconcile later (the conflict-resolution lesson). You reach for AP when stale data is better than an error — a social feed, a shopping cart, a "likes" count, a DNS record. (Dynamo-style stores like Cassandra sit here.)

Neither is "better." They're answers to "which failure hurts my users less: a wrong answer, or no answer?" — and the right answer differs per feature, sometimes within one product.

Set the two dials, then cause a partition and read from the far side. A CP system refuses (no wrong answer, but no answer); an AP one serves possibly-stale data. Heal the network and the choice shifts to the everyday one — latency vs consistency — which is the part CAP ignores.

If Partition (CAP)
Else, normal ops (PACELC)
Region Awrites here
Region Byou read here
This system is PC/EClike etcd / ZooKeeper (fully consistent)

CAP only speaks during a partition: you must give up availability (CP — refuse rather than serve wrong) or consistency (AP — stay up and diverge). But partitions are rare; PACELC adds the everyday clause — else, on a healthy network, you still trade latency vs consistency, because being certain a read is current means coordinating, and coordination is round trips. Neither choice is better; the right one differs per feature — sometimes within one product.

PACELC: the part that matters when nothing is broken

CAP has a serious blind spot: it only says anything during a partition. But partitions are rare — most systems run partition-free 99.9% of the time. What governs behavior on all those normal days? CAP is silent. PACELC fills the gap, and it's the more practically useful framing:

If there is a Partition, choose Availability or Consistency (that's CAP); Else (normal operation), choose Latency or Consistency.

The "else" clause is the everyday one. Even with a perfectly healthy network, a system that wants strong consistency across replicas must coordinate — wait for a write to reach a quorum of nodes, or read from enough nodes to be sure it's current — and that coordination is round trips, which is latency. So even when nothing is broken, you're constantly trading consistency against speed:

  • EL (else, latency) — don't wait for all replicas; answer fast from the nearest one, accept possible staleness. (Async replication, from the replication lesson, is exactly this.)
  • EC (else, consistency) — wait for coordination so reads are always current, and pay the latency on every operation. (Synchronous replication is this.)

PACELC is why the read-your-writes bug exists at all: choosing EL (fast reads from a lagging replica) is a consistency sacrifice you made for latency, and it bites on the very next read. Most systems are described by both letters — Dynamo/Cassandra are PA/EL (available and fast, consistency last), a synchronously-replicated Postgres is closer to PC/EC (consistent under partition and normally, latency paid).

Go deeper

Check yourself

Answer out loud, as if an interviewer asked. If you hand-wave, reread that section.

  1. State the CAP theorem correctly, and explain precisely why "pick two of three" is the wrong framing — including why partition tolerance is not actually a choice.
  2. Walk through the two-node partition thought experiment: a write arrives at one side of a partition, and show why the node must sacrifice either consistency or availability, with no third option.
  3. Define CP and AP by the question "which is worse for my users, a wrong answer or no answer?" Give a feature that should be CP and one that should be AP, and justify each.
  4. CAP is silent when there is no partition. Explain what PACELC adds, what the "else" clause trades, and why that clause is behind the read-your-writes bug from the replication lesson.
  5. Someone proposes a "CA" distributed system. Explain why that label is meaningless for anything with a network, and what a system so labeled will actually do when a partition occurs.
  6. Fable keeps its ledger on a single node and calls that correct. Explain how that sidesteps the CAP trade entirely, why a money ledger specifically wants CP rather than AP if it ever distributes, and where eventual consistency still legitimately lives in the app.