In one line

When a network partition splits a distributed system, it must choose between staying consistent (refuse some requests) and staying available (answer, possibly with stale data). Day to day, the real trade-off is consistency vs latency.

The idea

CAP says a distributed data system can’t guarantee all three at once during a network partition:

  • Consistency (linearisability): every read sees the latest write.
  • Availability: every request to a non-failed node gets a response.
  • Partition tolerance: the system keeps working despite lost messages between nodes.

Partitions will happen in any distributed system, so P isn’t optional. The real choice is: when partitioned, C or A?

flowchart TB
  N1[(Node 1<br/>value = v2)] -. network partition .- N2[(Node 2<br/>value = v1)]
  C1[Client reads from Node 2] --> N2
  N2 --> D{What should Node 2 do?}
  D -->|CP: refuse / error| CP[Consistent, not available]
  D -->|AP: return v1| AP[Available, possibly stale]

PACELC: the more useful version

If Partition → choose A or C; Else (normal operation) → choose Latency or Consistency.

Most of the time there’s no partition, but waiting for replicas to confirm every write still costs latency. That’s the trade-off you tune daily.

How it works

Consistency models (strong → weak)

ModelGuaranteeExample use
Strong / linearisableReads always see the latest committed writeJob state transitions, model promotion, payments
Sequential / causalOperations seen in an order that respects cause → effectComments and replies, label edits then review
Read-your-writesYou always see your own writesUser edits a dataset name and sees it immediately
Monotonic readsYou never see data go “back in time”Dashboards that don’t flicker between old and new
EventualIf writes stop, all replicas converge… eventuallyCounters, analytics, search indexes, caches

Where you choose it in practice

  • MongoDB: writeConcern: majority + readConcern: majority + read from primary → strong-ish; readPreference: secondary → eventual. Causal consistency is available via sessions.
  • Caches and read replicas → eventual by design (see 05 Caching, 07 Replication, Partitioning and Sharding).
  • Events between services → the other service’s view is eventually consistent (see 06 Message Queues and Pub-Sub).
  • Azure Cosmos DB famously offers five selectable levels: strong, bounded staleness, session, consistent prefix, eventual.

ACID vs BASE (quick contrast)

  • ACID (relational transactions): Atomic, Consistent, Isolated, Durable.
  • BASE: Basically Available, Soft state, Eventually consistent, typical of large distributed stores.

Note: the “C” in ACID (data satisfies invariants) is not the same “C” as in CAP (linearisability).

On the platform

Decide per piece of data, not per system:

DataNeedsWhy
Training job state, model promotion to productionStrongTwo people must not promote different models at once; a job must not run twice
Dataset version manifestStrong on write, then immutableLineage must be exact; once written it never changes, so reads can be cached
Annotation progress countersEventualA few seconds’ lag is fine
Dashboards, search, thumbnailsEventualSpeed matters more than freshness
User’s own edits in the UIRead-your-writesOtherwise users think saving failed

Common mistakes

  • Saying “we picked CA”. In a distributed system you can’t opt out of partitions.
  • Using eventual consistency for things that need a single source of truth (job ownership, locks).
  • Demanding strong consistency everywhere and paying for it in latency and availability.
  • Not telling users that data is eventually consistent, so they file bugs about “missing” images that appear 10 s later.

Check yourself

Apply it

  • For the five most important data items on the platform, write down the consistency each needs, then check what the code actually uses (read preference, write concern, caches).

Go deeper

Related: 07 Replication, Partitioning and Sharding · 05 Caching · 06 Message Queues and Pub-Sub