Cassandra is the database you choose when you need to absorb an enormous, relentless stream of writes across many data centers and never go down ā and you're willing to give up some of the conveniences of a relational database to get it. It runs the write-heavy backbones of companies that can't afford an outage. The trade-offs it makes are deliberate and unusual, and if you model data the way you would in Postgres, you'll fight it the whole way. Understanding the internals is how you stop fighting and start using it as designed.
Three pillars hold up everything Cassandra does: an LSM-tree storage engine that makes writes blazing fast, a masterless ring that distributes data with no single point of failure, and tunable consistency that lets you dial the consistency/availability trade-off per query. I'll take them in turn, then trace a write and a read.
The LSM-tree: why writes are so fast
A relational database typically uses a B-tree, which means a write often involves reading a page, modifying it, and writing it back ā random I/O, and an update in place. Cassandra uses a log-structured merge-tree (LSM), which is built around a simple insight: sequential writes are vastly faster than random writes, so never update in place ā only ever append. A write flows through three structures:
- Commit log ā the write is first appended to an on-disk commit log (sequential, fast). This is purely for durability: if the node crashes, the commit log is replayed on restart.
- Memtable ā simultaneously, the write goes into an in-memory sorted structure, the memtable. The write is now "done" from the client's perspective; nothing random touched disk.
- SSTable ā when a memtable fills, it's flushed to disk as an immutable Sorted String Table. Once written, an SSTable is never modified.
graph TD
W["Write request"]
CL["Commit log (append ā durability)"]
MT["Memtable (in-memory, sorted)"]
SS["SSTable (immutable, on disk)"]
CMP["Compaction:
merge SSTables, drop tombstones &
superseded versions ā fewer, larger SSTables"]
W --> CL
W --> MT
MT -->|"flush when full"| SS
SS --> CMP
CMP --> SS
The LSM write path. Every write is a sequential append to the commit log plus an in-memory memtable update ā no random I/O, no read-before-write ā which is why Cassandra ingests writes so fast. Immutable SSTables accumulate, so a background compaction process merges them, discarding tombstones (delete markers) and superseded values. The same memtable/SSTable/compaction pattern underlies many modern write-optimized stores.
The cost of never updating in place is paid on reads and on compaction. A single row's data can be spread across the memtable and several SSTables, so a read may have to merge fragments from multiple places and reconcile by timestamp (latest wins). Compaction keeps that bounded by periodically merging SSTables together. And deletes are especially subtle: you can't remove data from an immutable SSTable, so a delete writes a tombstone ā a marker that says "this is deleted" ā and the space is only reclaimed at compaction.
The tombstone trap bites every team eventually. Because deletes are tombstones that linger until compaction (and must outlive a grace period to avoid resurrecting data on a lagging replica), workloads that delete heavily or use collections/queues as deletion-churn patterns accumulate tombstones ā and a read that has to scan past thousands of tombstones to find live data gets slow or errors out. Modeling tip: design so you rarely delete, and never use Cassandra as a queue.
The ring: masterless distribution by consistent hashing
Now the distributed layer, and Cassandra's most distinctive choice: there is no master. Every node is equal ā any node can take any read or write and coordinate it. This is what gives Cassandra no single point of failure and near-linear scalability: add nodes, get more capacity, with no special node to bottleneck or fail.
Data is placed using consistent hashing. Imagine the hash space as a ring. Each node owns a range of the ring (in practice many small ranges, via virtual nodes, for smoother balancing). To place a row, Cassandra hashes its partition key to a token on the ring; the node owning that token's range is responsible for it, and the next Nā1 nodes clockwise hold the replicas, where N is the replication factor. The beauty of consistent hashing is that adding or removing a node only reshuffles the ranges adjacent to it, not the whole dataset.
graph TD
K["Row with partition key 'user-42'"]
H["hash(partition key) ā token on the ring"]
subgraph RING["The ring (RF = 3)"]
N1["Node A
(owns token range)"]
N2["Node B (replica)"]
N3["Node C (replica)"]
N4["Node D"]
end
K --> H --> N1
N1 -->|"replica"| N2
N2 -->|"replica"| N3
Consistent hashing on the ring. The partition key's hash picks a token; the owning node plus the next Nā1 clockwise hold the replicas. There is no master ā any node can coordinate any request. Because only neighboring ranges move when membership changes, the cluster scales out (and recovers) without a global data reshuffle. This is also why the partition key choice is the most important data-modeling decision: it decides data placement and the access patterns the table can serve efficiently.
Gossip: keeping a leaderless cluster coherent
With no master, how does every node know the cluster's state ā who's up, who's down, who owns what? Through gossip: once per second, each node exchanges state information with a few random peers. State spreads epidemically, so within a few rounds the whole cluster converges on a consistent view of membership and health ā without any central coordinator to query or to fail. It's the protocol that makes "everyone is equal" actually workable at scale.
Tunable consistency: you choose, per query
This is the lever that makes Cassandra flexible, and it's grounded in a simple piece of arithmetic. With a replication factor N, you set the consistency level for each read (R replicas must respond) and each write (W replicas must acknowledge). The guarantee:
If R + W > N then a read is guaranteed to see the latest write.
(Strong consistency from quorum overlap ā the read set and write
set must share at least one replica that has the newest value.)
Common setup: N = 3, W = QUORUM (2), R = QUORUM (2)
R + W = 4 > 3 ā strong consistency, tolerates 1 node down| Consistency level | Behavior | Trade-off |
|---|---|---|
ONE | One replica responds | Fastest, highest availability; may read stale |
QUORUM | Majority of replicas respond | Strong consistency with R+W>N; tolerates a minority down |
ALL | Every replica responds | Strongest read; zero tolerance for a down replica |
LOCAL_QUORUM | Majority within the local data center | Strong + low latency in multi-DC; avoids cross-DC round trips |
This is Cassandra living on the availability side of the CAP trade-off but letting you tune it: choose ONE and you favor speed and availability over freshness; choose QUORUM and you get strong consistency at the cost of needing a majority alive. The same cluster can serve a "must be correct" query at QUORUM and a "good enough, just be fast" query at ONE ā per statement.
Tracing a read
Reads are where the design's costs surface, so they're worth following. A coordinator (any node) routes the read to the replicas, waits for R of them per the consistency level, and on each replica the read must reconstruct the row from potentially several SSTables plus the memtable. To make that efficient, each SSTable has a bloom filter (a fast probabilistic check for "could this SSTable contain the key?", skipping SSTables that definitely don't) and a partition index for locating the key within the ones that might. If the contacted replicas disagree, the coordinator returns the newest version and triggers a read repair to update the stale replica in the background ā one of the mechanisms (alongside hinted handoff and anti-entropy repair) that pulls an eventually consistent system back toward agreement.
Cassandra 4.0 is arriving this year, and it's the most significant release in a long while ā a major focus on stability, faster streaming for node operations (rebuilds and bootstraps move data dramatically faster), audit logging, and a long list of correctness and performance hardening. If you're standing up a new cluster, it's the version to target.
What to carry away
Cassandra is fast and resilient because of three reinforcing choices. The LSM engine turns every write into a sequential append to a commit log and memtable, flushing immutable SSTables that compaction later merges ā superb write throughput, at the cost of read-side merging and the tombstone hazard. The masterless ring places data by consistent hashing with replication, coordinated by gossip, so there's no single point of failure and scaling out is near-linear. And tunable consistency lets you pick, per query, how many replicas must agree, dialing the availability/consistency trade-off to fit the request.
Model for it accordingly: choose partition keys for your access patterns (queries follow the key, not the other way around), avoid delete-heavy designs, and pick consistency levels deliberately. Do that, and Cassandra delivers exactly what it promises ā writes that don't flinch and a cluster that doesn't go down.