When your transaction commits, what does 'committed' physically mean — and how does the database survive a power cut mid-write without corrupting or losing it? The answer is write-ahead logging: append the change to a log, force it to disk with fsync, and only then call it committed. This lesson follows a write through the WAL, explains why sequential appends are the fastest thing a disk does, shows how checkpoints and crash recovery work, and ends on the 2018 bug where fsync itself lied.

The write path: WAL, fsync, and checkpoints

You've followed a read all the way down — pages, the buffer pool, B-trees. Now flip it. Your Prisma call resolves, COMMIT returns, your API sends 201 Created, and you tell the user their expense is saved. Then the server loses power half a second later. When it boots back up, is the expense there?

It had better be — you promised. That promise is the single hardest thing a database does, and the word for it is durability: once a transaction commits, its effects survive crashes, power loss, and the process being killed. The tricky part is that the obvious implementation — "when you commit, write the changed pages to disk" — is both too slow and not crash-safe. The real answer is a technique called write-ahead logging (WAL), and it's the same core idea behind every serious storage engine, from Postgres to MySQL's InnoDB to SQLite to filesystems themselves. Understand it once and a lot of systems stop being mysterious.

Why "just write the page" fails

Recall from the pages lesson that data lives in 8KB pages, cached in the buffer pool. When you UPDATE a row, Postgres modifies the page in the buffer pool — that page is now dirty (changed in RAM, not yet on disk). The naive durability plan is: on COMMIT, write every dirty page back to its home location on disk, then return.

Two things kill this plan:

It's slow in the worst possible way. A transaction might touch a handful of rows scattered across several pages — and those pages live at random spots in the table file. Writing them means random writes all over the disk. From D1 you know random I/O is the slow case; forcing several random page writes on the critical path of every commit would make writes crawl.
It's not even safe. A page write can be interrupted. If power dies while a single 8KB page is half-written (a torn page), you now have a page that's neither the old version nor the new one — silent corruption, with no log of what was supposed to happen.

So the very thing you'd do naively is both the slowest option and an unsafe one. WAL fixes both at once.

Write-ahead logging: log first, pages later

The rule is in the name — write ahead: before a change is considered durable, a record describing that change is written to a separate, append-only file called the WAL (also "the transaction log"; InnoDB calls its equivalent the redo log). The sequence for a commit is:

Modify the page in the buffer pool (dirty it) — fast, in RAM.
Append a small WAL record to the end of the WAL file: "in page X, at offset Y, this changed to this." Just the delta, appended after the last record.
fsync the WAL — force the operating system to actually flush those WAL bytes from its cache down onto physical disk (much more on this word in a moment).
Now return "committed."

The actual data pages are not written to their home locations at commit time. They stay dirty in the buffer pool and get flushed lazily, later, in the background. Commit durability rides entirely on the WAL being safely on disk.

Why is this fast? Because the WAL is append-only. Every WAL record for every transaction, in commit order, goes to the end of one file — the disk head (or the SSD's write path) writes one contiguous, ever-growing stream. That's sequential I/O, the fastest thing storage does — and it's the exact asymmetry from D1, now paying off: instead of several random page writes per commit, you do one sequential append. A slow random write becomes a fast sequential one, and that single change is what lets a database commit thousands of transactions a second.

COMMIT path (what's on the critical path):

  buffer pool                WAL file (append-only, sequential)
  [page 12*]  ---- record -->  ...|rec|rec|rec|NEW REC|  ==fsync==> disk ✔
  [page 88*]  ---- record -->                                      |
   (* = dirty, stays in RAM)                            "committed" returns here

  ...much later, in the background:
  checkpoint --- flush dirty pages 12, 88 to their random home locations on disk

fsync: the word your durability rests on

Step 3 hides the whole ballgame. When a program writes to a file, the bytes usually land in the OS's own write cache (the page cache from D1) and the write call returns immediately — the data is not on physical disk yet, it's queued. That's great for speed and fatal for durability: a power cut loses everything still in that cache. fsync(fd) is the system call that says "do not return until every byte for this file is truly on the physical medium." It is the difference between "the OS has my data" and "the disk has my data," and it is comparatively expensive — you're waiting on physical hardware to confirm.

This is why "committed" costs what it costs. The WAL fsync before returning is the price of the durability promise. Which raises an obvious optimization.

Group commit: amortizing the fsync

If ten transactions all want to commit within the same handful of milliseconds, Postgres doesn't have to fsync ten times. It can let their WAL records pile up in the buffer, then do one fsync that flushes all ten at once — and every one of those ten commits returns as soon as that single flush completes. This is group commit: under load, the fixed cost of an fsync is shared across many concurrent commits, so throughput climbs even though each individual commit still waits for a real flush. Higher concurrency actually makes the per-commit fsync overhead smaller, which is why a busy database can be more efficient per transaction than an idle one.

The escape hatch: synchronous_commit = off

Postgres lets you opt out of waiting for the fsync. With synchronous_commit = off, COMMIT returns as soon as the WAL record is written to the OS buffer — without waiting for it to reach physical disk. This makes commits noticeably faster, and the trade-off is precise and worth stating exactly:

What you risk: a crash can lose the last fraction of a second of already-acknowledged commits (the WAL records that were in the OS buffer but not yet flushed). The loss window is bounded and small.
What you do not risk: corruption. The WAL is still written in order and replayed correctly on recovery; you can lose recent commits but the database always comes back consistent, never torn.

That distinction — bounded recent loss, never corruption — is the entire reason it's a safe knob to reach for. For Fable's money writes it stays on (losing a settlement is unacceptable); for high-volume, reconstructable data like analytics events, off is a reasonable speed win. Knowing precisely what you're trading is the point.

Checkpoints and crash recovery

If dirty pages only get flushed "lazily," when does that actually happen, and how does recovery work? Both answers are the same mechanism: the checkpoint.

A checkpoint is a periodic event where Postgres flushes all currently-dirty pages to their home locations on disk and records a marker in the WAL: "as of here, every change up to WAL position P is safely in the data files." This is deliberately spread out over time to avoid a sudden storm of random writes. The checkpoint is what lets the WAL be trimmed — once the pages a WAL segment describes are safely flushed, that segment is no longer needed for recovery and can be recycled.

Now the payoff. On restart after a crash, Postgres does crash recovery (also called REDO):

Find the last completed checkpoint marker in the WAL.
Replay every WAL record after it, in order, re-applying each change to the pages. Records for pages already on disk are harmless to re-apply; records for changes that never made it to a data page get applied now.
When the WAL is exhausted, the database is back to the exact state of the last acknowledged commit — every committed transaction present, every in-flight-but-uncommitted one discarded.

The WAL is the source of truth for "what happened." The data files are just a cache of the WAL's cumulative effect, allowed to lag behind. A committed change might not be in the data file yet at crash time, but it's always in the WAL, so replay reconstructs it. This is why recovery time is bounded by "how much WAL since the last checkpoint" — more frequent checkpoints mean faster recovery but more constant background write work, a tuning trade-off you now understand from the inside.

One more thing the WAL is: the replication stream

Here's a thread to pull later. The WAL is a complete, ordered description of every change the database made. Feed that same stream to a second Postgres server and have it replay the records, and that server becomes a live, up-to-date replica — it stays in sync by replaying the primary's WAL. This is exactly how Postgres streaming replication works, and it's why the WAL sits at the center of high availability, read replicas, and point-in-time recovery. We'll build on this in the consistency track; for now just register that the log you write for durability is the same log you ship for replication. One mechanism, two payoffs.

fsyncgate: the day fsync itself lied

The entire durability argument above rests on one assumption: that when fsync returns success, the data is on disk, and if it fails, you find out and can retry. In 2018 the Postgres team discovered this assumption was wrong on Linux — the incident is remembered as fsyncgate. On some kernels, if a write-back error occurred (a transient disk problem), the OS would report the fsync failure once, then clear the error flag. Postgres, following decades-old Unix convention, assumed a failed fsync could be retried — so it would call fsync again, get a success back (because the error was already cleared), and conclude the data was safely flushed. But the dirty pages had been dropped on the failed attempt. The result: a COMMIT reported durable, confirmed durable on retry, and the data was simply gone — the worst possible failure, a silent lost write that every layer believed it had handled. The fix was brutal and correct: on any fsync failure, Postgres now PANICs — crashes the whole server on purpose — so that crash recovery replays the WAL from a known-good checkpoint rather than trusting a second fsync that might be lying. The lesson generalizes past this one bug: durability is only as strong as the weakest honest link in the stack, and "the OS told me it worked" is an assumption, not a guarantee.

Where this lives in Fable

Every expense and settlement write in Fable is a real database transaction, and this lesson is why I sleep at night about them. When the API returns 201 for a new expense, that response is gated on the WAL fsync having completed — "committed" means "on disk," so a server crash a moment later cannot lose a recorded expense. That durability guarantee is also what makes Fable's idempotency keys safe: every money write carries a client-generated ULID key protected by a unique constraint, so if the phone doesn't hear back and retries the request, the retry either finds the first write already durably committed (and returns it) or races and loses to the unique constraint. The whole retry-safe design assumes the underlying commit is truly durable and atomic — the WAL is what earns that assumption. synchronous_commit stays on for these tables for exactly the reason in the trade-off table: losing a settlement to a bounded crash window is not acceptable the way losing an analytics ping would be. The write path and the idempotency scheme are covered together in the data model chapter.

Go deeper

PostgreSQL docs — "Reliability and the Write-Ahead Log" — The authoritative explanation of why WAL guarantees durability and how checkpoints, fsync, and recovery fit together, from the people who wrote it.
Designing Data-Intensive Applications — ch. 3 (Storage and Retrieval) — Puts WAL, B-trees, and log-structured storage into one arc so you see write-ahead logging as a general technique, not a Postgres quirk — the book this whole track keeps citing.
The Internals of PostgreSQL — ch. 9 (Write-Ahead Logging) — The diagram-heavy deep dive into WAL records, LSNs, checkpoints, and the exact recovery algorithm when the paragraph here is not enough.

Check yourself

Answer out loud, as if an interviewer asked. If you hand-wave, reread that section.

Your API returns 201 for a new expense and the server loses power 200ms later; the expense survives. Trace exactly what had to be on disk at the moment 201 was sent — and note what was almost certainly still only in RAM.
Writing the changed data pages directly on every commit is rejected for two independent reasons — one about speed (tie it to the random-vs-sequential asymmetry from the pages lesson), one about safety. State both, and explain how appending to the WAL instead fixes each.
What does fsync actually guarantee that a plain write() does not, why is it the expensive step, and how does group commit make its cost shrink as concurrency rises?
A teammate sets synchronous_commit = off and says 'now we might lose data on a crash.' Be precise: what exactly can be lost, what is the bound on it, and what is guaranteed to NOT happen (that would happen with the naive 'just write pages' approach)?
After a crash, how does Postgres get back to a consistent state — what role does the last checkpoint play, and in what sense are the data files just a laggy cache of the WAL?
In fsyncgate, every layer believed the commit was durable and the data still vanished. What false assumption did Postgres make about a failed-then-retried fsync, and why is deliberately PANICking the *correct* fix rather than an overreaction?

The write path: WAL, fsync, and checkpoints

Go deeper

Check yourself

On this page