Crash Consistency and Journaling File Systems

Core Idea

A single file operation, such as appending one block, changes several on-disk structures: the data block, the inode and the free-space bitmap. The disk writes them one at a time, so a power loss or crash in between can leave the structures disagreeing with one another. Crash consistency is the problem of making sure the file system can always return to a consistent state. Journaling solves it by first writing a description of the planned updates to a log (the journal), marking it committed, and only then writing the updates in place, so recovery can simply replay committed entries.

How It's Best Learned

List the three writes needed to append a block, then enumerate every crash point and the state each leaves behind, as in the OSTEP chapter on crash consistency. Compare the cost of the older fix, a full-disk scan with `fsck`, against replaying a small journal. Finish by comparing data journaling with ordered (metadata-only) journaling, which ext4 uses by default.

Common Misconceptions

• Writing blocks in a careful order is enough to avoid all inconsistency. (A crash can still fall between any two writes; ordering alone can reduce harm but cannot make a multi-block update atomic.)
• Journaling avoids extra disk writes. (It writes journaled blocks twice, once to the log and once to their final location, and trades that cost for fast, reliable recovery.)
• A journaling file system never loses recent data. (It keeps the structures consistent, but updates that were not yet committed to the journal at the time of the crash are lost.)

Explainer

From file system implementation you know that a file is described by an inode, its contents live in data blocks, and a bitmap records which blocks are free. Now look at an ordinary operation: appending 4 KB to a file. Three on-disk structures must change. A new data block is written, the inode gains a pointer to it and a larger size, and the data bitmap marks the block used.

The disk can persist only one write at a time, and the system may crash or lose power between any two. Each crash point leaves a different mess. If only the data block is written, nothing points to it, so the data is lost but the structures still agree. If only the inode is written, it points to a block that may hold old garbage, and the bitmap still calls that block free. If only the bitmap is written, a block is marked used but belongs to no file, so space leaks. This is the crash-consistency problem.

The older answer was to let crashes happen and repair afterward. A tool called fsck scans every inode, directory and bitmap and fixes disagreements. It works, but it is slow, because its time grows with the size of the disk. On a large disk it can take many minutes or longer.

Journaling, a form of write-ahead logging, avoids the scan. Before touching the real structures, the file system writes a transaction to a reserved area called the journal: a begin block, copies of the blocks it is about to change, and finally a commit block. Once the commit block is on disk, the transaction is durable. The file system then writes the blocks to their real locations, a step called checkpointing, and later frees that journal space. After a crash, recovery reads the journal, replays every transaction that has a commit block, and ignores incomplete ones. Either the whole update happens or none of it does.

Copying user data into the journal doubles the cost of every write, so most systems journal only metadata. In ext4's default ordered mode, the data block is written to its final location before the metadata transaction commits. That way an inode never ends up pointing at garbage. Other file systems, such as ZFS and Btrfs, use copy-on-write instead. They never overwrite live blocks in place, and they switch to the new version with a single atomic pointer update.