I/O Buffering and Kernel Buffer Caches

Core Idea

The kernel maintains buffer caches between processes and storage devices to reduce I/O latency and optimize bandwidth utilization. Write-back caching defers writes to disk; read caching avoids repeated disk accesses for the same data. Buffer cache management, including dirty buffer flushing and page replacement policies, is critical to both performance and data durability guarantees.

Explainer

You know from I/O systems that storage devices are dramatically slower than the CPU and main memory — a disk read takes milliseconds while a memory access takes nanoseconds, a gap of roughly six orders of magnitude. From cache design principles, you know the strategy: put a faster, smaller storage layer in front of a slower, larger one and exploit temporal and spatial locality. The kernel's buffer cache (also called the page cache in modern Linux) applies this exact strategy to disk I/O, sitting in main memory between user processes and the storage device.

When a process calls `read()`, the kernel first checks whether the requested disk blocks are already in the buffer cache. If they are (a cache hit), the data is copied from memory to the process — no disk access needed. If not (a cache miss), the kernel reads the block from disk into the buffer cache, then copies it to the process. Subsequent reads of the same block by any process hit the cache. The kernel also performs read-ahead: when it detects sequential access patterns, it prefetches upcoming blocks into the cache before they are requested, turning what would be many small random reads into efficient sequential I/O.

Writing has two strategies with very different trade-offs. Write-through writes data to both the cache and the disk immediately — safe but slow, since every write blocks until the disk confirms. Write-back (the default in most operating systems) writes only to the cache and marks the buffer as dirty, deferring the actual disk write to a later time. This is dramatically faster for the application — `write()` returns almost immediately — but creates a durability risk: if the system crashes before dirty buffers are flushed to disk, that data is lost. The kernel mitigates this by periodically flushing dirty buffers (Linux's `pdflush`/`writeback` threads run every few seconds) and by honoring explicit `fsync()` calls from applications that need durability guarantees, such as databases.

Managing the buffer cache means deciding which blocks to keep and which to evict when memory is full. The kernel uses page replacement policies (like LRU approximations) to evict the least-recently-used cached blocks, making room for new ones. This is the same fundamental trade-off you studied in cache design: cache size versus hit rate, write-back versus write-through, and eviction policy. The buffer cache is why a program that reads the same configuration file a hundred times does not generate a hundred disk reads, and why a burst of small writes to a log file does not produce a burst of disk I/O — the kernel absorbs the writes into memory and flushes them efficiently in larger batches.