Folios, page cache, file mappings, and writeback

Before you start#

Read kernel allocation. This chapter uses Linux v6.12 as a historical source baseline. The guest demonstration requires Python 3 and an ordinary writable temporary directory; actual cache residency, readahead and page size depend on the kernel and workload.

Learning goals#

Explain the relationship between an inode, its address_space, the XArray and folios; follow a buffered read and an insertion failure; distinguish dirtying, writeback, invalidation and durable storage.

Source walkthrough#

For a regular cached file, inode->i_mapping leads to a struct address_space. Its i_pages XArray indexes cached entries by page index; its host points back to the inode, and a_ops supplies filesystem address-space operations. invalidate_lock coordinates certain changes to page-cache contents with operations such as truncation. Consult the actual structure in include/linux/fs.h. An XArray entry can represent more than a simple resident page, so a reader must use its documented lookup and locking rules rather than assuming every slot is a bare pointer.

A folio represents a physically contiguous power-of-two collection of base pages managed as a unit; a folio can comprise one base page. It has references and flags such as uptodate, dirty and writeback. A large folio spans multiple page indices. Its contents are file bytes, while the inode contains filesystem metadata and struct file describes an open instance. One inode can have many opens sharing the same mapping. The mapping's XArray does not make unsynchronized pointers safe forever: lookup code must obtain appropriate references and validate state. See mm/filemap.c and include/linux/mm_types.h.

For a buffered read using the generic file path, generic_file_read_iter calls filemap_read; the latter gets batches through filemap_get_pages, then copies available data to the iterator and advances the I/O position by bytes actually copied. At the start of filemap_get_pages, the source computes (GPL-2.0-only; other statements omitted):

	pgoff_t index = iocb->ki_pos >> PAGE_SHIFT;

and then:

	last_index = DIV_ROUND_UP(iocb->ki_pos + count, PAGE_SIZE);

The first expression converts byte position to a base-page index; the second is the exclusive page-index bound for the requested range. The batch lookup uses these indices; a miss can trigger readahead or folio creation and the filesystem's read_folio operation. A file's actual operations can choose a different path. See filemap_get_pages and filemap_read.

A worked cached read#

Suppose a configured machine has 4096-byte base pages, a sufficiently large file, and a read starts at byte 4090 for 20 bytes. This is a hypothetical calculation, not a trace. The first index is 0; the exclusive bound is ceil(4110/4096) = 2. Therefore the requested range intersects base-page indices 0 and 1. A single large folio may cover both, or separate folios may cover them; neither layout is guaranteed by the arithmetic. The reader copies six bytes from the first page's region and fourteen from the next if all bytes are available and the copy succeeds. EOF, an I/O error, interruption or a userspace-copy fault can instead lead to a short result or error under the implementation's rules.

If a cache miss needs a new folio, filemap_add_folio charges memory, marks the folio locked, and calls __filemap_add_folio. Its failure branch in the same file says:

	if (unlikely(ret)) {
		mem_cgroup_uncharge(folio);
		__folio_clear_locked(folio);
	} else {

The charge is undone and the lock bit cleared when insertion fails; the caller still has responsibilities for the folio it allocated. The else path handles successful insertion, including LRU treatment. Competing insertions can make a previously observed miss stale, so the code must handle the insertion result and retry where required. Treating a cache miss as exclusive ownership of an index would be a race.

Mapping, dirtying, and invalidation#

A buffered read copies bytes from cache into a userspace buffer. mmap establishes virtual mappings so userspace loads and stores can access mapped file pages through page faults; it does not copy the whole file at mmap time. Both may involve the same file mapping, but private mappings use copy-on-write for private modifications, while shared writable mappings can dirty file-backed data. Actual behavior depends on mapping flags and filesystem support. Concurrent truncation can invalidate mappings and a later access beyond the new file end can raise SIGBUS.

Dirty means the memory contents need to be reconciled with backing storage; writeback means writeout is in progress, not necessarily complete. A folio can be redirtied during writeback. folio_mark_dirty obtains the mapping and dispatches to the mapping's dirty_folio operation when present; its no-mapping path uses noop_dirty_folio. The implementation and state accounting are in mm/page-writeback.c. Reclaim aims to recover memory and may have to wait for or trigger writeback, but reclaim and persistence are distinct. Invalidation removes or discards relevant cached state under coordination with filesystem changes; it cannot simply discard dirty data without meeting its caller's contract. Completion of writeback is still not, by itself, proof that all application-required metadata and volatile device caches are durable.

In mm/page-writeback.c, v6.12, SPDX GPL-2.0-only (the flag handling and comments inside the mapped branch are elided):

bool folio_mark_dirty(struct folio *folio)
{
	struct address_space *mapping = folio_mapping(folio);

	if (likely(mapping)) {
		/* ... */
		return mapping->a_ops->dirty_folio(mapping, folio);
	}

	return noop_dirty_folio(mapping, folio);
}

Notice that the dirty decision is dispatched through the mapping's a_ops, and that a folio with no mapping takes a different branch. The function's only job here is to mark the folio; it does not start writeback or show that the bytes reached storage. The excerpt does not show the flag clearing that runs before the dispatch.

What cache state actually tells you#

A cache lookup, a folio reference, and the folio lock answer different questions. Finding an XArray entry tells the lookup code that an entry was present at a particular point; it does not grant permanent ownership. A reference keeps the folio from being freed while held. The folio lock serializes operations that require that lock, such as coordinating a not-yet-uptodate folio with a read operation; it is not a lock on every filesystem metadata structure. The code must also verify that the folio still belongs to the expected mapping after races with invalidation or truncation. A filesystem's read_folio method is responsible for starting the appropriate read and completing the folio state according to its contract; merely allocating a folio does not mean its contents are valid file data.

For a concrete state sequence, imagine a cached folio whose current bytes are abc and whose dirty bit is clear. A buffered write modifies the first byte to X and marks the folio dirty; the visible cache bytes are now Xbc, while backing storage may still contain abc. Writeback starts a write of a suitable version of the data. If a later writer changes the bytes again while I/O is in flight, the folio can need another writeback cycle. Completion of the first I/O does not authorize discarding the later modification. The actual lock and dirty-bit protocol is more detailed than this teaching sequence and varies with the filesystem's operations.

Truncating a file changes its logical size and invalidates relevant cached and mapped regions through coordinated paths. A reader must not assume that a reference to an old folio guarantees the old file size or that it still belongs at the same mapping index. invalidate_lock participates in coordinating cache filling and invalidation, while other locks and mapping-specific rules protect other parts of the operation. Invalidation is not a convenient synonym for flushing: a dirty folio may require writeback or another explicitly permitted treatment before removal. That distinction becomes important for direct I/O and hole punching as well as truncation.

Faults and observations are separate evidence#

For a mapped file, a page fault occurs when the process accesses an address whose current page-table state cannot satisfy the access. The fault handler checks the VMA and permissions, then the file's fault operation can locate or populate page-cache state and install a mapping. A later access may not fault, so counting faults is not the same as counting loads. A private writable mapping can initially refer to file-backed data, then a write fault can create private anonymous state. A shared mapping exposes modifications through the file mapping subject to synchronization and filesystem rules. Closing the file descriptor does not by itself unmap a mapping; the mapping has its own lifetime.

Now consider a reader holding a folio reference while another task truncates the file. The reference prevents that folio object from being freed while held, but it does not freeze the file size or prove that the folio is still attached to the original mapping. The reader must obey the code's validation and locking rules. An access to a shared mapping beyond the truncated file's end can signal SIGBUS; a normal read past EOF instead returns zero. This difference comes from the interfaces, not from an assumption that mmap always loads the whole file.

Writeback errors can be discovered later than the userspace write that dirtied a folio. Kernel bookkeeping associates errors with mappings and file descriptions so later synchronization can report relevant failures; the exact reporting rules require reading the relevant implementation and API. A successful read-back before synchronization proves only that the current visible view contains those bytes. It can be served from cache while the device still contains older data. Conversely, an observed cache miss in one run is not a guarantee that a subsequent run will miss: readahead, prior access, memory pressure and other tasks change residency.

This is why the lab's expected aXYdef output is intentionally narrower than a claim about disk persistence. To investigate cache behavior, an experiment would need controlled guest state, a specified filesystem and kernel configuration, instrumentation, repeated trials and a clear cache-control method. Dropping host caches would change unrelated workloads and is outside this lab. A single timing comparison does not by itself identify an individual folio or establish a universal cache algorithm.

XArray state and a concurrent truncate#

address_space.i_pages is an XArray indexed in base-page units. A large folio can cover a range of indices, so code must use XArray/folio helpers rather than treat every index as an independent base-page allocation. The mapping also tracks marked entries used by writeback, including dirty and writeback marks. A folio reference protects its lifetime, the folio lock serializes operations covered by that lock, and mapping->invalidate_lock coordinates cache filling and invalidation with operations such as truncation. These protections solve different problems; holding one does not automatically grant the others. See include/linux/fs.h, include/linux/xarray.h, and mm/truncate.c.

A concrete race occurs in filemap_fault. The fault can obtain a folio reference, then wait for its lock while another task truncates and removes it from the mapping. The v6.12 code rechecks after locking (GPL-2.0-only; surrounding code omitted):

	if (unlikely(folio->mapping != mapping)) {
		folio_unlock(folio);
		folio_put(folio);
		goto retry_find;
	}

The mapping comparison detects that this folio is no longer attached to the expected address space. The fault unlocks it, drops the reference and retries lookup; it does not install the stale folio as the current file page. The same function rechecks i_size after locking and returns VM_FAULT_SIGBUS if the fault index is now beyond the rounded-up file size. This is a specific mmap-fault path, not a claim that every buffered read follows the identical code. In filemap_update_page, a detached folio can instead produce the internal AOP_TRUNCATED_PAGE retry indication; NOWAIT/NOIO conditions can return -EAGAIN rather than waiting.

Suppose a file initially has two base pages and a fault starts on the second. A concurrent truncate removes that range before the fault locks the folio. The mapping recheck can detect the removal, release the stale reference, and retry; the size recheck can then reject a fault beyond the new end. A buffered read at or beyond EOF normally returns zero instead of delivering a mapping fault. A fault in the final partial page and filesystem-specific page_mkwrite behavior require the actual VMA and filesystem rules; the model must not claim every byte beyond EOF has identical semantics.

Dirty, writeback, and observable bytes#

A shared writable mapping can reach a filesystem's page_mkwrite path before the page becomes writable; the filesystem may need allocation, locking and timestamp or journal work. A private writable mapping instead has copy-on-write semantics for its modifications. Buffered writes can dirty folios through filesystem/address-space operations. Writeback selects dirty entries using mapping marks, starts I/O and later ends writeback; a folio redirtied during I/O can remain dirty for a subsequent cycle. An error can be recorded and reported by a later synchronization operation. Clean in cache is a state of that cache entry, not a proof that all directory metadata or a device's volatile cache is durable.

Readahead can fetch more than the immediately requested range. filemap_read can copy only the portion available to an iterator and can return a short count or error according to progress and fault conditions. Neither a readahead request nor a successful userspace read proves that the storage device was accessed during that call. A read can be served from an already cached folio, and a concurrent truncate can change the range before or during the operation. For durability and error reporting, continue to fsync and crash consistency after that later chapter is reached; this is not a prerequisite link.

A second safe observation uses a private temporary file, not a global cache control:

python3 - <<'PY'
import mmap, os, tempfile
with tempfile.TemporaryFile() as f:
    f.write(b'abcde'); f.flush()
    with mmap.mmap(f.fileno(), 5, access=mmap.ACCESS_WRITE) as m:
        m[2:4] = b'XY'; m.flush(); os.fsync(f.fileno())
        f.seek(0); print(f.read())
PY

Expected b'abXYe' if mapping and synchronization succeed. m.flush() requests that the mapped range be flushed to the filesystem; os.fsync() requests synchronization of file data and associated metadata for the descriptor. An exception is a failed observation to record. The program observes a file view and requests synchronization; it does not prove a cache hit, a particular folio size, a truncate race, or survival of power loss. The temporary file is removed on close.

A folio lock protects the folio-level transition being performed; it does not freeze the inode size for all time. In the fault example, the second size check is necessary because a truncate can change the file after the initial check. The shared invalidate lock coordinates the cache fill with invalidation, while a folio reference prevents reclamation during the reader's use. The code's retry is a response to a changed relationship, not an instruction for an application to retry indefinitely after a real I/O error.

Practice#

Source exercise: in an existing v6.12 checkout run rg -n -A35 'int filemap_add_folio\(' "$KERNEL_SRC/mm/filemap.c" and rg -n -A28 'bool folio_mark_dirty\(' "$KERNEL_SRC/mm/page-writeback.c". Identify the charge failure, insertion failure and mapping-dependent dirty callback. Also inspect rg -n -A130 'vm_fault_t filemap_fault\(' "$KERNEL_SRC/mm/filemap.c" and mark the mapping and size rechecks after the folio is locked. The expected result is source text, not a measured cache hit.

In the disposable guest, run this userland experiment:

python3 - <<'PY'
import mmap, tempfile
with tempfile.TemporaryFile() as f:
    f.write(b'abcdef'); f.flush()
    with mmap.mmap(f.fileno(), 6, access=mmap.ACCESS_WRITE) as m:
        m[1:3] = b'XY'
        f.seek(0)
        print(f.read(6))
        m.flush()
PY

Expected on a working regular temporary filesystem: b'aXYdef'. The shared mapping modification and ordinary read observe the same file contents; the precise residency, I/O and persistence sequence is not measured. If mmap or flush fails, report the filesystem and exception rather than assuming the output. TemporaryFile closes and removes the temporary object. Save the page-index calculation and a state sketch distinguishing clean, dirty and writeback; do not label the experiment a crash-durability test.

Knowledge check#

1. Which structure indexes a cached file's folios, and which field points to filesystem operations?

2. Why can the 20-byte example touch two page indices but only one folio?

3. What does the insertion error branch undo?

4. What do shared mmap modification, writeback completion, and durable storage each establish?

5. What does filemap_fault do if truncation detached its folio after lookup?

Answers#

1. The mapping's i_pages XArray indexes entries, and a_ops provides the address-space operations.

2. A multi-page folio can span multiple base-page indices; the byte calculation alone does not determine folio order.

3. It uncharges the folio and clears its locked bit; caller-side ownership and cleanup must still be checked.

4. A shared modification changes the mapped file data view; writeback completion concerns a submitted write; durability additionally depends on filesystem synchronization, ordering and storage guarantees.

5. After locking, it detects the mapping mismatch, unlocks and puts the stale folio, and retries lookup; it also rechecks file size before returning a locked page.

Checkpoint#

Save the page-index arithmetic, the filemap_add_folio error explanation and the observed Python output. Pass when they distinguish one mapping from an open file, a page index from a folio, and dirty data from durable data. If not, read the two functions again and retry. Progress is not certification.

Sources#

• address_space, Linux v6.12.
• File mapping and read implementation, Linux v6.12.
• Dirtying and writeback, Linux v6.12.
• Memory types and folios, Linux v6.12.
• Page cache documentation, Linux 6.12.