bio, scatter-gather, DMA, and I/O completion

Before you start#

Read blk-mq. This chapter uses Linux v6.12 as a historical source baseline. DMA details depend on the device, architecture, IOMMU, coherency, and configuration. The observation reads kernel-exposed block information without sending I/O to a selected disk.

Learning goals#

Follow bio ranges and completion ownership, explain splitting and chaining, distinguish CPU and DMA address spaces, and handle mapping and completion failures without confusing userspace vectors with device mappings.

Source walkthrough#

A useful mental path is: request data range → bio and active bio_vec range → driver scatterlist → mapped DMA segments → device access → DMA unmap/synchronization → bio completion. These are related stages, not interchangeable representations; each handoff has its own owner and lifetime. The sections below follow that path.

struct bio represents a block operation on bi_bdev, with operation flags, an iterator, completion state and callback. A bio_vec identifies a memory segment using a page, offset and length. The iterator (bi_iter) carries the current sector, remaining size and vector traversal position. It is a view of the remaining part of an operation; advancing it changes the current position and size, not the original meaning of the pages. Sector numbers use 512-byte units even when a device has a different logical block size. The structures are defined in include/linux/blk_types.h and helpers in include/linux/bio.h.

A bio can be referenced independently of its payload. A reference to a bio does not automatically give arbitrary code permission to free its pages, nor does a page reference make it safe to alter a buffer while a device owns it. The submitting layer, clone user, block layer and driver have different responsibilities. bi_end_io identifies a callback invoked when the bio's completion conditions are met; bi_status carries block-layer status. The callback must honor the bio's allocation and ownership contract, including references held by other users. An asynchronous bio cannot have its memory reused just because its submitter returned to userspace.

Consider a bio at sector 80, size 8192 bytes, backed by two 4096-byte vector segments. It covers sectors 80 through 95 inclusive. If a queue limit requires splitting off eight sectors, the front bio covers sector 80 through 87 and 4096 bytes; the remaining bio advances to sector 88 and 4096 bytes. The vectors are descriptions of memory, not necessarily individual on-wire DMA segments. Actual segment combining and device limits can produce a different layout. This is a calculated teaching example, not a captured request.

bio_split creates a clone, sets the front size and advances the original iterator. Its core fragment (GPL-2.0; intervening error handling omitted) includes:

	split->bi_iter.bi_size = sectors << 9;

and later:

	bio_advance(bio, split->bi_iter.bi_size);

The shift converts the 512-byte sector count to bytes; advancing adjusts the original to the remainder. The helper rejects invalid counts with assertions, handles zone-append differently, and can fail allocation. Its comment says that, except for discard, the split bio points to the original bio's vector storage and that the original bio and bioset must remain alive until the split bio is finished. Splitting is therefore not an independent deep copy of all pages or vector metadata.

Completion and error aggregation#

A chain ties child completion to a parent. In block/bio.c, bio_chain sets the child's private pointer and completion callback and increments the parent's remaining count. The child must not already have those callback fields set for this API. The chained end helper contains (GPL-2.0; preceding declaration omitted):

	if (bio->bi_status && !parent->bi_status)
		parent->bi_status = bio->bi_status;
	bio_put(bio);
	return parent;

A nonzero child status is propagated if the parent does not already have an error, the child reference is put, and the parent is returned for completion processing. The parent is not reported complete merely because one child ended. bio_endio first checks bio_remaining_done; it also handles integrity completion and other bookkeeping before invoking the final callback. A chained child is handled with an iterative goto again path to avoid unbounded recursion. The actual reported status is a block status, not a detailed list of every lower-device failure.

For the split example, imagine the front 4096 bytes fail and the remaining 4096 bytes complete successfully. A properly chained arrangement propagates the failing status to the parent and waits for the required completions before the parent's final callback. A caller must not claim an entirely successful 8192-byte operation because the second half succeeded. Nor may it free shared vector backing while the other half still uses it. A particular upper layer may have more detailed partial-I/O handling; this model does not assert that every filesystem split uses exactly this chain arrangement.

Scatter-gather is not the same as a userspace iovec#

A userspace iovec describes buffers to a syscall such as readv or writev. Those are userspace virtual addresses and lengths; they do not by themselves pin pages, produce a bio, establish physical contiguity, or map anything for a device. A kernel bio_vec describes pages and offsets for a block operation. A scatterlist describes segments supplied to the DMA API or other kernel interfaces. A driver can form a scatterlist from a request's segments, subject to its device limits, then map it for the specific device.

A CPU virtual address, a physical address, and a dma_addr_t are different address domains. An IOMMU may translate device-visible addresses, and bounce buffering or other implementation details can alter the mapping. The driver must use dma_map_* for the right device and direction and check the API's error result. For dma_map_sg, zero indicates mapping failure. A positive returned count is the number of mapped DMA segments to program; it can be smaller than the original scatterlist input count because entries may be merged. Keep both counts distinct: pass the mapped count to device programming, but pass the original input count to the matching unmap call. See kernel/dma/mapping.c and the DMA API HOWTO.

In kernel/dma/mapping.c, v6.12, SPDX GPL-2.0 (surrounding code omitted), the exported scatterlist wrapper returns zero for any negative internal result:

unsigned int dma_map_sg_attrs(struct device *dev, struct scatterlist *sg,
		    int nents, enum dma_data_direction dir, unsigned long attrs)
{
	int ret;

	ret = __dma_map_sg_attrs(dev, sg, nents, dir, attrs);
	if (ret < 0)
		return 0;
	return ret;
}

The wrapper discards the specific negative error and returns the zero used for failure, so a caller that needs to tell -ENOMEM from -EINVAL should look at the sgtable interface, whose return codes are documented in the same file. This excerpt does not show __dma_map_sg_attrs, the entry merging behind the returned count, or any IOMMU behavior.

Direction is from the device's perspective. DMA_TO_DEVICE means the device reads memory, as for writing data to storage. DMA_FROM_DEVICE means the device writes memory, as for a storage read. DMA_BIDIRECTIONAL has different synchronization requirements and is not a substitute for identifying the actual direction. Mapping and synchronization establish an ownership protocol: while a streaming DMA mapping is owned by the device, CPU access must obey the API's synchronization rules; after completion, unmap or synchronize as required before the CPU relies on the data. On noncoherent systems the API can perform cache maintenance. A coherent allocation avoids some cache-maintenance work but still requires ordering, device-specific barriers, and a lifetime that covers the device's access.

Imagine three input scatterlist entries mapped into two DMA segments. The driver programs the two DMA segments returned by the mapping. If mapping returns zero instead, it must not program the device with CPU pointers and must release any earlier resources. If the device times out, unmapping and freeing memory before stopping or otherwise synchronizing the device can permit a late DMA write into reused memory. The exact abort and reset procedure belongs to the driver and device; the DMA API alone does not stop hardware.

Race and teardown reasoning#

The bio_endio callback may execute in a completion context where sleeping is not allowed; exact context is determined by the submitting path and callback contract. Callback code should not infer process context from the identity of the task that originally submitted I/O. A callback that wants to perform sleepable work may need an appropriate deferred mechanism and must keep the necessary references until that work completes. Completion can race with cancellation or device removal, so an owner must arrange that exactly the appropriate completion and cleanup occur under its protocol.

A DMA unmap is also not synonymous with bio completion. The driver first needs hardware completion or a successful stop/reset protocol. It then unmaps its DMA mapping using the device, direction and original size/count required by the API, updates the request or bio status, and releases its own bookkeeping in the order required by that driver. Completion of an upper bio can happen only when all lower work on which it depends has been handled. In a stacked device, each layer may have its own clone, callback and references; a single callback is not necessarily the physical device interrupt handler.

Clone and iterator lifetime in detail#

A cloned bio is a new descriptor referring to an existing data description; the exact helper and flags determine what metadata and vectors are shared. It is not a second, independent copy of every data page. A stacking target may change its clone's device and sector without changing the user's original logical address. It must still retain whatever backing objects the clone needs until its completion. A bio_get reference and a page or vector lifetime solve related but different problems; the relevant allocation and clone contracts must be checked together.

The bvec_iter expresses progress through the vector array, including a current index and amount consumed within the current vector. Advancing a bio by 4096 bytes can move to the next vector or leave the iterator partway through a larger vector. A driver constructing a scatterlist must iterate the active range, not assume every entry in the underlying array is still part of the request. A split can adjust two views while both refer to the same underlying data. If the first view ends, the second may still need that data and the original vector storage.

A completion chain has a count because completion order is not guaranteed. In the two-piece example, the second piece could finish first. Its success does not authorize the parent's callback to run while the first piece is outstanding. If the first later fails, the parent's final status reflects the applicable error propagation. If the chain has more children, the same reasoning applies to each required completion. This is a relationship explicitly established by bio_chain; two unrelated bios do not automatically form such a group just because they originated from one application write.

Direction and synchronization example#

Consider a disk read into a memory buffer on a noncoherent platform. Before issuing the command, the driver maps the scatterlist for DMA_FROM_DEVICE and checks the result. The device writes through the returned DMA addresses. After the hardware indicates completion, the driver follows the DMA API's synchronization or unmap rules before the CPU consumes the bytes. The exact cache operations are architecture-specific and performed by the API as appropriate; a driver should not substitute a guessed cache flush instruction. A failed command can still require unmapping the mappings that succeeded before the error.

For a disk write, the device reads memory and the direction is DMA_TO_DEVICE. The caller must not modify bytes while the device is entitled to read them unless a specific synchronization protocol allows it. Reusing the buffer early can produce a mixture of old and new contents even if no memory fault occurs. A coherent allocation changes cache visibility properties, not the need to order descriptor writes before notifying the device or to prevent access after free. A memory barrier and a DMA mapping solve different problems; use the driver and API documentation for the relevant platform.

These examples are ownership sequences, not proof that all storage controllers use the same hardware descriptors. Some devices have segment boundaries, address-width restrictions, or maximum descriptor counts; DMA mapping can use an IOMMU or bounce buffers. A source-only learner can still trace who maps, who unmaps, and which failure path releases earlier mappings without access to a particular physical device.

Practice#

Source-only route:

git -C "$KERNEL_SRC" describe --tags --exact-match
rg -n -A35 'struct bio \*bio_split\(' "$KERNEL_SRC/block/bio.c"
rg -n -A25 'static struct bio \*__bio_chain_endio' "$KERNEL_SRC/block/bio.c"
rg -n -A35 'unsigned int dma_map_sg_attrs\(' "$KERNEL_SRC/kernel/dma/mapping.c"

Expect v6.12 and the split, status propagation and DMA map definitions. Calculate the sector-80 split above, mark shared vector ownership, and write a failure timeline in which mapping fails before the device is programmed. The source alone does not measure an actual DMA transaction.

Read-only Linux observation:

for q in /sys/block/*/queue; do
  [ -d "$q" ] || continue
  printf '%s\n' "$q"
  for f in logical_block_size physical_block_size max_segments max_segment_size; do
    [ ! -r "$q/$f" ] || printf '  %s = %s\n' "$f" "$(cat "$q/$f")"
  done
done

The files, when exposed and readable, describe queue limits in bytes or counts according to the named attribute. A segment limit is not a captured scatterlist and does not reveal a DMA address. Missing attributes or no visible block device are legitimate configuration/container outcomes; report them. No disk is opened for writing and no privileged operation is needed. Deliver the split calculation, a table of iovec/bio_vec/scatterlist/DMA-address meanings, and a map-failure cleanup sequence.

Knowledge check#

1. What changes when the 8192-byte example is split after eight sectors?
2. What does bio_chain do with child status and when can the parent callback run?
3. Why can the count returned by dma_map_sg differ from the input count, and which count is passed to unmap?
4. Why is DMA_TO_DEVICE the natural direction for a device writing memory contents to storage?
5. Why is unmapping immediately on timeout potentially unsafe?

Answers#

1. The front is 4096 bytes at sector 80; the original advances to sector 88 with 4096 bytes left. The split can share the original vector storage.
2. The helper propagates a child error when the parent has no status and puts the child; the parent's final callback waits for its remaining completion count.
3. Mapping can merge segments. The returned count is used to program mapped DMA segments, while unmapping uses the original input count.
4. The device reads the memory buffer, so the data direction is from memory to the device.
5. Hardware may still access the mapping; the driver must stop or synchronize it under its device-specific protocol before memory is reused.

Checkpoint#

Save the split and ownership diagrams and the queue-limit output or missing-device explanation. Pass when your diagram distinguishes data ownership, bio completion, DMA mapping and final resource release, and correctly handles a failed mapping and one failed child. Re-read bio_split, bio_chain, and the DMA API contract if any count or owner is ambiguous. This is practice, not certification.

Sources#

• bio splitting and completion, Linux v6.12.
• bio structures.
• bio helper interfaces.
• DMA mapping implementation.
• DMA mapping guide.