DRM/KMS, GPUs, dma-buf, and fences

Before you start#

Review DMA mappings and device ownership, device lifecycles, and locking. This chapter reads upstream Linux v6.12. A DRM-capable device, atomic modesetting support, appropriate driver, permissions, and compatible userspace are separate prerequisites; none is guaranteed by x86-64 QEMU or arm64. The worked examples are source reasoning, not captured execution.

Learning goals#

Explain the relationship between display configuration and buffer storage; trace validation and commit; account for one dma-buf reference across success and error; and distinguish a fence signaling completion from ownership of the memory it protects.

Source walkthrough#

The Direct Rendering Manager (DRM) supplies interfaces for graphics devices. Kernel Mode Setting (KMS) represents display pipelines. A framebuffer describes pixel storage and layout; a plane selects a framebuffer and placement; a CRTC combines plane data and drives display timing; a connector represents an output connection. The documented pipeline feeds framebuffer data through planes to a CRTC, then routes output toward a connector; encoders may also appear in the kernel model. Actual drivers and hardware have different arrangements. Rendering and scanout are logically distinct jobs even if integrated into one device. KMS object model

Atomic KMS groups proposed changes in a drm_atomic_state. “Atomic” here describes a coordinated configuration interface; it does not mean every hardware register changes in one CPU instruction, or that a nonblocking commit has physically finished before returning. In drm_atomic.c, drm_atomic_commit contains this exact excerpt (v6.12, permission notice in the source; copyright Red Hat, Intel and The Linux Foundation; preceding and following lines omitted):

	ret = drm_atomic_check_only(state);
	if (ret)
		return ret;

	drm_dbg_atomic(state->dev, "committing %p\n", state);

	return config->funcs->atomic_commit(state->dev, state, false);

The first line checks the proposed state. The following conditional propagates failure instead of committing. The debug message is diagnostic, not the operation. The final line calls the device's atomic_commit callback; false identifies this blocking entry point's argument. drm_atomic_check_only first checks individual old/new planes and CRTCs, then performs additional consistency and driver checks. Its loops return immediately on an error. A driver may reject a configuration because constraints such as supported formats, shared resources, or bandwidth are not met; the exact reason is driver specific. Consult the callback and helper documentation before interpreting completion or error cleanup for a specific driver.

A stateful display transaction

Imagine an application proposes a plane with framebuffer A on CRTC 0. This is a hypothetical state, not an observed run. The core checks A's new plane state against the old one. If the check returns -EINVAL, drm_atomic_commit returns that error and does not call the driver's commit callback on this path. The application can construct a corrected request, for example using a supported format discovered through the relevant UAPI. If validation succeeds, the callback receives the proposed state; the driver and its helpers implement the hardware-specific commit. Later an application proposes framebuffer B. Whether it may safely reuse A depends on the relevant completion and synchronization contract, not merely on having sent a request.

A test-only request checks whether a configuration is acceptable without applying it, but conditions can change before a later real commit. Locks protect state and may require retry: the documentation around drm_atomic_commit describes -EDEADLK and the wound/wait backoff protocol. That is different from ignoring an error and looping blindly. See atomic KMS documentation and the source comment for the exact caller obligations.

Sharing memory and completion#

A dma-buf is an object through which an exporter and importer cooperate over shared backing storage. Its file descriptor is a handle, not the physical address of a contiguous buffer. An importer attaches a device and maps an attachment for DMA according to the exporter and DMA APIs. Mappings, direction and cache synchronization matter, especially on non-coherent systems. The dma-buf documentation describes these contracts; the concrete code is in dma-buf.c.

Consider a second source-level scenario: dma_buf_get(fd) calls fget(fd). If that fails, it returns ERR_PTR(-EBADF). If the file is not a dma-buf, it calls fput(file) and returns ERR_PTR(-EINVAL). On success it returns file->private_data; the acquired file reference keeps the object alive. A successful caller eventually balances it with dma_buf_put, which calls fput(dmabuf->file). Therefore a wrong-type file does not leak the reference acquired by fget, and a caller must not call dma_buf_put on an error pointer. The final release depends on all references, not merely one process closing one descriptor.

In dma_buf_put, Linux v6.12, GPL-2.0-only, the release side is a single fput on the file that dma_buf_get acquired:

void dma_buf_put(struct dma_buf *dmabuf)
{
	if (WARN_ON(!dmabuf || !dmabuf->file))
		return;

	fput(dmabuf->file);
}

The WARN_ON branch refuses to release a null or file-less object, so the balance applies to a buffer that dma_buf_get returned successfully. The excerpt does not show other references, such as attachments, that can keep the object alive after this call.

A dma_fence represents completion of a particular asynchronous operation. It is not the buffer, its file descriptor, or a guarantee that all operations on that buffer are complete. A dma-resv object holds a set of fences for a buffer; implicit synchronization selects dependencies based on fence usage and access mode. Explicit synchronization passes a particular dependency, often using sync-file fds or KMS properties. A signaled fence can report an error, and an unsignaled fence can leave dependent work waiting. Fences order device work; they do not themselves perform CPU cache maintenance. dma-buf primitives and synchronization

Object names, files, and the ioctl boundary#

A process normally reaches DRM through a device node opened as a file descriptor. In the kernel, struct drm_device represents the DRM device and struct drm_file represents per-open DRM state. The file is significant: object handles and capability settings have per-file consequences. Opening two DRM files does not make an integer handle in one automatically meaningful in the other. Access to primary nodes, render nodes, modesetting, and DRM master or leasing is controlled by different rules; simply opening a render node does not authorize changing the display. The DRM ioctl dispatcher validates and routes requests, and drivers can add driver-specific ioctls. The atomic ioctl implementation is one concrete UAPI path, not a claim that every ioctl follows the atomic path.

GEM, the Graphics Execution Manager, gives drivers a common object and handle framework. A GEM object describes a buffer-related object, with driver-specific backing and behavior; the generic object is not necessarily physically contiguous memory. A GEM handle names an object in a particular DRM file's handle table. In drm_gem.c, drm_gem_handle_create delegates handle construction under the object-name lock. Closing a handle drops the handle's reference; it does not guarantee immediate destruction while framebuffers, mappings, exports or other references remain. A process that closes its file drops that file's handles, again subject to other references.

A GEM mmap offset is different from the GEM handle. It is a device-specific offset used in a mapping operation, checked against the object's mmap permissions; it is not a physical address and is not the pixel bytes themselves. A successful mmap creates a userspace virtual mapping whose lifetime and CPU access rules depend on the object and driver. Mapping is not implicit synchronization with a concurrent device. A framebuffer ID instead names a KMS framebuffer object describing scanout dimensions, pixel format, pitches, offsets and backing objects. It can reference one or more GEM handles at creation, depending on format and driver. The framebuffer ID is not an mmap offset or a dma-buf descriptor. A dma-buf fd is a transferable file descriptor naming a shareable buffer object; import or export creates appropriate relationships with GEM and drivers where supported. Its integer belongs to the process's fd table. These four integers can coincide numerically while naming entirely different resources.

Consider a compositor with DRM file F and GEM handle H. It creates framebuffer P using H and requests a plane change to P. The handle H is meaningful to F; P refers to the KMS framebuffer object and its backing references. Another process receiving a dma-buf fd D can import the shared object into its own DRM file, obtaining a different local handle. Closing D does not close F's H. Conversely, releasing H does not prove that scanout is complete or that the framebuffer and imported users have released their references. The correct question at each step is which object owns a reference and which API operation releases it. An fd, mmap, framebuffer, GEM object and hardware use may have overlapping but nonidentical lifetimes.

Atomic UAPI, locks, and helper completion#

drm_mode_atomic_ioctl receives object IDs and property arrays in struct drm_mode_atomic. It verifies atomic support and the per-file atomic capability, rejects unsupported flags and a nonzero reserved field, and rejects combining test-only with a page-flip event. It allocates drm_atomic_state, initializes a modeset acquisition context and builds new state from userspace properties. A failed userspace copy, lookup, property validation, allocation or lock acquisition can stop the request. When the modeset wound/wait mechanism reports -EDEADLK, the ioctl cleanup and backoff path clears the state, backs off and retries; other failures must not be disguised as a retry. The exact implementation and its retry label are in the pinned UAPI source. The check stage is where core and driver validation establish whether the proposed collection is acceptable.

A driver can implement its own atomic callbacks or use helpers. In the helper implementation, plane preparation happens before the software state swap. For the ordinary path, drm_atomic_helper_commit uses this short v6.12 excerpt from drm_atomic_helper.c; the source carries a permissive copyright and license notice, and surrounding lines are omitted:

	ret = drm_atomic_helper_prepare_planes(dev, state);
	if (ret)
		return ret;

Preparation can fail before the helper crosses into the later commit sequence, and this snippet propagates that error rather than pretending the framebuffer was successfully applied. In the blocking case the helper subsequently waits for relevant fences before swapping state; after the swap it obtains a state reference and invokes or queues commit-tail work depending on the nonblocking flag. The helper's asynchronous-update branch differs, so this is not a universal sequence for every commit. The helper tail and driver callbacks perform hardware programming and later cleanup according to their contracts. State and framebuffer references must survive asynchronous work. A page-flip event communicates a particular completion event to userspace; it is not a global assertion that all rendering on all buffers is complete. Userspace should follow the explicit event and fence rules for its own request and not free or reuse data merely because a nonblocking ioctl returned.

In the same drm_atomic_helper_commit, Linux v6.12, the blocking branch waits for fences and then swaps state (MIT permission notice in the source; the asynchronous-update branch is omitted, and the point-of-no-return comment is elided):

	if (!nonblock) {
		ret = drm_atomic_helper_wait_for_fences(dev, state, true);
		if (ret)
			goto err;
	}
	/* ... point-of-no-return comment omitted ... */
	ret = drm_atomic_helper_swap_state(state, true);
	if (ret)
		goto err;

The wait runs only for a blocking commit. The swap is the software point of no return, and the source comment treats what follows it as committed. The excerpt does not show the nonblocking branch or the commit tail, so it does not describe every helper commit.

Synchronization and a failed import#

With implicit synchronization, participating drivers use fences associated with a buffer's dma_resv reservation object to express relevant dependencies. Reservation usage and access mode determine which fences matter; “wait on one fence” is not equivalent to “all work everywhere finished.” With explicit synchronization, a protocol passes a particular fence, commonly via a sync-file descriptor or KMS fence properties, so a recipient can wait on the specified dependency or receive an output fence. The two approaches can coexist. Neither by itself provides CPU cache maintenance or proves a producer wrote meaningful pixels.

For an imported dma-buf, the importer attaches its device, maps the attachment in the appropriate DMA direction, uses the returned scatter-gather mapping under the exporter and DMA contracts, then unmaps and detaches in the required order. That mapping belongs to the relevant attachment and direction; a physical address or scatterlist from one device is not automatically valid for another. CPU access to shared mappings must follow the applicable begin/end CPU access or userspace DMA-BUF synchronization contract, especially for non-coherent systems. Fences order asynchronous operations; cache synchronization addresses visibility. They solve related but distinct problems.

The v6.12 dma_buf_get implementation in dma-buf.c contains this exact GPL-2.0-only excerpt; surrounding lines are omitted:

	if (!is_dma_buf_file(file)) {
		fput(file);
		return ERR_PTR(-EINVAL);
	}

The type check tests the file acquired earlier by fget; fput releases that temporary reference before the encoded error pointer is returned. This is a failure cleanup path, not a successful dma-buf reference.

A concrete error case: process X passes an ordinary file fd to dma_buf_get. fget can succeed, adding a file reference, but is_dma_buf_file fails. The function calls fput and returns ERR_PTR(-EINVAL); it must not retain that temporary reference. If fget itself fails, the error is -EBADF and there is no acquired file reference to drop. On success, the dma_buf_put call balances the acquired reference. This matters independently of whether the later attachment or fence operations succeed; each successful acquisition must have a cleanup path if a later stage fails. See dma-buf.c and the sharing documentation.

Read-only Linux observation lab#

On your own authorized Linux host or in a disposable guest, first identify the running kernel with uname -r, then inspect DRM class entries without changing display state:

uname -r
ls -l /sys/class/drm
for p in /sys/class/drm/card*-*/status; do
  [ -r "$p" ] && printf '%s: %s\n' "$p" "$(cat "$p")"
done
if command -v modetest >/dev/null 2>&1; then modetest -c; else echo 'modetest unavailable'; fi

The class can contain card and connector entries; readable connector status values such as connected, disconnected, or unknown report the driver's view, not proof of pixels being displayed. modetest -c asks for connector information and does not request a modeset, but it needs a usable DRM device and permission. Absence of a DRM node, an empty class, an unavailable tool, or permission denial is a legitimate environment result; do not use sudo or change permissions to force success. Record the actual kernel version and whether it is upstream v6.12, a distribution build, or unknown. modetest belongs to libdrm, a separate userspace project: record modetest --help or the installed package version if available, rather than attributing its behavior to the kernel tag.

For a source-only alternative, use the supplied dma_buf_get wrong-file case and atomic preparation excerpt above. Your evidence table should show: fget succeeds; type check fails; fput balances the acquired reference; -EINVAL returns. Separately, prepare_planes fails, so the helper returns before its later swap. These are two source branches, not a claim that a live machine executed them. Deliver either the observed connector table or this source table, with a stated limitation and the distinction between a connector status and a fence.

Practice#

Source-based transaction simulation; no GPU or privileged changes. With Python 3 on the host, paste this complete program into a shell; it models only the documented check/commit decision and reference accounting, not a DRM implementation:

python3 - <<'PY'
def transaction(check_ok, valid_fd, dma_buf):
    events = ['check']
    if not check_ok:
        return events + ['reject'], 0
    events += ['commit', 'fget']
    if not valid_fd:
        return events + ['EBADF'], 0
    refs = 1
    if not dma_buf:
        return events + ['EINVAL', 'fput'], refs - 1
    return events + ['use', 'dma_buf_put'], refs - 1
for args in [(False, True, True), (True, True, False), (True, True, True)]:
    print(args, transaction(*args))
PY

The first case must not print commit; the second ends in EINVAL, fput with zero retained references; the last ends with dma_buf_put and zero. This synthetic sequence puts the two independently discussed source examples together for accounting; it is not a claim that drm_atomic_commit itself calls dma_buf_get. Extend the table on paper with an invalid descriptor and a successful buffer whose fence remains unsignaled. Deliver the table, showing which fact comes from which function and what the model omits. The program creates no files; a failed assertion in your table calls for re-reading the two source paths, not changing host graphics settings.

Knowledge check#

1. Why can a test-only atomic check pass even though a later commit fails?
2. What happens to the fget reference for a valid file descriptor that is not a dma-buf?
3. Does a signaled rendering fence prove the buffer object has been freed?
4. Why can a GEM handle and a dma-buf file descriptor with the same numeric value refer to different things?

Answers#

1. A check concerns the proposed state at that point; intervening state, locking and driver/hardware conditions can affect a later transaction. Check and application are separate operations.
2. dma_buf_get calls fput before returning ERR_PTR(-EINVAL). The caller must recognize the error pointer and must not put it as a successful buffer.
3. No. The fence concerns completion of an operation; object lifetime depends on references, and other operations or users may still retain the buffer.
4. A GEM handle belongs to a DRM file’s handle table, while an fd belongs to a process file-descriptor table. Their integer values do not identify a common namespace; import and export create relationships explicitly.

Checkpoint#

Produce a four-row transaction/reference table with the failed-check, wrong-file, valid-file and unsignaled-fence cases. Ready means no commit follows a failed check, every acquired reference has an owner or release, and completion is kept separate from object lifetime. If not, step through the two quoted functions again and redraw the transitions; this is readiness practice, not certification.

Sources#

• v6.12 drm_atomic.c
• v6.12 dma-buf.c
• DRM/KMS documentation
• dma-buf documentation