PipeWire graphs, buffers, and real-time scheduling

Before you start#

Read PipeWire core and objects and ALSA userspace. This chapter uses PipeWire 1.2.0 as a source baseline. Its numbers are models, not measurements of an installed graph or sound device.

Learning goals#

Trace a stream buffer from dequeue through application processing and queueing. Interpret SPA data and chunk bounds, calculate a graph quantum, and distinguish scheduling deadline, buffer ownership and end-to-end latency. Design an error and shutdown path that does not block a real-time callback.

Source walkthrough#

A PipeWire graph connects nodes through ports and links. A driver node or graph scheduling mechanism supplies a cycle and clock context; follower nodes process according to the graph's dependencies. The quantum is the number of frames processed in a cycle at the graph rate. A node can run at or adapt to different rates depending on configuration and negotiation. A cycle boundary is a scheduling concept, not proof that a hardware converter completed playback at that instant. The engine's behavior depends on graph topology, device clocking, driver and scheduling configuration.

A pw_stream offers an application-facing way to exchange buffers with a graph. A stream has a direction, negotiated parameters, state and callbacks. Control operations establish or change a stream, while a process callback handles data when the graph expects work. The actual callback context depends on stream flags and loop integration. In a real-time data-loop callback, blocking, allocation, contended locks, synchronous logging and unbounded work can cause missed deadlines. The control thread should prepare resources and exchange bounded state with the data path using an appropriate synchronization design. See PipeWire 1.2.0 src/pipewire/stream.c.

A pw_buffer wraps a SPA buffer and stream-specific metadata. A spa_buffer contains metadata entries and data blocks. Each spa_data describes a storage type, pointer or fd-related fields, maximum size and a spa_chunk describing the valid region. Types include memory pointer, memfd and DMA-BUF where supported. The data type determines how storage is mapped and synchronized; an fd is not a CPU pointer, and a DMA-BUF mapping may have additional synchronization rules. A chunk's offset and size must be checked against the mapped maximum and any ring or plane conventions before access. Consumers commonly normalize an offset and clamp the size to the remaining mapped region; the struct declaration describes fields but does not enforce those bounds. Its stride can describe a layout relationship; it must not be guessed from the media type alone. The SPA buffer header defines the fields.

Dequeue and queue ownership

An output application dequeues an available buffer, writes or otherwise supplies the negotiated data, updates the appropriate valid-region metadata, and queues it back to the stream. A capture application interprets the direction and valid data according to its stream contract. While a buffer is queued for graph use, the application must not keep modifying it as though it still owns exclusive access. After the buffer is returned and dequeued again, the application can use it under the new cycle's rules. Destruction, format renegotiation and buffer removal invalidate old pointers; a stream callback and teardown must be coordinated so no thread accesses a freed or unmapped buffer.

From PipeWire 1.2.0 src/pipewire/stream.c, pw_stream_dequeue_buffer, MIT SPDX file license; declarations and later busy-buffer handling are omitted:

	if ((b = queue_pop(impl, &impl->dequeued)) == NULL) {
		res = -errno;
		pw_log_trace_fp("%p: no more buffers: %m", stream);
		errno = -res;
		return NULL;
	}

The first line removes an available internal buffer or detects that none is available. The next lines preserve the failure's error value across the trace operation and return a null pointer. A process callback must handle this outcome rather than dereference the result. The surrounding implementation also checks an output buffer's busy count and can put it back and report EBUSY; a buffer can therefore be unavailable even when the application expected one. Logging shown in library source does not imply an application should perform unbounded synchronous logging on a real-time thread.

From pw_stream_queue_buffer in the same source and license, with earlier busy-count handling and later driver-trigger logic omitted:

	if ((res = queue_push(impl, &impl->queued, b)) < 0)
		return res;

The first statement tries to put the buffer on the stream's queued list. The second propagates a negative error if that operation fails. An application should not silently count a failed queue as successfully delivered media. The rest of the function includes direction- and driver-dependent behavior; this excerpt does not establish one universal scheduling chain. The application must follow the API's ownership rules for successful queue and handle an error without double-queueing or freeing memory still used by the stream.

The node implementation provides a separate view of graph processing. From PipeWire 1.2.0 src/pipewire/impl-node.c, process_node, MIT SPDX file license, with input mixer processing and output tee logic omitted:

		/* process the actual node */
		status = spa_node_process_fast(this->node);

The call invokes the SPA node's processing operation and stores its returned status. The surrounding function is called from a data loop, first atomically changes a triggered activation to awake, and processes this node only when its real-time state is prepared. It can then process output mixers if the status indicates data. This is one node implementation path, not proof that every stream callback runs on an identical thread or that the driver is physical hardware.

SPA status, metadata and bounds

A graph's node I/O can include spa_io_buffers, with a status and buffer ID used to coordinate availability and processing. Status values distinguish conditions such as needing data, having data or an error under the relevant SPA interface. The buffer ID indexes a negotiated set; it is not a pointer or a permanent global graph ID. An implementation must validate the ID against the current buffer set and respond to the state machine rather than assume every process call supplies new data. See PipeWire 1.2.0 spa/include/spa/node/io.h.

In spa/include/spa/node/io.h, PipeWire 1.2.0, MIT SPDX (surrounding code omitted):

struct spa_io_buffers {
#define SPA_STATUS_OK			0
#define SPA_STATUS_NEED_DATA		(1<<0)
#define SPA_STATUS_HAVE_DATA		(1<<1)
#define SPA_STATUS_STOPPED		(1<<2)
#define SPA_STATUS_DRAINED		(1<<3)
	int32_t status;			/**< the status code */
	uint32_t buffer_id;		/**< a buffer id */
};

status takes values such as SPA_STATUS_NEED_DATA and SPA_STATUS_HAVE_DATA, and buffer_id names an entry in the negotiated buffer set. The excerpt does not show how a node reacts to each status, so it does not establish any particular driver behavior.

A spa_chunk describes the valid bytes in a spa_data region. For a simple contiguous example, if maxsize is 4096 bytes, offset is 512 and size is 1024, the interval ends at 1536 and fits. The pinned header specifies taking offset modulo maxsize and clamping size to maxsize. If the resulting region wraps, a consumer must handle the split according to the negotiated layout rather than assume one contiguous interval; it must not blindly access beyond the mapping. Integer overflow is another reason to validate bounds before addition. A size of zero can represent no valid payload; it is not permission to read a full buffer. For planar audio, each data plane and its chunk must be handled separately.

In spa/include/spa/buffer/buffer.h, PipeWire 1.2.0, MIT SPDX (surrounding code omitted):

struct spa_chunk {
	uint32_t offset;		/**< offset of valid data. Should be taken
					  *  modulo the data maxsize to get the offset
					  *  in the data memory. */
	uint32_t size;			/**< size of valid data. Should be clamped to
					  *  maxsize. */
	int32_t stride;			/**< stride of valid data */
#define SPA_CHUNK_FLAG_NONE		0
#define SPA_CHUNK_FLAG_CORRUPTED	(1u<<0)	/**< chunk data is corrupted in some way */
#define SPA_CHUNK_FLAG_EMPTY		(1u<<1)	/**< chunk data is empty with media specific
						  *  neutral data such as silence or black. This
						  *  could be used to optimize processing. */
	int32_t flags;			/**< chunk flags */
};

The field comments state two bounds rules that the struct cannot enforce by itself: take offset modulo the data maxsize, and clamp size to that maximum. The declaration shows no consumer code, so it does not show how a wrapped region is split or how a planar chunk is handled.

Metadata can carry timing, sequence or other information. A timestamp needs its stated clock domain and units; comparing it directly with an unrelated wall clock can create false latency claims. An I/O position and clock describe graph timing, but hardware clocks can drift and a client can receive discontinuities. The application should check metadata presence and type, not dereference a missing structure because another backend supplied it in a previous run. A buffer reference protects a particular object only under the API's lifetime contract; retaining a pointer after queueing or after a buffer removal event can cause a race or use-after-free.

Two buffers and two cycles

Use a synthetic graph rate of 48,000 frames per second and quantum of 256 frames. One quantum is 256/48,000 seconds, about 5.333 milliseconds. Let A and B be separate buffers each large enough for one negotiated quantum. At cycle 1 the application dequeues A, fills 256 frames, sets a valid chunk for those bytes, and queues A. The application no longer writes A while the graph owns that use. At cycle 2 the application dequeues B, fills and queues it while A is still in flight. Later the graph returns A; only after the application successfully dequeues it again can it reuse A for another cycle under the stream contract.

If the producer misses the cycle 2 deadline, the engine or endpoint can report an xrun, insert silence, repeat data or take another configured recovery action. Which outcome actually occurs depends on the nodes and implementation; this model does not claim a specific PipeWire server did any of them. The application should record or report a discontinuity through an appropriate non-real-time mechanism, restore a coherent buffer state and resume only when the graph and stream permit it. It must not label B as played merely because it was queued, nor free A because a frame callback or unrelated control reply arrived.

The quantum is not total round-trip latency. A signal can encounter capture hardware buffering, graph cycles, application processing, playback buffering and converter delay. Links, adapters and resamplers can add further latency. A graph configured for 5.333 ms cycles can have a much larger input-to-output delay. A useful measurement specifies where timestamps were taken, their clock domains and the load and configuration. No such measurement is performed in this lesson.

Real-time constraints and control handoff

Real-time scheduling is a policy for allocating CPU time, not a guarantee that every deadline will be met. The graph's data thread may have elevated scheduling priority when permitted by system policy and resource limits; RTKit can mediate such requests in some setups. Permissions, limits and implementation determine whether that succeeds. An application should function or fail clearly when it cannot obtain the requested scheduling. This exercise never asks RTKit or changes scheduling priority.

A callback with a 5.333 ms budget has less than that interval available after accounting for all other graph work and scheduler delay. Waiting on a mutex held by a control thread, allocating memory that faults, performing filesystem I/O, or printing synchronously can exceed the budget unpredictably. A common design preallocates buffers and uses a bounded single-producer/single-consumer queue or another carefully specified handoff to exchange compact commands and metrics. The exact atomic ordering and overflow behavior of such a queue must be designed and tested; merely naming it lock-free does not make application data safe. A full queue needs a defined drop or coalescing policy.

Control changes can race with processing. Suppose the control thread asks to change format while a process callback still holds a pointer to the old buffer layout. The implementation must stop or synchronize processing and wait for the relevant buffer-removal and parameter events before freeing old mappings. A callback using the old pointer after unmap can crash even if the new format was negotiated correctly. A safe shutdown prevents new work, coordinates callback completion, returns or retires owned buffers under the API, removes listeners, destroys the stream and then releases the context and mappings. Exact ordering depends on the loop and API version.

Drift, resampling and recovery

Two physical devices rarely have perfectly identical clocks. Even if both report a nominal 48 kHz, one may produce or consume slightly more frames per second than the other. Without adaptation, a buffer connecting them eventually fills or empties. A rate-matching or resampling component can adjust the relationship using measured timing feedback, subject to its own quality and stability limits. Nominal sample rates are labels for configuration, not proof of a shared oscillator. A timestamp discontinuity, device suspend or clock reset can require resetting the estimator rather than extrapolating indefinitely.

For illustration, a 100-parts-per-million difference at nominal 48,000 frames per second is 4.8 frames per second. Over 100 seconds, ignoring correction and other effects, the relative difference is about 480 frames, or 10 ms at 48 kHz. This is synthetic arithmetic, not a hardware measurement. A bounded buffer cannot absorb an indefinitely accumulating mismatch; an adaptive strategy or another clocking arrangement is needed. The selected resampler, graph policy and device determine actual behavior.

An xrun is a symptom with several possible causes: scheduling delay, a slow node, resource starvation, clock mismatch or a device-specific issue. Raising priority is not a universal cure, and a system can deny it. Diagnose with appropriately authorized measurements and controlled experiments, while protecting user audio data. The present exercises deliberately do not inspect the host graph or request elevated scheduling.

Two independent shutdown events

A stream can receive a buffer-removal event while a control request to disconnect is already pending. The application must not free a mapping twice or infer that a previously queued process callback has vanished. Keep a clear state for each buffer and arrange for the data loop to stop using it before unmapping or releasing the backing descriptor. If a control-thread cancellation races with a process callback, a flag alone is insufficient unless its visibility and lifetime are correctly synchronized. The last callback may still need a stable stream and callback-data reference until it returns. This is a general ownership design requirement; the particular loop and teardown APIs determine the implementation details.

Practice#

This private model calculates two buffer cycles and drift. It neither creates a stream nor measures real-time behavior:

python3 - <<'PY'
rate, quantum = 48000, 256
print('quantum ms:', round(1000 * quantum / rate, 3))
free = ['A', 'B']
a = free.pop(0)
b = free.pop(0)
print('in flight:', a, b, 'free:', free)
free.append(a)
print('returned:', free)
print('100 ppm frames in 100 s:', rate * 100 / 1_000_000 * 100)
PY

Expected quantum is 5.333 ms, the free list is empty after A and B leave it, and the illustrative drift is 480 frames. A counterexample is to reuse A before it returns: that would let producer and consumer access the same buffer concurrently. As a source-only alternative, use the A/B timeline above and record owner, chunk validity and error handling at each transition. Include the null dequeue and failed queue source branches. No server, device or scheduling policy is involved.

Knowledge check#

1. What should a process callback do if dequeue returns null?
2. Why is a chunk's offset and size not the same as its allocation capacity?
3. Can the application keep writing A after successfully queueing it?
4. Is a 256-frame quantum at 48 kHz the total round-trip latency?
5. Why can nominally identical device rates still require adaptation?

Answers#

1. It must handle the absence of an available buffer, avoiding a null dereference and following its bounded error or retry policy.
2. Capacity describes allocated storage; offset and size describe the valid region for this use. Bounds and layout still require validation.
3. No. Queueing hands the buffer to the stream/graph under its ownership contract; reuse waits for its return and a new dequeue.
4. No. It is about 5.333 ms for one cycle, while buffering, processing and device delays add to end-to-end latency.
5. Separate clocks can differ from their nominal rates, accumulating a mismatch that eventually fills or empties a finite buffer.

Checkpoint#

Submit the two-cycle ownership table, the 5.333 ms calculation and a control-to-data handoff sketch. Pass when no queued buffer is modified, null dequeue and queue errors have paths, and quantum is not labeled total latency. If not, reread the stream source and replay the private model. This is practice, not certification.

Sources#

See stream implementation, SPA buffer fields, SPA I/O status, and node implementation. No server or audio hardware was used.