The Linux userspace map and project baselines

Before you start#

Review the kernel system map, file descriptors, system calls, and userspace contracts. This chapter adds separately maintained userspace projects to that map. Linux v6.12 remains the historical kernel reading baseline; the glibc source below is a separate release and is not evidence that a distribution shipped that exact combination.

Learning goals#

Explain which project owns each part of a simple application request, identify the interfaces at process and privilege boundaries, and record enough version and configuration information to investigate an error without assuming that a source path executed. Distinguish a source reading, a model, and an observation. Identify where a service is optional and where authority is checked.

Source walkthrough#

Suppose an ordinary application writes a short request to an already-connected Unix stream socket and reads a reply. This deliberately small example is not a claim about the D-Bus wire protocol. The application has a descriptor number in its own descriptor table, a request buffer in its virtual address space, and a protocol that it and its peer have agreed upon. The descriptor refers to a kernel open-file object; its numeric value is meaningful only in that process's table. A service in another process has its own descriptor and memory. It cannot simply dereference the application's pointer. The kernel mediates the transfer, but the meaning of the bytes is established by the applications' protocol.

The actual GNU C Library 2.40 Linux implementation of __libc_write in sysdeps/unix/sysv/linux/write.c carries an LGPL-2.1-or-later notice. This is the whole function, omitting its surrounding includes and aliases:

ssize_t
__libc_write (int fd, const void *buf, size_t nbytes)
{
  return SYSCALL_CANCEL (write, fd, buf, nbytes);
}

The signed result type can represent a byte count or a failure indication. fd identifies an open descriptor in the calling process; buf points to bytes the caller must keep readable during the call, and nbytes is the requested length. The function passes all three to a cancellation-aware syscall macro. Its one source line is not itself a description of every machine instruction: macro expansion, architecture and build determine that implementation. It also does not ensure that the peer has processed the message. An accepted write may transfer fewer bytes than requested, and an error requires handling. In particular, software should not interpret an acknowledgment from the kernel that some bytes were accepted as a protocol-level acknowledgment from a service.

The corresponding actual __libc_read in sysdeps/unix/sysv/linux/read.c, also LGPL-2.1-or-later, is equally compact; surrounding aliases and comments are omitted:

ssize_t
__libc_read (int fd, void *buf, size_t nbytes)
{
  return SYSCALL_CANCEL (read, fd, buf, nbytes);
}

Here the buffer is writable by the operation. A positive return identifies how many bytes were delivered to it, zero represents end of stream for this connected stream socket, and -1 signals an error whose details are reported using errno (read(2)). A stream does not preserve an application's message boundaries: one read can produce part of a reply or combine bytes from several writes. A protocol therefore needs framing, such as an agreed length field, a delimiter with an escaping rule, or another unambiguous format. The receiving program must validate a length before allocating or indexing memory; a peer can disconnect halfway through a message. The wrapper does not turn an untrusted message into a trustworthy object.

The cited files are from the glibc tag glibc-2.40, resolved to commit 3d1aed874918c466a4477af1da35983ab036690e in the recorded mirror. They show an implementation for Linux, not every libc or every architecture's instruction sequence. The Linux v6.12 read implementation supplies a separate kernel-side entry point and should be read with its own configuration and file-operation dispatch conditions. A file descriptor can refer to many object types; a successful descriptor lookup alone cannot prove a Unix socket or a service is involved.

In Linux v6.12, fs/read_write.c, SPDX GPL-2.0, the kernel-side ksys_read resolves the descriptor in the calling task's table, calls vfs_read and then releases that reference (surrounding code omitted):

ssize_t ksys_read(unsigned int fd, char __user *buf, size_t count)
{
	struct fd f = fdget_pos(fd);
	ssize_t ret = -EBADF;

	if (fd_file(f)) {
		loff_t pos, *ppos = file_ppos(fd_file(f));
		if (ppos) {
			pos = *ppos;
			ppos = &pos;
		}
		ret = vfs_read(fd_file(f), buf, count, ppos);
		if (ret >= 0 && ppos)
			fd_file(f)->f_pos = pos;
		fdput_pos(f);
	}
	return ret;
}

The excerpt shows that an unresolved descriptor starts from -EBADF, and that the file position is updated only when the read returned a nonnegative count. It does not show vfs_read's permission and operation checks, the socket receive path, or the libc conversion of a negative kernel result into -1 with errno.

A worked request with ownership and failure#

Imagine process A owns descriptor 5, a connected stream socket. Its private buffer contains the four bytes PING. It calls write(5, buffer, 4). Suppose this particular call returns 2. That result means the caller must advance its pointer by two bytes and attempt to send the remaining NG if its protocol and error policy require the whole request. It must not resend all four bytes as though nothing happened. If the next call returns 2, four bytes have been accepted in total. This is a synthetic timeline for reasoning, not an observed trace and not a promise that a four-byte write normally splits.

Process B may read one byte and then three, or four together. It keeps an input buffer and a count of initialized bytes. If its protocol expects exactly four bytes, B should not compare the request before enough data has arrived. After consuming PING, B might generate PONG; A similarly accumulates the reply. A buffer allocated by A is owned by A, and B's buffer is owned by B. The connected endpoints and any queued kernel data have their own lifetimes. Closing A's descriptor releases A's descriptor reference, but other duplicated descriptors or references may still keep the underlying object alive. The peer can eventually observe end of stream after the relevant write-side references close and buffered data is consumed. Descriptor numbers can later be reused, so storing just the integer after close is not a durable object identity.

Consider the error branch. If A uses a descriptor that has already been closed, a write ordinarily fails with EBADF. Its buffer still exists; the failed call did not transfer ownership of that memory to the kernel. A should preserve the error immediately if it must perform other library calls before reporting it, since a later operation may change errno. If B exits first, A's write to a shut-down connection-oriented socket can generate SIGPIPE and fail with EPIPE on Linux (socket(7)); an application must choose an explicit signal policy. A request can also fail after a partial transfer, leaving the peer with a prefix. Retrying a protocol operation without defining whether it is idempotent can duplicate an action. These cases are why application protocol state, not simply a successful syscall, determines whether a request completed.

A real service protocol may add authentication, serial numbers, authorization, file-descriptor passing, message headers and a broker. If an application uses D-Bus, the bus implementation and the particular service define their relevant policy and object interfaces; do not infer a particular broker or service from the existence of a socket. Bus authorization, a service's own authorization and the kernel's filesystem or socket checks are distinct decisions. A successful connection does not grant unlimited authority to every operation. The D-Bus specification describes protocol concepts, but a later chapter will pin and inspect a particular implementation rather than treating that specification as runtime evidence.

How the components fit together#

An executable file may name an ELF interpreter in its PT_INTERP program header. On a typical dynamically linked GNU system the kernel starts that interpreter as part of executing the program; the dynamic loader maps required shared objects and resolves relocations before application control reaches ordinary startup. The loader is userspace code, even though the kernel sets up the initial process image. The exact interpreter path and libraries depend on architecture and packaging. A missing interpreter can produce a confusing missing-file error even when the executable itself exists. Static executables and other executable formats take different paths. The glibc dynamic loader source is an entry point for library loading, not evidence that every program calls dlopen explicitly.

A C library exposes interfaces that application code can call. Some operations perform a system call, some run entirely in userspace, and others combine both approaches. The write and read wrappers above show two specific paths; they do not establish that every library function has the same cancellation behavior. glibc and musl are distinct implementations with different internals and release histories. An application may also use additional libraries for a user interface, cryptography or a network protocol. Those libraries are not automatically part of the kernel merely because their functions are available on Linux.

System services are programs too. A service manager may start and supervise a daemon according to configured policy, but a container, a minimal image or a different distribution can use a different arrangement. A system manager and a per-user manager have different authority and scope. A broker can route messages between clients, but it does not replace each service's application-level validation. A graphical compositor processes clients' display requests and coordinates presentation; an audio service can manage devices and media graphs. Neither service is guaranteed to be present on a headless server. Their kernel-facing operations ultimately use supported interfaces such as descriptors, sockets, memory mappings and device APIs, but the semantics of their userspace protocols must be investigated in their own repositories.

Wine adds another compatibility boundary. A Windows-oriented program can invoke APIs implemented by Wine components, which in turn use Unix-side facilities and host services. A Wine prefix organizes state; it is not a security sandbox, and it does not make an unknown program safe. The implementation, ABI, architecture and optional graphics or audio backends matter. This chapter does not claim a Wine path was inspected or executed; later chapters must pin and cite the exact Wine revision before tracing it.

A distribution brings these parts together by selecting versions, applying patches, configuring services and delivering updates. Its package version can include security backports absent from a similarly named upstream release. The live host's libc and kernel may differ from both source baselines in this book. A container image can supply its own libraries while sharing a host kernel, subject to runtime and isolation configuration. Therefore record the executable, its interpreter and libraries, the service version if a service is involved, the kernel revision, architecture and relevant environment. Record only information you are entitled to inspect; paths and logs may expose private data.

State, concurrency and trust#

A synchronous API does not imply a single-threaded system. While A blocks waiting for data, B can run, terminate or replace its own internal state. A signal may interrupt an operation; a descriptor can be shared by threads, and closing it concurrently requires an application-level lifetime strategy. A library callback may run later than the code that registered it. Ownership questions are concrete: who may close the descriptor, who frees the buffer, how is a pending callback cancelled, and which lock or event-loop rule protects the state? A reference can keep an object alive without serializing modifications to it. A mutex can serialize state changes without automatically retaining a referenced object.

When diagnosing an error, identify the last confirmed boundary. A loader error before main is not a service policy denial. A successful connect followed by a rejected method call suggests a different stage from a missing socket path. A write return does not prove the service parsed or acted on the message. Application-level timeouts can occur even if kernel I/O succeeded, for example if the peer is waiting on a resource or never sends a valid response. Gather the protocol result, descriptor and process context, and pertinent configuration before assigning blame. Logging or tracing itself can alter timing and expose private content; ask what the measurement includes and omit secrets.

Practice#

This is an unprivileged, bounded own-process experiment using Python 3 and a private socket pair. It does not contact an existing daemon or modify host services. Run it in a terminal on a Linux system:

python3 - <<'PY'
import socket
left, right = socket.socketpair()
try:
    left.sendall(b'PING')
    request = right.recv(4)
    right.sendall(b'PONG')
    reply = left.recv(4)
    print('request:', request, 'reply:', reply)
    left.close()
    try:
        left.send(b'X')
    except OSError as error:
        print('closed descriptor:', type(error).__name__)
finally:
    left.close()
    right.close()
PY

For this small local test, the expected output contains request: b'PING' reply: b'PONG' and an exception class for the closed socket. Python may reject use of the closed object before a syscall; that is still a useful distinction between a library-level error and kernel evidence. Socket reads are allowed to be short in general; this minimal demonstration is intentionally bounded and is not a production message parser. If Python is absent, do not install it for the exercise: use the supplied four-byte state timeline above. If output differs, note the Python version and error rather than changing system services. Both endpoints close in finally, including after an exception. No guest or v6.12 runtime is claimed.

Deliver a one-page diagram showing A's buffer and descriptor, kernel socket endpoints, B's separate buffer, and the two return values. Annotate the short-write and disconnected-peer branches. Add the host kernel and Python version if you ran the test. As a source-only alternative, use the synthetic 2 + 2 write and 1 + 3 read sequence above; explain why accepted bytes and a complete service-level response are different events. The alternative establishes a reasoning artifact, not a measured execution.

Knowledge check#

1. Why does the numeric descriptor 5 not identify a system-wide socket on its own?
2. What must A do after the first write returns 2 for a four-byte request, and why is the service's action still unknown?
3. What is the distinction between the kernel's initial ELF setup, a dynamic loader, and a service manager?
4. How can a connection succeed but a later service operation fail authorization?
5. Why does recording the glibc version not identify the exact kernel or distribution behavior?

Answers#

1. Each process has a descriptor table. The integer selects an entry there; another process can use the same number for a different object. Duplicated descriptors can also refer to the same open-file object, so neither numeric equality nor inequality alone proves object identity.
2. A advances past the two accepted bytes and handles the remaining two according to its protocol, including errors and interruption. The return reports transfer progress at the syscall boundary, not that B read, validated, or committed an action. A full application acknowledgment needs its own protocol semantics.
3. The kernel establishes the executable process image and hands control to the appropriate entry point. A dynamically linked executable's userspace interpreter maps and relocates shared objects under its rules. A service manager supervises services and activation; none of these responsibilities implies that every Linux system uses the same loader or manager.
4. A transport connection establishes a communication channel under applicable access checks. A broker or service may apply additional policy to a named operation, credentials or object. Those decisions occur at separate boundaries and can reject a validly connected client.
5. libc, the kernel and distribution packages have independent versions and patches. The glibc source gives evidence about one implementation at one revision; a live system may use another libc build, kernel, architecture or backport. A runtime trace must record the actual environment and cannot be inferred from the tag alone.

Checkpoint#

Submit the labeled request/reply diagram, a short-write/error state table and either the own-process output with versions or the source-only explanation. Ready means you can name each buffer owner, distinguish a descriptor from the underlying object, show how the remaining two bytes are handled, and identify at least two independent policy or failure boundaries. If you cannot, return to file descriptors, redraw the 2 + 2 timeline, and retry the questions. This source-only gate is not proof of runtime expertise.

Sources#

• glibc 2.40 read.c, pinned commit
• glibc 2.40 write.c, pinned commit
• glibc 2.40 loader dl-open.c, pinned commit
• Linux read(2) documentation
• Linux write(2) documentation
• Linux socket(7) documentation
• Linux execve(2) documentation
• Linux v6.12 file I/O source
• D-Bus specification