initcalls, initramfs, PID 1, and userspace

Before you start#

Read x86 boot and init, arm64 firmware and boot, and the disposable kernel lab. This chapter reads upstream v6.12 and provides a source-only exercise. Optional observations inspect only your own process and a disposable guest. A host's PID 1, kernel version, namespace and configuration may differ; never restart or signal host PID 1 for this lesson.

Learning goals#

Trace the boot task's transition through rest_init, kernel_init_freeable, initcalls and init-memory cleanup to userspace execution. Distinguish rootfs, initramfs and legacy initrd, explain when prepare_namespace runs, and diagnose an init failure without assuming that successful unpacking or a running PID 1 proves every device is ready.

Two tasks and a synchronization point#

In v6.12 init/main.c, start_kernel eventually calls rest_init. The latter starts the task that runs kernel_init first, so that it receives PID 1 in the initial PID namespace, then creates kthreadd. kthreadd supports creation of kernel threads. Creating the init task does not immediately mean that userspace /init has executed. At this point it runs kernel code and must wait for other prerequisites.

In init/main.c, v6.12, SPDX GPL-2.0-only (lines omitted):

	/*
	 * We need to spawn init first so that it obtains pid 1, however
	 * the init task will end up wanting to create kthreads, which, if
	 * we schedule it before we create kthreadd, will OOPS.
	 */
	pid = user_mode_thread(kernel_init, NULL, CLONE_FS);
/* ... */
	pid = kernel_thread(kthreadd, NULL, NULL, CLONE_FS | CLONE_FILES);
/* ... */
	complete(&kthreadd_done);

The first call creates kernel_init with CLONE_FS, and the comment gives the reason it is created before kthreadd. complete(&kthreadd_done) is the signal that kernel_init waits for. The excerpt does not show what kthreadd does or whether kernel_init succeeds later.

The following excerpt from kernel_init is upstream v6.12 (SPDX GPL-2.0-only; surrounding statements omitted):

wait_for_completion(&kthreadd_done);

kernel_init_freeable();

The completion wait prevents this task from continuing past the synchronization point until rest_init signals that kthreadd is established. Then kernel_init_freeable performs the next group of initialization work. A completion is a kernel synchronization mechanism, not a userspace process or a promise that all possible devices are ready. The boot task's context changes as scheduling, memory allocation and interrupt services become available; do not assume that a call safe here was also safe at the first machine instruction.

Source walkthrough#

In the same file, kernel_init_freeable prepares CPUs, workqueues and memory-related facilities, invokes early initcalls, brings up SMP where configured, and calls do_basic_setup. This excerpt shows the later transition (same SPDX notice and tag; intervening calls and comment omitted):

do_basic_setup();

wait_for_initramfs();
console_on_rootfs();

do_basic_setup initializes driver infrastructure and calls do_initcalls; it is a function call, not a guarantee that every future device has finished probing. wait_for_initramfs waits for the rootfs population work to finish before early userspace is examined. console_on_rootfs attempts to open /dev/console and duplicate the resulting file for the initial standard descriptors. Its source warns on failure; an absent console can make later diagnostics invisible or incomplete. The initial console descriptors are file references held by the new process, not an assertion that every terminal device is operational.

After kernel_init_freeable returns, kernel_init calls async_synchronize_full before freeing init memory. It then performs init-memory cleanup, makes relevant mappings read-only and moves toward execution of the selected userspace program. Functions and data marked for initialization can be discarded after their permitted lifetime. Saving an unprotected pointer to an __init function for a later callback can therefore lead to execution of freed memory. A driver that schedules asynchronous initialization must follow the appropriate lifetime and synchronization rules; the existence of a synchronization point does not make every unrelated work item in the system permanently finished.

Registration, levels and ordering#

An initcall is a function registered for a stage of kernel initialization. Macros in v6.12 include/linux/init.h place references into specially named sections. The linker script machinery in v6.12 include/asm-generic/vmlinux.lds.h collects ordered section ranges and boundary symbols. do_initcall_level walks the entries between level boundaries and calls do_one_initcall; do_initcalls iterates the levels after restoring a command-line copy for each level. The early range is handled separately by do_pre_smp_initcalls. Registration is therefore a build-and-link relationship, not a list of ordinary calls all visible in do_basic_setup.

In init/main.c, v6.12, SPDX GPL-2.0-only (surrounding code omitted):

	trace_initcall_level(initcall_level_names[level]);
	for (fn = initcall_levels[level]; fn < initcall_levels[level+1]; fn++)
		do_one_initcall(initcall_from_entry(fn));

The loop walks the entries between the level boundaries in initcall_levels. The excerpt does not show how the registration macros fill those boundaries, or the link-order rules that decide what appears in each range.

The level names in this revision include pure, core, postcore, arch, subsys, fs, device and late. A level gives an ordering framework, but it is not a general dependency solver for all hardware. Link order, Kconfig selection and built-in versus module status affect which functions exist and their order within relevant ranges. A driver can register successfully yet find no matching device. A probe may be deferred because a regulator, clock, bus or other provider is not ready. Some initialization and discovery can run asynchronously. A return from the initcall walk therefore does not establish that all possible hardware has appeared or every deferred probe succeeded. Consult driver infrastructure documentation when investigating a particular driver.

A concrete source-only example illustrates the distinction. Suppose initcall A registers a bus and returns zero. Later initcall B registers a driver, whose probe observes that a required provider is absent and returns -EPROBE_DEFER. The two registration functions may have completed, while the device is still not bound. If the provider appears later and the driver core retries, the device may eventually bind; if it never appears, waiting indefinitely is not a repair. This scenario is illustrative and does not assert that every driver uses this exact order. Record the driver's actual registration, probe result and dependency when studying a real boot.

Rootfs, initramfs and the other root path#

Rootfs is the initial filesystem instance available during early boot. An initramfs is a cpio archive whose contents are unpacked into rootfs, either built into the kernel or supplied externally when configured. It can contain directories, device nodes and an executable /init; it does not have to be a block device. An old-style initrd is a different historical mechanism involving a RAM-disk image and associated loading behavior. The names are often used loosely by bootloader interfaces; inspect the actual artifact and kernel configuration rather than inferring its format from an option name. The rootfs and initramfs guide explains the distinction.

After waiting for initramfs population, the source checks whether the configured early userspace command is accessible. When it is not, it clears that command and calls prepare_namespace; this does not mean that a later successful access check guarantees exec will succeed. A script can be present and executable but name a missing interpreter, and an ELF binary can require an absent dynamic loader. In those cases the attempted execution can still fail. The exact fallback path depends on the boot arguments and the point of failure.

In init/main.c, v6.12, SPDX GPL-2.0-only (surrounding code omitted):

	/*
	 * check if there is an early userspace init.  If yes, let it do all
	 * the work
	 */
	if (init_eaccess(ramdisk_execute_command) != 0) {
		ramdisk_execute_command = NULL;
		prepare_namespace();
	}

The test uses init_eaccess on the configured early command, which is /init by default. When that check fails, the kernel clears the command and takes the kernel-managed root path. Passing the check does not prove that the later exec succeeds, and the excerpt does not show what prepare_namespace mounts.

prepare_namespace in v6.12 init/do_mounts.c handles the kernel-managed root-mount path. It considers root delay, waits for known device probing, interprets the saved root name and can wait for a root device before calling mount_root. A rootwait option can be useful when a device appears asynchronously; in this revision it can wait indefinitely or with a specified timeout depending on the option. An invalid root specification, missing block driver, unsupported filesystem or permanently absent device is not repaired merely by waiting. The kernel parameter reference is the place to check exact option semantics for the chosen release.

If the kernel must mount a block root directly, support needed to reach it—including the relevant controller, block and filesystem code—must be available early enough, commonly built into the kernel. If an initramfs supplies userspace and modules, it can load suitable modules and perform its own discovery and mounts before handing off. That is a different dependency chain: the module loader, module files and their dependencies must themselves be present and compatible. A root= argument alone does not provide a missing driver. Chapter 08 deliberately uses an initramfs-only root, so it does not prove a virtual disk or root filesystem driver works.

An early userspace program can mount a real root and use an appropriate switch_root-style operation before executing a new init. This is a userspace procedure with its own requirements about mount points, console and the files that remain reachable. It is not the same operation as the kernel's prepare_namespace path, and it must not be attempted on the host for this lesson. After a handoff, the PID can remain 1 while the executed program and root view change; exec replaces a process image rather than creating a new PID.

PID 1 is a role, not a particular product#

The kernel starts an initial userspace program; that program or its successors implement userspace initialization policy. A distribution may use a service manager, a small init or a custom embedded program. Service configuration, dependency ordering, log handling and login policy are userspace responsibilities, not universal behavior of Linux's kernel. PID 1 in a PID namespace also has special process-lifecycle responsibilities and signal semantics. An init process must be designed to supervise or reap children where required; simply naming any shell PID 1 does not make it a complete service manager.

In the chapter 08 teaching guest, /init mounts a few virtual filesystems and execs a shell. That is intentionally small, not a production init system. If PID 1 exits in the initial PID namespace, the guest kernel can panic. Do not use that behavior as a host experiment. A container may show a different PID 1 from the initial namespace and impose permissions on /proc; a command run in a container cannot establish who owns PID 1 on the host.

A worked failure chain#

Consider two synthetic boots using the same kernel configuration. In case A, the external archive unpacks and contains /init plus its interpreter /bin/sh. console_on_rootfs has a valid console node, the script starts, mounts proc and uses exec to replace itself with a shell. An observation of the marker followed by a shell reporting PID 1 supports this particular sequence, provided the serial log and command line are captured. It does not prove that every mount succeeded; inspect the mount diagnostics and /proc separately.

In case B, the archive unpacks, but /bin/sh is absent. /init can exist and have execute permission, yet the script interpreter cannot be opened. Execution fails after unpacking. Inspecting only the initramfs filename or a successful gzip -t would miss that failure. Conversely, a broken console can hide successful execution. Keep the archive listing, executable format, configuration, console arguments and earliest errors together before deciding which hypothesis fits. These are model cases, not a claim that either boot was observed.

Practice#

Source-only route: create a table with four states: init task waiting for kthreadd_done; initcall walk; initramfs populated and console opened or failed; userspace exec attempted. For each state record one source symbol, one owned or referenced object and one failure or uncertainty. Apply it to cases A and B above and identify the first different condition: availability of the script interpreter. This is supplied evidence-based reasoning, not a guest trace.

Read-only host observation: in an ordinary Linux shell, inspect the current process and, if permissions allow, PID 1 in your current PID namespace:

printf 'shell PID=%s\n' "$$"
readlink "/proc/$$/exe"
head -c 160 "/proc/$$/cmdline" | tr '\0' ' '; printf '\n'
sed -n '1,12p' /proc/1/status
readlink /proc/1/exe || printf 'PID 1 executable not visible here\n'

These commands reveal process paths and arguments, which can contain private information; do not publish output without reviewing it. $$ identifies the shell, whereas /proc/1 is the PID 1 visible in this namespace, possibly a container init. The executable link may be inaccessible or disappear due to permissions or process lifetime; record that error rather than escalating privileges. Reading these files does not prove the host booted this v6.12 source, reveal the original initramfs, or identify the host's initial namespace from a container. There is no cleanup because no files are created.

Optional guest: boot the disposable chapter 08 guest with initcall_debug appended to its existing guest kernel command line, retaining the serial log. This boot parameter and diagnostics depend on build and release; log volume and timing will change. Compare reported initcall names and return values to the source and separately inspect /proc/1/status in the guest. A failure, absent console or missing initcall message is not proof that every earlier stage was absent. Stop only that guest as described in chapter 08. Never restart, signal or replace the host's PID 1.

Knowledge check#

1. Why is the task running kernel_init created before kthreadd, yet required to wait for it?
2. How do initcall macros and linker section ranges relate to the loop in do_initcalls?
3. Why can a completed initcall walk coexist with a device whose probe is still deferred?
4. When does the v6.12 early boot path call prepare_namespace, and why does rootwait not install a missing driver?
5. Why can an executable /init still fail with a missing-file error?

Answers#

1. Init must obtain PID 1, but its later work may need kernel-thread support. The completion coordinates progress after the task is created; it is not itself a userspace exec.
2. Registration macros emit entries into selected sections, the linker collects level boundaries, and the runtime loop walks those ranges. Configuration and link selection determine which entries exist.
3. Registration, probing and readiness are different events. A probe can depend on a later provider and be deferred or asynchronous; a successful registration return is not universal hardware readiness.
4. After waiting for initramfs and opening the console, the source takes that path when the early userspace command is not accessible. rootwait waits for a root-device condition; it cannot create missing driver code or correct an invalid filesystem.
5. A script's interpreter or an ELF binary's dynamic interpreter may be absent even when the named file exists. Check the archive and interpreter path, architecture and permissions in the guest's own root.

Checkpoint#

Deliver the four-state source table and a labeled observation or source-only case comparison. Pass when it distinguishes kernel task creation from userspace exec, explains initcall registration and deferred probe, and gives a concrete root or interpreter failure with a safe diagnostic. If unsure, trace kernel_init_freeable, do_initcall_level and prepare_namespace again and revise the table. This is learning evidence, not certification.

Sources#

• v6.12 init/main.c: tasks, initcalls, initramfs wait and init execution.
• v6.12 init/do_mounts.c: root selection and waiting.
• v6.12 initcall macros and linker section definitions.
• Rootfs and initramfs.
• Kernel parameters.