Kbuild, toolchains, and reproducible builds

Before you start#

Read the disposable QEMU lab and source navigation. This chapter explains upstream v6.12. Its host experiment uses the chapter 08 x86-64 source and output directories, a working compiler and the same configuration; no running guest or privileged operation is required. If you cannot build, use the supplied source-only worked example. An architecture, configuration and toolchain are part of every result.

Learning goals#

Trace how Kbuild chooses and builds objects, assembles archives and links an x86-64 boot image. Distinguish host tools from target code and source files from generated configuration. Predict why an incremental build does or does not rebuild an object, and record the evidence required to reproduce a build rather than relying only on a version tag.

A build is a graph of decisions#

A kernel image is not created by compiling every .c file in the repository. GNU make starts from the top-level v6.12 Makefile, reads configuration and descends into selected directories using Kbuild rules. Individual directory Makefiles describe targets, while common scripts implement compilation and linking. The selected architecture supplies its own Makefiles, headers, entry code and linker scripts. Configuration controls whether many features are absent, compiled into the image, or built as modules. A file existing in Git is not proof that it was compiled or executed.

A target is an output that make can create from prerequisites; the commands that create it form a rule. Kbuild extends this familiar idea with saved command lines, generated configuration dependencies and recursive directory handling. A change to a compiler option can therefore matter even if the source file's timestamp is unchanged. Conversely, a source edit in an unselected driver normally need not change the image. The graph depends on configuration, architecture, make goals and the tools actually invoked. The Kbuild makefiles guide explains the conventions.

An output directory is an important boundary. With O=/some/path, the source tree remains the input, while .config, generated headers, object files and build products live in the output tree. Use the same O= for every subsequent command; otherwise make may read a different configuration or place artifacts in an unexpected tree. A source checkout can support separate output directories for x86-64 and arm64, but those directories must not share generated files. A clean source tree also makes it easier to identify your edits; it does not by itself make the output reproducible.

Source walkthrough#

At the beginning of v6.12 scripts/Makefile.build, this excerpt loads generated configuration and the directory's rules (SPDX GPL-2.0; surrounding lines omitted):

-include include/config/auto.conf

include $(srctree)/scripts/Kbuild.include
include $(srctree)/scripts/Makefile.compiler
include $(kbuild-file)

The leading - on the first include permits that file to be absent at this particular inclusion site; the overall build still has configuration preparation requirements. auto.conf carries configuration values in a form make can consume. srctree identifies source-side scripts, and kbuild-file selects the local rules for the directory being built. Other generated files, notably include/generated/autoconf.h, convey configuration to C preprocessing. .config is the input record; it is not a C header, and editing it without regenerating the derived files can leave the build inconsistent. Normal make targets arrange the necessary synchronization. The separate Kconfig documentation describes configuration targets.

The same Makefile.build contains the built-in archive rule (SPDX GPL-2.0; adjacent comments omitted):

$(obj)/built-in.a: $(real-obj-y) FORCE
	$(call if_changed,ar_builtin)

$(obj) is the output directory for this part of the build. real-obj-y identifies the selected built-in prerequisites after Kbuild expands composite objects. FORCE causes make to consider the rule, while if_changed uses Kbuild's change detection to decide whether its command must run; it does not mean that every invocation blindly reconstructs the archive. The archive command immediately above the rule uses ar with flags including thin-archive and deterministic options; its entries reference member object files rather than producing an ordinary standalone shared library. The top-level link combines selected built-in objects into vmlinux, the kernel ELF image. Linker scripts arrange sections and addresses; their architecture and generated inputs matter. For x86, a further architecture-specific build makes the bootable compressed arch/x86/boot/bzImage. vmlinux, its archive inputs and bzImage are related but not interchangeable artifacts.

The command behind that rule is in the same file. In v6.12 scripts/Makefile.build, SPDX GPL-2.0 (the comment above the first line omitted):

quiet_cmd_ar_builtin = AR      $@
      cmd_ar_builtin = rm -f $@; \
	$(if $(real-prereqs), printf "$(obj)/%s " $(patsubst $(obj)/%,%,$(real-prereqs)) | xargs) \
	$(AR) cDPrST $@

Read as shell text, the xargs pipeline and the following $(AR) cDPrST invocation form one command, so xargs supplies the member list to ar. The omitted comment concerns the argument-list length limit. The excerpt does not show how real-prereqs is expanded or which objects a particular build selected; those come from the obj-y lists and the earlier Kbuild expansion.

Follow a selected object#

Suppose a directory's Kbuild contains the illustrative line obj-$(CONFIG_EXAMPLE) += demo.o. This is synthetic make syntax, not a claim that Linux defines that symbol. If the effective configuration is y, expansion contributes demo.o to obj-y, and Kbuild can place it in the appropriate built-in archive. If it is m and the option supports modules, the object contributes to obj-m and participates in module construction. If the option is n, the assignment generally contributes to neither selected list. A composite target can have its own demo-y list of component objects. Directory entries ending in / cause descent; v6.12 scripts/Makefile.lib transforms built-in subdirectories into references to their built-in.a files and handles module ordering separately.

Now change demo.c while that object is selected. A subsequent make sees a changed prerequisite, recompiles it, updates the relevant archive and relinks affected outputs. If nothing changed and the saved command agrees, a second invocation may do little. If a header used by the object changes, the dependency information should also trigger the relevant rebuild. Kbuild's fixdep processes compiler-generated dependencies and tracks configuration references; inspect v6.12 scripts/basic/fixdep.c and a generated .cmd file to see the actual recorded command and prerequisites. .cmd files are build metadata, not source files to hand-edit. A generated file's existence does not prove it is current after an interrupted or failed build.

modpost performs module-related checks and generates metadata from the build's symbol information. It can detect unresolved symbols or licensing and section-related problems under relevant settings, but a successful modpost is not proof that a module will load or behave safely on a different kernel. Module.symvers, the configuration and the exact target build matter, especially with symbol versioning. See external module documentation. Chapter 10 follows the runtime loader and its errors.

Host tools, target tools and cross compilation#

Some executables created during a kernel build run on the host: configuration utilities, fixdep, and gen_init_cpio are examples. They must match the build machine. The target compiler, assembler and linker create the kernel image and modules for the target architecture. Confusing these roles produces recognizable errors: a host utility accidentally compiled for another CPU may fail with an executable-format error before any kernel image exists; an x86 executable called vmlinux is not an arm64 kernel merely because the output directory is named arm64.

ARCH selects the kernel architecture's build rules; CROSS_COMPILE traditionally supplies a prefix such as aarch64-linux-gnu- for target GNU tools. ARCH=arm64 does not itself install a cross compiler or select a firmware and QEMU machine. With LLVM, LLVM=1 selects supported Clang/LLVM tools using Kbuild's documented conventions; cross compilation still needs compatible target support and any required host tools. Follow the LLVM build guide for the release and installed tool versions. GCC and LLVM are alternatives with overlapping but not identical supported configurations. Do not mix output directories or change toolchains mid-investigation without recording and checking the resulting rebuild.

Errors have different meanings. A missing bc is a missing host program, not a guest boot failure. An unsupported compiler option can indicate an incompatible toolchain; an undefined reference during vmlinux linking is not fixed by running QEMU. A successful link does not establish that the image boots with your firmware, root filesystem or hardware. Retain the first error and the preceding command, not only the final make exit status.

Reproducibility and provenance#

A tag selects a source snapshot only after it is resolved, and it does not capture uncommitted changes. Record the full commit, dirty status, configuration, target ARCH, compiler and binutils versions, host tools, environment, command line and output directory. Also record source patches and generated inputs. Timestamps, build user and host identity, absolute paths and other metadata can change output bytes. The reproducible builds documentation explains controls such as KBUILD_BUILD_TIMESTAMP and related variables. A reproducible byte comparison requires independently prepared environments and a specified procedure; a successful second make in one output directory only demonstrates incremental behavior there.

Keep an artifact inventory: .config says what was requested and resolved, include/config/auto.conf and generated headers reflect configuration consumed by the build, .o files are intermediate objects, built-in.a organizes selected built-ins, vmlinux is a linked ELF image, and x86 bzImage is a boot artifact. Some products appear only after specific goals complete. A partial or failed build can leave older outputs in place: check timestamps, command output and exit status before calling an image newly built.

Reading the link boundary#

A linker script is an input to the link, not a bootloader command. It specifies how input sections are arranged into the output image and defines symbols that kernel code or later build stages can use. The architecture supplies the relevant script and may generate it from preprocessed inputs, so a link failure involving an undefined section boundary or an unexpected size must be investigated against that architecture's generated script and link command. readelf -S can show output sections in a successfully linked vmlinux; it cannot establish that the bootloader actually placed a compressed x86 image where early code expects it. Keep the build-time section layout separate from the runtime memory map handed in by firmware.

A dependency mismatch in context#

Imagine two output directories built from the same clean checkout. Directory A has CONFIG_MODULES=y; directory B disables modules. Even if the compiler version and a particular C source are identical, their generated configuration inputs can differ and the build graph can select different targets. Copying a .o or .cmd file from A into B bypasses the assumptions under which its command and prerequisites were recorded. The safe investigation is to rebuild B using B's own configuration and output tree, then compare results with the provenance attached. Similarly, changing ARCH while reusing a directory can leave architecture-specific generated material that does not describe the new target. Separate output directories make that boundary visible.

For debugging a rebuild, first distinguish why make considered a target from why the compile command actually ran. Save the verbose log, the object's .cmd file and the timestamps or hashes of the relevant inputs. If the command changed, the change itself can explain a rebuild even when file contents did not. If a generated configuration dependency changed, determine which effective option changed before blaming an unrelated source file. This investigation is narrower and more useful than assuming that every subsequent build either fully recompiles or does no work at all.

Practice#

Host-only controlled incremental build. Use the private chapter 08 checkout and output directory, with LAB set to that lab's directory. The commands write only build artifacts there. First establish a baseline and confirm that you are not changing source:

cd "$LAB/linux" || exit 1
git -C linux status --short
git -C linux rev-parse 'v6.12^{commit}'
make -C linux O="$LAB/build" V=1 init/main.o > "$LAB/first-build.log" 2>&1 || exit 1
sha256sum "$LAB/build/init/main.o"
make -C linux O="$LAB/build" V=1 init/main.o > "$LAB/second-build.log" 2>&1 || exit 1
sha256sum "$LAB/build/init/main.o"
grep -E '(^| )(-c|CC )' "$LAB/second-build.log" || printf 'No visible compile line in second log.\n'

A successful first command produces the object. With no changed inputs and unchanged command, the second normally does not recompile it, though preparatory commands or tool-version differences may appear. Equal hashes show byte equality of this one artifact at those moments, not a reproducible full build. Inspect build/init/.main.o.cmd and the logs for the recorded command and dependencies. If a build fails, stop and preserve the first error; don't use an old object as evidence of the failed build's success.

Now force a single output artifact stale without touching tracked source: remove only the lab output's init/main.o, then rebuild the same target. This is a controlled counterexample to “make always does nothing when source is unchanged”: a missing target must be recreated. Verify LAB and the path first; never remove a host kernel or another project's files.

printf '%s\n' "$LAB/build/init/main.o"
rm -- "$LAB/build/init/main.o"
make -C "$LAB/linux" O="$LAB/build" V=1 init/main.o > "$LAB/rebuild.log" 2>&1 || exit 1
ls -l "$LAB/build/init/main.o"

The rebuilt object should reappear and the log should show a compile. It is an output artifact; the source is untouched. Keep the three logs and hashes. If you cannot build, use the source-only state table: unchanged source + existing target + matching saved command → no necessary recompilation; missing selected target → a rule must recreate it; an unselected source edit → no implication that the image changes. Cite the archive rule and explain why this table predicts rather than measures behavior. Cleaning the entire lab is described in chapter 08; keep your evidence first.

Knowledge check#

1. Why do obj-y, built-in.a, vmlinux and bzImage describe different stages rather than four names for one file?
2. What does if_changed contribute to the archive rule, and why does FORCE not imply unconditional rebuilding?
3. What does ARCH=arm64 select, and why does it not make an x86 host compiler an arm64 cross compiler?
4. What would equal object hashes on two successive local builds establish, and what would they not establish?
5. Why can a file named bzImage after a failed make be misleading?

Answers#

1. obj-y selects built-in objects, archives collect them, vmlinux is the linked ELF kernel and x86 bzImage is a further boot image. The exact graph is architecture and configuration dependent.
2. It compares relevant saved command and dependency state when make considers the rule. FORCE ensures evaluation; Kbuild's conditional logic can still skip the command if nothing relevant changed.
3. It chooses the arm64 kernel build rules. Target-capable compiler, assembler and linker, compatible host tools, configuration and boot environment must still be supplied.
4. They establish equality of that artifact at those two observations. They do not independently reproduce a full build across environments or establish a successful boot.
5. A failed invocation can leave a prior output in place. Preserve the exit status, build log, configuration and artifact timestamps or hashes to distinguish old from newly produced output.

Checkpoint#

Deliver a graph from configuration through one selected object, archive, vmlinux and x86 bzImage, plus the incremental logs or the source-only state table. Pass when you explain the missing-object rebuild, the role of a host tool and three inputs beyond the Git tag needed for reproduction. If not, repeat the target build and inspect its .cmd file and the two quoted Kbuild rules. This is learning evidence, not certification.

Sources#

• v6.12 scripts/Makefile.build: config include and archive rule.
• v6.12 scripts/Makefile.lib: subdirectory and object handling.
• v6.12 scripts/basic/fixdep.c: dependency processing.
• Kbuild makefiles.
• Reproducible builds.
• LLVM builds.