Before you start#
Read x86 boot and init and CPU, memory and I/O. This chapter reads upstream v6.12 arm64 source and its boot protocol; it does not require an arm64 board, a cross compiler or a virtual machine. The chapter 08 x86-64 image is not an arm64 image. The supplied source-only exercise is sufficient for the checkpoint; no arm64 runtime behavior is claimed.
Learning goals#
Describe the arm64 loader's handoff, including the device-tree address, processor state and memory placement. Follow primary_entry through early mapping and exception-level setup to start_kernel. Separate the boot CPU from secondary CPU release, and compare this contract with x86 without assuming that firmware or discovery mechanisms are universal.
The loader's obligations#
An architecture boot protocol is an agreement between the software handing off control and the kernel receiving it. For arm64, the v6.12 boot protocol describes placement of the kernel Image, the device tree and optional initramfs, and processor state at entry. The loader must ensure these objects do not overlap in prohibited ways and remain accessible as required. A device tree blob, or DTB, is a binary description of hardware and boot information; it is not executable driver code and it is not interchangeable with the kernel image.
For the documented primary entry, register x0 contains the physical address of the DTB, while x1, x2 and x3 are reserved and must be zero. Interrupts are masked, the CPU is in non-secure EL1 or EL2, and the documented normal protocol requires the MMU off. The protocol also specifies cache maintenance, coherency, timer and system-register conditions. The instruction cache may be on or off under its stated constraints; saying simply “all caches must be off” loses an important part of the contract. A correct address placed in x0 is not enough if it points to an invalid or overwritten blob.
Read the text version
- x0: Physical address of the DTB under the documented primary entry. → x21: Keeps the FDT pointer through the early primary path. (mov x21, x0 in preserve_boot_args)
- x0: Physical address of the DTB under the documented primary entry. → boot_args: Array that preserve_boot_args fills from x0 to x3 at kernel entry. (first stp stores x21 into boot_args)
- x1 to x3: Reserved by the protocol and expected to be zero at entry. → boot_args: Array that preserve_boot_args fills from x0 to x3 at kernel entry. (stp instructions store x1 to x3)
- x0: Physical address of the DTB under the documented primary entry. → __fdt_pointer: Saved in __primary_switched and read by setup_arch in setup.c. (str_l saves x21 as __fdt_pointer)
The uncompressed Image is different from a compressed Image.gz. The protocol notes that the AArch64 kernel image does not itself provide a decompressor for that compressed image: a loader using Image.gz must decompress it before entering the kernel. This contrasts with the x86 compressed startup path discussed in chapter 11. A bootloader, firmware and virtual machine are separate components; which one loads or decompresses an image depends on the environment. A direct QEMU configuration must satisfy the applicable handoff just as a physical platform must, but its emulated devices and firmware are not a description of every board.
Source walkthrough#
- Loaderprimary_entry
x0 identifies the DTB; enter under boot protocol
- primary_entryprimary_entry
preserve boot arguments
- primary_entryearly MMU and EL
initialize EL and CPU, create early mapping
- early MMU and ELstart_kernel
primary switch and common initialization
Time runs downward. Dashed arrows show returns.
Read the text version
- Loader → primary_entry: x0 identifies the DTB; enter under boot protocol
- primary_entry → primary_entry: preserve boot arguments
- primary_entry → early MMU and EL: initialize EL and CPU, create early mapping
- early MMU and EL → start_kernel: primary switch and common initialization
In v6.12 arch/arm64/kernel/head.S, primary_entry begins (SPDX GPL-2.0-only; surrounding instructions omitted):
bl record_mmu_state
bl preserve_boot_args
bl branches to a subroutine and records a return address. The first call records information about the entry MMU and cache state, and the second preserves boot arguments before later work can overwrite the argument registers. In preserve_boot_args, the source moves the incoming x0 into x21 and saves the entry registers. The file's comments describe x21 as carrying the FDT pointer on the primary low-level path. This is an address to a physical boot-time object under the protocol; it is not a userspace pointer or a guarantee that the blob is valid.
In arch/arm64/kernel/head.S, v6.12, SPDX GPL-2.0-only (surrounding code omitted):
SYM_CODE_START_LOCAL(preserve_boot_args)
mov x21, x0 // x21=FDT
adr_l x0, boot_args // record the contents of
stp x21, x1, [x0] // x0 .. x3 at kernel entry
The first instruction copies the FDT pointer from x0 into x21, and the stp at line 173 stores x21 and x1 into boot_args. The next line (omitted here) stores x2 and x3. The excerpt does not show whether the pointer is valid or whether the blob stays in memory until it is mapped; those depend on the loader and the later setup_machine_fdt path.
The same primary_entry later includes these instructions (same file and SPDX notice; cache handling and setup comments omitted):
bl init_kernel_el
mov x20, x0
The call initializes exception-level state and returns a boot-mode value in x0; the next instruction preserves that value in x20. The source then calls __cpu_setup and branches to __primary_switch. That path enables the necessary translation setup and reaches __primary_switched, where the code establishes initial task and vector state, saves the FDT pointer and eventually calls start_kernel. Follow the exact labels and conditional code in the source before drawing a complete instruction-level trace. There are alternatives for boot environment and configuration, and these excerpts do not prove which instructions ran on any particular machine.
In arch/arm64/kernel/head.S, v6.12, SPDX GPL-2.0-only (cache, KASAN and other lines omitted):
SYM_FUNC_START_LOCAL(__primary_switched)
adr_l x4, init_task
init_cpu_task x4, x5, x6
adr_l x8, vectors // load VBAR_EL1 with virtual
msr vbar_el1, x8 // vector table address
isb
stp x29, x30, [sp, #-16]!
mov x29, sp
str_l x21, __fdt_pointer, x5 // Save FDT pointer
/* ... */
ldp x29, x30, [sp], #16
bl start_kernel
The FDT pointer saved in __fdt_pointer is read later by setup_arch in arch/arm64/kernel/setup.c, which passes it to setup_machine_fdt. The excerpt does not show the mapping step or the boot-mode handling in the omitted lines.
The code also handles a case in which it observes entry with MMU and caches on, including cleaning an identity-mapped region before transitioning. That implementation branch does not amend the normal documented boot protocol's requirement that a conforming direct handoff enter with the MMU off. Do not generalize a defensive or special entry path into permission for arbitrary loaders to violate the published contract. In early execution, cache maintenance, barriers and translation tables protect the transition between address interpretations; they are not cosmetic performance hints.
- record_mmu_state
Record whether the MMU was on at entry; the result is kept in x19.
- preserve_boot_args
Copy the FDT pointer from x0 into x21 and store x0 to x3 in boot_args.
- init_kernel_el
Set up exception-level state; the boot mode returns in x0 and is kept in x20.
- __cpu_setup
Prepare processor state, including TCR, before the MMU is enabled.
- __primary_switch
Enable the MMU with the early page tables and move to the early stack.
- __primary_switched
Set up the initial task and vectors, save __fdt_pointer, then call start_kernel.
Exception levels and early mappings#
An exception level defines a privilege context in the architecture. EL0 normally hosts user programs; Linux normally executes at EL1, while EL2 provides a hypervisor context when present and configured. EL3, if implemented, is associated with higher-level secure firmware responsibilities. Exact virtualization behavior depends on CPU features and firmware. init_kernel_el establishes the kernel's early execution mode from the allowed entry state, and subsequent code records the mode for later initialization. It is wrong to infer from an arm64 label alone that EL2 is available, that KVM is enabled or that the kernel is running a guest.
The memory-management unit translates addresses under page tables. Early identity mappings permit a controlled transition while code and data must remain reachable as translation is enabled or adjusted. The source creates an initial identity map and performs cache handling based on the entry state. Later mappings support the kernel's normal virtual-address layout. A physical DTB address passed in x0 must remain valid until the kernel maps and consumes it. In v6.12 arch/arm64/kernel/setup.c, setup_machine_fdt remaps the blob and reserves its physical memory with memblock when mapping succeeds. Failure to find or validate a usable boot description is an early boot problem, not an ordinary userspace file-not-found error.
A practical thought experiment: the loader places the image and DTB in non-overlapping RAM, sets x0 to the DTB physical address, zeroes the reserved registers, and enters at a permitted exception level with the documented MMU and interrupt state. The kernel preserves that pointer, builds its early mappings, switches into its intended execution environment and later parses the boot data. If the loader instead overwrites the DTB with the initramfs before handoff, the register can still contain the old numeric address while the bytes there no longer describe a valid tree. Correct register value and correct object lifetime are separate conditions. This is a conceptual trace, not a captured machine boot.
Device Tree, ACPI and firmware services#
Device Tree and ACPI are different ways to describe platform hardware and configuration. On arm64 the early FDT can also convey boot information in ACPI-capable environments; the existence of a DTB does not by itself prove that every hardware device will be enumerated from Device Tree. In the v6.12 arm64 setup_arch source, setup_machine_fdt occurs early, acpi_boot_table_init runs later, and the full device tree is unflattened when ACPI is disabled. This ordering is a specific source observation, not an assertion that all firmware supplies identical tables or all platforms support both discovery modes. See the arm64 ACPI documentation.
UEFI is a firmware interface that can provide boot services and a memory map; ACPI is a set of tables and interfaces used to describe and manage platform features. They are related on many systems but are not synonyms. A DT-based embedded system may use a different loader and firmware arrangement. A boot image and a plausible hardware description still need the right drivers built in or available as modules, correct interrupt and clock information, and a root or initramfs. A missing GIC description, wrong interrupt routing, invalid timer frequency or incompatible driver can cause a failure well after the first kernel instruction.
PSCI, the Power State Coordination Interface, is a firmware interface commonly used on arm systems for CPU power and related operations. It is one possible mechanism for bringing secondary CPUs online; the precise mechanism must be described by platform firmware and supported by the kernel. GIC denotes an interrupt-controller family, with version and configuration differences. Neither a PSCI name in a description nor a GIC driver compiled into the image proves that firmware passed valid parameters or that interrupts are correctly routed. For a real investigation, preserve the exact platform description and kernel configuration along with logs.
Boot CPU, secondary CPUs and hotplug#
The boot CPU executes the primary entry and establishes core state. Secondary CPUs follow architecture-specific startup paths and are brought into a coherent system under the platform's release mechanism. The arm64 source has separate secondary labels and secondary_start_kernel; they should not be conflated with the primary primary_entry sequence. A secondary CPU must receive a valid entry address and suitable architectural state, and the kernel must coordinate its online transition with shared memory and interrupt setup.
CPU hotplug later removes or reintroduces CPUs under a running kernel's synchronization and subsystem rules. Some low-level code is shared with hotplug, but initial firmware handoff and a later online operation are not identical events. A system that boots on its primary CPU but fails to online a secondary CPU has passed some earlier milestones; it has not proved the PSCI, release, coherency or interrupt configuration correct. Conversely, a boot-time failure before the first console message may prevent any useful secondary-CPU evidence from appearing. No hotplug commands are needed in this chapter.
Comparing two contracts#
| Question | x86-64 example | arm64 example |
|---|---|---|
| Early image | bzImage includes a compressed startup path in the discussed configuration. | The protocol's uncompressed Image is entered after any required external decompression. |
| Boot data | x86 boot parameters and loader protocol carry the command line and related information. | The primary entry receives the DTB physical address in x0 under the documented contract. |
| Initial transition | The discussed compressed startup_64 path uses x86-specific registers and identity mapping requirements. | primary_entry records state, preserves arguments and prepares arm64 translation and exception-level state. |
| Later common code | Architecture-specific entry reaches start_kernel. | Architecture-specific entry reaches start_kernel. |
The common function name is a convergence point, not proof that the machines performed the same earlier work or have the same page tables, firmware, interrupt controller or boot log. The x86 boot protocol and arm64 boot protocol are the primary contracts for this comparison.
Practice#
Source-only, no cross build or hardware needed. Construct an evidence table for the hypothetical handoff below. It is teaching data, not output from a machine:
Image: uncompressed arm64 Image in RAM
x0: physical address of a non-overlapping DTB
x1, x2, x3: zero
entry: non-secure EL1; MMU off; interrupts masked
For each row, write the requirement, the evidence a loader or firmware would have to supply, and the consequence of a violated assumption. Add two states: before preserve_boot_args, x0 carries the DTB address; after the cited mov x21, x0, x21 retains that address for the early path. Then record the subsequent source steps init_kernel_el, __cpu_setup, __primary_switch, __primary_switched and start_kernel, explicitly marking configuration-dependent details as unresolved. This exercise does not claim to test image placement or execute assembly.
For a concrete counterexample, suppose the same address is passed in x0, but the DTB memory has been overwritten by the initramfs. The register test alone passes while the object contents and lifetime fail. Your diagnosis should ask for a memory map, blob validation and loader logs; it should not conclude that start_kernel ran. A second counterexample is attempting to boot an x86 bzImage as arm64 Image: naming or copying it does not change its instruction set or boot format.
If a local v6.12 source checkout from chapter 07 exists, the following host read-only commands corroborate the source map without building an arm64 kernel:
git -C "$LAB/linux" rev-parse 'v6.12^{commit}'
git -C "$LAB/linux" grep -n -e 'SYM_CODE_START(primary_entry)' -e 'bl[[:space:]]*init_kernel_el' -e 'bl[[:space:]]*start_kernel' v6.12 -- arch/arm64/kernel/head.S
git -C "$LAB/linux" show v6.12:Documentation/arch/arm64/booting.rst | sed -n '155,185p'
Expect source matches and the documented register and MMU requirements; line numbers and presentation should correspond to the exact tag. If the checkout is absent or LAB was not retained, use the linked source instead. A shallow repository can show tagged file content but has limited history. No generated files or host settings are changed, so no cleanup is needed. An arm64 QEMU virt build is a later optional exercise requiring a target compiler, arm64 kernel configuration, suitable Image, firmware or direct-loader arrangement and arm64 userspace; the x86 lab artifacts do not satisfy those requirements.
Knowledge check#
- What does
x0contain at the documented arm64 primary handoff, and why can the numeric value be correct while boot still fails? - Why does the MMU-on handling in
head.Snot establish that an arbitrary direct loader may ignore the normal MMU-off protocol? - What does
init_kernel_elcontribute, and why does entering at EL2 not prove KVM is active? - How do the initial FDT and later ACPI decision relate in the v6.12 setup source?
- Why are secondary CPU release and later CPU hotplug related but distinct?
Answers#
- It holds the DTB physical address. The memory must contain a valid, non-overwritten blob at the time it is mapped and parsed; a correct number cannot guarantee the pointed-to object's lifetime or contents.
- The published handoff contract states the normal MMU-off requirement. Source handling of an observed alternate or special environment is not a universal replacement for that contract.
- It establishes early exception-level state and returns a boot-mode value recorded by the primary path. CPU capability, firmware setup, kernel configuration and later virtualization initialization are additional conditions for KVM.
setup_archinvokes early FDT setup, then ACPI table initialization; it unflattens the device tree when ACPI is disabled. Early FDT presence is therefore not proof that all devices use DT enumeration.- Both involve architecture-specific CPU state, but initial secondary startup occurs during boot using a platform release mechanism; hotplug changes CPU availability after the system is already running and requires its own synchronization.
Checkpoint#
Deliver the source-only handoff table and the two counterexamples, or a separately documented arm64 guest observation if you have one. Pass when you can account for x0 and DTB lifetime, the documented MMU state, the primary entry sequence and one secondary-CPU uncertainty without claiming an execution you did not observe. If not, revisit the protocol and head.S in order and correct the table. This checkpoint is not certification.
Sources#
- v6.12 arm64 entry code: primary and secondary paths.
- v6.12 arm64 setup: FDT and ACPI setup.
- arm64 boot protocol.
- arm64 ACPI documentation.
- x86 boot protocol.