KUnit, kselftest, syzkaller, and regression evidence

Before you start#

Review source navigation, build and configuration, kernel debugging, and sanitizers. The baseline is upstream Linux v6.12. The chapter 08 QEMU image remains unbooted because of the observed GCC 16/C23 compatibility issue; no KUnit guest run or fuzzing result is claimed here.

Learning goals#

Distinguish KUnit from kselftest and fuzzing, trace a KUnit case failure and cleanup, interpret skip separately from pass, and design reproducible regression evidence with an appropriate isolation boundary.

Source walkthrough#

KUnit is an in-kernel unit-testing framework. A suite groups cases, and a case can exercise internal kernel functions and structures that are not exposed as a userspace ABI. Fixtures arrange per-test setup and teardown; assertions express expected conditions. The kernel emits KTAP-formatted results, which userspace tooling can parse. A test compiled into a kernel is not evidence that it ran: configuration, suite selection, boot, filtering and output collection must all be recorded.

In lib/kunit/test.c, kunit_run_case_internal invokes a suite's optional initialization before the case body. This exact v6.12 SPDX GPL-2.0 excerpt omits surrounding declarations and timing code:

		ret = suite->init(test);
		if (ret) {
			kunit_err(test, "failed to initialize: %d\n", ret);
			kunit_set_failure(test);
			return;
		}

The fixture initializer returns a status. If it fails, the framework records a failure and returns from the internal case runner rather than calling the case body as if setup succeeded. The enclosing kunit_run_case_catch_errors uses try/catch machinery to run the case and separately run cleanup; it propagates a test failure to the case result. A test author must understand which resources were acquired before failure and which cleanup mechanism owns them. A case abort, timeout, failed initializer, skipped case and assertion failure do not all establish the same result.

KUnit assertions can stop the current case or record a failure depending on the assertion or expectation API. They do not automatically prove that a device, concurrency interleaving or every configuration is covered. A fixture must not leave work, references or allocations active after its owning test state is released. Tests that manipulate global state need isolation and cleanup, including failure paths, to avoid affecting later cases.

kselftest and a real skip path#

kselftest generally exercises kernel-facing behavior from userspace, though its collections also include shell scripts, helper modules and tests with special prerequisites. It can test system calls, procfs, networking and many other UAPIs from a process context. That makes it complementary to KUnit's internal white-box access, not automatically an end-to-end test of every distribution or device. Some selftests require privileges, namespaces, hardware, network setup or configuration and can modify their environment; read the individual test before running it.

In tools/testing/selftests/pidfd/pidfd_test.c, test_pidfd_send_signal_recycled_pid_fail checks prerequisites before attempting its PID-namespace scenario. This exact v6.12 excerpt is from the unshare failure branch; SPDX GPL-2.0; surrounding lines are omitted:

		if (errno == EPERM) {
			ksft_test_result_skip("%s test: Unsharing pid namespace not permitted\n",
					      test_name);
			return 0;
		}

When creating the PID namespace fails with EPERM, this branch records a skip and returns from that test function. A different PID-namespace error reaches a failure exit. If PID namespace creation succeeds, the function next requests a mount namespace; EPERM there is also reported as a skip, while other errors are fatal. After that it makes / recursively private before changing /proc inside a child namespace. This namespace and mount setup is a reason to use an isolated test environment. A skip means the intended assertion was not exercised. It is neither a pass nor a failure of the tested kernel behavior. The helper's return 0 only returns from that helper; the file later calls ksft_exit_pass(), and the harness also sees the emitted per-test skip. Interpret per-test records together with the process exit status. KSFT_SKIP is exit code 4 for tests that use that convention.

KTAP evidence and fixtures#

Consider this synthetic result, not output from a kernel test run:

KTAP version 1
1..3
ok 1 - basic_case
ok 2 - optional_case # SKIP prerequisite unavailable
not ok 3 - boundary_case

The plan declares three results. The first line reports a pass for one case, the second reports a skip despite beginning with ok, and the third reports a failure. The skipped case provides no evidence that its behavior works. The failing case needs its diagnostics, configuration and reproduction; the four lines do not say whether the cause is a kernel regression, a test bug, an unavailable dependency or corrupted output. If a suite crashes before reporting all planned cases, missing results and timeout must not be silently converted to passes.

A fixture might allocate a test object, initialize a lock and register cleanup for the allocation. If initialization fails before the object exists, teardown must not dereference an uninitialized pointer. If it succeeds and the case later fails, registered resources and suite cleanup must still be handled according to the framework's actual contract. Workqueues, timers and references created by the case may need cancellation and synchronization before the fixture is freed. The kunit_run_case_catch_errors source's separate cleanup invocation is useful evidence, but it does not automatically repair an incorrectly written fixture.

The run and cleanup stages are separate calls in lib/kunit/test.c, v6.12, SPDX GPL-2.0 (the function's opening and its result propagation after this excerpt are omitted):

	kunit_try_catch_init(try_catch,
			     test,
			     kunit_try_run_case,
			     kunit_catch_run_case);
	context.test = test;
	context.suite = suite;
	context.test_case = test_case;
	kunit_try_catch_run(try_catch, &context);

	/* Now run the cleanup */
	kunit_try_catch_init(try_catch,
			     test,
			     kunit_try_run_case_cleanup,
			     kunit_catch_run_case_cleanup);
	kunit_try_catch_run(try_catch, &context);

The cleanup call is a second try/catch run over the same context, so a case that fails still reaches its cleanup stage. The excerpt does not show whether a particular fixture's resources were released correctly, which is still the test author's responsibility. It also does not show what happens to a case that times out, since no timeout branch appears here.

v6.12 runners and environment#

The v6.12 KUnit wrapper supports run, a --kunitconfig fragment, --build_dir, --arch, --jobs and --timeout. The checked lib/kunit/.kunitconfig enables KUnit's own tests and example tests. In a disposable build environment with a clean v6.12 checkout and a verified compatible compiler, an optional UML run is:

./tools/testing/kunit/kunit.py run --arch=um --kunitconfig=lib/kunit --build_dir=.kunit-lab --jobs=2 --timeout=120

This command builds and runs a UML kernel; it is not a read-only host command and has not been run here. UML is a separate kernel process but still consumes host resources and needs a trusted build environment. For non-UML architectures the wrapper uses configured QEMU support; guest images, toolchains and hardware emulation must match the architecture. A compiler failure, unavailable QEMU, missing configuration symbol or timeout means no passing kernel test result exists. The previous GCC 16/C23 issue must be resolved and the resulting build verified before treating the chapter 08 lab as operational.

The v6.12 selftests Makefile supports selected targets. For example, in a dedicated disposable guest with the necessary build dependencies and after reviewing the pidfd test's namespace and mount behavior:

make -C tools/testing/selftests TARGETS=pidfd
make -C tools/testing/selftests TARGETS=pidfd run_tests

Build and run are separate operations. This collection may skip cases if prerequisites or permissions are missing and can create namespaces and alter mount propagation in its own test environment. Use a throwaway guest with recovery and cleanup, not a shared host or ordinary container. The command is documented for that future environment and was not run here. Preserve the runner output, exit status, per-case results and configuration; a successful build is not a successful test run.

Fuzzing and reproducer evidence#

Fuzzing generates and mutates inputs to explore behavior beyond a fixed set of hand-written cases. syzkaller at commit d02c3949702ccb7c6e269ca3d78fe55190b3c415 is a separately versioned project; that immutable reference identifies a source snapshot, not a claim that it is compatible with every v6.12 build. Its syscall descriptions constrain generated programs, an executor performs operations in a target environment, and coverage feedback can guide exploration when configured. A crash report can be grouped with similar reports, and a reproducer can be minimized, but deduplication and minimization are heuristics: two similar stacks can have different causes, while one cause can yield different stacks.

A useful reproducer includes the exact kernel revision and config, architecture, compiler, boot arguments, executor or userspace version, sequence, timing assumptions and expected failure. Sanitizer reports add evidence but also alter timing and memory use. A syzbot report is an externally produced observation with its own build and reproduction status; read its linked config, report and current state before treating it as applicable to your build. An untrusted reproducer can crash the target or manipulate resources and must not be run without review in a shared environment.

Never fuzz a host kernel or a container that shares that kernel. A bounded future fuzz experiment would require a dedicated disposable VM, a throwaway image and snapshot, explicit CPU, memory, disk and time limits, a controlled network boundary, a compatible kernel and executor, a designated crash-output directory, and a recovery plan to terminate and recreate the guest. Disable or constrain access to real devices and sensitive data. Review generated or external reproducers before execution, stop at the bound, retain the artifacts and discard the guest. No guest fuzzing or untrusted reproducer is run by this lesson.

Regression evidence, not a single green line#

A regression workflow starts with a precise symptom and an environment in which the failure can be reproduced. Run the smallest trusted reproducer against the suspected bad revision, record configuration and output, then test an appropriate earlier revision or negative control. Apply the proposed fix and repeat the same test, plus neighboring behavior and relevant configurations. A test that only passes after the fix is useful only if it also demonstrates the intended failure on the unfixed system or otherwise establishes why it would detect the bug.

Bisection requires consistent good and bad classifications. A revision that fails to build, lacks a dependency or cannot run the test is not automatically good or bad; mark it untestable and record why. Changes in compiler, QEMU, config, userspace or workload can create apparent regressions. A flaky race requires repeated trials and reported frequencies, not a single pass. Timeouts must identify the limit and whether the system was still making progress. An environmental difference between baseline and fix can explain an apparent improvement without the code change being responsible.

Test outcomes have scope: a KUnit pass for a pure helper does not prove a device teardown race is fixed, and a kselftest skip does not prove the UAPI behavior. Cross-check with source reasoning, targeted tests, sanitizer evidence where applicable and review. Preserve raw output and negative cases so another engineer can challenge the result. A fixed kernel can still contain unrelated errors; the evidence supports a bounded claim about the tested conditions.

Reading a failing test as a state machine#

Consider a KUnit case that initializes a fixture, creates a kernel object, performs one operation, and checks an invariant. If the initializer returns an error, the cited source marks failure and does not execute the case body. If initialization succeeds but an assertion fails, the case has executed and its diagnostics can identify the violated expectation. If the test times out, neither a pass nor an ordinary assertion failure can be inferred; the worker or kernel might still need cleanup or termination. The framework's try/catch wrapper and cleanup stage make those paths observable, but test-specific resources still need correct ownership.

For the pidfd selftest, a denied PID-namespace creation causes the explicit skip branch. If namespace creation succeeds, later setup can still fail: the source also requests a mount namespace and makes mount propagation private before continuing the scenario. The first skip therefore provides no evidence about PID reuse or signal delivery. A test report should preserve the prerequisite result, per-case status and exact kernel/environment, rather than counting the process's zero return from this helper as a successful assertion. A different non-EPERM unshare error follows the failure path in the source.

A reproducible regression package can contain a short command sequence, the test binary and source revision, kernel configuration, console log, timestamps, exit status and a table of outcomes. For a race, retain each trial's result and timeout rather than reporting only the one that failed or passed. When a reproducer is minimized, rerun both the original and minimized cases on the same known-bad and candidate-fixed builds. Minimization can remove environmental setup that was accidentally masking or contributing to a failure; a smaller reproducer is valuable only if the observed behavior remains attributable to the same problem.

A fuzzing campaign adds resource and trust boundaries. The manager and executor version, enabled syscall descriptions, kernel coverage configuration, guest snapshots and crash collection determine what inputs were actually explored. A crash deduplication key based on a top frame can group unrelated lifetime errors; changing compiler or sanitizer configuration can split reports for one bug. Treat such grouping as triage guidance, then compare the actual accesses, allocation/free histories and minimized sequences. The pinned syzkaller commit is a reference for future investigation, not a claim that this workspace ran that version.

Practice#

Safe userspace result interpretation

The following program processes the supplied synthetic outcomes in its own process; it does not run a kernel test or alter the host:

python3 - <<'PY'
results = [('basic_case', 'PASS'), ('optional_case', 'SKIP'), ('boundary_case', 'FAIL')]
for state in ('PASS', 'SKIP', 'FAIL'):
    print(state, sum(result == state for _, result in results))
print('kernel behavior tested: no; input is synthetic')
PY

Expected counts are one pass, one skip and one fail. The counterexample to “all ok lines are passes” is the KTAP skip line above: it starts with ok but has the skip directive. Deliver a short report distinguishing the synthetic result from a real test, identify the unexercised prerequisite and list the artifacts needed to reproduce the failure. Missing Python is a valid limitation; use the supplied table without installing anything.

Source-only alternative: trace suite initialization failure through the quoted KUnit branch and explain why the case body does not run. Separately trace EPERM from the pidfd test's namespace setup to its skip. These branches are supplied source evidence; they are not a claim that either test ran on this host.

Knowledge check#

1. What happens when the KUnit suite initializer returns an error in the quoted path?
2. Why is the pidfd test's EPERM result a skip rather than a pass?
3. Why is a build failure during bisection not automatically a good or bad result?
4. What additional evidence makes a fix test more convincing than a single green run?
5. Why is a container not sufficient isolation for host-kernel fuzzing?

Answers#

1. The framework records a failure and returns before invoking the case body. The enclosing runner separately handles cleanup and result propagation.
2. The requested PID namespace was not created, so the intended scenario was not exercised. The test explicitly records the missing permission as a skip.
3. The target behavior was not observed under that revision. A compiler or dependency failure is a separate classification and should be recorded as untestable.
4. Reproduction before the fix, the same controlled test after it, negative and neighboring cases, configuration identity and repeated trials for nondeterministic bugs support a stronger bounded claim.
5. Containers share the host kernel. Generated syscalls or a crash can affect the host and other containers; a disposable VM supplies a separate guest kernel boundary.

Checkpoint#

Submit the synthetic pass/skip/fail report and a before/after regression plan with an untestable classification and guest recovery boundary. Ready means a skip is not called a pass, fixture failure does not run the case body, and no unrun KUnit or fuzz result is claimed. Retry the two source branches if the evidence is ambiguous. This is not certification.

Sources#

• KUnit case runner v6.12
• pidfd kselftest v6.12
• KUnit wrapper v6.12
• KUnit runner documentation
• kselftest documentation