Lockdep, preemption, and PREEMPT_RT

Before you start#

Read locks and sleeping, real-time and deadline, and interrupts and softirqs. This chapter describes upstream v6.12. Its ABBA graph is synthetic and its optional host program analyzes a small graph without acquiring kernel locks or changing scheduling policy. No deliberate host deadlock or lockdep warning is required.

Learning goals#

Explain how lockdep learns lock-class dependencies and context usage, and why its reports and coverage have limits. Analyze ABBA and interrupt reentry with a real ordering fix. Distinguish PREEMPT_RT ordinary spinlock behavior from raw locks, IRQ masking and priority-inheritance protocols.

Classes, instances and observations#

Lockdep is a runtime lock correctness validator when built and enabled with relevant instrumentation. A lock instance is a particular lock object; a lock class groups instances that share a locking role and therefore participate in the same dependency rules. A class key, initialization site and annotations affect that grouping. Nested instances can need subclasses or other documented annotations when a genuine hierarchy permits them. Incorrectly assigning classes can hide a real problem or produce a misleading report, so class annotations must describe an actual invariant rather than suppress a warning.

As a task acquires locks, lockdep tracks held-lock state and can record an ordering edge from a previously held class to a newly acquired class. If a path exists in the opposite direction, the combined graph can describe a potential deadlock even if the particular run did not hang. Read and write acquisitions, trylocks, nesting and context affect the rules; a simple directed graph is a useful beginning, not the entire lockdep algorithm. The lockdep design documentation explains the validator's model.

Coverage is bounded by executed and instrumented paths, configuration, class annotations and lockdep's own resource limits. A clean run is not proof that all possible lock orders or device paths are safe. A warning is valuable evidence of a potential violation, but assess whether the reported classes, context and annotation assumptions reflect the actual design. Reproducing a warning in a disposable guest and checking the code's invariants is different from declaring every report either certainly exploitable or automatically false. Do not disable lockdep or invent a subclass merely to make the report disappear.

Source walkthrough#

In v6.12 kernel/locking/lockdep.c, the public lock_acquire entry contains this excerpt (SPDX GPL-2.0-only; tracing and the disabled/NMI branch omitted):

raw_local_irq_save(flags);
check_flags(flags);

__lock_acquire(lock, subclass, trylock, read, check,
	       irqs_disabled_flags(flags), nest_lock, ip, 0, 0, 0);

The entry first traces and checks whether debugging and lockdep are enabled. On the shown normal path it saves local IRQ state, checks flags and invokes internal acquisition tracking with the class map, subclass, acquisition type, nesting information, instruction pointer and prior IRQ state. The surrounding recursion guard and IRQ restore are essential; these two excerpts are not a complete call. The validator's own raw IRQ handling is an implementation detail and is not a rule that every validated ordinary spin_lock_irqsave necessarily masks hardware IRQs under every configuration.

The same file's check_noncircular searches for an existing conflicting path. The following excerpt is v6.12 (same SPDX notice; report branches and closing braces omitted):

ret = check_path(target, &src_entry, hlock_conflict, NULL, &target_entry);

if (unlikely(ret == BFS_RMATCH)) {

The search result indicates whether a conflicting path was found; the following branches arrange trace information and print a deadlock or circular-dependency report depending on the classes. This check is part of chain validation, not a simulation of every possible CPU interleaving. A failure to exercise an acquisition path means its dependency may never be observed. The validator also tracks IRQ context usage and can detect incompatible hardirq/softirq-safe and unsafe uses under its rules.

ABBA and an interrupt counterexample#

Take two lock classes A and B. On path 1, a task holds A and then acquires B, producing edge A → B. On path 2, another task holds B and then acquires A, producing B → A. The graph has a cycle. In one possible execution, task 1 holds A while task 2 holds B; each waits for the other lock, so neither can progress. Lockdep can identify the dependency cycle from observed orders even if the exact simultaneous deadlock did not occur in the test. The source-only graph does not claim a warning was emitted on this host.

A real fix is to establish and enforce a consistent order, for example always take A before B, and redesign paths that cannot obey it. Sometimes the correct fix is to avoid holding one lock across the other operation, split a critical section or change ownership. First verify that the proposed order preserves the protected invariants and lifetime; mechanically reversing a lock can introduce a different race. A legitimate nested hierarchy may need a subclass only when its ordering is real, bounded and documented. Reclassifying two locks as unrelated merely to silence a genuine cycle is not a fix.

IRQ reentry creates another possible cycle even with one lock instance. Suppose process context holds lock A while local interrupts remain enabled, and a hard IRQ on that same CPU takes A. The interrupt cannot return until it gets A, while the interrupted task cannot release A until the interrupt returns. The appropriate design may use an IRQ-safe acquisition protocol, defer work or otherwise avoid sharing that lock in the hard handler. The exact choice depends on context and PREEMPT_RT rules. Disabling local interrupts alone also does not exclude a different CPU; cross-CPU mutual exclusion still needs the correct lock.

Lockdep records context usage so that a class used in interrupt context and acquired with interrupts enabled elsewhere can reveal unsafe combinations. Hardirq and softirq contexts have distinct rules; NMI nesting can impose further constraints. A lockdep report does not prove that a physical interrupt actually arrived during the test. Conversely, a path absent from testing or compiled out of the selected configuration is not covered by the absence of a report.

PREEMPT_RT changes the design constraints#

The upstream v6.12 preemption Kconfig defines PREEMPT_RT as a boolean choice dependent on EXPERT and ARCH_SUPPORTS_RT, selecting preemption support. Whether an architecture and configuration offer it is a build-time question; a release string alone does not establish that a running kernel uses it. RT reduces many sources of non-preemptible latency through changes in locking and interrupt execution, but it does not promise zero latency or turn an application into a correctly analyzed real-time system.

On PREEMPT_RT, many ordinary spinlock_t and rwlock_t operations are implemented using RT-aware lock mechanisms with priority inheritance and different preemption behavior. raw_spinlock_t remains a raw spinning lock for the limited low-level contexts that require it. This is a semantic change, not merely a faster implementation. In particular, code must not assume that ordinary spin_lock_irqsave necessarily disables hardware interrupts on RT as it does in the familiar non-RT mental model. Use the actual primitive and lock-type documentation for the context and configuration.

In include/linux/spinlock_rt.h, SPDX GPL-2.0-only, this PREEMPT_RT branch of spin_lock_irqsave reads as follows (the header's other declarations omitted):

#define spin_lock_irqsave(lock, flags)			 \
	do {						 \
		typecheck(unsigned long, flags);	 \
		flags = 0;				 \
		spin_lock(lock);			 \
	} while (0)

On this branch the macro zeroes flags as a placeholder and calls spin_lock, which on RT is the RT-aware lock, so no hardware IRQ mask is taken here. Code that depends on the saved state in flags has to be audited for RT rather than assumed portable. The excerpt does not show the raw_spinlock_t definitions or the non-RT branch, so it cannot tell you what a given driver's interrupt context needs.

local_lock supplies a named per-CPU protection scope. On non-RT it can map to preemption or local interrupt disabling according to the variant; on RT it maps to a per-CPU spinlock_t with the corresponding transformed semantics. It is not a general inter-CPU lock, and a source pattern that relies on implicit preemption or interrupt masking must be audited when moved to RT. Migration control and preemption control are related but distinct: a task may be prevented from migrating while still being preemptible under applicable RT rules. Check the primitive's documented guarantees instead of substituting one for the other.

In include/linux/local_lock_internal.h, SPDX GPL-2.0, the PREEMPT_RT branch's __local_lock_irqsave reads as follows (the non-RT branch and other macros omitted):

#define __local_lock_irqsave(lock, flags)			\
	do {							\
		typecheck(unsigned long, flags);		\
		flags = 0;					\
		__local_lock(lock);				\
	} while (0)

On RT this variant does not call local_irq_save; it zeroes flags and takes the per-CPU lock through __local_lock. A section that relied on local interrupt masking for its local_lock_irqsave protection therefore needs a fresh audit on RT. The excerpt does not show the body of __local_lock, so it does not establish what a preempted holder may do.

Most device interrupts are forced into threads on an RT configuration where eligible, allowing more work to be preemptible. Exceptions, explicitly non-threaded handlers, per-CPU interrupts and architecture-specific paths remain; an NMI is not converted into an ordinary sleeping task merely by enabling RT. A raw hard-IRQ path still needs short bounded work and appropriate raw locking. Allocations and user accesses can sleep or fault depending on flags and API, so moving code into a threaded handler does not make every allocation or lock nesting safe. Raw locks are reserved for genuinely low-level requirements because they retain non-preemptible sections and can increase latency.

Priority inheritance is a progress aid#

An RT mutex can raise an owner's effective priority when a higher-priority waiter depends on it. If the owner itself waits for another PI-aware lock, the implementation can propagate priority through a chain subject to its rules. When waiters change or locks are released, the effective priority can be adjusted downward again. The v6.12 RT mutex source uses waiter and owner state protected by the relevant wait and PI locks. The boost does not transfer lock ownership or guarantee a bounded critical section.

For example, low-priority L holds A, high-priority H waits for A, and medium-priority M is runnable. With an applicable PI mutex, H's priority can be donated to L so M does not indefinitely prevent L from running to release A. If L is blocked on unbounded I/O, holding an unrelated raw spinlock too long, or waiting in a lock cycle, PI alone does not solve the problem. After L releases A and no other waiter requires the boost, deboosting restores the effective priority according to the remaining dependencies. This is a conditional model, not a host scheduling experiment.

Diagnostics have different scopes#

might_sleep and CONFIG_DEBUG_ATOMIC_SLEEP help detect operations that may sleep in invalid contexts when the relevant checks are enabled and exercised. They are not a complete proof of all lock lifetimes. Lockstat measures selected lock contention and timing when configured and enabled; it answers a different question from lockdep's dependency correctness model. Tracing and instrumentation can alter timing and add overhead. A lockdep splat, an atomic-sleep warning and a contention measurement each require their own interpretation and source context.

A useful review records the lock class and instance, the held-lock stack, context, CPU, acquisition sites and configuration. For an IRQ-safe/unsafe report, determine whether the lock is actually shared with an interrupt and whether the intended masking or threading rules apply. For an ABBA report, draw the cycle and locate both acquisitions. Then make a change that preserves ordering and object lifetime and test it under a bounded workload. A quiet log after exercising only one path is evidence only about that limited run.

Coverage has a lifecycle#

A boot-time test can exercise initialization locks but miss removal, error cleanup, suspend or a rare interrupt path. Even if lockdep records A before B during initialization, it cannot invent an unexecuted B-before-A path from absent runtime evidence. Conversely, once both class relationships are established, the validator can reason about the cycle without waiting for a production hang. A meaningful validation plan therefore identifies the configuration and paths exercised, including failure and teardown paths, and keeps logs from a disposable guest. It should also record if lockdep disabled itself after a diagnostic or reached a resource limit: later silence may no longer represent active checking. Instrumentation and class assignment are part of the experiment, not mere presentation details.

Practice#

Source-only route: draw nodes A and B with A → B and B → A; list a schedule in which task 1 owns A and task 2 owns B, then each attempts the other. Remove the cycle by specifying a consistent acquisition order and explain what data each lock protects. Separately draw the process/IRQ reentry case and name the context condition that makes it dangerous. This graph is teaching data, not an actual lockdep report.

Safe host route: run a single-thread Python graph check; it does not acquire kernel locks or create a deadlock:

python3 - <<'PY'
edges = [('A', 'B'), ('B', 'A')]
graph = {}
for src, dst in edges:
    graph.setdefault(src, set()).add(dst)
def reachable(start, target, seen=None):
    seen = set() if seen is None else seen
    if start in seen:
        return False
    seen.add(start)
    return target in graph.get(start, ()) or any(reachable(n, target, seen) for n in graph.get(start, ()))
print('cycle in synthetic graph:', reachable('A', 'A'))
graph['B'].remove('A')
print('cycle after removing B->A:', reachable('A', 'A'))
PY

Expect True then False. This models only reachability; it does not reproduce lockdep classes, shared locks, IRQ-state validation or a real execution. Removing an edge in the model is not evidence that a source change safely removed the corresponding acquisition. As a counterexample, if the second path still acquires B then A in untested code, the real cycle remains even though this small table says otherwise.

Optionally inspect an exposed running-kernel config for CONFIG_PROVE_LOCKING, CONFIG_DEBUG_ATOMIC_SLEEP and CONFIG_PREEMPT_RT using /proc/config.gz or a matching /boot/config-$(uname -r) as in the previous chapter. A missing or stale config is not proof a feature is absent. Do not enable lockdep, change priorities, inject a deadlock or write tracing knobs on the host. No files are created by the graph model and no cleanup is required.

Knowledge check#

1. How can lockdep detect an ABBA possibility without observing both tasks deadlocked at once?
2. Why is assigning a new subclass not automatically a valid fix for a circular dependency?
3. Why can a task and a hard interrupt deadlock on the same lock even on one CPU?
4. What changes about ordinary spinlock_t under PREEMPT_RT, and what remains true of raw_spinlock_t?
5. Why does a clean lockdep run not establish complete correctness?

Answers#

1. It records observed class-order dependencies; observing both A → B and B → A can expose a cycle that admits a deadlocking interleaving.
2. A subclass must represent a real documented nesting hierarchy. An annotation that merely hides contradictory acquisition orders leaves the underlying race or deadlock intact.
3. The interrupt can preempt the holder and wait for a lock that the interrupted task cannot release until the interrupt returns. The correct IRQ/context protocol must exclude or avoid that reentry.
4. Many ordinary spinlocks become RT-aware sleeping lock mechanisms with changed preemption and IRQ assumptions. Raw spinlocks retain raw spinning semantics and strict non-sleeping constraints.
5. Only enabled, instrumented and exercised paths contribute evidence; configuration, annotations and resource limits affect what can be checked.

Checkpoint#

Deliver the ABBA and IRQ-reentry graphs and the labeled synthetic graph output. Pass when you propose a real ordering or ownership change, explain why a subclass is not a suppression tool, and state the RT spinlock_t/raw-lock distinction. If uncertain, retrace lock_acquire, check_noncircular and the RT lock documentation before revising the graph. This is not certification.

Sources#

• v6.12 lockdep implementation: acquisition and circular-path checking.
• v6.12 RT mutex implementation.
• v6.12 preemption configuration.
• Lockdep design.
• Lock types and PREEMPT_RT.