Clocksources, timers, NO_HZ, and timekeeping

Before you start#

Read interrupts and softirqs, scheduler core, and CPU, memory and I/O. The source baseline is upstream v6.12; clock hardware, suspend behavior, architecture and configuration affect runtime observations. The source-only arithmetic and timer timeline need no guest. The optional host commands read clocks and clocksource information without setting time or changing kernel parameters.

Learning goals#

Distinguish a clocksource from a clockevent device and a clock value from a timer deadline. Follow a sequence-counted timekeeper read and compare timer deletion with shutdown. Explain wraparound, periodic deadlines, tick suppression and callback lifetime without treating clock resolution as a latency guarantee.

Reading time versus scheduling an event#

A clocksource provides a monotonically advancing hardware or virtual cycle counter under its documented properties. The timekeeping code turns cycle deltas into time using a multiplier and shift, while handling the counter's mask and wrap. A clockevent device is used to arrange an interrupt at a future time. A platform may use related hardware for both, but they answer different questions: “what time is it?” versus “notify me when this deadline is reached.” Selecting a high-resolution clocksource does not ensure that an interrupt or a scheduled task will run at a precise instant.

The kernel timekeeper maintains base values, conversion parameters and offsets for different clocks. Readers need a consistent snapshot while writers update those values. In v6.12 kernel/time/timekeeping.c, ktime_get contains this excerpt (SPDX GPL-2.0; declarations and surrounding statements omitted):

do {
	seq = read_seqcount_begin(&tk_core.seq);
	base = tk->tkr_mono.base;
	nsecs = timekeeping_get_ns(&tk->tkr_mono);
} while (read_seqcount_retry(&tk_core.seq, seq));

The reader begins a sequence-counted snapshot, reads the monotonic base and converted current contribution, then retries if a concurrent writer changed the sequence. A retry avoids combining values from inconsistent updates. A sequence counter does not by itself protect object lifetime or permit a reader to sleep arbitrarily; writers require their documented serialization, and data read optimistically must be safe to retry. The subsequent ktime_add_ns combines the values. ktime_get warns when timekeeping is suspended in the relevant code; it is not a universal interface for reading a running clock during suspend.

The clocksource_cyc2ns helper in v6.12 include/linux/clocksource.h expresses a conversion as multiplication followed by a right shift. The actual timekeeper's calculations and overflow constraints include additional details; using a small worked example is safer than assuming arbitrary cycle and multiplier values cannot overflow. Clocksource quality, stability, watchdog decisions and virtualization can affect which source is selected. Consult timekeeping accessors and the architecture's clocksource implementation for a real system.

In v6.12 include/linux/clocksource.h, SPDX GPL-2.0 (kernel-doc comment omitted):

static inline s64 clocksource_cyc2ns(u64 cycles, u32 mult, u32 shift)
{
	return ((u64) cycles * mult) >> shift;
}

The multiplication is done in u64 before the shift, so a large cycles value times a large mult can exceed 64 bits; the comment above the helper describes the inputs it is meant for. The excerpt does not show how the timekeeper chooses mult and shift, which is why the worked example in this chapter uses synthetic values only.

Source walkthrough#

A low-resolution timer_list and an hrtimer are different timer facilities. Timer-wheel structures group many timers efficiently at coarser granularity, while high-resolution timers use ordered expiry queues and suitable clockevent support for finer deadlines when enabled. A timer expiry is a request to invoke a callback no earlier than an applicable deadline under its semantics, not a guarantee that application work completes then. Interrupt masking, scheduling, hardware, callback backlog and configuration introduce delay. A timer's owner must preserve both the timer object and everything its callback can access until the appropriate cancellation and synchronization protocol is complete.

In v6.12 kernel/time/timer.c, these two short functions have different final arguments (SPDX GPL-2.0; declarations and exports omitted):

return __timer_delete_sync(timer, false);

return __timer_delete_sync(timer, true);

The first is the body of timer_delete_sync; it deactivates a timer and synchronizes with a running callback under its documented rules, but another actor can rearm it unless the caller prevents that. del_timer_sync is a compatibility name encountered in existing code; use the exact v6.12 declarations and documentation when reading it. The second is the body of timer_shutdown_sync; its shutdown mode also prevents subsequent rearming of that timer, with attempted rearm ignored as documented. These functions can wait and have locking/context constraints. Neither is a substitute for stopping every other callback that holds a reference to the surrounding object.

For a concrete race, imagine a timer callback queues work, while the work callback rearms the timer. A teardown that merely calls timer_delete_sync and then frees the object can race with work that rearms after deletion. The source documentation gives the shutdown ordering for this circular relationship: shut down the timer synchronously, destroy or otherwise synchronize the workqueue according to its contract, then free the enclosing object after all other references are gone. Do not hold a lock needed by the timer callback while waiting for that callback to finish; the documented deadlock example in the source shows why. This is a conditional design example, not a universal teardown recipe for every timer owner.

Clock domains and adjustments#

CLOCK_REALTIME represents wall-clock time and can be adjusted, including discontinuous steps. CLOCK_MONOTONIC is intended for elapsed-time ordering and is not subject to discontinuous wall-clock jumps, though its rate can be adjusted by time discipline. Linux documents that CLOCK_MONOTONIC stops during suspend; CLOCK_BOOTTIME follows the same monotonic reference but includes suspended time. CLOCK_MONOTONIC_RAW exposes a rawer hardware-based time scale without the same frequency adjustments. CLOCK_TAI represents a distinct timescale related to wall time and leap-second handling. Exact availability and suspend semantics depend on the platform and kernel. See the v6.12 ktime accessor documentation.

NTP-related time discipline can slew the clock's rate or, through authorized clock-setting operations and policy, step wall time. Leap-second handling and the TAI offset are not identical to “all clocks jump together.” A userspace daemon may obtain network time and request adjustments, while the kernel implements its timekeeping and adjustment interfaces; neither the mere presence of a daemon nor a single clock reading proves synchronization quality. Do not set host time for this lesson. A monotonic deadline is generally more suitable for elapsed timeout logic than subtracting two realtime readings that might straddle a wall-clock step, but applications must select a clock whose suspend and adjustment semantics match their requirements.

The vDSO can provide userspace clock reads without a system call on supported architectures and configurations, with fallback paths where needed. A strace that does not show clock_gettime therefore does not prove the program did not read time. Conversely a measured difference between two readings contains execution and scheduling overhead. Clock resolution is the granularity exposed by an interface, not the accuracy of the hardware, the precision of a network synchronization source or the worst-case timer wakeup latency.

A cycle and a periodic deadline#

For a simplified 32-bit cycle counter, let the previous reading be 0xfffffff0 and the new reading be 0x00000010. Assuming at most one wrap and a mask of 0xffffffff, (new - old) & mask is 32 cycles. If the conversion parameters are mult=1000 and shift=0, the delta is 32 × 1000 = 32,000 ns, or 32 microseconds. This synthetic example corresponds to 1,000 ns per cycle. If more than one wrap occurred between samples, the masked subtraction alone cannot reveal how many wraps happened; a real clocksource must satisfy its sampling and stability requirements. Other multipliers and shifts approximate the conversion with integer arithmetic and can introduce rounding.

For a periodic task, suppose an ideal absolute schedule has deadlines at 10, 20 and 30 ms from a chosen start. The first wake occurs at 10.4 ms and the callback or work takes 3 ms. Scheduling the next absolute deadline at 20 ms preserves the intended phase, though it may still wake late. Sleeping a further 10 ms after finishing at 13.4 ms instead targets about 23.4 ms, accumulating the callback and wakeup delay into the phase. If work overruns multiple periods, the application must decide whether to skip, catch up or report missed periods; an absolute deadline alone cannot do that. These are illustrative values, not measured host latency.

Tick suppression and callback context#

jiffies is a kernel time base associated with HZ, the configured tick frequency, but modern kernels do not necessarily receive a periodic tick on every CPU continuously. NO_HZ_IDLE can suppress unnecessary ticks while a CPU is idle. NO_HZ_FULL, where configured and usable, can reduce scheduling-clock ticks for suitable workloads on selected CPUs, while housekeeping CPUs and other events still perform needed work. Interrupts, timers, RCU, scheduler requirements, multiple runnable tasks and configuration can prevent complete tick suppression. “Tickless” does not mean that the CPU receives no interrupts, no timers expire, or RCU and scheduler accounting disappear. See NO_HZ documentation.

Timer callbacks execute in contexts defined by their timer type, flags and configuration. Ordinary timer callbacks commonly execute through timer softirq processing; high-resolution timers can have hard or soft execution modes and PREEMPT_RT changes relevant defaults and constraints. A callback must obey the exact API's locking and sleepability rules, not assume that every timer is a normal process-context thread. Pinned timers, CPU hotplug and migration have specific behavior; a timer associated with one CPU should not be assumed to remain there across all transitions. Timer slack can coalesce wakeups under applicable userspace and kernel interfaces, trading exactness for efficiency.

hrtimer_cancel and timer-wheel shutdown operations have different interfaces; do not apply the timer_list shutdown guarantee to an hrtimer by name alone. A callback that rearms itself or schedules work needs a teardown state and synchronization protocol appropriate to the type. A timeout return also does not prove that an asynchronous callback cannot later touch a stack object; lifetime belongs to the whole producer/consumer relationship. Later chapters examine workqueues and completions in more detail.

Clock selection changes the question#

Suppose an application records a wall-clock timestamp before a machine suspends and another afterward. The difference includes both the time while the system was active and the suspended interval, subject to realtime adjustments and the platform's ability to maintain time. A monotonic clock that excludes suspend answers a different elapsed-time question. If an application needs to expire a lease even across suspend, a clock with boottime semantics may be more appropriate than one that pauses during suspend; its correctness still depends on the API, platform and application contract. Comparing values from different clock domains without converting or identifying their origins is not meaningful.

Now suppose an administrator steps wall time backward while a program waits for a deadline. A timeout based on absolute realtime can be affected differently from one based on a monotonic clock. A frequency correction can also make a disciplined monotonic clock progress at a slightly different rate from a raw clock without creating a discontinuous backward jump. A single pair of readings cannot establish which adjustment occurred. Record the selected clock ID and whether the timeout is absolute or relative before drawing conclusions from a late wakeup.

Rearming and cancellation are separate decisions#

A timer can be pending while its callback is not running, or its callback can be running after the pending instance was removed from the timer queue. Another CPU or a work item may try to rearm it concurrently. A non-synchronous deletion result alone therefore cannot prove that a callback has stopped. Even a synchronized deletion needs the caller to prevent a separate producer from rearming afterward. Final shutdown adds the no-rearm behavior for that timer_list, but it does not cancel a different hrtimer or wait for unrelated work queued by the callback. A teardown record should name each producer, timer, work item and owner reference instead of using the vague phrase “the timer is gone.”

Practice#

Source-only route: calculate the 32-cycle wrap and 32,000 ns conversion above, showing the mask and units. Draw the 10/20/30 ms absolute schedule and the 23.4 ms relative-sleep counterexample. Add a timer → work → rearm cycle and mark where timer_shutdown_sync and work synchronization must occur before freeing the enclosing object. This is supplied reasoning, not a guest trace.

Read-only host route: run the following as an ordinary Linux user. It does not adjust any clock or kernel setting:

python3 - <<'PY'
import time
for name in ('time', 'monotonic', 'perf_counter'):
    info = time.get_clock_info(name)
    print(name, 'implementation=', info.implementation,
          'monotonic=', info.monotonic, 'adjustable=', info.adjustable,
          'resolution=', info.resolution)
a = time.monotonic_ns()
time.sleep(0.02)
b = time.monotonic_ns()
print('observed monotonic elapsed ns:', b-a)
PY
cat /sys/devices/system/clocksource/clocksource0/current_clocksource 2>/dev/null || printf 'clocksource not exposed\n'
cat /sys/devices/system/clocksource/clocksource0/available_clocksource 2>/dev/null || printf 'available sources not exposed\n'

The sleep is bounded and applies only to this Python process. Elapsed time should normally be at least the requested 20 ms, but scheduler delay can make it longer; unusual clock or environment behavior should be recorded, not hidden. Python's adjustable field describes that interface and is not a complete report of frequency slewing. Sysfs may be hidden, virtualized or absent. Clocksource names can reveal system information, so review them before sharing. No host clock, timezone, governor or timer parameter is changed and no files need cleanup.

For a counterexample, a reported nanosecond resolution does not prove that the 20 ms sleep woke within one nanosecond of its target. Deliver the arithmetic table, observed clock metadata if available and one reason the observed delay is not a direct measurement of hardware clock accuracy.

Knowledge check#

1. What does the sequence-count retry in ktime_get protect against, and what does it not protect by itself?
2. Why is a clockevent device not interchangeable with a clocksource?
3. What does timer_shutdown_sync add beyond synchronous deletion in the shown v6.12 API?
4. Why can relative sleeps drift even when every individual sleep is approximately correct?
5. Why does NO_HZ_FULL not mean a CPU can never receive a timer interrupt?

Answers#

1. It prevents a reader from accepting a mixture of timekeeper values across a concurrent update. Object lifetime and writer serialization still need their own rules.
2. A clocksource supplies a counter used to compute time; a clockevent device arranges a future notification. Their hardware can be related but their roles differ.
3. It also prevents later rearming of that timer after shutdown, while waiting for a running callback under its documented rules. Other work and references still require synchronization.
4. A relative delay scheduled after work completes incorporates work and wakeup lateness into the next target. An absolute schedule preserves phase but still must handle overruns.
5. Tick suppression is conditional and other interrupts, timers, RCU or scheduler requirements can still need service. Housekeeping and configuration also matter.

Checkpoint#

Deliver the wraparound calculation, periodic schedule and timer/work teardown diagram, plus labeled host observations if performed. Pass when you distinguish source from event, state the timer rearm race and explain one clock-domain or tick-suppression limitation. If uncertain, retrace ktime_get and the two timer deletion functions and recalculate the units. This is not certification.

Sources#

• v6.12 timekeeping: ktime_get.
• v6.12 timer implementation: synchronous deletion and shutdown.
• v6.12 clocksource conversion and hrtimer implementation.
• Timekeeping accessors.
• NO_HZ documentation.