x86/vdso: Make delta calculation overflow safe
authorAdrian Hunter <adrian.hunter@intel.com>
Mon, 25 Mar 2024 06:40:12 +0000 (08:40 +0200)
committerThomas Gleixner <tglx@linutronix.de>
Mon, 8 Apr 2024 13:03:07 +0000 (15:03 +0200)
commit7e90ffb716d289b3b82fb41892bb52a11bdadfd9
treefa2d674efac6e51898351ae8e6320d815922a050
parent456e3788bc7164c1c8298045e04068b8e3d8e413
x86/vdso: Make delta calculation overflow safe

Kernel timekeeping is designed to keep the change in cycles (since the last
timer interrupt) below max_cycles, which prevents multiplication overflow
when converting cycles to nanoseconds. However, if timer interrupts stop,
the calculation will eventually overflow.

Add protection against that. Select GENERIC_VDSO_OVERFLOW_PROTECT so that
max_cycles is made available in the VDSO data page. Check against
max_cycles, falling back to a slower higher precision calculation. Take
advantage of the opportunity to move masking and negative motion check
into the slow path.

The result is a calculation that has similar performance as before. Newer
machines showed performance benefit, whereas older Skylake-based hardware
such as Intel Kaby Lake was seen <1% worse.

Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/r/20240325064023.2997-9-adrian.hunter@intel.com
arch/x86/Kconfig
arch/x86/include/asm/vdso/gettimeofday.h