x86/static_call: Add support for Jcc tail-calls
authorPeter Zijlstra <peterz@infradead.org>
Thu, 26 Jan 2023 15:34:27 +0000 (16:34 +0100)
committerIngo Molnar <mingo@kernel.org>
Tue, 31 Jan 2023 14:05:31 +0000 (15:05 +0100)
commit923510c88d2b7d947c4217835fd9ca6bd65cc56c
tree3339b2c1e4a8b7b0832c1b27573cb1e70e04e6f1
parentac0ee0a9560c97fa5fe1409e450c2425d4ebd17a
x86/static_call: Add support for Jcc tail-calls

Clang likes to create conditional tail calls like:

  0000000000000350 <amd_pmu_add_event>:
  350:       0f 1f 44 00 00          nopl   0x0(%rax,%rax,1) 351: R_X86_64_NONE      __fentry__-0x4
  355:       48 83 bf 20 01 00 00 00         cmpq   $0x0,0x120(%rdi)
  35d:       0f 85 00 00 00 00       jne    363 <amd_pmu_add_event+0x13>     35f: R_X86_64_PLT32     __SCT__amd_pmu_branch_add-0x4
  363:       e9 00 00 00 00          jmp    368 <amd_pmu_add_event+0x18>     364: R_X86_64_PLT32     __x86_return_thunk-0x4

Where 0x35d is a static call site that's turned into a conditional
tail-call using the Jcc class of instructions.

Teach the in-line static call text patching about this.

Notably, since there is no conditional-ret, in that case patch the Jcc
to point at an empty stub function that does the ret -- or the return
thunk when needed.

Reported-by: "Erhard F." <erhard_f@mailbox.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Link: https://lore.kernel.org/r/Y9Kdg9QjHkr9G5b5@hirez.programming.kicks-ass.net
arch/x86/kernel/static_call.c