The kernel needs to explicitly enable FSGSBASE. So, the application needs
to know if it can safely use these instructions. Just looking at the CPUID
bit is not enough because it may be running in a kernel that does not
enable the instructions.
One way for the application would be to just try and catch the SIGILL.
But that is difficult to do in libraries which may not want to overwrite
the signal handlers of the main application.
Enumerate the enabled FSGSBASE capability in bit 1 of AT_HWCAP2 in the ELF
aux vector. AT_HWCAP2 is already used by PPC for similar purposes.
The application can access it open coded or by using the getauxval()
function in newer versions of glibc.
Signed-off-by: Andi Kleen <email address hidden>
Signed-off-by: Chang S. Bae <email address hidden>
Reviewed-by: Tony Luck <email address hidden>
Cc: Thomas Gleixner <email address hidden>
Cc: Borislav Petkov <email address hidden>
Cc: Andy Lutomirski <email address hidden>
Cc: H. Peter Anvin <email address hidden>
Cc: Dave Hansen <email address hidden>
Cc: Tony Luck <email address hidden>
Cc: Andi Kleen <email address hidden>
Signed-off-by: Marcelo Henrique Cerri <email address hidden>
19cac2e...
by
Andy Lutomirski <email address hidden>
UBUNTU: SAUCE: x86/fsgsbase/64: Enable FSGSBASE on 64bit by default and add a chicken bit
Before enabling FSGSBASE the kernel could safely assume that the content
of GS base was a user address. Thus any speculative access as the result
of a mispredicted branch controlling the execution of SWAPGS would be to
a user address. So systems with speculation-proof SMAP did not need to
add additional LFENCE instructions to mitigate.
With FSGSBASE enabled a hostile user can set GS base to a kernel address.
So they can make the kernel speculatively access data they wish to leak
via a side channel. This means that SMAP provides no protection.
Add FSGSBASE as an additional condition to enable the fence-based SWAPGS
mitigation.
Signed-off-by: Tony Luck <email address hidden>
Signed-off-by: Chang S. Bae <email address hidden>
Cc: Thomas Gleixner <email address hidden>
Cc: Borislav Petkov <email address hidden>
Cc: Andy Lutomirski <email address hidden>
Cc: H. Peter Anvin <email address hidden>
Cc: Dave Hansen <email address hidden>
Cc: Tony Luck <email address hidden>
Cc: Andi Kleen <email address hidden>
Signed-off-by: Marcelo Henrique Cerri <email address hidden>
e60ed41...
by
"Chang S. Bae" <email address hidden>
UBUNTU: SAUCE: x86/fsgsbase/64: Use FSGSBASE instructions on thread copy and ptrace
When FSGSBASE is enabled, copying threads and reading FS/GS base using
ptrace must read the actual values.
When copying a thread, use fsgs_save() and copy the saved values. For
ptrace, the bases must be read from memory regardless of the selector
if FSGSBASE is enabled.
Suggested-by: Andy Lutomirski <email address hidden>
Signed-off-by: Chang S. Bae <email address hidden>
Reviewed-by: Tony Luck <email address hidden>
Cc: Thomas Gleixner <email address hidden>
Cc: Borislav Petkov <email address hidden>
Cc: Andy Lutomirski <email address hidden>
Cc: H. Peter Anvin <email address hidden>
Cc: Dave Hansen <email address hidden>
Cc: Tony Luck <email address hidden>
Cc: Andi Kleen <email address hidden>
Signed-off-by: Marcelo Henrique Cerri <email address hidden>
d997024...
by
Andy Lutomirski <email address hidden>
UBUNTU: SAUCE: x86/fsgsbase/64: Use FSGSBASE in switch_to() if available
With the new FSGSBASE instructions, FS/GS base can be efficiently read
and written in __switch_to(). Use that capability to preserve the full
state.
This will enable user code to do whatever it wants with the new
instructions without any kernel-induced gotchas. (There can still be
architectural gotchas: movl %gs,%eax; movl %eax,%gs may change GS base
if WRGSBASE was used, but users are expected to read the CPU manual
before doing things like that.)
This is a considerable speedup. It seems to save about 100 cycles per
context switch compared to the baseline 4.6-rc1 behavior on a Skylake
laptop.
[ chang: 5~10% performance improvements were seen by a context switch
benchmark that ran threads with different FS/GS base values (to the
baseline 4.16). ]
Signed-off-by: Andy Lutomirski <email address hidden>
Signed-off-by: Chang S. Bae <email address hidden>
Reviewed-by: Tony Luck <email address hidden>
Cc: Thomas Gleixner <email address hidden>
Cc: Borislav Petkov <email address hidden>
Cc: Andy Lutomirski <email address hidden>
Cc: H. Peter Anvin <email address hidden>
Cc: Dave Hansen <email address hidden>
Cc: Tony Luck <email address hidden>
Cc: Andi Kleen <email address hidden>
Signed-off-by: Marcelo Henrique Cerri <email address hidden>
697f2a9...
by
"Chang S. Bae" <email address hidden>
UBUNTU: SAUCE: x86/fsgsbase/64: Enable FSGSBASE instructions in helper functions
Add CPU feature conditional FS/GS base access to the relevant helper
functions. That allows accelerating certain FS/GS base operations in
subsequent changes.
Note, that while possible, the user space entry/exit GS base operations are
not going to use the new FSGSBASE instructions. The reason is that it would
require additional storage for the user space value which adds more
complexity to the low level code and experiments have shown marginal
benefit. This may be revisited later but for now the SWAPGS based handling
in the entry code is preserved except for the paranoid entry/exit code.
Suggested-by: Tony Luck <email address hidden>
Signed-off-by: Chang S. Bae <email address hidden>
Reviewed-by: Tony Luck <email address hidden>
Cc: Thomas Gleixner <email address hidden>
Cc: Borislav Petkov <email address hidden>
Cc: Andy Lutomirski <email address hidden>
Cc: H. Peter Anvin <email address hidden>
Cc: Dave Hansen <email address hidden>
Cc: Tony Luck <email address hidden>
Cc: Andi Kleen <email address hidden>
Cc: Andrew Cooper <email address hidden>
Signed-off-by: Marcelo Henrique Cerri <email address hidden>
Without FSGSBASE, user space cannot change GS base other than through a
PRCTL. The kernel enforces that the user space GS base value is positive
as negative values are used for detecting the kernel space GS base value
in the paranoid entry code.
If FSGSBASE is enabled, user space can set arbitrary GS base values without
kernel intervention, including negative ones, which breaks the paranoid
entry assumptions.
To avoid this, paranoid entry needs to unconditionally save the current
GS base value independent of the interrupted context, retrieve and write
the kernel GS base and unconditionally restore the saved value on exit.
The restore happens either in paranoid exit or in the special exit path of
the NMI low level code.
All other entry code paths which use unconditional SWAPGS are not affected
as they do not depend on the actual content.
The new logic for paranoid entry, when FSGSBASE is enabled, removes SWAPGS
and replaces with unconditional WRGSBASE. Hence no fences are needed.
Suggested-by: H. Peter Anvin <email address hidden>
Suggested-by: Andy Lutomirski <email address hidden>
Suggested-by: Thomas Gleixner <email address hidden>
Signed-off-by: Chang S. Bae <email address hidden>
Reviewed-by: Tony Luck <email address hidden>
Acked-by: Tom Lendacky <email address hidden>
Cc: Thomas Gleixner <email address hidden>
Cc: Borislav Petkov <email address hidden>
Cc: Andy Lutomirski <email address hidden>
Cc: H. Peter Anvin <email address hidden>
Cc: Dave Hansen <email address hidden>
Cc: Tony Luck <email address hidden>
Cc: Andi Kleen <email address hidden>
Cc: Tom Lendacky <email address hidden>
Cc: Vegard Nossum <email address hidden>
[<email address hidden>: Fix context. 4.15 doesn't have the
macro STACKLEAK_ERASE_NOCLOBBER]
Signed-off-by: Marcelo Henrique Cerri <email address hidden>
4104a02...
by
"Chang S. Bae" <email address hidden>
UBUNTU: SAUCE: x86/entry/64: Introduce the FIND_PERCPU_BASE macro
GS base is used to find per-CPU data in the kernel. But when GS base is
unknown, the per-CPU base can be found from the per_cpu_offset table with a
CPU NR. The CPU NR is extracted from the limit field of the CPUNODE entry
in GDT, or by the RDPID instruction. This is a prerequisite for using
FSGSBASE in the low level entry code.
Also, add the GAS-compatible RDPID macro as binutils 2.21 does not support
it. Support is added in version 2.27.
Suggested-by: H. Peter Anvin <email address hidden>
Signed-off-by: Chang S. Bae <email address hidden>
Reviewed-by: Tony Luck <email address hidden>
Cc: Thomas Gleixner <email address hidden>
Cc: Borislav Petkov <email address hidden>
Cc: Andy Lutomirski <email address hidden>
Cc: H. Peter Anvin <email address hidden>
Cc: Dave Hansen <email address hidden>
Cc: Tony Luck <email address hidden>
Cc: Andi Kleen <email address hidden>
Cc: Vegard Nossum <email address hidden>
[<email address hidden>: Fix context. 4.15 doesn't have the
macro STACKLEAK_ERASE]
Signed-off-by: Marcelo Henrique Cerri <email address hidden>
