- Jul 29, 2022
-
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 CVE: CVE-2022-23816 CVE: CVE-2022-23825 CVE: CVE-2022-29900 CVE: CVE-2022-29901 Conflicts: Fuzz in the first hunk due to missing upstream commit 8ad7e8f6 ("x86/fpu/xsave: Support XSAVEC in the kernel"). commit a883d624 Author: Peter Zijlstra <peterz@infradead.org> Date: Tue, 14 Jun 2022 23:15:33 +0200 x86/cpufeatures: Move RETPOLINE flags to word 11 In order to extend the RETPOLINE features to 4, move them to word 11 where there is still room. This mostly keeps DISABLE_RETPOLINE simple. Signed-off-by:
Peter Zijlstra (Intel) <peterz@infradead.org> Signed-off-by:
Borislav Petkov <bp@suse.de> Reviewed-by:
Josh Poimboeuf <jpoimboe@kernel.org> Signed-off-by:
Waiman Long <longman@redhat.com>
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 CVE: CVE-2022-23816 CVE: CVE-2022-23825 CVE: CVE-2022-29900 CVE: CVE-2022-29901 commit 742ab6df Author: Peter Zijlstra <peterz@infradead.org> Date: Tue, 14 Jun 2022 23:15:32 +0200 x86/kvm/vmx: Make noinstr clean The recent mmio_stale_data fixes broke the noinstr constraints: vmlinux.o: warning: objtool: vmx_vcpu_enter_exit+0x15b: call to wrmsrl.constprop.0() leaves .noinstr.text section vmlinux.o: warning: objtool: vmx_vcpu_enter_exit+0x1bf: call to kvm_arch_has_assigned_device() leaves .noinstr.text section make it all happy again. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 036c07c0 Author: Josh Poimboeuf <jpoimboe@kernel.org> Date: Thu, 19 May 2022 11:46:58 -0700 x86/entry: Fix register corruption in compat syscall A panic was reported in the init process on AMD: Run /sbin/init as init process init[1]: segfault at f7fd5ca0 ip 00000000f7f5bbc7 sp 00000000ffa06aa0 error 7 in libc.so[f7f51000+4e000] Code: 8a 44 24 10 88 41 ff 8b 44 24 10 83 c4 2c 5b 5e 5f 5d c3 53 83 ec 08 8b 5c 24 10 81 fb 00 f0 ff ff 76 0c e8 ba dc ff ff f7 db <89> 18 83 cb ff 83 c4 08 89 d8 5b c3 e8 81 60 ff ff 05 28 84 07 00 Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b CPU: 1 PID: 1 Comm: init Tainted: G W 5.18.0-rc7-next-20220519 #1 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x57/0x7d panic+0x10f/0x28d do_exit.cold+0x18/0x48 do_group_exit+0x2e/0xb0 get_signal+0xb6d/0xb80 arch_do_signal_or_restart+0x31/0x760 ? show_opcodes.cold+0x1c/0x21 ? force_sig_fault+0x49/0x70 exit_to_user_mode_prepare+0x131/0x1a0 irqentry_exit_to_user_mode+0x5/0x30 asm_exc_page_fault+0x27/0x30 RIP: 0023:0xf7f5bbc7 Code: 8a 44 24 10 88 41 ff 8b 44 24 10 83 c4 2c 5b 5e 5f 5d c3 53 83 ec 08 8b 5c 24 10 81 fb 00 f0 ff ff 76 0c e8 ba dc ff ff f7 db <89> 18 83 cb ff 83 c4 08 89 d8 5b c3 e8 81 60 ff ff 05 28 84 07 00 RSP: 002b:00000000ffa06aa0 EFLAGS: 00000217 RAX: 00000000f7fd5ca0 RBX: 000000000000000c RCX: 0000000000001000 RDX: 0000000000000001 RSI: 00000000f7fd5b60 RDI: 00000000f7fd5b60 RBP: 00000000f7fd1c1c R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000 R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 </TASK> The task's CX register got corrupted by commit 8c42819b ("x86/entry: Use PUSH_AND_CLEAR_REGS for compat"), which overlooked the fact that compat SYSCALL apparently stores the user's CX value in BP. Before that commit, CX was saved from its stashed value in BP: pushq %rbp /* pt_regs->cx (stashed in bp) */ But then it got changed to: pushq %rcx /* pt_regs->cx */ So the wrong value got saved and later restored back to the user. Fix it by pushing the correct value again (BP) for regs->cx. Fixes: 8c42819b ("x86/entry: Use PUSH_AND_CLEAR_REGS for compat") Reported-by:
Guenter Roeck <linux@roeck-us.net> Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 1b331eee Author: Peter Zijlstra <peterz@infradead.org> Date: Fri, 6 May 2022 14:14:35 +0200 x86/entry: Remove skip_r11rcx Yes, r11 and rcx have been restored previously, but since they're being popped anyway (into rsi) might as well pop them into their own regs -- setting them to the value they already are. Less magical code. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 8c42819b Author: Peter Zijlstra <peterz@infradead.org> Date: Fri, 6 May 2022 14:14:34 +0200 x86/entry: Use PUSH_AND_CLEAR_REGS for compat Since the upper regs don't exist for ia32 code, preserving them doesn't hurt and it simplifies the code. This doesn't add any attack surface that would not already be available through INT80. Notably: - 32bit SYSENTER: didn't clear si, dx, cx. - 32bit SYSCALL, INT80: *do* clear si since the C functions don't take a second argument. - 64bit: didn't clear si since the C functions take a second argument; except the error_entry path might have only one argument, so clearing si was missing here. 32b SYSENTER should be clearing all those 3 registers, nothing uses them and selftests pass. Unconditionally clear rsi since it simplifies code. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit d205222e Author: Linus Torvalds <torvalds@linux-foundation.org> Date: Fri, 6 May 2022 14:14:33 +0200 x86/entry: Simplify entry_INT80_compat() Instead of playing silly games with rdi, use rax for simpler and more consistent code. Signed-off-by:
Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 64cbd0ac Author: Lai Jiangshan <jiangshan.ljs@antgroup.com> Date: Tue, 3 May 2022 11:21:06 +0800 x86/entry: Don't call error_entry() for XENPV XENPV guests enter already on the task stack and they can't fault for native_iret() nor native_load_gs_index() since they use their own pvop for IRET and load_gs_index(). A CR3 switch is not needed either. So there is no reason to call error_entry() in XENPV. [ bp: Massage commit message. ] Signed-off-by:
Lai Jiangshan <jiangshan.ljs@antgroup.com> Signed-off-by:
Juergen Gross <jgross@suse.com> Link: https://lore.kernel.org/r/20220503032107.680190-6-jiangshanlai@gmail.com Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit c64cc280 Author: Lai Jiangshan <jiangshan.ljs@antgroup.com> Date: Thu, 21 Apr 2022 22:10:51 +0800 x86/entry: Move CLD to the start of the idtentry macro Move it after CLAC. Suggested-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit ee774dac Author: Lai Jiangshan <jiangshan.ljs@antgroup.com> Date: Thu, 21 Apr 2022 22:10:50 +0800 x86/entry: Move PUSH_AND_CLEAR_REGS out of error_entry() The macro idtentry() (through idtentry_body()) calls error_entry() unconditionally even on XENPV. But XENPV needs to only push and clear regs. PUSH_AND_CLEAR_REGS in error_entry() makes the stack not return to its original place when the function returns, which means it is not possible to convert it to a C function. Carve out PUSH_AND_CLEAR_REGS out of error_entry() and into a separate function and call it before error_entry() in order to avoid calling error_entry() on XENPV. It will also allow for error_entry() to be converted to C code that can use inlined sync_regs() and save a function call. [ bp: Massage commit message. ] Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 520a7e80 Author: Lai Jiangshan <jiangshan.ljs@antgroup.com> Date: Thu, 21 Apr 2022 22:10:49 +0800 x86/entry: Switch the stack after error_entry() returns error_entry() calls fixup_bad_iret() before sync_regs() if it is a fault from a bad IRET, to copy pt_regs to the kernel stack. It switches to the kernel stack directly after sync_regs(). But error_entry() itself is also a function call, so it has to stash the address it is going to return to, in %r12 which is unnecessarily complicated. Move the stack switching after error_entry() and get rid of the need to handle the return address. [ bp: Massage commit message. ] Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 0aca53c6 Author: Lai Jiangshan <jiangshan.ljs@antgroup.com> Date: Thu, 21 Apr 2022 22:10:48 +0800 x86/traps: Use pt_regs directly in fixup_bad_iret() Always stash the address error_entry() is going to return to, in %r12 and get rid of the void *error_entry_ret; slot in struct bad_iret_stack which was supposed to account for it and pt_regs pushed on the stack. After this, both fixup_bad_iret() and sync_regs() can work on a struct pt_regs pointer directly. [ bp: Rewrite commit message, touch ups. ] Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 1c0513de Author: Josh Poimboeuf <jpoimboe@kernel.org> Date: Mon, 18 Apr 2022 09:50:23 -0700 x86/retpoline: Add ANNOTATE_NOENDBR for retpolines The retpolines are exported, so they're referenced by ksymtab sections. But they're never indirect-branched to, so add ANNOTATE_NOENDBR. Fixes: ed53a0d9 ("x86/alternative: Use .ibt_endbr_seal to seal indirect calls") Signed-off-by:
Josh Poimboeuf <jpoimboe@redhat.com> Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 613871cd Author: Josh Poimboeuf <jpoimboe@kernel.org> Date: Mon, 18 Apr 2022 09:50:22 -0700 x86/static_call: Add ANNOTATE_NOENDBR to static call trampoline The static call trampoline is never indirect-branched to, but is referenced by the static call key. Add ANNOTATE_NOENDBR. Fixes: ed53a0d9 ("x86/alternative: Use .ibt_endbr_seal to seal indirect calls") Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 7a53f408 Author: Peter Zijlstra <peterz@infradead.org> Date: Wed, 23 Mar 2022 23:35:01 +0100 objtool: Fix SLS validation for kcov tail-call replacement Since not all compilers have a function attribute to disable KCOV instrumentation, objtool can rewrite KCOV instrumentation in noinstr functions as per commit: f56dae88 ("objtool: Handle __sanitize_cov*() tail calls") However, this has subtle interaction with the SLS validation from commit: 1cc1e4c8 ("objtool: Add straight-line-speculation validation") In that when a tail-call instrucion is replaced with a RET an additional INT3 instruction is also written, but is not represented in the decoded instruction stream. This then leads to false positive missing INT3 objtool warnings in noinstr code. Instead of adding additional struct instruction objects, mark the RET instruction with retpoline_safe to suppress the warning (since we know there really is an INT3). Fixes: 1cc1e4c8 ("objtool: Add straight-line-speculation validation") Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 1cd5f059 Author: Peter Zijlstra <peterz@infradead.org> Date: Fri, 18 Mar 2022 21:24:38 +0100 x86,static_call: Fix __static_call_return0 for i386 Paolo reported that the instruction sequence that is used to replace: call __static_call_return0 namely: 66 66 48 31 c0 data16 data16 xor %rax,%rax decodes to something else on i386, namely: 66 66 48 data16 dec %ax 31 c0 xor %eax,%eax Which is a nonsensical sequence that happens to have the same outcome. *However* an important distinction is that it consists of 2 instructions which is a problem when the thing needs to be overwriten with a regular call instruction again. As such, replace the instruction with something that decodes the same on both i386 and x86_64. Fixes: 3f2a8fc4 ("static_call/x86: Add __static_call_return0()") Reported-by:
Paolo Bonzini <pbonzini@redhat.com> Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 7ed7aa4d Author: Peter Zijlstra <peterz@infradead.org> Date: Thu, 24 Mar 2022 00:05:55 +0100 crypto: x86/poly1305 - Fixup SLS Due to being a perl generated asm file, it got missed by the mass convertion script. arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_init_x86_64()+0x3a: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_x86_64()+0xf2: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_emit_x86_64()+0x37: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: __poly1305_block()+0x6d: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: __poly1305_init_avx()+0x1e8: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx()+0x18a: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx()+0xaf8: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_emit_avx()+0x99: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx2()+0x18a: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx2()+0x776: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx512()+0x18a: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx512()+0x796: missing int3 after ret arch/x86/crypto/poly1305-x86_64-cryptogams.o: warning: objtool: poly1305_blocks_avx512()+0x10bd: missing int3 after ret Fixes: f94909ce ("x86: Prepare asm files for straight-line-speculation") Signed-off-by:
Herbert Xu <herbert@gondor.apana.org.au> Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 3986f65d Author: Peter Zijlstra <peterz@infradead.org> Date: Mon, 21 Mar 2022 10:13:12 +0100 kvm/emulate: Fix SETcc emulation for ENDBR Companion patch for commit fe83f5ea ("kvm/emulate: Fix SETcc emulation function offsets with SLS"), now extending it to cover the additional ENDBR instruction. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 3e3f0695 Author: Peter Zijlstra <peterz@infradead.org> Date: Tue, 8 Mar 2022 16:30:40 +0100 x86/ibt: Annotate text references Annotate away some of the generic code references. This is things where we take the address of a symbol for exception handling or return addresses (eg. context switch). Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 99c95c5d Author: Peter Zijlstra <peterz@infradead.org> Date: Tue, 8 Mar 2022 16:30:37 +0100 x86/alternative: Simplify int3_selftest_ip Similar to ibt_selftest_ip, apply the same pattern. Suggested-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 Conflicts: A fuzz due to the presence of a later uptream commit fe83f5ea ("Fix SETcc emulation function offsets with SLS"). commit 6649fa87 Author: Peter Zijlstra <peterz@infradead.org> Date: Tue, 8 Mar 2022 16:30:28 +0100 x86/ibt,kvm: Add ENDBR to fastops Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 8f93402b Author: Peter Zijlstra <peterz@infradead.org> Date: Tue, 8 Mar 2022 16:30:24 +0100 x86/ibt,entry: Sprinkle ENDBR dust Kernel entry points should be having ENDBR on for IBT configs. The SYSCALL entry points are found through taking their respective address in order to program them in the MSRs, while the exception entry points are found through UNWIND_HINT_IRET_REGS. The rule is that any UNWIND_HINT_IRET_REGS at sym+0 should have an ENDBR, see the later objtool ibt validation patch. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 5b2fc515 Author: Peter Zijlstra <peterz@infradead.org> Date: Tue, 8 Mar 2022 16:30:23 +0100 x86/ibt,xen: Sprinkle the ENDBR Even though Xen currently doesn't advertise IBT, prepare for when it will eventually do so and sprinkle the ENDBR dust accordingly. Even though most of the entry points are IRET like, the CPL0 Hypervisor can set WAIT-FOR-ENDBR and demand ENDBR at these sites. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 6cf3e4c0 Author: Peter Zijlstra <peterz@infradead.org> Date: Tue, 8 Mar 2022 16:30:21 +0100 x86/entry: Cleanup PARAVIRT Since commit 5c8f6a2e ("x86/xen: Add xenpv_restore_regs_and_return_to_usermode()") Xen will no longer reach this code and we can do away with the paravirt SWAPGS/INTERRUPT_RETURN. Suggested-by:
Andrew Cooper <Andrew.Cooper3@citrix.com> Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit c8c301ab Author: Peter Zijlstra <peterz@infradead.org> Date: Tue, 8 Mar 2022 16:30:18 +0100 x86/ibt: Add ANNOTATE_NOENDBR In order to have objtool warn about code references to !ENDBR instruction, we need an annotation to allow this for non-control-flow instances -- consider text range checks, text patching, or return trampolines etc. Signed-off-by:
Kees Cook <keescook@chromium.org> Acked-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 Upstream Status: RHEL-only As we are not backporting the complete set of X86_KERNEL_IBT patches, we can't enable it at the moment. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 156ff4a5 Author: Peter Zijlstra <peterz@infradead.org> Date: Tue, 8 Mar 2022 16:30:17 +0100 x86/ibt: Base IBT bits Add Kconfig, Makefile and basic instruction support for x86 IBT. (Ab)use __DISABLE_EXPORTS to disable IBT since it's already employed to mark compressed and purgatory. Additionally mark realmode with it as well to avoid inserting ENDBR instructions there. While ENDBR is technically a NOP, inserting them was causing some grief due to code growth. There's also a problem with using __noendbr in code compiled without -fcf-protection=branch. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 537da1ed Author: Peter Zijlstra <peterz@infradead.org> Date: Tue, 8 Mar 2022 16:30:15 +0100 objtool,efi: Update __efi64_thunk annotation The current annotation relies on not running objtool on the file; this won't work when running objtool on vmlinux.o. Instead explicitly mark __efi64_thunk() to be ignored. This preserves the status quo, which is somewhat unfortunate. Luckily this code is hardly ever used. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 82880283 Author: Sergei Trofimovich <slyich@gmail.com> Date: Thu, 20 Jan 2022 23:37:48 +0000 objtool: Fix truncated string warning On GCC 12, the build fails due to a possible truncated string: check.c: In function 'validate_call': check.c:2865:58: error: '%d' directive output may be truncated writing between 1 and 10 bytes into a region of size 9 [-Werror=format-truncation=] 2865 | snprintf(pvname, sizeof(pvname), "pv_ops[%d]", idx); | ^~ In theory it's a valid bug: static char pvname[16]; int idx; ... idx = (rel->addend / sizeof(void *)); snprintf(pvname, sizeof(pvname), "pv_ops[%d]", idx); There are only 7 chars for %d while it could take up to 9, so the printed "pv_ops[%d]" string could get truncated. In reality the bug should never happen, because pv_ops only has ~80 entries, so 7 chars for the integer is more than enough. Still, it's worth fixing. Bump the buffer size by 2 bytes to silence the warning. [ jpoimboe: changed size to 19; massaged changelog ] Fixes: db2b0c5d ("objtool: Support pv_opsindirect calls for noinstr") Reported-by:
Adam Borowski <kilobyte@angband.pl> Reported-by:
Martin Liška <mliska@suse.cz> Signed-off-by:
Sergei Trofimovich <slyich@gmail.com> Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 Upstream Status: RHEL-only SLS (Straight Line Speculation) can be enabled as the compiler do have support for that. However, it is default to n. With a future patch that adds and enable CONFIG_RETHUNK, it does mitigates any straight line speculation problem with the return instruction. So it is not enabled for now. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 Conflicts: Merge conflict in arch/x86/lib/retpoline.S due to the presence of a later upstream commit d45476d9 ("x86/speculation: Rename RETPOLINE_AMD to RETPOLINE_LFENCE"). commit e463a09a Author: Peter Zijlstra <peterz@infradead.org> Date: Sat, 4 Dec 2021 14:43:44 +0100 x86: Add straight-line-speculation mitigation Make use of an upcoming GCC feature to mitigate straight-line-speculation for x86: https://gcc.gnu.org/g:53a643f8568067d7700a9f2facc8ba39974973d3 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102952 https://bugs.llvm.org/show_bug.cgi?id=52323 It's built tested on x86_64-allyesconfig using GCC-12 and GCC-11. Maintenance overhead of this should be fairly low due to objtool validation. Size overhead of all these additional int3 instructions comes to: text data bss dec hex filename 22267751 6933356 2011368 31212475 1dc43bb defconfig-build/vmlinux 22804126 6933356 1470696 31208178 1dc32f2 defconfig-build/vmlinux.sls Or roughly 2.4% additional text. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 26c44b77 Author: Peter Zijlstra <peterz@infradead.org> Date: Sat, 4 Dec 2021 14:43:43 +0100 x86/alternative: Relax text_poke_bp() constraint Currently, text_poke_bp() is very strict to only allow patching a single instruction; however with straight-line-speculation it will be required to patch: ret; int3, which is two instructions. As such, relax the constraints a little to allow int3 padding for all instructions that do not imply the execution of the next instruction, ie: RET, JMP.d8 and JMP.d32. While there, rename the text_poke_loc::rel32 field to ::disp. Note: this fills up the text_poke_loc structure which is now a round 16 bytes big. [ bp: Put comments ontop instead of on the side. ] Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 1cc1e4c8 Author: Peter Zijlstra <peterz@infradead.org> Date: Sat, 4 Dec 2021 14:43:42 +0100 objtool: Add straight-line-speculation validation Teach objtool to validate the straight-line-speculation constraints: - speculation trap after indirect calls - speculation trap after RET Notable: when an instruction is annotated RETPOLINE_SAFE, indicating speculation isn't a problem, also don't care about sls for that instruction. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 Conflicts: Include an additional kvm_crash_shutdown() hunk for arch/x86/kernel/kvm.c to replace ret to ASM_RET in upstream merge commit a4cfff3f ("Merge branch 'kvm-older-features' into HEAD"). commit b17c2baa Author: Peter Zijlstra <peterz@infradead.org> Date: Sat, 4 Dec 2021 14:43:41 +0100 x86: Prepare inline-asm for straight-line-speculation Replace all ret/retq instructions with ASM_RET in preparation of making it more than a single instruction. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 Conflicts: 1) Merge conflicts in arch/x86/xen/xen-asm.S due to missing upstream commit d7bfc7d5 ("x86/xen: Make irq_enable() noinstr") and commit 20125c87 ("x86/xen: Make save_fl() noinstr"). 2) The hunks for arch/x86/crypto/sm4-aesni-avx-asm_64.S and arch/x86/crypto/sm4-aesni-avx2-asm_64.S are dropped as RHEL9 doesn't have those. 3) Fuzz in arch/x86/lib/copy_user_64.S due to missing upstream commit acba44d2 ("x86/copy_user_64: Remove .fixup usage"). 4) Fuzzes in arch/x86/kernel/ftrace_64.S and arch/x86/kernel/irqflags.S. 5) Include a hunk in arch/x86/kernel/ftrace_64.S caught by running the script in the patch description. commit f94909ce Author: Peter Zijlstra <peterz@infradead.org> Date: Sat, 4 Dec 2021 14:43:40 +0100 x86: Prepare asm files for straight-line-speculation Replace all ret/retq instructions with RET in preparation of making RET a macro. Since AS is case insensitive it's a big no-op without RET defined. find arch/x86/ -name \*.S | while read file do sed -i 's/\<ret[q]*\>/RET/' $file done Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 33761363 Author: Youquan Song <youquan.song@intel.com> Date: Thu, 23 Dec 2021 12:07:01 -0800 x86/mce: Reduce number of machine checks taken during recovery When any of the copy functions in arch/x86/lib/copy_user_64.S take a fault, the fixup code copies the remaining byte count from %ecx to %edx and unconditionally jumps to .Lcopy_user_handle_tail to continue the copy in case any more bytes can be copied. If the fault was #PF this may copy more bytes (because the page fault handler might have fixed the fault). But when the fault is a machine check the original copy code will have copied all the way to the poisoned cache line. So .Lcopy_user_handle_tail will just take another machine check for no good reason. Every code path to .Lcopy_user_handle_tail comes from an exception fixup path, so add a check there to check the trap type (in %eax) and simply return the count of remaining bytes if the trap was a machine check. Doing this reduces the number of machine checks taken during synthetic tests from four to three. As well as reducing the number of machine checks, this also allows Skylake generation Xeons to recover some cases that currently fail. The is because REP; MOVSB is only recoverable when source and destination are well aligned and the byte count is large. That useless call to .Lcopy_user_handle_tail may violate one or more of these conditions and generate a fatal machine check. [ Tony: Add more details to commit message. ] [ bp: Fixup comment. Also, another tip patchset which is adding straight-line speculation mitigation changes the "ret" instruction to an all-caps macro "RET". But, since gas is case-insensitive, use "RET" in the newly added asm block already in order to simplify tip branch merging on its way upstream. ] Signed-off-by:
Youquan Song <youquan.song@intel.com> Signed-off-by:
Tony Luck <tony.luck@intel.com> Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 22da5a07 Author: Peter Zijlstra <peterz@infradead.org> Date: Sat, 4 Dec 2021 14:43:39 +0100 x86/lib/atomic64_386_32: Rename things Principally, in order to get rid of #define RET in this code to make place for a new RET, but also to clarify the code, rename a bunch of things: s/UNLOCK/IRQ_RESTORE/ s/LOCK/IRQ_SAVE/ s/BEGIN/BEGIN_IRQ_SAVE/ s/\<RET\>/RET_IRQ_RESTORE/ s/RET_ENDP/\tRET_IRQ_RESTORE\rENDP/ which then leaves RET unused so it can be removed. Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 68cf4f2a Author: Peter Zijlstra <peterz@infradead.org> Date: Fri, 19 Nov 2021 17:50:25 +0100 x86: Use -mindirect-branch-cs-prefix for RETPOLINE builds In order to further enable commit: bbe2df3f ("x86/alternative: Try inline spectre_v2=retpoline,amd") add the new GCC flag -mindirect-branch-cs-prefix: https://gcc.gnu.org/g:2196a681d7810ad8b227bf983f38ba716620545e https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102952 https://bugs.llvm.org/show_bug.cgi?id=52323 to RETPOLINE=y builds. This should allow fully inlining retpoline,amd for GCC builds. Signed-off-by:
Nick Desaulniers <ndesaulniers@google.com> Link: https://lkml.kernel.org/r/20211119165630.276205624@infradead.org Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit b2f825bf Author: Peter Zijlstra <peterz@infradead.org> Date: Fri, 19 Nov 2021 17:50:24 +0100 x86: Move RETPOLINE*_CFLAGS to arch Makefile Currently, RETPOLINE*_CFLAGS are defined in the top-level Makefile but only x86 makes use of them. Move them there. If ever another architecture finds the need, it can be reconsidered. [ bp: Massage a bit. ] Suggested-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 Conflcits: Fuzz in arch/x86/xen/xen-asm.S due to missing upstream commit 09c41307 ("x86/xen: Make irq_disable() noinstr"). commit 5c8f6a2e Author: Lai Jiangshan <laijs@linux.alibaba.com> Date: Fri, 26 Nov 2021 18:11:23 +0800 x86/xen: Add xenpv_restore_regs_and_return_to_usermode() In the native case, PER_CPU_VAR(cpu_tss_rw + TSS_sp0) is the trampoline stack. But XEN pv doesn't use trampoline stack, so PER_CPU_VAR(cpu_tss_rw + TSS_sp0) is also the kernel stack. In that case, source and destination stacks are identical, which means that reusing swapgs_restore_regs_and_return_to_usermode() in XEN pv would cause %rsp to move up to the top of the kernel stack and leave the IRET frame below %rsp. This is dangerous as it can be corrupted if #NMI / #MC hit as either of these events occurring in the middle of the stack pushing would clobber data on the (original) stack. And, with XEN pv, swapgs_restore_regs_and_return_to_usermode() pushing the IRET frame on to the original address is useless and error-prone when there is any future attempt to modify the code. [ bp: Massage commit message. ] Fixes: 7f2590a1 ("x86/entry/64: Use a per-CPU trampoline stack for IDT entries") Signed-off-by:
Lai Jiangshan <laijs@linux.alibaba.com> Signed-off-by:
Boris Ostrovsky <boris.ostrovsky@oracle.com> Link: https://lkml.kernel.org/r/20211126101209.8613-4-jiangshanlai@gmail.com Signed-off-by:
-
Waiman Long authored
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231 commit 1367afaa Author: Lai Jiangshan <laijs@linux.alibaba.com> Date: Fri, 26 Nov 2021 18:11:22 +0800 x86/entry: Use the correct fence macro after swapgs in kernel CR3 The commit c7589070 ("x86/entry/64: Remove unneeded kernel CR3 switching") removed a CR3 write in the faulting path of load_gs_index(). But the path's FENCE_SWAPGS_USER_ENTRY has no fence operation if PTI is enabled, see spectre_v1_select_mitigation(). Rather, it depended on the serializing CR3 write of SWITCH_TO_KERNEL_CR3 and since it got removed, add a FENCE_SWAPGS_KERNEL_ENTRY call to make sure speculation is blocked. [ bp: Massage commit message and comment. ] Fixes: c7589070 ("x86/entry/64: Remove unneeded kernel CR3 switching") Signed-off-by:
-
