sugar/linux
1
0
Fork 0
Code Activity

Merge tag 'x86_bugs_for_v6.17_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Browse Source 
Pull x86 CPU mitigation updates from Borislav Petkov:

 - Untangle the Retbleed from the ITS mitigation on Intel. Allow for ITS
   to enable stuffing independently from Retbleed, do some cleanups to
   simplify and streamline the code

 - Simplify SRSO and make mitigation types selection more versatile
   depending on the Retbleed mitigation selection. Simplify code some

 - Add the second part of the attack vector controls which provide a lot
   friendlier user interface to the speculation mitigations than
   selecting each one by one as it is now.

   Instead, the selection of whole attack vectors which are relevant to
   the system in use can be done and protection against only those
   vectors is enabled, thus giving back some performance to the users

* tag 'x86_bugs_for_v6.17_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (31 commits)
  x86/bugs: Print enabled attack vectors
  x86/bugs: Add attack vector controls for TSA
  x86/pti: Add attack vector controls for PTI
  x86/bugs: Add attack vector controls for ITS
  x86/bugs: Add attack vector controls for SRSO
  x86/bugs: Add attack vector controls for L1TF
  x86/bugs: Add attack vector controls for spectre_v2
  x86/bugs: Add attack vector controls for BHI
  x86/bugs: Add attack vector controls for spectre_v2_user
  x86/bugs: Add attack vector controls for retbleed
  x86/bugs: Add attack vector controls for spectre_v1
  x86/bugs: Add attack vector controls for GDS
  x86/bugs: Add attack vector controls for SRBDS
  x86/bugs: Add attack vector controls for RFDS
  x86/bugs: Add attack vector controls for MMIO
  x86/bugs: Add attack vector controls for TAA
  x86/bugs: Add attack vector controls for MDS
  x86/bugs: Define attack vectors relevant for each bug
  x86/Kconfig: Add arch attack vector support
  cpu: Define attack vectors
  ...
This commit is contained in:
Linus Torvalds
2025-07-29 16:34:45 -07:00
parent d7223aed30 a026dc61cf
commit 04d29e3609
9 changed files with 713 additions and 154 deletions
Download Patch File Download Diff File Expand all files Collapse all files

238
Documentation/admin-guide/hw-vuln/attack_vector_controls.rst Normal file
View File

@@ -0,0 +1,238 @@
.. SPDX-License-Identifier: GPL-2.0
Attack Vector Controls
======================
Attack vector controls provide a simple method to configure only the mitigations
for CPU vulnerabilities which are relevant given the intended use of a system.
Administrators are encouraged to consider which attack vectors are relevant and
disable all others in order to recoup system performance.
When new relevant CPU vulnerabilities are found, they will be added to these
attack vector controls so administrators will likely not need to reconfigure
their command line parameters as mitigations will continue to be correctly
applied based on the chosen attack vector controls.
Attack Vectors
--------------
There are 5 sets of attack-vector mitigations currently supported by the kernel:
#. :ref:`user_kernel`
#. :ref:`user_user`
#. :ref:`guest_host`
#. :ref:`guest_guest`
#. :ref:`smt`
To control the enabled attack vectors, see :ref:`cmdline`.
.. _user_kernel:
User-to-Kernel
^^^^^^^^^^^^^^
The user-to-kernel attack vector involves a malicious userspace program
attempting to leak kernel data into userspace by exploiting a CPU vulnerability.
The kernel data involved might be limited to certain kernel memory, or include
all memory in the system, depending on the vulnerability exploited.
If no untrusted userspace applications are being run, such as with single-user
systems, consider disabling user-to-kernel mitigations.
Note that the CPU vulnerabilities mitigated by Linux have generally not been
shown to be exploitable from browser-based sandboxes. User-to-kernel
mitigations are therefore mostly relevant if unknown userspace applications may
be run by untrusted users.
*user-to-kernel mitigations are enabled by default*
.. _user_user:
User-to-User
^^^^^^^^^^^^
The user-to-user attack vector involves a malicious userspace program attempting
to influence the behavior of another unsuspecting userspace program in order to
exfiltrate data. The vulnerability of a userspace program is based on the
program itself and the interfaces it provides.
If no untrusted userspace applications are being run, consider disabling
user-to-user mitigations.
Note that because the Linux kernel contains a mapping of all physical memory,
preventing a malicious userspace program from leaking data from another
userspace program requires mitigating user-to-kernel attacks as well for
complete protection.
*user-to-user mitigations are enabled by default*
.. _guest_host:
Guest-to-Host
^^^^^^^^^^^^^
The guest-to-host attack vector involves a malicious VM attempting to leak
hypervisor data into the VM. The data involved may be limited, or may
potentially include all memory in the system, depending on the vulnerability
exploited.
If no untrusted VMs are being run, consider disabling guest-to-host mitigations.
*guest-to-host mitigations are enabled by default if KVM support is present*
.. _guest_guest:
Guest-to-Guest
^^^^^^^^^^^^^^
The guest-to-guest attack vector involves a malicious VM attempting to influence
the behavior of another unsuspecting VM in order to exfiltrate data. The
vulnerability of a VM is based on the code inside the VM itself and the
interfaces it provides.
If no untrusted VMs, or only a single VM is being run, consider disabling
guest-to-guest mitigations.
Similar to the user-to-user attack vector, preventing a malicious VM from
leaking data from another VM requires mitigating guest-to-host attacks as well
due to the Linux kernel phys map.
*guest-to-guest mitigations are enabled by default if KVM support is present*
.. _smt:
Cross-Thread
^^^^^^^^^^^^
The cross-thread attack vector involves a malicious userspace program or
malicious VM either observing or attempting to influence the behavior of code
running on the SMT sibling thread in order to exfiltrate data.
Many cross-thread attacks can only be mitigated if SMT is disabled, which will
result in reduced CPU core count and reduced performance.
If cross-thread mitigations are fully enabled ('auto,nosmt'), all mitigations
for cross-thread attacks will be enabled. SMT may be disabled depending on
which vulnerabilities are present in the CPU.
If cross-thread mitigations are partially enabled ('auto'), mitigations for
cross-thread attacks will be enabled but SMT will not be disabled.
If cross-thread mitigations are disabled, no mitigations for cross-thread
attacks will be enabled.
Cross-thread mitigation may not be required if core-scheduling or similar
techniques are used to prevent untrusted workloads from running on SMT siblings.
*cross-thread mitigations default to partially enabled*
.. _cmdline:
Command Line Controls
---------------------
Attack vectors are controlled through the mitigations= command line option. The
value provided begins with a global option and then may optionally include one
or more options to disable various attack vectors.
Format:
| ``mitigations=[global]``
| ``mitigations=[global],[attack vectors]``
Global options:
============ =============================================================
Option Description
============ =============================================================
'off' All attack vectors disabled.
'auto' All attack vectors enabled, partial cross-thread mitigations.
'auto,nosmt' All attack vectors enabled, full cross-thread mitigations.
============ =============================================================
Attack vector options:
================= =======================================
Option Description
================= =======================================
'no_user_kernel' Disables user-to-kernel mitigations.
'no_user_user' Disables user-to-user mitigations.
'no_guest_host' Disables guest-to-host mitigations.
'no_guest_guest' Disables guest-to-guest mitigations
'no_cross_thread' Disables all cross-thread mitigations.
================= =======================================
Multiple attack vector options may be specified in a comma-separated list. If
the global option is not specified, it defaults to 'auto'. The global option
'off' is equivalent to disabling all attack vectors.
Examples:
| ``mitigations=auto,no_user_kernel``
Enable all attack vectors except user-to-kernel. Partial cross-thread
mitigations.
| ``mitigations=auto,nosmt,no_guest_host,no_guest_guest``
Enable all attack vectors and cross-thread mitigations except for
guest-to-host and guest-to-guest mitigations.
| ``mitigations=,no_cross_thread``
Enable all attack vectors but not cross-thread mitigations.
Interactions with command-line options
--------------------------------------
Vulnerability-specific controls (e.g. "retbleed=off") take precedence over all
attack vector controls. Mitigations for individual vulnerabilities may be
turned on or off via their command-line options regardless of the attack vector
controls.
Summary of attack-vector mitigations
------------------------------------
When a vulnerability is mitigated due to an attack-vector control, the default
mitigation option for that particular vulnerability is used. To use a different
mitigation, please use the vulnerability-specific command line option.
The table below summarizes which vulnerabilities are mitigated when different
attack vectors are enabled and assuming the CPU is vulnerable.
=============== ============== ============ ============= ============== ============ ========
Vulnerability User-to-Kernel User-to-User Guest-to-Host Guest-to-Guest Cross-Thread Notes
=============== ============== ============ ============= ============== ============ ========
BHI X X
ITS X X
GDS X X X X * (Note 1)
L1TF X X * (Note 2)
MDS X X X X * (Note 2)
MMIO X X X X * (Note 2)
Meltdown X
Retbleed X X * (Note 3)
RFDS X X X X
Spectre_v1 X
Spectre_v2 X X
Spectre_v2_user X X * (Note 1)
SRBDS X X X X
SRSO X X
SSB (Note 4)
TAA X X X X * (Note 2)
TSA X X X X
=============== ============== ============ ============= ============== ============ ========
Notes:
1 -- Can be mitigated without disabling SMT.
2 -- Disables SMT if cross-thread mitigations are fully enabled and the CPU
is vulnerable
3 -- Disables SMT if cross-thread mitigations are fully enabled, the CPU is
vulnerable, and STIBP is not supported
4 -- Speculative store bypass is always enabled by default (no kernel
mitigation applied) unless overridden with spec_store_bypass_disable option
When an attack-vector is disabled, all mitigations for the vulnerabilities
listed in the above table are disabled, unless mitigation is required for a
different enabled attack-vector or a mitigation is explicitly selected via a
vulnerability-specific command line option.

1
Documentation/admin-guide/hw-vuln/index.rst
View File

@@ -9,6 +9,7 @@ are configurable at compile, boot or run time.
.. toctree::
:maxdepth: 1
attack_vector_controls
spectre
l1tf
mds

4
Documentation/admin-guide/kernel-parameters.txt
View File

@@ -3797,6 +3797,10 @@
mmio_stale_data=full,nosmt [X86]
retbleed=auto,nosmt [X86]
[X86] After one of the above options, additionally
supports attack-vector based controls as documented in
Documentation/admin-guide/hw-vuln/attack_vector_controls.rst
mminit_loglevel=
[KNL,EARLY] When CONFIG_DEBUG_MEMORY_INIT is set, this
parameter allows control of the logging verbosity for

3
arch/Kconfig
View File

@@ -1772,4 +1772,7 @@ config ARCH_WANTS_PRE_LINK_VMLINUX
An architecture can select this if it provides arch/<arch>/tools/Makefile
with .arch.vmlinux.o target to be linked into vmlinux.
config ARCH_HAS_CPU_ATTACK_VECTORS
bool
endmenu

1
arch/x86/Kconfig
View File

@@ -75,6 +75,7 @@ config X86
select ARCH_ENABLE_SPLIT_PMD_PTLOCK if (PGTABLE_LEVELS > 2) && (X86_64 || X86_PAE)
select ARCH_ENABLE_THP_MIGRATION if X86_64 && TRANSPARENT_HUGEPAGE
select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI
select ARCH_HAS_CPU_ATTACK_VECTORS if CPU_MITIGATIONS
select ARCH_HAS_CACHE_LINE_SIZE
select ARCH_HAS_CPU_CACHE_INVALIDATE_MEMREGION
select ARCH_HAS_CPU_FINALIZE_INIT

465
arch/x86/kernel/cpu/bugs.c
View File

@@ -115,10 +115,9 @@ void (*x86_return_thunk)(void) __ro_after_init = __x86_return_thunk;
static void __init set_return_thunk(void *thunk)
{
if (x86_return_thunk != __x86_return_thunk)
pr_warn("x86/bugs: return thunk changed\n");
x86_return_thunk = thunk;
pr_info("active return thunk: %ps\n", thunk);
}
/* Update SPEC_CTRL MSR and its cached copy unconditionally */
@@ -190,6 +189,39 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush);
DEFINE_STATIC_KEY_FALSE(cpu_buf_vm_clear);
EXPORT_SYMBOL_GPL(cpu_buf_vm_clear);
#undef pr_fmt
#define pr_fmt(fmt) "mitigations: " fmt
static void __init cpu_print_attack_vectors(void)
{
pr_info("Enabled attack vectors: ");
if (cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL))
pr_cont("user_kernel, ");
if (cpu_attack_vector_mitigated(CPU_MITIGATE_USER_USER))
pr_cont("user_user, ");
if (cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST))
pr_cont("guest_host, ");
if (cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_GUEST))
pr_cont("guest_guest, ");
pr_cont("SMT mitigations: ");
switch (smt_mitigations) {
case SMT_MITIGATIONS_OFF:
pr_cont("off\n");
break;
case SMT_MITIGATIONS_AUTO:
pr_cont("auto\n");
break;
case SMT_MITIGATIONS_ON:
pr_cont("on\n");
}
}
void __init cpu_select_mitigations(void)
{
/*
@@ -210,6 +242,8 @@ void __init cpu_select_mitigations(void)
x86_arch_cap_msr = x86_read_arch_cap_msr();
cpu_print_attack_vectors();
/* Select the proper CPU mitigations before patching alternatives: */
spectre_v1_select_mitigation();
spectre_v2_select_mitigation();
@@ -333,6 +367,62 @@ static void x86_amd_ssb_disable(void)
#undef pr_fmt
#define pr_fmt(fmt) "MDS: " fmt
/*
* Returns true if vulnerability should be mitigated based on the
* selected attack vector controls.
*
* See Documentation/admin-guide/hw-vuln/attack_vector_controls.rst
*/
static bool __init should_mitigate_vuln(unsigned int bug)
{
switch (bug) {
/*
* The only runtime-selected spectre_v1 mitigations in the kernel are
* related to SWAPGS protection on kernel entry. Therefore, protection
* is only required for the user->kernel attack vector.
*/
case X86_BUG_SPECTRE_V1:
return cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL);
case X86_BUG_SPECTRE_V2:
case X86_BUG_RETBLEED:
case X86_BUG_SRSO:
case X86_BUG_L1TF:
case X86_BUG_ITS:
return cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL) ||
cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST);
case X86_BUG_SPECTRE_V2_USER:
return cpu_attack_vector_mitigated(CPU_MITIGATE_USER_USER) ||
cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_GUEST);
/*
* All the vulnerabilities below allow potentially leaking data
* across address spaces. Therefore, mitigation is required for
* any of these 4 attack vectors.
*/
case X86_BUG_MDS:
case X86_BUG_TAA:
case X86_BUG_MMIO_STALE_DATA:
case X86_BUG_RFDS:
case X86_BUG_SRBDS:
return cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL) ||
cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST) ||
cpu_attack_vector_mitigated(CPU_MITIGATE_USER_USER) ||
cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_GUEST);
case X86_BUG_GDS:
return cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL) ||
cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST) ||
cpu_attack_vector_mitigated(CPU_MITIGATE_USER_USER) ||
cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_GUEST) ||
(smt_mitigations != SMT_MITIGATIONS_OFF);
default:
WARN(1, "Unknown bug %x\n", bug);
return false;
}
}
/* Default mitigation for MDS-affected CPUs */
static enum mds_mitigations mds_mitigation __ro_after_init =
IS_ENABLED(CONFIG_MITIGATION_MDS) ? MDS_MITIGATION_AUTO : MDS_MITIGATION_OFF;
@@ -386,13 +476,17 @@ static bool verw_clear_cpu_buf_mitigation_selected __ro_after_init;
static void __init mds_select_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_MDS) || cpu_mitigations_off()) {
if (!boot_cpu_has_bug(X86_BUG_MDS)) {
mds_mitigation = MDS_MITIGATION_OFF;
return;
}
if (mds_mitigation == MDS_MITIGATION_AUTO)
mds_mitigation = MDS_MITIGATION_FULL;
if (mds_mitigation == MDS_MITIGATION_AUTO) {
if (should_mitigate_vuln(X86_BUG_MDS))
mds_mitigation = MDS_MITIGATION_FULL;
else
mds_mitigation = MDS_MITIGATION_OFF;
}
if (mds_mitigation == MDS_MITIGATION_OFF)
return;
@@ -402,7 +496,7 @@ static void __init mds_select_mitigation(void)
static void __init mds_update_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_MDS) || cpu_mitigations_off())
if (!boot_cpu_has_bug(X86_BUG_MDS))
return;
/* If TAA, MMIO, or RFDS are being mitigated, MDS gets mitigated too. */
@@ -423,7 +517,7 @@ static void __init mds_apply_mitigation(void)
mds_mitigation == MDS_MITIGATION_VMWERV) {
setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
if (!boot_cpu_has(X86_BUG_MSBDS_ONLY) &&
(mds_nosmt || cpu_mitigations_auto_nosmt()))
(mds_nosmt || smt_mitigations == SMT_MITIGATIONS_ON))
cpu_smt_disable(false);
}
}
@@ -479,12 +573,13 @@ static void __init taa_select_mitigation(void)
return;
}
if (cpu_mitigations_off())
taa_mitigation = TAA_MITIGATION_OFF;
/* Microcode will be checked in taa_update_mitigation(). */
if (taa_mitigation == TAA_MITIGATION_AUTO)
taa_mitigation = TAA_MITIGATION_VERW;
if (taa_mitigation == TAA_MITIGATION_AUTO) {
if (should_mitigate_vuln(X86_BUG_TAA))
taa_mitigation = TAA_MITIGATION_VERW;
else
taa_mitigation = TAA_MITIGATION_OFF;
}
if (taa_mitigation != TAA_MITIGATION_OFF)
verw_clear_cpu_buf_mitigation_selected = true;
@@ -492,7 +587,7 @@ static void __init taa_select_mitigation(void)
static void __init taa_update_mitigation(void)
{
if (!taa_vulnerable() || cpu_mitigations_off())
if (!taa_vulnerable())
return;
if (verw_clear_cpu_buf_mitigation_selected)
@@ -533,7 +628,7 @@ static void __init taa_apply_mitigation(void)
*/
setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
if (taa_nosmt || cpu_mitigations_auto_nosmt())
if (taa_nosmt || smt_mitigations == SMT_MITIGATIONS_ON)
cpu_smt_disable(false);
}
}
@@ -579,8 +674,12 @@ static void __init mmio_select_mitigation(void)
}
/* Microcode will be checked in mmio_update_mitigation(). */
if (mmio_mitigation == MMIO_MITIGATION_AUTO)
mmio_mitigation = MMIO_MITIGATION_VERW;
if (mmio_mitigation == MMIO_MITIGATION_AUTO) {
if (should_mitigate_vuln(X86_BUG_MMIO_STALE_DATA))
mmio_mitigation = MMIO_MITIGATION_VERW;
else
mmio_mitigation = MMIO_MITIGATION_OFF;
}
if (mmio_mitigation == MMIO_MITIGATION_OFF)
return;
@@ -595,7 +694,7 @@ static void __init mmio_select_mitigation(void)
static void __init mmio_update_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA) || cpu_mitigations_off())
if (!boot_cpu_has_bug(X86_BUG_MMIO_STALE_DATA))
return;
if (verw_clear_cpu_buf_mitigation_selected)
@@ -643,7 +742,7 @@ static void __init mmio_apply_mitigation(void)
if (!(x86_arch_cap_msr & ARCH_CAP_FBSDP_NO))
static_branch_enable(&cpu_buf_idle_clear);
if (mmio_nosmt || cpu_mitigations_auto_nosmt())
if (mmio_nosmt || smt_mitigations == SMT_MITIGATIONS_ON)
cpu_smt_disable(false);
}
@@ -684,13 +783,17 @@ static inline bool __init verw_clears_cpu_reg_file(void)
static void __init rfds_select_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_RFDS) || cpu_mitigations_off()) {
if (!boot_cpu_has_bug(X86_BUG_RFDS)) {
rfds_mitigation = RFDS_MITIGATION_OFF;
return;
}
if (rfds_mitigation == RFDS_MITIGATION_AUTO)
rfds_mitigation = RFDS_MITIGATION_VERW;
if (rfds_mitigation == RFDS_MITIGATION_AUTO) {
if (should_mitigate_vuln(X86_BUG_RFDS))
rfds_mitigation = RFDS_MITIGATION_VERW;
else
rfds_mitigation = RFDS_MITIGATION_OFF;
}
if (rfds_mitigation == RFDS_MITIGATION_OFF)
return;
@@ -701,7 +804,7 @@ static void __init rfds_select_mitigation(void)
static void __init rfds_update_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_RFDS) || cpu_mitigations_off())
if (!boot_cpu_has_bug(X86_BUG_RFDS))
return;
if (verw_clear_cpu_buf_mitigation_selected)
@@ -802,13 +905,19 @@ void update_srbds_msr(void)
static void __init srbds_select_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_SRBDS) || cpu_mitigations_off()) {
if (!boot_cpu_has_bug(X86_BUG_SRBDS)) {
srbds_mitigation = SRBDS_MITIGATION_OFF;
return;
}
if (srbds_mitigation == SRBDS_MITIGATION_AUTO)
srbds_mitigation = SRBDS_MITIGATION_FULL;
if (srbds_mitigation == SRBDS_MITIGATION_AUTO) {
if (should_mitigate_vuln(X86_BUG_SRBDS))
srbds_mitigation = SRBDS_MITIGATION_FULL;
else {
srbds_mitigation = SRBDS_MITIGATION_OFF;
return;
}
}
/*
* Check to see if this is one of the MDS_NO systems supporting TSX that
@@ -956,12 +1065,15 @@ static void __init gds_select_mitigation(void)
return;
}
if (cpu_mitigations_off())
gds_mitigation = GDS_MITIGATION_OFF;
/* Will verify below that mitigation _can_ be disabled */
if (gds_mitigation == GDS_MITIGATION_AUTO)
gds_mitigation = GDS_MITIGATION_FULL;
if (gds_mitigation == GDS_MITIGATION_AUTO) {
if (should_mitigate_vuln(X86_BUG_GDS))
gds_mitigation = GDS_MITIGATION_FULL;
else {
gds_mitigation = GDS_MITIGATION_OFF;
return;
}
}
/* No microcode */
if (!(x86_arch_cap_msr & ARCH_CAP_GDS_CTRL)) {
@@ -1067,13 +1179,16 @@ static bool smap_works_speculatively(void)
static void __init spectre_v1_select_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V1) || cpu_mitigations_off())
if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V1))
spectre_v1_mitigation = SPECTRE_V1_MITIGATION_NONE;
if (!should_mitigate_vuln(X86_BUG_SPECTRE_V1))
spectre_v1_mitigation = SPECTRE_V1_MITIGATION_NONE;
}
static void __init spectre_v1_apply_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V1) || cpu_mitigations_off())
if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V1))
return;
if (spectre_v1_mitigation == SPECTRE_V1_MITIGATION_AUTO) {
@@ -1124,6 +1239,20 @@ early_param("nospectre_v1", nospectre_v1_cmdline);
enum spectre_v2_mitigation spectre_v2_enabled __ro_after_init = SPECTRE_V2_NONE;
/* Depends on spectre_v2 mitigation selected already */
static inline bool cdt_possible(enum spectre_v2_mitigation mode)
{
if (!IS_ENABLED(CONFIG_MITIGATION_CALL_DEPTH_TRACKING) ||
!IS_ENABLED(CONFIG_MITIGATION_RETPOLINE))
return false;
if (mode == SPECTRE_V2_RETPOLINE ||
mode == SPECTRE_V2_EIBRS_RETPOLINE)
return true;
return false;
}
#undef pr_fmt
#define pr_fmt(fmt) "RETBleed: " fmt
@@ -1162,6 +1291,21 @@ static enum retbleed_mitigation retbleed_mitigation __ro_after_init =
static int __ro_after_init retbleed_nosmt = false;
enum srso_mitigation {
SRSO_MITIGATION_NONE,
SRSO_MITIGATION_AUTO,
SRSO_MITIGATION_UCODE_NEEDED,
SRSO_MITIGATION_SAFE_RET_UCODE_NEEDED,
SRSO_MITIGATION_MICROCODE,
SRSO_MITIGATION_NOSMT,
SRSO_MITIGATION_SAFE_RET,
SRSO_MITIGATION_IBPB,
SRSO_MITIGATION_IBPB_ON_VMEXIT,
SRSO_MITIGATION_BP_SPEC_REDUCE,
};
static enum srso_mitigation srso_mitigation __ro_after_init = SRSO_MITIGATION_AUTO;
static int __init retbleed_parse_cmdline(char *str)
{
if (!str)
@@ -1204,7 +1348,7 @@ early_param("retbleed", retbleed_parse_cmdline);
static void __init retbleed_select_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_RETBLEED) || cpu_mitigations_off()) {
if (!boot_cpu_has_bug(X86_BUG_RETBLEED)) {
retbleed_mitigation = RETBLEED_MITIGATION_NONE;
return;
}
@@ -1241,6 +1385,11 @@ static void __init retbleed_select_mitigation(void)
if (retbleed_mitigation != RETBLEED_MITIGATION_AUTO)
return;
if (!should_mitigate_vuln(X86_BUG_RETBLEED)) {
retbleed_mitigation = RETBLEED_MITIGATION_NONE;
return;
}
/* Intel mitigation selected in retbleed_update_mitigation() */
if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD ||
boot_cpu_data.x86_vendor == X86_VENDOR_HYGON) {
@@ -1251,35 +1400,36 @@ static void __init retbleed_select_mitigation(void)
retbleed_mitigation = RETBLEED_MITIGATION_IBPB;
else
retbleed_mitigation = RETBLEED_MITIGATION_NONE;
} else if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) {
/* Final mitigation depends on spectre-v2 selection */
if (boot_cpu_has(X86_FEATURE_IBRS_ENHANCED))
retbleed_mitigation = RETBLEED_MITIGATION_EIBRS;
else if (boot_cpu_has(X86_FEATURE_IBRS))
retbleed_mitigation = RETBLEED_MITIGATION_IBRS;
else
retbleed_mitigation = RETBLEED_MITIGATION_NONE;
}
}
static void __init retbleed_update_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_RETBLEED) || cpu_mitigations_off())
if (!boot_cpu_has_bug(X86_BUG_RETBLEED))
return;
if (retbleed_mitigation == RETBLEED_MITIGATION_NONE)
goto out;
/* ITS can also enable stuffing */
if (its_mitigation == ITS_MITIGATION_RETPOLINE_STUFF)
retbleed_mitigation = RETBLEED_MITIGATION_STUFF;
/*
* retbleed=stuff is only allowed on Intel. If stuffing can't be used
* then a different mitigation will be selected below.
*
* its=stuff will also attempt to enable stuffing.
*/
if (retbleed_mitigation == RETBLEED_MITIGATION_STUFF ||
its_mitigation == ITS_MITIGATION_RETPOLINE_STUFF) {
if (spectre_v2_enabled != SPECTRE_V2_RETPOLINE) {
pr_err("WARNING: retbleed=stuff depends on spectre_v2=retpoline\n");
retbleed_mitigation = RETBLEED_MITIGATION_AUTO;
} else {
if (retbleed_mitigation != RETBLEED_MITIGATION_STUFF)
pr_info("Retbleed mitigation updated to stuffing\n");
/* If SRSO is using IBPB, that works for retbleed too */
if (srso_mitigation == SRSO_MITIGATION_IBPB)
retbleed_mitigation = RETBLEED_MITIGATION_IBPB;
retbleed_mitigation = RETBLEED_MITIGATION_STUFF;
}
if (retbleed_mitigation == RETBLEED_MITIGATION_STUFF &&
!cdt_possible(spectre_v2_enabled)) {
pr_err("WARNING: retbleed=stuff depends on retpoline\n");
retbleed_mitigation = RETBLEED_MITIGATION_NONE;
}
/*
* Let IBRS trump all on Intel without affecting the effects of the
* retbleed= cmdline option except for call depth based stuffing
@@ -1298,15 +1448,11 @@ static void __init retbleed_update_mitigation(void)
if (retbleed_mitigation != RETBLEED_MITIGATION_STUFF)
pr_err(RETBLEED_INTEL_MSG);
}
/* If nothing has set the mitigation yet, default to NONE. */
if (retbleed_mitigation == RETBLEED_MITIGATION_AUTO)
retbleed_mitigation = RETBLEED_MITIGATION_NONE;
}
out:
pr_info("%s\n", retbleed_strings[retbleed_mitigation]);
}
static void __init retbleed_apply_mitigation(void)
{
bool mitigate_smt = false;
@@ -1362,7 +1508,7 @@ static void __init retbleed_apply_mitigation(void)
}
if (mitigate_smt && !boot_cpu_has(X86_FEATURE_STIBP) &&
(retbleed_nosmt || cpu_mitigations_auto_nosmt()))
(retbleed_nosmt || smt_mitigations == SMT_MITIGATIONS_ON))
cpu_smt_disable(false);
}
@@ -1407,13 +1553,17 @@ early_param("indirect_target_selection", its_parse_cmdline);
static void __init its_select_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_ITS) || cpu_mitigations_off()) {
if (!boot_cpu_has_bug(X86_BUG_ITS)) {
its_mitigation = ITS_MITIGATION_OFF;
return;
}
if (its_mitigation == ITS_MITIGATION_AUTO)
its_mitigation = ITS_MITIGATION_ALIGNED_THUNKS;
if (its_mitigation == ITS_MITIGATION_AUTO) {
if (should_mitigate_vuln(X86_BUG_ITS))
its_mitigation = ITS_MITIGATION_ALIGNED_THUNKS;
else
its_mitigation = ITS_MITIGATION_OFF;
}
if (its_mitigation == ITS_MITIGATION_OFF)
return;
@@ -1444,15 +1594,17 @@ static void __init its_select_mitigation(void)
static void __init its_update_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_ITS) || cpu_mitigations_off())
if (!boot_cpu_has_bug(X86_BUG_ITS))
return;
switch (spectre_v2_enabled) {
case SPECTRE_V2_NONE:
pr_err("WARNING: Spectre-v2 mitigation is off, disabling ITS\n");
if (its_mitigation != ITS_MITIGATION_OFF)
pr_err("WARNING: Spectre-v2 mitigation is off, disabling ITS\n");
its_mitigation = ITS_MITIGATION_OFF;
break;
case SPECTRE_V2_RETPOLINE:
case SPECTRE_V2_EIBRS_RETPOLINE:
/* Retpoline+CDT mitigates ITS */
if (retbleed_mitigation == RETBLEED_MITIGATION_STUFF)
its_mitigation = ITS_MITIGATION_RETPOLINE_STUFF;
@@ -1466,13 +1618,8 @@ static void __init its_update_mitigation(void)
break;
}
/*
* retbleed_update_mitigation() will try to do stuffing if its=stuff.
* If it can't, such as if spectre_v2!=retpoline, then fall back to
* aligned thunks.
*/
if (its_mitigation == ITS_MITIGATION_RETPOLINE_STUFF &&
retbleed_mitigation != RETBLEED_MITIGATION_STUFF)
!cdt_possible(spectre_v2_enabled))
its_mitigation = ITS_MITIGATION_ALIGNED_THUNKS;
pr_info("%s\n", its_strings[its_mitigation]);
@@ -1480,15 +1627,24 @@ static void __init its_update_mitigation(void)
static void __init its_apply_mitigation(void)
{
/* its=stuff forces retbleed stuffing and is enabled there. */
if (its_mitigation != ITS_MITIGATION_ALIGNED_THUNKS)
return;
switch (its_mitigation) {
case ITS_MITIGATION_OFF:
case ITS_MITIGATION_AUTO:
case ITS_MITIGATION_VMEXIT_ONLY:
break;
case ITS_MITIGATION_ALIGNED_THUNKS:
if (!boot_cpu_has(X86_FEATURE_RETPOLINE))
setup_force_cpu_cap(X86_FEATURE_INDIRECT_THUNK_ITS);
if (!boot_cpu_has(X86_FEATURE_RETPOLINE))
setup_force_cpu_cap(X86_FEATURE_INDIRECT_THUNK_ITS);
setup_force_cpu_cap(X86_FEATURE_RETHUNK);
set_return_thunk(its_return_thunk);
setup_force_cpu_cap(X86_FEATURE_RETHUNK);
set_return_thunk(its_return_thunk);
break;
case ITS_MITIGATION_RETPOLINE_STUFF:
setup_force_cpu_cap(X86_FEATURE_RETHUNK);
setup_force_cpu_cap(X86_FEATURE_CALL_DEPTH);
set_return_thunk(call_depth_return_thunk);
break;
}
}
#undef pr_fmt
@@ -1536,28 +1692,43 @@ early_param("tsa", tsa_parse_cmdline);
static void __init tsa_select_mitigation(void)
{
if (cpu_mitigations_off() || !boot_cpu_has_bug(X86_BUG_TSA)) {
if (!boot_cpu_has_bug(X86_BUG_TSA)) {
tsa_mitigation = TSA_MITIGATION_NONE;
return;
}
if (tsa_mitigation == TSA_MITIGATION_AUTO) {
bool vm = false, uk = false;
tsa_mitigation = TSA_MITIGATION_NONE;
if (cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL) ||
cpu_attack_vector_mitigated(CPU_MITIGATE_USER_USER)) {
tsa_mitigation = TSA_MITIGATION_USER_KERNEL;
uk = true;
}
if (cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST) ||
cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_GUEST)) {
tsa_mitigation = TSA_MITIGATION_VM;
vm = true;
}
if (uk && vm)
tsa_mitigation = TSA_MITIGATION_FULL;
}
if (tsa_mitigation == TSA_MITIGATION_NONE)
return;
if (!boot_cpu_has(X86_FEATURE_VERW_CLEAR)) {
if (!boot_cpu_has(X86_FEATURE_VERW_CLEAR))
tsa_mitigation = TSA_MITIGATION_UCODE_NEEDED;
goto out;
}
if (tsa_mitigation == TSA_MITIGATION_AUTO)
tsa_mitigation = TSA_MITIGATION_FULL;
/*
* No need to set verw_clear_cpu_buf_mitigation_selected - it
* doesn't fit all cases here and it is not needed because this
* is the only VERW-based mitigation on AMD.
*/
out:
pr_info("%s\n", tsa_strings[tsa_mitigation]);
}
@@ -1701,7 +1872,7 @@ static enum spectre_v2_user_cmd __init spectre_v2_parse_user_cmdline(void)
char arg[20];
int ret, i;
if (cpu_mitigations_off() || !IS_ENABLED(CONFIG_MITIGATION_SPECTRE_V2))
if (!IS_ENABLED(CONFIG_MITIGATION_SPECTRE_V2))
return SPECTRE_V2_USER_CMD_NONE;
ret = cmdline_find_option(boot_command_line, "spectre_v2_user",
@@ -1739,6 +1910,13 @@ static void __init spectre_v2_user_select_mitigation(void)
spectre_v2_user_stibp = SPECTRE_V2_USER_STRICT;
break;
case SPECTRE_V2_USER_CMD_AUTO:
if (!should_mitigate_vuln(X86_BUG_SPECTRE_V2_USER))
break;
spectre_v2_user_ibpb = SPECTRE_V2_USER_PRCTL;
if (smt_mitigations == SMT_MITIGATIONS_OFF)
break;
spectre_v2_user_stibp = SPECTRE_V2_USER_PRCTL;
break;
case SPECTRE_V2_USER_CMD_PRCTL:
spectre_v2_user_ibpb = SPECTRE_V2_USER_PRCTL;
spectre_v2_user_stibp = SPECTRE_V2_USER_PRCTL;
@@ -1890,8 +2068,7 @@ static enum spectre_v2_mitigation_cmd __init spectre_v2_parse_cmdline(void)
int ret, i;
cmd = IS_ENABLED(CONFIG_MITIGATION_SPECTRE_V2) ? SPECTRE_V2_CMD_AUTO : SPECTRE_V2_CMD_NONE;
if (cmdline_find_option_bool(boot_command_line, "nospectre_v2") ||
cpu_mitigations_off())
if (cmdline_find_option_bool(boot_command_line, "nospectre_v2"))
return SPECTRE_V2_CMD_NONE;
ret = cmdline_find_option(boot_command_line, "spectre_v2", arg, sizeof(arg));
@@ -2094,11 +2271,20 @@ early_param("spectre_bhi", spectre_bhi_parse_cmdline);
static void __init bhi_select_mitigation(void)
{
if (!boot_cpu_has(X86_BUG_BHI) || cpu_mitigations_off())
if (!boot_cpu_has(X86_BUG_BHI))
bhi_mitigation = BHI_MITIGATION_OFF;
if (bhi_mitigation == BHI_MITIGATION_AUTO)
bhi_mitigation = BHI_MITIGATION_ON;
if (bhi_mitigation != BHI_MITIGATION_AUTO)
return;
if (cpu_attack_vector_mitigated(CPU_MITIGATE_GUEST_HOST)) {
if (cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL))
bhi_mitigation = BHI_MITIGATION_ON;
else
bhi_mitigation = BHI_MITIGATION_VMEXIT_ONLY;
} else {
bhi_mitigation = BHI_MITIGATION_OFF;
}
}
static void __init bhi_update_mitigation(void)
@@ -2154,8 +2340,11 @@ static void __init spectre_v2_select_mitigation(void)
case SPECTRE_V2_CMD_NONE:
return;
case SPECTRE_V2_CMD_FORCE:
case SPECTRE_V2_CMD_AUTO:
if (!should_mitigate_vuln(X86_BUG_SPECTRE_V2))
break;
fallthrough;
case SPECTRE_V2_CMD_FORCE:
if (boot_cpu_has(X86_FEATURE_IBRS_ENHANCED)) {
spectre_v2_enabled = SPECTRE_V2_EIBRS;
break;
@@ -2209,7 +2398,7 @@ static void __init spectre_v2_update_mitigation(void)
}
}
if (boot_cpu_has_bug(X86_BUG_SPECTRE_V2) && !cpu_mitigations_off())
if (boot_cpu_has_bug(X86_BUG_SPECTRE_V2))
pr_info("%s\n", spectre_v2_strings[spectre_v2_enabled]);
}
@@ -2861,17 +3050,23 @@ static void override_cache_bits(struct cpuinfo_x86 *c)
static void __init l1tf_select_mitigation(void)
{
if (!boot_cpu_has_bug(X86_BUG_L1TF) || cpu_mitigations_off()) {
if (!boot_cpu_has_bug(X86_BUG_L1TF)) {
l1tf_mitigation = L1TF_MITIGATION_OFF;
return;
}
if (l1tf_mitigation == L1TF_MITIGATION_AUTO) {
if (cpu_mitigations_auto_nosmt())
l1tf_mitigation = L1TF_MITIGATION_FLUSH_NOSMT;
else
l1tf_mitigation = L1TF_MITIGATION_FLUSH;
if (l1tf_mitigation != L1TF_MITIGATION_AUTO)
return;
if (!should_mitigate_vuln(X86_BUG_L1TF)) {
l1tf_mitigation = L1TF_MITIGATION_OFF;
return;
}
if (smt_mitigations == SMT_MITIGATIONS_ON)
l1tf_mitigation = L1TF_MITIGATION_FLUSH_NOSMT;
else
l1tf_mitigation = L1TF_MITIGATION_FLUSH;
}
static void __init l1tf_apply_mitigation(void)
@@ -2945,31 +3140,18 @@ early_param("l1tf", l1tf_cmdline);
#undef pr_fmt
#define pr_fmt(fmt) "Speculative Return Stack Overflow: " fmt
enum srso_mitigation {
SRSO_MITIGATION_NONE,
SRSO_MITIGATION_AUTO,
SRSO_MITIGATION_UCODE_NEEDED,
SRSO_MITIGATION_SAFE_RET_UCODE_NEEDED,
SRSO_MITIGATION_MICROCODE,
SRSO_MITIGATION_SAFE_RET,
SRSO_MITIGATION_IBPB,
SRSO_MITIGATION_IBPB_ON_VMEXIT,
SRSO_MITIGATION_BP_SPEC_REDUCE,
};
static const char * const srso_strings[] = {
[SRSO_MITIGATION_NONE] = "Vulnerable",
[SRSO_MITIGATION_UCODE_NEEDED] = "Vulnerable: No microcode",
[SRSO_MITIGATION_SAFE_RET_UCODE_NEEDED] = "Vulnerable: Safe RET, no microcode",
[SRSO_MITIGATION_MICROCODE] = "Vulnerable: Microcode, no safe RET",
[SRSO_MITIGATION_NOSMT] = "Mitigation: SMT disabled",
[SRSO_MITIGATION_SAFE_RET] = "Mitigation: Safe RET",
[SRSO_MITIGATION_IBPB] = "Mitigation: IBPB",
[SRSO_MITIGATION_IBPB_ON_VMEXIT] = "Mitigation: IBPB on VMEXIT only",
[SRSO_MITIGATION_BP_SPEC_REDUCE] = "Mitigation: Reduced Speculation"
};
static enum srso_mitigation srso_mitigation __ro_after_init = SRSO_MITIGATION_AUTO;
static int __init srso_parse_cmdline(char *str)
{
if (!str)
@@ -2996,35 +3178,44 @@ early_param("spec_rstack_overflow", srso_parse_cmdline);
static void __init srso_select_mitigation(void)
{
bool has_microcode;
if (!boot_cpu_has_bug(X86_BUG_SRSO) || cpu_mitigations_off())
if (!boot_cpu_has_bug(X86_BUG_SRSO)) {
srso_mitigation = SRSO_MITIGATION_NONE;
if (srso_mitigation == SRSO_MITIGATION_NONE)
return;
}
if (srso_mitigation == SRSO_MITIGATION_AUTO)
srso_mitigation = SRSO_MITIGATION_SAFE_RET;
has_microcode = boot_cpu_has(X86_FEATURE_IBPB_BRTYPE);
if (has_microcode) {
/*
* Zen1/2 with SMT off aren't vulnerable after the right
* IBPB microcode has been applied.
*/
if (boot_cpu_data.x86 < 0x19 && !cpu_smt_possible()) {
setup_force_cpu_cap(X86_FEATURE_SRSO_NO);
if (srso_mitigation == SRSO_MITIGATION_AUTO) {
if (should_mitigate_vuln(X86_BUG_SRSO)) {
srso_mitigation = SRSO_MITIGATION_SAFE_RET;
} else {
srso_mitigation = SRSO_MITIGATION_NONE;
return;
}
} else {
}
/* Zen1/2 with SMT off aren't vulnerable to SRSO. */
if (boot_cpu_data.x86 < 0x19 && !cpu_smt_possible()) {
srso_mitigation = SRSO_MITIGATION_NOSMT;
return;
}
if (!boot_cpu_has(X86_FEATURE_IBPB_BRTYPE)) {
pr_warn("IBPB-extending microcode not applied!\n");
pr_warn(SRSO_NOTICE);
/*
* Safe-RET provides partial mitigation without microcode, but
* other mitigations require microcode to provide any
* mitigations.
*/
if (srso_mitigation == SRSO_MITIGATION_SAFE_RET)
srso_mitigation = SRSO_MITIGATION_SAFE_RET_UCODE_NEEDED;
else
srso_mitigation = SRSO_MITIGATION_UCODE_NEEDED;
}
switch (srso_mitigation) {
case SRSO_MITIGATION_SAFE_RET:
case SRSO_MITIGATION_SAFE_RET_UCODE_NEEDED:
if (boot_cpu_has(X86_FEATURE_SRSO_USER_KERNEL_NO)) {
srso_mitigation = SRSO_MITIGATION_IBPB_ON_VMEXIT;
goto ibpb_on_vmexit;
@@ -3034,9 +3225,6 @@ static void __init srso_select_mitigation(void)
pr_err("WARNING: kernel not compiled with MITIGATION_SRSO.\n");
srso_mitigation = SRSO_MITIGATION_NONE;
}
if (!has_microcode)
srso_mitigation = SRSO_MITIGATION_SAFE_RET_UCODE_NEEDED;
break;
ibpb_on_vmexit:
case SRSO_MITIGATION_IBPB_ON_VMEXIT:
@@ -3051,9 +3239,6 @@ ibpb_on_vmexit:
pr_err("WARNING: kernel not compiled with MITIGATION_IBPB_ENTRY.\n");
srso_mitigation = SRSO_MITIGATION_NONE;
}
if (!has_microcode)
srso_mitigation = SRSO_MITIGATION_UCODE_NEEDED;
break;
default:
break;
@@ -3068,8 +3253,7 @@ static void __init srso_update_mitigation(void)
srso_mitigation = SRSO_MITIGATION_IBPB;
if (boot_cpu_has_bug(X86_BUG_SRSO) &&
!cpu_mitigations_off() &&
!boot_cpu_has(X86_FEATURE_SRSO_NO))
!cpu_mitigations_off())
pr_info("%s\n", srso_strings[srso_mitigation]);
}
@@ -3365,9 +3549,6 @@ static ssize_t retbleed_show_state(char *buf)
static ssize_t srso_show_state(char *buf)
{
if (boot_cpu_has(X86_FEATURE_SRSO_NO))
return sysfs_emit(buf, "Mitigation: SMT disabled\n");
return sysfs_emit(buf, "%s\n", srso_strings[srso_mitigation]);
}

4
arch/x86/mm/pti.c
View File

@@ -38,6 +38,7 @@
#include <asm/desc.h>
#include <asm/sections.h>
#include <asm/set_memory.h>
#include <asm/bugs.h>
#undef pr_fmt
#define pr_fmt(fmt) "Kernel/User page tables isolation: " fmt
@@ -84,7 +85,8 @@ void __init pti_check_boottime_disable(void)
return;
}
if (cpu_mitigations_off())
if (pti_mode == PTI_AUTO &&
!cpu_attack_vector_mitigated(CPU_MITIGATE_USER_KERNEL))
pti_mode = PTI_FORCE_OFF;
if (pti_mode == PTI_FORCE_OFF) {
pti_print_if_insecure("disabled on command line.");

21
include/linux/cpu.h
View File

@@ -198,9 +198,25 @@ void cpuhp_report_idle_dead(void);
static inline void cpuhp_report_idle_dead(void) { }
#endif /* #ifdef CONFIG_HOTPLUG_CPU */
enum cpu_attack_vectors {
CPU_MITIGATE_USER_KERNEL,
CPU_MITIGATE_USER_USER,
CPU_MITIGATE_GUEST_HOST,
CPU_MITIGATE_GUEST_GUEST,
NR_CPU_ATTACK_VECTORS,
};
enum smt_mitigations {
SMT_MITIGATIONS_OFF,
SMT_MITIGATIONS_AUTO,
SMT_MITIGATIONS_ON,
};
#ifdef CONFIG_CPU_MITIGATIONS
extern bool cpu_mitigations_off(void);
extern bool cpu_mitigations_auto_nosmt(void);
extern bool cpu_attack_vector_mitigated(enum cpu_attack_vectors v);
extern enum smt_mitigations smt_mitigations;
#else
static inline bool cpu_mitigations_off(void)
{
@@ -210,6 +226,11 @@ static inline bool cpu_mitigations_auto_nosmt(void)
{
return false;
}
static inline bool cpu_attack_vector_mitigated(enum cpu_attack_vectors v)
{
return false;
}
#define smt_mitigations SMT_MITIGATIONS_OFF
#endif
#endif /* _LINUX_CPU_H_ */

130
kernel/cpu.c
View File

@@ -37,6 +37,7 @@
#include <linux/cpuset.h>
#include <linux/random.h>
#include <linux/cc_platform.h>
#include <linux/parser.h>
#include <trace/events/power.h>
#define CREATE_TRACE_POINTS
@@ -3174,8 +3175,38 @@ void __init boot_cpu_hotplug_init(void)
#ifdef CONFIG_CPU_MITIGATIONS
/*
* These are used for a global "mitigations=" cmdline option for toggling
* optional CPU mitigations.
* All except the cross-thread attack vector are mitigated by default.
* Cross-thread mitigation often requires disabling SMT which is expensive
* so cross-thread mitigations are only partially enabled by default.
*
* Guest-to-Host and Guest-to-Guest vectors are only needed if KVM support is
* present.
*/
static bool attack_vectors[NR_CPU_ATTACK_VECTORS] __ro_after_init = {
[CPU_MITIGATE_USER_KERNEL] = true,
[CPU_MITIGATE_USER_USER] = true,
[CPU_MITIGATE_GUEST_HOST] = IS_ENABLED(CONFIG_KVM),
[CPU_MITIGATE_GUEST_GUEST] = IS_ENABLED(CONFIG_KVM),
};
bool cpu_attack_vector_mitigated(enum cpu_attack_vectors v)
{
if (v < NR_CPU_ATTACK_VECTORS)
return attack_vectors[v];
WARN_ONCE(1, "Invalid attack vector %d\n", v);
return false;
}
/*
* There are 3 global options, 'off', 'auto', 'auto,nosmt'. These may optionally
* be combined with attack-vector disables which follow them.
*
* Examples:
* mitigations=auto,no_user_kernel,no_user_user,no_cross_thread
* mitigations=auto,nosmt,no_guest_host,no_guest_guest
*
* mitigations=off is equivalent to disabling all attack vectors.
*/
enum cpu_mitigations {
CPU_MITIGATIONS_OFF,
@@ -3183,19 +3214,96 @@ enum cpu_mitigations {
CPU_MITIGATIONS_AUTO_NOSMT,
};
enum {
NO_USER_KERNEL,
NO_USER_USER,
NO_GUEST_HOST,
NO_GUEST_GUEST,
NO_CROSS_THREAD,
NR_VECTOR_PARAMS,
};
enum smt_mitigations smt_mitigations __ro_after_init = SMT_MITIGATIONS_AUTO;
static enum cpu_mitigations cpu_mitigations __ro_after_init = CPU_MITIGATIONS_AUTO;
static const match_table_t global_mitigations = {
{ CPU_MITIGATIONS_AUTO_NOSMT, "auto,nosmt"},
{ CPU_MITIGATIONS_AUTO, "auto"},
{ CPU_MITIGATIONS_OFF, "off"},
};
static const match_table_t vector_mitigations = {
{ NO_USER_KERNEL, "no_user_kernel"},
{ NO_USER_USER, "no_user_user"},
{ NO_GUEST_HOST, "no_guest_host"},
{ NO_GUEST_GUEST, "no_guest_guest"},
{ NO_CROSS_THREAD, "no_cross_thread"},
{ NR_VECTOR_PARAMS, NULL},
};
static int __init mitigations_parse_global_opt(char *arg)
{
int i;
for (i = 0; i < ARRAY_SIZE(global_mitigations); i++) {
const char *pattern = global_mitigations[i].pattern;
if (!strncmp(arg, pattern, strlen(pattern))) {
cpu_mitigations = global_mitigations[i].token;
return strlen(pattern);
}
}
return 0;
}
static int __init mitigations_parse_cmdline(char *arg)
{
if (!strcmp(arg, "off"))
cpu_mitigations = CPU_MITIGATIONS_OFF;
else if (!strcmp(arg, "auto"))
cpu_mitigations = CPU_MITIGATIONS_AUTO;
else if (!strcmp(arg, "auto,nosmt"))
cpu_mitigations = CPU_MITIGATIONS_AUTO_NOSMT;
else
pr_crit("Unsupported mitigations=%s, system may still be vulnerable\n",
arg);
char *s, *p;
int len;
len = mitigations_parse_global_opt(arg);
if (cpu_mitigations_off()) {
memset(attack_vectors, 0, sizeof(attack_vectors));
smt_mitigations = SMT_MITIGATIONS_OFF;
} else if (cpu_mitigations_auto_nosmt()) {
smt_mitigations = SMT_MITIGATIONS_ON;
}
p = arg + len;
if (!*p)
return 0;
/* Attack vector controls may come after the ',' */
if (*p++ != ',' || !IS_ENABLED(CONFIG_ARCH_HAS_CPU_ATTACK_VECTORS)) {
pr_crit("Unsupported mitigations=%s, system may still be vulnerable\n", arg);
return 0;
}
while ((s = strsep(&p, ",")) != NULL) {
switch (match_token(s, vector_mitigations, NULL)) {
case NO_USER_KERNEL:
attack_vectors[CPU_MITIGATE_USER_KERNEL] = false;
break;
case NO_USER_USER:
attack_vectors[CPU_MITIGATE_USER_USER] = false;
break;
case NO_GUEST_HOST:
attack_vectors[CPU_MITIGATE_GUEST_HOST] = false;
break;
case NO_GUEST_GUEST:
attack_vectors[CPU_MITIGATE_GUEST_GUEST] = false;
break;
case NO_CROSS_THREAD:
smt_mitigations = SMT_MITIGATIONS_OFF;
break;
default:
pr_crit("Unsupported mitigations options %s\n", s);
return 0;
}
}
return 0;
}