| 1 | Recommendations for KVM CPU model configuration on x86 hosts |
| 2 | ============================================================ |
| 3 | |
| 4 | The information that follows provides recommendations for configuring |
| 5 | CPU models on x86 hosts. The goals are to maximise performance, while |
| 6 | protecting guest OS against various CPU hardware flaws, and optionally |
| 7 | enabling live migration between hosts with heterogeneous CPU models. |
| 8 | |
| 9 | |
| 10 | Two ways to configure CPU models with QEMU / KVM |
| 11 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 12 | |
| 13 | (1) **Host passthrough** |
| 14 | |
| 15 | This passes the host CPU model features, model, stepping, exactly to |
| 16 | the guest. Note that KVM may filter out some host CPU model features |
| 17 | if they cannot be supported with virtualization. Live migration is |
| 18 | unsafe when this mode is used as libvirt / QEMU cannot guarantee a |
| 19 | stable CPU is exposed to the guest across hosts. This is the |
| 20 | recommended CPU to use, provided live migration is not required. |
| 21 | |
| 22 | (2) **Named model** |
| 23 | |
| 24 | QEMU comes with a number of predefined named CPU models, that |
| 25 | typically refer to specific generations of hardware released by |
| 26 | Intel and AMD. These allow the guest VMs to have a degree of |
| 27 | isolation from the host CPU, allowing greater flexibility in live |
| 28 | migrating between hosts with differing hardware. @end table |
| 29 | |
| 30 | In both cases, it is possible to optionally add or remove individual CPU |
| 31 | features, to alter what is presented to the guest by default. |
| 32 | |
| 33 | Libvirt supports a third way to configure CPU models known as "Host |
| 34 | model". This uses the QEMU "Named model" feature, automatically picking |
| 35 | a CPU model that is similar the host CPU, and then adding extra features |
| 36 | to approximate the host model as closely as possible. This does not |
| 37 | guarantee the CPU family, stepping, etc will precisely match the host |
| 38 | CPU, as they would with "Host passthrough", but gives much of the |
| 39 | benefit of passthrough, while making live migration safe. |
| 40 | |
| 41 | |
| 42 | ABI compatibility levels for CPU models |
| 43 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 44 | |
| 45 | The x86_64 architecture has a number of `ABI compatibility levels`_ |
| 46 | defined. Traditionally most operating systems and toolchains would |
| 47 | only target the original baseline ABI. It is expected that in |
| 48 | future OS and toolchains are likely to target newer ABIs. The |
| 49 | table that follows illustrates which ABI compatibility levels |
| 50 | can be satisfied by the QEMU CPU models. Note that the table only |
| 51 | lists the long term stable CPU model versions (eg Haswell-v4). |
| 52 | In addition to what is listed, there are also many CPU model |
| 53 | aliases which resolve to a different CPU model version, |
| 54 | depending on the machine type is in use. |
| 55 | |
| 56 | .. _ABI compatibility levels: https://gitlab.com/x86-psABIs/x86-64-ABI/ |
| 57 | |
| 58 | .. csv-table:: x86-64 ABI compatibility levels |
| 59 | :file: cpu-models-x86-abi.csv |
| 60 | :widths: 40,15,15,15,15 |
| 61 | :header-rows: 1 |
| 62 | |
| 63 | |
| 64 | Preferred CPU models for Intel x86 hosts |
| 65 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 66 | |
| 67 | The following CPU models are preferred for use on Intel hosts. |
| 68 | Administrators / applications are recommended to use the CPU model that |
| 69 | matches the generation of the host CPUs in use. In a deployment with a |
| 70 | mixture of host CPU models between machines, if live migration |
| 71 | compatibility is required, use the newest CPU model that is compatible |
| 72 | across all desired hosts. |
| 73 | |
| 74 | ``DiamondRapids`` |
| 75 | Intel Xeon Processor (DiamondRapids, 2026) |
| 76 | |
| 77 | This does not include SMT but allows the module (dual core module |
| 78 | - DCM) and die (core building block - CBB) topology levels. The |
| 79 | cache hierarchy is L1 i/d cache per thread, L2 cache per module, |
| 80 | and L3 cache per die, which can be emulated using the smp-cache |
| 81 | option: |
| 82 | |
| 83 | :: |
| 84 | |
| 85 | -machine smp-cache.0.cache=l1d,smp-cache.0.topology=thread,\ |
| 86 | smp-cache.1.cache=l1i,smp-cache.1.topology=thread,\ |
| 87 | smp-cache.2.cache=l2,smp-cache.2.topology=module,\ |
| 88 | smp-cache.3.cache=l3,smp-cache.3.topology=die |
| 89 | |
| 90 | ``ClearwaterForest`` |
| 91 | Intel Xeon Processor (ClearwaterForest, 2025) |
| 92 | |
| 93 | ``SierraForest``, ``SierraForest-v2`` |
| 94 | Intel Xeon Processor (SierraForest, 2024), SierraForest-v2 mitigates |
| 95 | the GDS and RFDS vulnerabilities with stepping 3. |
| 96 | |
| 97 | ``GraniteRapids``, ``GraniteRapids-v2`` |
| 98 | Intel Xeon Processor (GraniteRapids, 2024) |
| 99 | |
| 100 | ``Cascadelake-Server``, ``Cascadelake-Server-noTSX`` |
| 101 | Intel Xeon Processor (Cascade Lake, 2019), with "stepping" levels 6 |
| 102 | or 7 only. (The Cascade Lake Xeon processor with *stepping 5 is |
| 103 | vulnerable to MDS variants*.) |
| 104 | |
| 105 | ``Skylake-Server``, ``Skylake-Server-IBRS``, ``Skylake-Server-IBRS-noTSX`` |
| 106 | Intel Xeon Processor (Skylake, 2016) |
| 107 | |
| 108 | ``Skylake-Client``, ``Skylake-Client-IBRS``, ``Skylake-Client-noTSX-IBRS}`` |
| 109 | Intel Core Processor (Skylake, 2015) |
| 110 | |
| 111 | ``Broadwell``, ``Broadwell-IBRS``, ``Broadwell-noTSX``, ``Broadwell-noTSX-IBRS`` |
| 112 | Intel Core Processor (Broadwell, 2014) |
| 113 | |
| 114 | ``Haswell``, ``Haswell-IBRS``, ``Haswell-noTSX``, ``Haswell-noTSX-IBRS`` |
| 115 | Intel Core Processor (Haswell, 2013) |
| 116 | |
| 117 | ``IvyBridge``, ``IvyBridge-IBR`` |
| 118 | Intel Xeon E3-12xx v2 (Ivy Bridge, 2012) |
| 119 | |
| 120 | ``SandyBridge``, ``SandyBridge-IBRS`` |
| 121 | Intel Xeon E312xx (Sandy Bridge, 2011) |
| 122 | |
| 123 | ``Westmere``, ``Westmere-IBRS`` |
| 124 | Westmere E56xx/L56xx/X56xx (Nehalem-C, 2010) |
| 125 | |
| 126 | ``Nehalem``, ``Nehalem-IBRS`` |
| 127 | Intel Core i7 9xx (Nehalem Class Core i7, 2008) |
| 128 | |
| 129 | ``Penryn`` |
| 130 | Intel Core 2 Duo P9xxx (Penryn Class Core 2, 2007) |
| 131 | |
| 132 | ``Conroe`` |
| 133 | Intel Celeron_4x0 (Conroe/Merom Class Core 2, 2006) |
| 134 | |
| 135 | |
| 136 | Important CPU features for Intel x86 hosts |
| 137 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 138 | |
| 139 | The following are important CPU features that should be used on Intel |
| 140 | x86 hosts, when available in the host CPU. Some of them require explicit |
| 141 | configuration to enable, as they are not included by default in some, or |
| 142 | all, of the named CPU models listed above. In general all of these |
| 143 | features are included if using "Host passthrough" or "Host model". |
| 144 | |
| 145 | ``pcid`` |
| 146 | Recommended to mitigate the cost of the Meltdown (CVE-2017-5754) fix. |
| 147 | |
| 148 | Included by default in Haswell, Broadwell & Skylake Intel CPU models. |
| 149 | |
| 150 | Should be explicitly turned on for Westmere, SandyBridge, and |
| 151 | IvyBridge Intel CPU models. Note that some desktop/mobile Westmere |
| 152 | CPUs cannot support this feature. |
| 153 | |
| 154 | ``spec-ctrl`` |
| 155 | Required to enable the Spectre v2 (CVE-2017-5715) fix. |
| 156 | |
| 157 | Included by default in Intel CPU models with -IBRS suffix. |
| 158 | |
| 159 | Must be explicitly turned on for Intel CPU models without -IBRS |
| 160 | suffix. |
| 161 | |
| 162 | Requires the host CPU microcode to support this feature before it |
| 163 | can be used for guest CPUs. |
| 164 | |
| 165 | ``stibp`` |
| 166 | Required to enable stronger Spectre v2 (CVE-2017-5715) fixes in some |
| 167 | operating systems. |
| 168 | |
| 169 | Must be explicitly turned on for all Intel CPU models. |
| 170 | |
| 171 | Requires the host CPU microcode to support this feature before it can |
| 172 | be used for guest CPUs. |
| 173 | |
| 174 | ``ssbd`` |
| 175 | Required to enable the CVE-2018-3639 fix. |
| 176 | |
| 177 | Not included by default in any Intel CPU model. |
| 178 | |
| 179 | Must be explicitly turned on for all Intel CPU models. |
| 180 | |
| 181 | Requires the host CPU microcode to support this feature before it |
| 182 | can be used for guest CPUs. |
| 183 | |
| 184 | ``pdpe1gb`` |
| 185 | Recommended to allow guest OS to use 1GB size pages. |
| 186 | |
| 187 | Not included by default in any Intel CPU model. |
| 188 | |
| 189 | Should be explicitly turned on for all Intel CPU models. |
| 190 | |
| 191 | Note that not all CPU hardware will support this feature. |
| 192 | |
| 193 | ``md-clear`` |
| 194 | Required to confirm the MDS (CVE-2018-12126, CVE-2018-12127, |
| 195 | CVE-2018-12130, CVE-2019-11091) fixes. |
| 196 | |
| 197 | Not included by default in any Intel CPU model. |
| 198 | |
| 199 | Must be explicitly turned on for all Intel CPU models. |
| 200 | |
| 201 | Requires the host CPU microcode to support this feature before it |
| 202 | can be used for guest CPUs. |
| 203 | |
| 204 | ``mds-no`` |
| 205 | Recommended to inform the guest OS that the host is *not* vulnerable |
| 206 | to any of the MDS variants ([MFBDS] CVE-2018-12130, [MLPDS] |
| 207 | CVE-2018-12127, [MSBDS] CVE-2018-12126). |
| 208 | |
| 209 | This is an MSR (Model-Specific Register) feature rather than a CPUID feature, |
| 210 | therefore it will not appear in the Linux ``/proc/cpuinfo`` in the host or |
| 211 | guest. Instead, the host kernel uses it to populate the MDS |
| 212 | vulnerability file in ``sysfs``. |
| 213 | |
| 214 | So it should only be enabled for VMs if the host reports @code{Not |
| 215 | affected} in the ``/sys/devices/system/cpu/vulnerabilities/mds`` file. |
| 216 | |
| 217 | ``taa-no`` |
| 218 | Recommended to inform the guest that the host is ``not`` |
| 219 | vulnerable to CVE-2019-11135, TSX Asynchronous Abort (TAA). |
| 220 | |
| 221 | This is also an MSR feature, therefore it does not show up in the Linux |
| 222 | ``/proc/cpuinfo`` in the host or guest. |
| 223 | |
| 224 | It should only be enabled for VMs if the host reports ``Not affected`` |
| 225 | in the ``/sys/devices/system/cpu/vulnerabilities/tsx_async_abort`` |
| 226 | file. |
| 227 | |
| 228 | ``tsx-ctrl`` |
| 229 | Recommended to inform the guest that it can disable the Intel TSX |
| 230 | (Transactional Synchronization Extensions) feature; or, if the |
| 231 | processor is vulnerable, use the Intel VERW instruction (a |
| 232 | processor-level instruction that performs checks on memory access) as |
| 233 | a mitigation for the TAA vulnerability. (For details, refer to |
| 234 | Intel's `deep dive into MDS |
| 235 | <https://software.intel.com/security-software-guidance/insights/deep-dive-intel-analysis-microarchitectural-data-sampling>`_.) |
| 236 | |
| 237 | Expose this to the guest OS if and only if: (a) the host has TSX |
| 238 | enabled; *and* (b) the guest has ``rtm`` CPU flag enabled. |
| 239 | |
| 240 | By disabling TSX, KVM-based guests can avoid paying the price of |
| 241 | mitigating TSX-based attacks. |
| 242 | |
| 243 | Note that ``tsx-ctrl`` is also an MSR feature, therefore it does not show |
| 244 | up in the Linux ``/proc/cpuinfo`` in the host or guest. |
| 245 | |
| 246 | To validate that Intel TSX is indeed disabled for the guest, there are |
| 247 | two ways: (a) check for the *absence* of ``rtm`` in the guest's |
| 248 | ``/proc/cpuinfo``; or (b) the |
| 249 | ``/sys/devices/system/cpu/vulnerabilities/tsx_async_abort`` file in |
| 250 | the guest should report ``Mitigation: TSX disabled``. |
| 251 | |
| 252 | ``bhi-no`` |
| 253 | Recommended to inform the guest that the host is ``not`` |
| 254 | vulnerable to CVE-2022-0001, Branch History Injection (BHI). |
| 255 | |
| 256 | This is also an MSR feature, therefore it does not show up in the Linux |
| 257 | ``/proc/cpuinfo`` in the host or guest. |
| 258 | |
| 259 | It should only be enabled for VMs if the host reports |
| 260 | ``BHI: Not affected`` in the |
| 261 | ``/sys/devices/system/cpu/vulnerabilities/spectre_v2`` file. |
| 262 | |
| 263 | ``gds-no`` |
| 264 | Recommended to inform the guest that the host is ``not`` |
| 265 | vulnerable to CVE-2022-40982, Gather Data Sampling (GDS). |
| 266 | |
| 267 | This is also an MSR feature, therefore it does not show up in the Linux |
| 268 | ``/proc/cpuinfo`` in the host or guest. |
| 269 | |
| 270 | It should only be enabled for VMs if the host reports ``Not affected`` |
| 271 | in the ``/sys/devices/system/cpu/vulnerabilities/gather_data_sampling`` |
| 272 | file. |
| 273 | |
| 274 | ``rfds-no`` |
| 275 | Recommended to inform the guest that the host is ``not`` |
| 276 | vulnerable to CVE-2023-28746, Register File Data Sampling (RFDS). |
| 277 | |
| 278 | This is also an MSR feature, therefore it does not show up in the Linux |
| 279 | ``/proc/cpuinfo`` in the host or guest. |
| 280 | |
| 281 | It should only be enabled for VMs if the host reports ``Not affected`` |
| 282 | in the ``/sys/devices/system/cpu/vulnerabilities/reg_file_data_sampling`` |
| 283 | file. |
| 284 | |
| 285 | Preferred CPU models for AMD x86 hosts |
| 286 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 287 | |
| 288 | The following CPU models are preferred for use on AMD hosts. |
| 289 | Administrators / applications are recommended to use the CPU model that |
| 290 | matches the generation of the host CPUs in use. In a deployment with a |
| 291 | mixture of host CPU models between machines, if live migration |
| 292 | compatibility is required, use the newest CPU model that is compatible |
| 293 | across all desired hosts. |
| 294 | |
| 295 | ``EPYC``, ``EPYC-IBPB`` |
| 296 | AMD EPYC Processor (2017) |
| 297 | |
| 298 | ``Opteron_G5`` |
| 299 | AMD Opteron 63xx class CPU (2012) |
| 300 | |
| 301 | ``Opteron_G4`` |
| 302 | AMD Opteron 62xx class CPU (2011) |
| 303 | |
| 304 | ``Opteron_G3`` |
| 305 | AMD Opteron 23xx (Gen 3 Class Opteron, 2009) |
| 306 | |
| 307 | ``Opteron_G2`` |
| 308 | AMD Opteron 22xx (Gen 2 Class Opteron, 2006) |
| 309 | |
| 310 | ``Opteron_G1`` |
| 311 | AMD Opteron 240 (Gen 1 Class Opteron, 2004) |
| 312 | |
| 313 | |
| 314 | Important CPU features for AMD x86 hosts |
| 315 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 316 | |
| 317 | The following are important CPU features that should be used on AMD x86 |
| 318 | hosts, when available in the host CPU. Some of them require explicit |
| 319 | configuration to enable, as they are not included by default in some, or |
| 320 | all, of the named CPU models listed above. In general all of these |
| 321 | features are included if using "Host passthrough" or "Host model". |
| 322 | |
| 323 | ``ibpb`` |
| 324 | Required to enable the Spectre v2 (CVE-2017-5715) fix. |
| 325 | |
| 326 | Included by default in AMD CPU models with -IBPB suffix. |
| 327 | |
| 328 | Must be explicitly turned on for AMD CPU models without -IBPB suffix. |
| 329 | |
| 330 | Requires the host CPU microcode to support this feature before it |
| 331 | can be used for guest CPUs. |
| 332 | |
| 333 | ``stibp`` |
| 334 | Required to enable stronger Spectre v2 (CVE-2017-5715) fixes in some |
| 335 | operating systems. |
| 336 | |
| 337 | Must be explicitly turned on for all AMD CPU models. |
| 338 | |
| 339 | Requires the host CPU microcode to support this feature before it |
| 340 | can be used for guest CPUs. |
| 341 | |
| 342 | ``virt-ssbd`` |
| 343 | Required to enable the CVE-2018-3639 fix |
| 344 | |
| 345 | Not included by default in any AMD CPU model. |
| 346 | |
| 347 | Must be explicitly turned on for all AMD CPU models. |
| 348 | |
| 349 | This should be provided to guests, even if amd-ssbd is also provided, |
| 350 | for maximum guest compatibility. |
| 351 | |
| 352 | Note for some QEMU / libvirt versions, this must be force enabled when |
| 353 | when using "Host model", because this is a virtual feature that |
| 354 | doesn't exist in the physical host CPUs. |
| 355 | |
| 356 | ``amd-ssbd`` |
| 357 | Required to enable the CVE-2018-3639 fix |
| 358 | |
| 359 | Not included by default in any AMD CPU model. |
| 360 | |
| 361 | Must be explicitly turned on for all AMD CPU models. |
| 362 | |
| 363 | This provides higher performance than ``virt-ssbd`` so should be |
| 364 | exposed to guests whenever available in the host. ``virt-ssbd`` should |
| 365 | none the less also be exposed for maximum guest compatibility as some |
| 366 | kernels only know about ``virt-ssbd``. |
| 367 | |
| 368 | ``amd-no-ssb`` |
| 369 | Recommended to indicate the host is not vulnerable CVE-2018-3639 |
| 370 | |
| 371 | Not included by default in any AMD CPU model. |
| 372 | |
| 373 | Future hardware generations of CPU will not be vulnerable to |
| 374 | CVE-2018-3639, and thus the guest should be told not to enable |
| 375 | its mitigations, by exposing amd-no-ssb. This is mutually |
| 376 | exclusive with virt-ssbd and amd-ssbd. |
| 377 | |
| 378 | ``pdpe1gb`` |
| 379 | Recommended to allow guest OS to use 1GB size pages |
| 380 | |
| 381 | Not included by default in any AMD CPU model. |
| 382 | |
| 383 | Should be explicitly turned on for all AMD CPU models. |
| 384 | |
| 385 | Note that not all CPU hardware will support this feature. |
| 386 | |
| 387 | |
| 388 | Default x86 CPU models |
| 389 | ^^^^^^^^^^^^^^^^^^^^^^ |
| 390 | |
| 391 | The default QEMU CPU models are designed such that they can run on all |
| 392 | hosts. If an application does not wish to do perform any host |
| 393 | compatibility checks before launching guests, the default is guaranteed |
| 394 | to work. |
| 395 | |
| 396 | The default CPU models will, however, leave the guest OS vulnerable to |
| 397 | various CPU hardware flaws, so their use is strongly discouraged. |
| 398 | Applications should follow the earlier guidance to setup a better CPU |
| 399 | configuration, with host passthrough recommended if live migration is |
| 400 | not needed. |
| 401 | |
| 402 | ``qemu32``, ``qemu64`` |
| 403 | QEMU Virtual CPU version 2.5+ (32 & 64 bit variants) |
| 404 | |
| 405 | ``qemu64`` is used for x86_64 guests and ``qemu32`` is used for i686 |
| 406 | guests, when no ``-cpu`` argument is given to QEMU, or no ``<cpu>`` is |
| 407 | provided in libvirt XML. |
| 408 | |
| 409 | Other non-recommended x86 CPUs |
| 410 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 411 | |
| 412 | The following CPUs models are compatible with most AMD and Intel x86 |
| 413 | hosts, but their usage is discouraged, as they expose a very limited |
| 414 | featureset, which prevents guests having optimal performance. |
| 415 | |
| 416 | ``kvm32``, ``kvm64`` |
| 417 | Common KVM processor (32 & 64 bit variants). |
| 418 | |
| 419 | Legacy models just for historical compatibility with ancient QEMU |
| 420 | versions. |
| 421 | |
| 422 | ``486``, ``athlon``, ``phenom``, ``coreduo``, ``core2duo``, ``n270``, ``pentium``, ``pentium2``, ``pentium3`` |
| 423 | Various very old x86 CPU models, mostly predating the introduction |
| 424 | of hardware assisted virtualization, that should thus not be |
| 425 | required for running virtual machines. |
| 426 | |
| 427 | |
| 428 | Syntax for configuring CPU models |
| 429 | ================================= |
| 430 | |
| 431 | The examples below illustrate the approach to configuring the various |
| 432 | CPU models / features in QEMU and libvirt. |
| 433 | |
| 434 | QEMU command line |
| 435 | ^^^^^^^^^^^^^^^^^ |
| 436 | |
| 437 | Host passthrough: |
| 438 | |
| 439 | .. parsed-literal:: |
| 440 | |
| 441 | |qemu_system| -cpu host |
| 442 | |
| 443 | Host passthrough with feature customization: |
| 444 | |
| 445 | .. parsed-literal:: |
| 446 | |
| 447 | |qemu_system| -cpu host,vmx=off,... |
| 448 | |
| 449 | Named CPU models: |
| 450 | |
| 451 | .. parsed-literal:: |
| 452 | |
| 453 | |qemu_system| -cpu Westmere |
| 454 | |
| 455 | Named CPU models with feature customization: |
| 456 | |
| 457 | .. parsed-literal:: |
| 458 | |
| 459 | |qemu_system| -cpu Westmere,pcid=on,... |
| 460 | |
| 461 | Libvirt guest XML |
| 462 | ^^^^^^^^^^^^^^^^^ |
| 463 | |
| 464 | Host passthrough:: |
| 465 | |
| 466 | <cpu mode='host-passthrough'/> |
| 467 | |
| 468 | Host passthrough with feature customization:: |
| 469 | |
| 470 | <cpu mode='host-passthrough'> |
| 471 | <feature name="vmx" policy="disable"/> |
| 472 | ... |
| 473 | </cpu> |
| 474 | |
| 475 | Host model:: |
| 476 | |
| 477 | <cpu mode='host-model'/> |
| 478 | |
| 479 | Host model with feature customization:: |
| 480 | |
| 481 | <cpu mode='host-model'> |
| 482 | <feature name="vmx" policy="disable"/> |
| 483 | ... |
| 484 | </cpu> |
| 485 | |
| 486 | Named model:: |
| 487 | |
| 488 | <cpu mode='custom'> |
| 489 | <model name="Westmere"/> |
| 490 | </cpu> |
| 491 | |
| 492 | Named model with feature customization:: |
| 493 | |
| 494 | <cpu mode='custom'> |
| 495 | <model name="Westmere"/> |
| 496 | <feature name="pcid" policy="require"/> |
| 497 | ... |
| 498 | </cpu> |