| 1 | .. _arm-virt: |
| 2 | |
| 3 | 'virt' generic virtual platform (``virt``) |
| 4 | ========================================== |
| 5 | |
| 6 | The ``virt`` board is a platform which does not correspond to any |
| 7 | real hardware; it is designed for use in virtual machines. |
| 8 | It is the recommended board type if you simply want to run |
| 9 | a guest such as Linux and do not care about reproducing the |
| 10 | idiosyncrasies and limitations of a particular bit of real-world |
| 11 | hardware. |
| 12 | |
| 13 | This is a "versioned" board model, so as well as the ``virt`` machine |
| 14 | type itself (which may have improvements, bugfixes and other minor |
| 15 | changes between QEMU versions) a version is provided that guarantees |
| 16 | to have the same behaviour as that of previous QEMU releases, so |
| 17 | that VM migration will work between QEMU versions. For instance the |
| 18 | ``virt-5.0`` machine type will behave like the ``virt`` machine from |
| 19 | the QEMU 5.0 release, and migration should work between ``virt-5.0`` |
| 20 | of the 5.0 release and ``virt-5.0`` of the 5.1 release. Migration |
| 21 | is not guaranteed to work between different QEMU releases for |
| 22 | the non-versioned ``virt`` machine type. |
| 23 | |
| 24 | VM migration is not guaranteed when using ``-cpu max``, as features |
| 25 | supported may change between QEMU versions. To ensure your VM can be |
| 26 | migrated, it is recommended to use another cpu model instead. |
| 27 | |
| 28 | Supported devices |
| 29 | """"""""""""""""" |
| 30 | |
| 31 | The virt board supports: |
| 32 | |
| 33 | - PCI/PCIe devices |
| 34 | - CXL Fixed memory windows, root bridges and devices. |
| 35 | - Flash memory |
| 36 | - Either one or two PL011 UARTs for the NonSecure World |
| 37 | - An RTC |
| 38 | - The fw_cfg device that allows a guest to obtain data from QEMU |
| 39 | - A PL061 GPIO controller |
| 40 | - An optional machine-wide SMMUv3 IOMMU |
| 41 | - User-creatable SMMUv3 devices (see below for example) |
| 42 | - An optional SBSA Generic Watchdog Timer (see below) |
| 43 | - hotpluggable DIMMs |
| 44 | - hotpluggable NVDIMMs |
| 45 | - An MSI controller (GICv2m or ITS). |
| 46 | - When using GICv3, ITS is selected by default when available on the platform. |
| 47 | - If using GICv2, a GICv2m is provided by default instead. |
| 48 | - When ITS is not available on a GICv3 platform, a GICv2m is provided by default. |
| 49 | - 32 virtio-mmio transport devices |
| 50 | - running guests using the KVM accelerator on aarch64 hardware |
| 51 | - large amounts of RAM (at least 255GB, and more if using highmem) |
| 52 | - many CPUs (up to 512 if using a GICv3 and highmem) |
| 53 | - Secure-World-only devices if the CPU has TrustZone: |
| 54 | |
| 55 | - A second PL011 UART |
| 56 | - A second PL061 GPIO controller, with GPIO lines for triggering |
| 57 | a system reset or system poweroff |
| 58 | - A secure flash memory |
| 59 | - 16MB of secure RAM |
| 60 | |
| 61 | The second NonSecure UART only exists if a backend is configured |
| 62 | explicitly (e.g. with a second -serial command line option) and |
| 63 | TrustZone emulation is not enabled. |
| 64 | |
| 65 | Supported guest CPU types: |
| 66 | |
| 67 | - ``cortex-a7`` (32-bit) |
| 68 | - ``cortex-a15`` (32-bit; the default) |
| 69 | - ``cortex-a35`` (64-bit) |
| 70 | - ``cortex-a53`` (64-bit) |
| 71 | - ``cortex-a55`` (64-bit) |
| 72 | - ``cortex-a57`` (64-bit) |
| 73 | - ``cortex-a72`` (64-bit) |
| 74 | - ``cortex-a76`` (64-bit) |
| 75 | - ``cortex-a710`` (64-bit) |
| 76 | - ``a64fx`` (64-bit) |
| 77 | - ``host`` (with KVM and HVF only) |
| 78 | - ``neoverse-n1`` (64-bit) |
| 79 | - ``neoverse-v1`` (64-bit) |
| 80 | - ``neoverse-n2`` (64-bit) |
| 81 | - ``max`` (same as ``host`` for KVM and HVF; best possible emulation with TCG) |
| 82 | |
| 83 | Note that the default is ``cortex-a15``, so for an AArch64 guest you must |
| 84 | specify a CPU type. |
| 85 | |
| 86 | Also, please note that passing ``max`` CPU (i.e. ``-cpu max``) won't |
| 87 | enable all the CPU features for a given ``virt`` machine. Where a CPU |
| 88 | architectural feature requires support in both the CPU itself and in the |
| 89 | wider system (e.g. the MTE feature), it may not be enabled by default, |
| 90 | but instead requires a machine option to enable it. |
| 91 | |
| 92 | For example, MTE support must be enabled with ``-machine virt,mte=on``, |
| 93 | as well as by selecting an MTE-capable CPU (e.g., ``max``) with the |
| 94 | ``-cpu`` option. |
| 95 | |
| 96 | See the machine-specific options below, or check them for a given machine |
| 97 | by passing the ``help`` suboption, like: ``-machine virt-9.0,help``. |
| 98 | |
| 99 | Graphics output is available, but unlike the x86 PC machine types |
| 100 | there is no default display device enabled: you should select one from |
| 101 | the Display devices section of "-device help". The recommended option |
| 102 | is ``virtio-gpu-pci``; this is the only one which will work correctly |
| 103 | with KVM. You may also need to ensure your guest kernel is configured |
| 104 | with support for this; see below. |
| 105 | |
| 106 | Machine-specific options |
| 107 | """""""""""""""""""""""" |
| 108 | |
| 109 | The following machine-specific options are supported: |
| 110 | |
| 111 | secure |
| 112 | Set ``on``/``off`` to enable/disable emulating a guest CPU which implements the |
| 113 | Arm Security Extensions (TrustZone). The default is ``off``. |
| 114 | |
| 115 | virtualization |
| 116 | Set ``on``/``off`` to enable/disable emulating a guest CPU which implements the |
| 117 | Arm Virtualization Extensions. The default is ``off``. |
| 118 | |
| 119 | mte |
| 120 | Set ``on``/``off`` to enable/disable emulating a guest CPU which implements the |
| 121 | Arm Memory Tagging Extensions. The default is ``off``. |
| 122 | |
| 123 | highmem |
| 124 | Set ``on``/``off`` to enable/disable placing devices and RAM in physical |
| 125 | address space above 32 bits. The default is ``on`` for machine types |
| 126 | later than ``virt-2.12`` when the CPU supports an address space |
| 127 | bigger than 32 bits (i.e. 64-bit CPUs, and 32-bit CPUs with the |
| 128 | Large Physical Address Extension (LPAE) feature). If you want to |
| 129 | boot a 32-bit kernel which does not have ``CONFIG_LPAE`` enabled on |
| 130 | a CPU type which implements LPAE, you will need to manually set |
| 131 | this to ``off``; otherwise some devices, such as the PCI controller, |
| 132 | will not be accessible. |
| 133 | |
| 134 | compact-highmem |
| 135 | Set ``on``/``off`` to enable/disable the compact layout for high memory regions. |
| 136 | The default is ``on`` for machine types later than ``virt-7.2``. |
| 137 | |
| 138 | highmem-redists |
| 139 | Set ``on``/``off`` to enable/disable the high memory region for GICv3 or |
| 140 | GICv4 redistributor. The default is ``on``. Setting this to ``off`` will |
| 141 | limit the maximum number of CPUs when GICv3 or GICv4 is used. |
| 142 | |
| 143 | highmem-ecam |
| 144 | Set ``on``/``off`` to enable/disable the high memory region for PCI ECAM. |
| 145 | The default is ``on`` for machine types later than ``virt-3.0``. |
| 146 | |
| 147 | highmem-mmio |
| 148 | Set ``on``/``off`` to enable/disable the high memory region for PCI MMIO. |
| 149 | The default is ``on``. |
| 150 | |
| 151 | highmem-mmio-size |
| 152 | Set the high memory region size for PCI MMIO. Must be a power of 2 and |
| 153 | greater than or equal to the default size (512G). |
| 154 | |
| 155 | gic-version |
| 156 | Specify the version of the Generic Interrupt Controller (GIC) to provide. |
| 157 | Valid values are: |
| 158 | |
| 159 | ``2`` |
| 160 | GICv2. Note that this limits the number of CPUs to 8. |
| 161 | ``3`` |
| 162 | GICv3. This allows up to 512 CPUs. |
| 163 | ``4`` |
| 164 | GICv4. Requires ``virtualization`` to be ``on``; allows up to 317 CPUs. |
| 165 | ``x-5`` |
| 166 | GICv5 (experimental). This is an experimental emulation of the GICv5, |
| 167 | based on the EAC release of the GICv5 architecture specification. |
| 168 | Experimental means: |
| 169 | |
| 170 | - guest-visible behaviour may change when the final version of |
| 171 | the specification is released and QEMU implements it |
| 172 | - migration support is not yet implemented |
| 173 | - the GICv5 is not exposed to the guest via ACPI tables, only via DTB |
| 174 | - the way the interrupt controller is exposed to the guest and the |
| 175 | command line syntax for enabling it may change |
| 176 | |
| 177 | The current implementation supports only an EL1 guest (no EL2 or |
| 178 | EL3 and no Realm support), and does not implement the ITS (no |
| 179 | MSI support). |
| 180 | |
| 181 | Note that as the GICv5 is an Armv9 feature, enabling it will |
| 182 | automatically disable support for AArch32 at all exception levels |
| 183 | except for EL0 (userspace). |
| 184 | ``host`` |
| 185 | Use the same GIC version the host provides, when using KVM |
| 186 | ``max`` |
| 187 | Use the best GIC version possible (same as host when using KVM; |
| 188 | with TCG this is currently ``3`` if ``virtualization`` is ``off`` and |
| 189 | ``4`` if ``virtualization`` is ``on``, but this may change in future) |
| 190 | |
| 191 | msi |
| 192 | Specify the MSI and MSI-X controller (GIC) to provide. |
| 193 | Valid values are: |
| 194 | |
| 195 | ``auto`` |
| 196 | Use the best available MSI-X controller option. ITS when supported, GICv2m otherwise. |
| 197 | ``gicv2m`` |
| 198 | GICv2m. Typically used with a GICv2. Also available with a newer GIC. |
| 199 | ``its`` |
| 200 | GICv3 ITS. This is the default option when using a GICv3 or GICv4 with a supported |
| 201 | accelerator. |
| 202 | ``off`` |
| 203 | Disable support for MSI/MSI-X interrupts. |
| 204 | |
| 205 | its |
| 206 | Set ``on``/``off`` to control ITS instantiation. This is a deprecated option, use ``msi`` instead. |
| 207 | |
| 208 | iommu |
| 209 | Set the IOMMU type to create for the guest. Valid values are: |
| 210 | |
| 211 | ``none`` |
| 212 | Don't create an IOMMU (the default) |
| 213 | ``smmuv3`` |
| 214 | Create a machine-wide SMMUv3. |
| 215 | |
| 216 | default-bus-bypass-iommu |
| 217 | Set ``on``/``off`` to enable/disable `bypass_iommu |
| 218 | <https://gitlab.com/qemu-project/qemu/-/blob/master/docs/bypass-iommu.txt>`_ |
| 219 | for default root bus. |
| 220 | |
| 221 | ras |
| 222 | Set ``on``/``off`` to enable/disable reporting host memory errors to a guest |
| 223 | using ACPI and guest external abort exceptions. The default is off. |
| 224 | |
| 225 | acpi |
| 226 | Set ``on``/``off``/``auto`` to enable/disable ACPI. |
| 227 | |
| 228 | cxl |
| 229 | Set ``on``/``off`` to enable/disable CXL. More details in |
| 230 | :doc:`../devices/cxl`. The default is off. |
| 231 | |
| 232 | cxl-fmw |
| 233 | Array of CXL fixed memory windows describing fixed address routing to |
| 234 | target CXL host bridges. See :doc:`../devices/cxl`. |
| 235 | |
| 236 | dtb-randomness |
| 237 | Set ``on``/``off`` to pass random seeds via the guest DTB |
| 238 | rng-seed and kaslr-seed nodes (in both "/chosen" and |
| 239 | "/secure-chosen") to use for features like the random number |
| 240 | generator and address space randomisation. The default is |
| 241 | ``on``. You will want to disable it if your trusted boot chain |
| 242 | will verify the DTB it is passed, since this option causes the |
| 243 | DTB to be non-deterministic. It would be the responsibility of |
| 244 | the firmware to come up with a seed and pass it on if it wants to. |
| 245 | |
| 246 | dtb-kaslr-seed |
| 247 | A deprecated synonym for dtb-randomness. |
| 248 | |
| 249 | virtio-mmio-transports |
| 250 | Set the number of virtio-mmio transports to create (between 0 and 32; |
| 251 | the default is 32). Unused transports are harmless, but you can |
| 252 | use this property to avoid exposing them to the guest if you wish. |
| 253 | |
| 254 | x-oem-id |
| 255 | Set string (up to 6 bytes) to override the default value of field OEMID in ACPI |
| 256 | table header. |
| 257 | |
| 258 | x-oem-table-id |
| 259 | Set string (up to 8 bytes) to override the default value of field OEM Table ID |
| 260 | in ACPI table header. |
| 261 | |
| 262 | SMMU configuration |
| 263 | """""""""""""""""" |
| 264 | |
| 265 | Machine-wide SMMUv3 IOMMU |
| 266 | Setting the machine-specific option ``iommu=smmuv3`` causes QEMU to |
| 267 | create a single, machine-wide SMMUv3 instance that applies to all |
| 268 | devices in the PCIe topology. |
| 269 | |
| 270 | For information about selectively bypassing devices, refer to |
| 271 | ``docs/bypass-iommu.txt``. |
| 272 | |
| 273 | User-creatable SMMUv3 devices |
| 274 | You can use the ``-device arm-smmuv3`` option to create multiple |
| 275 | user-defined SMMUv3 devices, each associated with a separate PCIe |
| 276 | root complex. This is only permitted if the machine-wide SMMUv3 |
| 277 | (``iommu=smmuv3``) option is not used. Each ``arm-smmuv3`` device |
| 278 | uses the ``primary-bus`` sub-option to specify which PCIe root |
| 279 | complex it is associated with. |
| 280 | |
| 281 | This model is useful when you want to mirror a host configuration where |
| 282 | each NUMA node typically has its own SMMU, allowing the VM topology to |
| 283 | align more closely with the host’s hardware layout. |
| 284 | |
| 285 | Example:: |
| 286 | |
| 287 | -device arm-smmuv3,primary-bus=pcie.0,id=smmuv3.0 |
| 288 | ... |
| 289 | -device pxb-pcie,id=pcie.1,numa_node=1 |
| 290 | -device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.1 |
| 291 | |
| 292 | *Accelerated SMMUv3 (nested translation)* |
| 293 | |
| 294 | The ``accel=on`` option enables hardware-accelerated nested translation |
| 295 | for vfio-pci passthrough devices. In this mode the guest SMMU driver |
| 296 | programs its own Stage-1 page tables, with the host SMMUv3 handling both |
| 297 | Stage-1 (guest) and Stage-2 (host) translations in hardware. The host |
| 298 | SMMUv3 must support nested translation. This mode requires the iommufd |
| 299 | backend and is only supported when booting with ACPI (not device tree). |
| 300 | |
| 301 | When ``accel=on``, QEMU automatically derives the values for the |
| 302 | ``ril``, ``ats``, ``oas``, ``ssidsize`` and ``cmdqv`` sub-options |
| 303 | from the host SMMUv3 capabilities unless they are set explicitly. |
| 304 | |
| 305 | Example:: |
| 306 | |
| 307 | -device arm-smmuv3,primary-bus=pcie.0,id=smmuv3.0,accel=on |
| 308 | |
| 309 | *Accelerated SMMUv3 command queues (Tegra241 CMDQV)* |
| 310 | |
| 311 | The ``cmdqv`` sub-option enables NVIDIA Tegra241 Command Queue |
| 312 | Virtualization (CMDQV) on supported hosts. With CMDQV, each accelerated |
| 313 | ``arm-smmuv3`` device gets dedicated hardware command queues and the |
| 314 | guest issues SMMU invalidation commands directly to real hardware, |
| 315 | bypassing QEMU and improving throughput for workloads that issue many |
| 316 | invalidations. Without it, every invalidation command traps into QEMU. |
| 317 | |
| 318 | SBSA Generic Watchdog |
| 319 | """"""""""""""""""""" |
| 320 | |
| 321 | The SBSA Generic Watchdog Timer (GWDT) can be added to the virt machine |
| 322 | using ``-device sbsa-gwdt``. It is only supported on the virt machine, |
| 323 | which wires up statically assigned MMIO regions and IRQs via |
| 324 | machine-specific plug handlers. |
| 325 | |
| 326 | Two modes are available: |
| 327 | |
| 328 | Native mode (default) |
| 329 | The watchdog is described via the ACPI GTDT table and FDT, using |
| 330 | the system counter frequency. Example:: |
| 331 | |
| 332 | -device sbsa-gwdt |
| 333 | |
| 334 | WDAT mode |
| 335 | The watchdog is described via the ACPI WDAT table (no FDT node), |
| 336 | using a 1 kHz timer frequency. WDAT and GTDT watchdog entries are |
| 337 | mutually exclusive. Example:: |
| 338 | |
| 339 | -device sbsa-gwdt,wdat=on |
| 340 | |
| 341 | Linux guest kernel configuration |
| 342 | """""""""""""""""""""""""""""""" |
| 343 | |
| 344 | The 'defconfig' for Linux arm and arm64 kernels should include the |
| 345 | right device drivers for virtio and the PCI controller; however some older |
| 346 | kernel versions, especially for 32-bit Arm, did not have everything |
| 347 | enabled by default. If you're not seeing PCI devices that you expect, |
| 348 | then check that your guest config has:: |
| 349 | |
| 350 | CONFIG_PCI=y |
| 351 | CONFIG_VIRTIO_PCI=y |
| 352 | CONFIG_PCI_HOST_GENERIC=y |
| 353 | |
| 354 | If you want to use the ``virtio-gpu-pci`` graphics device you will also |
| 355 | need:: |
| 356 | |
| 357 | CONFIG_DRM=y |
| 358 | CONFIG_DRM_VIRTIO_GPU=y |
| 359 | |
| 360 | Hardware configuration information for bare-metal programming |
| 361 | """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""" |
| 362 | |
| 363 | The ``virt`` board automatically generates a device tree blob ("dtb") |
| 364 | which it passes to the guest. This provides information about the |
| 365 | addresses, interrupt lines and other configuration of the various devices |
| 366 | in the system. Guest code can rely on and hard-code the following |
| 367 | addresses: |
| 368 | |
| 369 | - Flash memory starts at address 0x0000_0000 |
| 370 | |
| 371 | - RAM starts at 0x4000_0000 |
| 372 | |
| 373 | All other information about device locations may change between |
| 374 | QEMU versions, so guest code must look in the DTB. |
| 375 | |
| 376 | QEMU supports two types of guest image boot for ``virt``, and |
| 377 | the way for the guest code to locate the dtb binary differs: |
| 378 | |
| 379 | - For guests using the Linux kernel boot protocol (this means any |
| 380 | non-ELF file passed to the QEMU ``-kernel`` option) the address |
| 381 | of the DTB is passed in a register (``r2`` for 32-bit guests, |
| 382 | or ``x0`` for 64-bit guests) |
| 383 | |
| 384 | - For guests booting as "bare-metal" (any other kind of boot), |
| 385 | the DTB is at the start of RAM (0x4000_0000) |