| 1 | PowerNV family boards (``powernv8``, ``powernv9``, ``powernv10``, ``powernv11``) |
| 2 | ================================================================================ |
| 3 | |
| 4 | PowerNV (as Non-Virtualized) is the "bare metal" platform using the |
| 5 | OPAL firmware. It runs Linux on IBM and OpenPOWER systems and it can |
| 6 | be used as an hypervisor OS, running KVM guests, or simply as a host |
| 7 | OS. |
| 8 | |
| 9 | The PowerNV QEMU machine tries to emulate a PowerNV system at the |
| 10 | level of the skiboot firmware, which loads the OS and provides some |
| 11 | runtime services. Power Systems have a lower firmware (HostBoot) that |
| 12 | does low level system initialization, like DRAM training. This is |
| 13 | beyond the scope of what QEMU addresses today. |
| 14 | |
| 15 | Supported devices |
| 16 | ----------------- |
| 17 | |
| 18 | * Multi processor support for POWER8, POWER9, Power10 and Power11. |
| 19 | * XSCOM, serial communication sideband bus to configure chiplets. |
| 20 | * Simple LPC Controller. |
| 21 | * Processor Service Interface (PSI) Controller. |
| 22 | * Interrupt Controller, XICS (POWER8) and XIVE (POWER9) and XIVE2 (Power10 & |
| 23 | Power11). |
| 24 | * POWER8 PHB3 PCIe Host bridge and POWER9 PHB4 PCIe Host bridge. |
| 25 | * Simple OCC is an on-chip micro-controller used for power management tasks. |
| 26 | * iBT device to handle BMC communication, with the internal BMC simulator |
| 27 | provided by QEMU or an external BMC such as an Aspeed QEMU machine. |
| 28 | * PNOR containing the different firmware partitions. |
| 29 | |
| 30 | Missing devices |
| 31 | --------------- |
| 32 | |
| 33 | A lot is missing, among which : |
| 34 | |
| 35 | * I2C controllers (yet to be merged). |
| 36 | * NPU/NPU2/NPU3 controllers. |
| 37 | * EEH support for PCIe Host bridge controllers. |
| 38 | * NX controller. |
| 39 | * VAS controller. |
| 40 | * chipTOD (Time Of Day). |
| 41 | * Self Boot Engine (SBE). |
| 42 | * FSI bus. |
| 43 | |
| 44 | Firmware |
| 45 | -------- |
| 46 | |
| 47 | The OPAL firmware (OpenPower Abstraction Layer) for OpenPower systems |
| 48 | includes the runtime services ``skiboot`` and the bootloader kernel and |
| 49 | initramfs ``skiroot``. Source code can be found on the `OpenPOWER account at |
| 50 | GitHub <https://github.com/open-power>`_. |
| 51 | |
| 52 | Prebuilt images of ``skiboot`` and ``skiroot`` are made available on the |
| 53 | `OpenPOWER <https://github.com/open-power/op-build/releases/>`__ site. |
| 54 | |
| 55 | QEMU includes a prebuilt image of ``skiboot`` which is updated when a |
| 56 | more recent version is required by the models. |
| 57 | |
| 58 | Current acceleration status |
| 59 | --------------------------- |
| 60 | |
| 61 | KVM acceleration in Linux Power hosts is provided by the kvm-hv and |
| 62 | kvm-pr modules. kvm-hv is adherent to PAPR and it's not compliant with |
| 63 | powernv. kvm-pr in theory could be used as a valid accel option but |
| 64 | this isn't supported by kvm-pr at this moment. |
| 65 | |
| 66 | To spare users from dealing with not so informative errors when attempting |
| 67 | to use accel=kvm, the powernv machine will throw an error informing that |
| 68 | KVM is not supported. This can be revisited in the future if kvm-pr (or |
| 69 | any other KVM alternative) is usable as KVM accel for this machine. |
| 70 | |
| 71 | Boot options |
| 72 | ------------ |
| 73 | |
| 74 | Here is a simple setup with one e1000e NIC : |
| 75 | |
| 76 | .. code-block:: bash |
| 77 | |
| 78 | $ qemu-system-ppc64 -m 2G -machine powernv9 -smp 2,cores=2,threads=1 \ |
| 79 | -accel tcg,thread=single \ |
| 80 | -device e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0 \ |
| 81 | -netdev user,id=net0,hostfwd=::20022-:22,hostname=pnv \ |
| 82 | -kernel ./zImage.epapr \ |
| 83 | -initrd ./rootfs.cpio.xz \ |
| 84 | -nographic |
| 85 | |
| 86 | and a SATA disk : |
| 87 | |
| 88 | .. code-block:: bash |
| 89 | |
| 90 | -device ich9-ahci,id=sata0,bus=pcie.1,addr=0x0 \ |
| 91 | -drive file=./ubuntu-ppc64le.qcow2,if=none,id=drive0,format=qcow2,cache=none \ |
| 92 | -device ide-hd,bus=sata0.0,unit=0,drive=drive0,id=ide,bootindex=1 \ |
| 93 | |
| 94 | Complex PCIe configuration |
| 95 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 96 | |
| 97 | Six PHBs are defined per chip (POWER9) but no default PCI layout is |
| 98 | provided (to be compatible with libvirt). One PCI device can be added |
| 99 | on any of the available PCIe slots using command line options such as: |
| 100 | |
| 101 | .. code-block:: bash |
| 102 | |
| 103 | -device e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0 |
| 104 | -netdev bridge,id=net0,helper=/usr/libexec/qemu-bridge-helper,br=virbr0,id=hostnet0 |
| 105 | |
| 106 | -device megasas,id=scsi0,bus=pcie.0,addr=0x0 |
| 107 | -drive file=./ubuntu-ppc64le.qcow2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=none |
| 108 | -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=2 |
| 109 | |
| 110 | Here is a full example with two different storage controllers on |
| 111 | different PHBs, each with a disk, the second PHB is empty : |
| 112 | |
| 113 | .. code-block:: bash |
| 114 | |
| 115 | $ qemu-system-ppc64 -m 2G -machine powernv9 -smp 2,cores=2,threads=1 -accel tcg,thread=single \ |
| 116 | -kernel ./zImage.epapr -initrd ./rootfs.cpio.xz -bios ./skiboot.lid \ |
| 117 | \ |
| 118 | -device megasas,id=scsi0,bus=pcie.0,addr=0x0 \ |
| 119 | -drive file=./rhel7-ppc64le.qcow2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=none \ |
| 120 | -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=2 \ |
| 121 | \ |
| 122 | -device pcie-pci-bridge,id=bridge1,bus=pcie.1,addr=0x0 \ |
| 123 | \ |
| 124 | -device ich9-ahci,id=sata0,bus=bridge1,addr=0x1 \ |
| 125 | -drive file=./ubuntu-ppc64le.qcow2,if=none,id=drive0,format=qcow2,cache=none \ |
| 126 | -device ide-hd,bus=sata0.0,unit=0,drive=drive0,id=ide,bootindex=1 \ |
| 127 | -device e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=bridge1,addr=0x2 \ |
| 128 | -netdev bridge,helper=/usr/libexec/qemu-bridge-helper,br=virbr0,id=net0 \ |
| 129 | -device nec-usb-xhci,bus=bridge1,addr=0x7 \ |
| 130 | \ |
| 131 | -serial mon:stdio -nographic |
| 132 | |
| 133 | You can also use VIRTIO devices : |
| 134 | |
| 135 | .. code-block:: bash |
| 136 | |
| 137 | -drive file=./fedora-ppc64le.qcow2,if=none,snapshot=on,id=drive0 \ |
| 138 | -device virtio-blk-pci,drive=drive0,id=blk0,bus=pcie.0 \ |
| 139 | \ |
| 140 | -netdev tap,helper=/usr/lib/qemu/qemu-bridge-helper,br=virbr0,id=netdev0 \ |
| 141 | -device virtio-net-pci,netdev=netdev0,id=net0,bus=pcie.1 \ |
| 142 | \ |
| 143 | -fsdev local,id=fsdev0,path=$HOME,security_model=passthrough \ |
| 144 | -device virtio-9p-pci,fsdev=fsdev0,mount_tag=host,bus=pcie.2 |
| 145 | |
| 146 | Multi sockets |
| 147 | ~~~~~~~~~~~~~ |
| 148 | |
| 149 | The number of sockets is deduced from the number of CPUs and the |
| 150 | number of cores. ``-smp 2,cores=1`` will define a machine with 2 |
| 151 | sockets of 1 core, whereas ``-smp 2,cores=2`` will define a machine |
| 152 | with 1 socket of 2 cores. ``-smp 8,cores=2``, 4 sockets of 2 cores. |
| 153 | |
| 154 | BMC configuration |
| 155 | ~~~~~~~~~~~~~~~~~ |
| 156 | |
| 157 | OpenPOWER systems negotiate the shutdown and reboot with their |
| 158 | BMC. The QEMU PowerNV machine embeds an IPMI BMC simulator using the |
| 159 | iBT interface and should offer the same power features. |
| 160 | |
| 161 | If you want to define your own BMC, use ``-nodefaults`` and specify |
| 162 | one on the command line : |
| 163 | |
| 164 | .. code-block:: bash |
| 165 | |
| 166 | -device ipmi-bmc-sim,id=bmc0 -device isa-ipmi-bt,bmc=bmc0,irq=10 |
| 167 | |
| 168 | The files `palmetto-SDR.bin <http://www.kaod.org/qemu/powernv/palmetto-SDR.bin>`__ |
| 169 | and `palmetto-FRU.bin <http://www.kaod.org/qemu/powernv/palmetto-FRU.bin>`__ |
| 170 | define a Sensor Data Record repository and a Field Replaceable Unit |
| 171 | inventory for a Palmetto BMC. They can be used to extend the QEMU BMC |
| 172 | simulator. |
| 173 | |
| 174 | .. code-block:: bash |
| 175 | |
| 176 | -device ipmi-bmc-sim,sdrfile=./palmetto-SDR.bin,fruareasize=256,frudatafile=./palmetto-FRU.bin,id=bmc0 \ |
| 177 | -device isa-ipmi-bt,bmc=bmc0,irq=10 |
| 178 | |
| 179 | The PowerNV machine can also be run with an external IPMI BMC device |
| 180 | connected to a remote QEMU machine acting as BMC, using these options |
| 181 | : |
| 182 | |
| 183 | .. code-block:: bash |
| 184 | |
| 185 | -chardev socket,id=ipmi0,host=localhost,port=9002,reconnect-ms=10000 \ |
| 186 | -device ipmi-bmc-extern,id=bmc0,chardev=ipmi0 \ |
| 187 | -device isa-ipmi-bt,bmc=bmc0,irq=10 \ |
| 188 | -nodefaults |
| 189 | |
| 190 | NVRAM |
| 191 | ~~~~~ |
| 192 | |
| 193 | Use a MTD drive to add a PNOR to the machine, and get a NVRAM : |
| 194 | |
| 195 | .. code-block:: bash |
| 196 | |
| 197 | -drive file=./witherspoon.pnor,format=raw,if=mtd |
| 198 | |
| 199 | If no mtd drive is provided, the powernv platform will create a default |
| 200 | PNOR device using a tiny formatted PNOR in pc-bios/pnv-pnor.bin opened |
| 201 | read-only (PNOR changes will be persistent across reboots but not across |
| 202 | invocations of QEMU). If no defaults are used, an erased 128MB PNOR is |
| 203 | provided (which skiboot will probably not recognize since it is not |
| 204 | formatted). |
| 205 | |
| 206 | Maintainer contact information |
| 207 | ------------------------------ |
| 208 | |
| 209 | Cédric Le Goater <clg@kaod.org> |