| 1 | VirtIO Persistent Memory |
| 2 | ======================== |
| 3 | |
| 4 | This document explains the setup and usage of the virtio pmem device. |
| 5 | The virtio pmem device is a paravirtualized persistent memory device |
| 6 | on regular (i.e non-NVDIMM) storage. |
| 7 | |
| 8 | Usecase |
| 9 | ------- |
| 10 | |
| 11 | Virtio pmem allows to bypass the guest page cache and directly use |
| 12 | host page cache. This reduces guest memory footprint as the host can |
| 13 | make efficient memory reclaim decisions under memory pressure. |
| 14 | |
| 15 | How does virtio-pmem compare to the nvdimm emulation? |
| 16 | ----------------------------------------------------- |
| 17 | |
| 18 | NVDIMM emulation on regular (i.e. non-NVDIMM) host storage does not |
| 19 | persist the guest writes as there are no defined semantics in the device |
| 20 | specification. The virtio pmem device provides guest write persistence |
| 21 | on non-NVDIMM host storage. |
| 22 | |
| 23 | virtio pmem usage |
| 24 | ----------------- |
| 25 | |
| 26 | A virtio pmem device backed by a memory-backend-file can be created on |
| 27 | the QEMU command line as in the following example:: |
| 28 | |
| 29 | -object memory-backend-file,id=mem1,share,mem-path=./virtio_pmem.img,size=4G |
| 30 | -device virtio-pmem-pci,memdev=mem1,id=nv1 |
| 31 | |
| 32 | where: |
| 33 | |
| 34 | - "object memory-backend-file,id=mem1,share,mem-path=<image>, size=<image size>" |
| 35 | creates a backend file with the specified size. |
| 36 | |
| 37 | - "device virtio-pmem-pci,id=nvdimm1,memdev=mem1" creates a virtio pmem |
| 38 | pci device whose storage is provided by above memory backend device. |
| 39 | |
| 40 | Multiple virtio pmem devices can be created if multiple pairs of "-object" |
| 41 | and "-device" are provided. |
| 42 | |
| 43 | Hotplug |
| 44 | ------- |
| 45 | |
| 46 | Virtio pmem devices can be hotplugged via the QEMU monitor. First, the |
| 47 | memory backing has to be added via 'object_add'; afterwards, the virtio |
| 48 | pmem device can be added via 'device_add'. |
| 49 | |
| 50 | For example, the following commands add another 4GB virtio pmem device to |
| 51 | the guest:: |
| 52 | |
| 53 | (qemu) object_add memory-backend-file,id=mem2,share=on,mem-path=virtio_pmem2.img,size=4G |
| 54 | (qemu) device_add virtio-pmem-pci,id=virtio_pmem2,memdev=mem2 |
| 55 | |
| 56 | Guest Data Persistence |
| 57 | ---------------------- |
| 58 | |
| 59 | Guest data persistence on non-NVDIMM requires guest userspace applications |
| 60 | to perform fsync/msync. This is different from a real nvdimm backend where |
| 61 | no additional fsync/msync is required. This is to persist guest writes in |
| 62 | host backing file which otherwise remains in host page cache and there is |
| 63 | risk of losing the data in case of power failure. |
| 64 | |
| 65 | With virtio pmem device, MAP_SYNC mmap flag is not supported. This provides |
| 66 | a hint to application to perform fsync for write persistence. |
| 67 | |
| 68 | Limitations |
| 69 | ----------- |
| 70 | |
| 71 | - Real nvdimm device backend is not supported. |
| 72 | - virtio pmem hotunplug is not supported. |
| 73 | - ACPI NVDIMM features like regions/namespaces are not supported. |
| 74 | - ndctl command is not supported. |