| 1 | .. _storage-daemon: |
| 2 | |
| 3 | =================== |
| 4 | QEMU Storage Daemon |
| 5 | =================== |
| 6 | |
| 7 | Synopsis |
| 8 | -------- |
| 9 | |
| 10 | **qemu-storage-daemon** [options] |
| 11 | |
| 12 | Description |
| 13 | ----------- |
| 14 | |
| 15 | ``qemu-storage-daemon`` provides disk image functionality from QEMU, |
| 16 | ``qemu-img``, and ``qemu-nbd`` in a long-running process controlled via QMP |
| 17 | commands without running a virtual machine. |
| 18 | It can export disk images, run block job operations, and |
| 19 | perform other disk-related operations. The daemon is controlled via a QMP |
| 20 | monitor and initial configuration from the command-line. |
| 21 | |
| 22 | The daemon offers the following subset of QEMU features: |
| 23 | |
| 24 | * Block nodes |
| 25 | * Block jobs |
| 26 | * Block exports |
| 27 | * Throttle groups |
| 28 | * Character devices |
| 29 | * Crypto and secrets |
| 30 | * QMP |
| 31 | * IOThreads |
| 32 | |
| 33 | Commands can be sent over a QEMU Monitor Protocol (QMP) connection. See the |
| 34 | :manpage:`qemu-storage-daemon-qmp-ref(7)` manual page for a description of the |
| 35 | commands. |
| 36 | |
| 37 | The daemon runs until it is stopped using the ``quit`` QMP command or |
| 38 | SIGINT/SIGHUP/SIGTERM. |
| 39 | |
| 40 | **Warning:** Never modify images in use by a running virtual machine or any |
| 41 | other process; this may destroy the image. Also, be aware that querying an |
| 42 | image that is being modified by another process may encounter inconsistent |
| 43 | state. |
| 44 | |
| 45 | Options |
| 46 | ------- |
| 47 | |
| 48 | .. program:: qemu-storage-daemon |
| 49 | |
| 50 | Standard options: |
| 51 | |
| 52 | .. option:: -h, --help |
| 53 | |
| 54 | Display help and exit |
| 55 | |
| 56 | .. option:: -V, --version |
| 57 | |
| 58 | Display version information and exit |
| 59 | |
| 60 | .. option:: -T, --trace [[enable=]PATTERN][,events=FILE][,file=FILE] |
| 61 | |
| 62 | .. include:: ../qemu-option-trace.rst.inc |
| 63 | |
| 64 | .. option:: --blockdev BLOCKDEVDEF |
| 65 | |
| 66 | is a block node definition. See the :manpage:`qemu(1)` manual page for a |
| 67 | description of block node properties and the :manpage:`qemu-block-drivers(7)` |
| 68 | manual page for a description of driver-specific parameters. |
| 69 | |
| 70 | .. option:: --chardev CHARDEVDEF |
| 71 | |
| 72 | is a character device definition. See the :manpage:`qemu(1)` manual page for |
| 73 | a description of character device properties. A common character device |
| 74 | definition configures a UNIX domain socket:: |
| 75 | |
| 76 | --chardev socket,id=char1,path=/var/run/qsd-qmp.sock,server=on,wait=off |
| 77 | |
| 78 | .. option:: --export [type=]nbd,id=<id>,node-name=<node-name>[,name=<export-name>][,writable=on|off][,bitmap=<name>] |
| 79 | --export [type=]vhost-user-blk,id=<id>,node-name=<node-name>,addr.type=unix,addr.path=<socket-path>[,writable=on|off][,logical-block-size=<block-size>][,num-queues=<num-queues>] |
| 80 | --export [type=]vhost-user-blk,id=<id>,node-name=<node-name>,addr.type=fd,addr.str=<fd>[,writable=on|off][,logical-block-size=<block-size>][,num-queues=<num-queues>] |
| 81 | --export [type=]fuse,id=<id>,node-name=<node-name>,mountpoint=<file>[,growable=on|off][,writable=on|off][,allow-other=on|off|auto] |
| 82 | --export [type=]vduse-blk,id=<id>,node-name=<node-name>,name=<vduse-name>[,writable=on|off][,num-queues=<num-queues>][,queue-size=<queue-size>][,logical-block-size=<block-size>][,serial=<serial-number>] |
| 83 | |
| 84 | is a block export definition. ``node-name`` is the block node that should be |
| 85 | exported. ``writable`` determines whether or not the export allows write |
| 86 | requests for modifying data (the default is off). |
| 87 | |
| 88 | The ``nbd`` export type requires ``--nbd-server`` (see below). ``name`` is |
| 89 | the NBD export name (if not specified, it defaults to the given |
| 90 | ``node-name``). ``bitmap`` is the name of a dirty bitmap reachable from the |
| 91 | block node, so the NBD client can use NBD_OPT_SET_META_CONTEXT with the |
| 92 | metadata context name "qemu:dirty-bitmap:BITMAP" to inspect the bitmap. |
| 93 | |
| 94 | The ``vhost-user-blk`` export type takes a vhost-user socket address on which |
| 95 | it accept incoming connections. Both |
| 96 | ``addr.type=unix,addr.path=<socket-path>`` for UNIX domain sockets and |
| 97 | ``addr.type=fd,addr.str=<fd>`` for file descriptor passing are supported. |
| 98 | ``logical-block-size`` sets the logical block size in bytes (the default is |
| 99 | 512). ``num-queues`` sets the number of virtqueues (the default is 1). |
| 100 | |
| 101 | The ``fuse`` export type takes a mount point, which must be a regular file, |
| 102 | on which to export the given block node. That file will not be changed, it |
| 103 | will just appear to have the block node's content while the export is active |
| 104 | (very much like mounting a filesystem on a directory does not change what the |
| 105 | directory contains, it only shows a different content while the filesystem is |
| 106 | mounted). Consequently, applications that have opened the given file before |
| 107 | the export became active will continue to see its original content. If |
| 108 | ``growable`` is set, writes after the end of the exported file will grow the |
| 109 | block node to fit. The ``allow-other`` option controls whether users other |
| 110 | than the user running the process will be allowed to access the export. Note |
| 111 | that enabling this option as a non-root user requires enabling the |
| 112 | user_allow_other option in the global fuse.conf configuration file. Setting |
| 113 | ``allow-other`` to auto (the default) will try enabling this option, and on |
| 114 | error fall back to disabling it. |
| 115 | |
| 116 | The ``vduse-blk`` export type takes a ``name`` (must be unique across the host) |
| 117 | to create the VDUSE device. |
| 118 | ``num-queues`` sets the number of virtqueues (the default is 1). |
| 119 | ``queue-size`` sets the virtqueue descriptor table size (the default is 256). |
| 120 | |
| 121 | The instantiated VDUSE device must then be added to the vDPA bus using the |
| 122 | vdpa(8) command from the iproute2 project:: |
| 123 | |
| 124 | # vdpa dev add name <id> mgmtdev vduse |
| 125 | |
| 126 | The device can be removed from the vDPA bus later as follows:: |
| 127 | |
| 128 | # vdpa dev del <id> |
| 129 | |
| 130 | For more information about attaching vDPA devices to the host with |
| 131 | virtio_vdpa.ko or attaching them to guests with vhost_vdpa.ko, see |
| 132 | https://vdpa-dev.gitlab.io/. |
| 133 | |
| 134 | For more information about VDUSE, see |
| 135 | https://docs.kernel.org/userspace-api/vduse.html. |
| 136 | |
| 137 | .. option:: --monitor MONITORDEF |
| 138 | |
| 139 | is a QMP monitor definition. See the :manpage:`qemu(1)` manual page for |
| 140 | a description of QMP monitor properties. A common QMP monitor definition |
| 141 | configures a monitor on character device ``char1``:: |
| 142 | |
| 143 | --monitor chardev=char1 |
| 144 | |
| 145 | .. option:: --nbd-server addr.type=inet,addr.host=<host>,addr.port=<port>[,tls-creds=<id>][,tls-authz=<id>][,max-connections=<n>] |
| 146 | --nbd-server addr.type=unix,addr.path=<path>[,tls-creds=<id>][,tls-authz=<id>][,max-connections=<n>] |
| 147 | --nbd-server addr.type=fd,addr.str=<fd>[,tls-creds=<id>][,tls-authz=<id>][,max-connections=<n>] |
| 148 | |
| 149 | is a server for NBD exports. Both TCP and UNIX domain sockets are supported. |
| 150 | A listen socket can be provided via file descriptor passing (see Examples |
| 151 | below). TLS encryption can be configured using ``--object`` tls-creds-* and |
| 152 | authz-* secrets (see below). |
| 153 | |
| 154 | To configure an NBD server on UNIX domain socket path |
| 155 | ``/var/run/qsd-nbd.sock``:: |
| 156 | |
| 157 | --nbd-server addr.type=unix,addr.path=/var/run/qsd-nbd.sock |
| 158 | |
| 159 | .. option:: --object help |
| 160 | --object <type>,help |
| 161 | --object <type>[,<property>=<value>...] |
| 162 | |
| 163 | is a QEMU user creatable object definition. List object types with ``help``. |
| 164 | List object properties with ``<type>,help``. See the :manpage:`qemu(1)` |
| 165 | manual page for a description of the object properties. |
| 166 | |
| 167 | .. option:: --pidfile PATH |
| 168 | |
| 169 | is the path to a file where the daemon writes its pid. This allows scripts to |
| 170 | stop the daemon by sending a signal:: |
| 171 | |
| 172 | $ kill -SIGTERM $(<path/to/qsd.pid) |
| 173 | |
| 174 | A file lock is applied to the file so only one instance of the daemon can run |
| 175 | with a given pid file path. The daemon unlinks its pid file when terminating. |
| 176 | |
| 177 | The pid file is written after chardevs, exports, and NBD servers have been |
| 178 | created but before accepting connections. The daemon has started successfully |
| 179 | when the pid file is written and clients may begin connecting. |
| 180 | |
| 181 | .. option:: --daemonize |
| 182 | |
| 183 | Daemonize the process. The parent process will exit once startup is complete |
| 184 | (i.e., after the pid file has been or would have been written) or failure |
| 185 | occurs. Its exit code reflects whether the child has started up successfully |
| 186 | or failed to do so. |
| 187 | |
| 188 | Examples |
| 189 | -------- |
| 190 | Launch the daemon with QMP monitor socket ``qmp.sock`` so clients can execute |
| 191 | QMP commands:: |
| 192 | |
| 193 | $ qemu-storage-daemon \ |
| 194 | --chardev socket,path=qmp.sock,server=on,wait=off,id=char1 \ |
| 195 | --monitor chardev=char1 |
| 196 | |
| 197 | Launch the daemon from Python with a QMP monitor socket using file descriptor |
| 198 | passing so there is no need to busy wait for the QMP monitor to become |
| 199 | available:: |
| 200 | |
| 201 | #!/usr/bin/env python3 |
| 202 | import subprocess |
| 203 | import socket |
| 204 | |
| 205 | sock_path = '/var/run/qmp.sock' |
| 206 | |
| 207 | with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as listen_sock: |
| 208 | listen_sock.bind(sock_path) |
| 209 | listen_sock.listen() |
| 210 | |
| 211 | fd = listen_sock.fileno() |
| 212 | |
| 213 | subprocess.Popen( |
| 214 | ['qemu-storage-daemon', |
| 215 | '--chardev', f'socket,fd={fd},server=on,id=char1', |
| 216 | '--monitor', 'chardev=char1'], |
| 217 | pass_fds=[fd], |
| 218 | ) |
| 219 | |
| 220 | # listen_sock was automatically closed when leaving the 'with' statement |
| 221 | # body. If the daemon process terminated early then the following connect() |
| 222 | # will fail with "Connection refused" because no process has the listen |
| 223 | # socket open anymore. Launch errors can be detected this way. |
| 224 | |
| 225 | qmp_sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) |
| 226 | qmp_sock.connect(sock_path) |
| 227 | ...QMP interaction... |
| 228 | |
| 229 | The same socket spawning approach also works with the ``--nbd-server |
| 230 | addr.type=fd,addr.str=<fd>`` and ``--export |
| 231 | type=vhost-user-blk,addr.type=fd,addr.str=<fd>`` options. |
| 232 | |
| 233 | Export raw image file ``disk.img`` over NBD UNIX domain socket ``nbd.sock``:: |
| 234 | |
| 235 | $ qemu-storage-daemon \ |
| 236 | --blockdev driver=file,node-name=disk,filename=disk.img \ |
| 237 | --nbd-server addr.type=unix,addr.path=nbd.sock \ |
| 238 | --export type=nbd,id=export,node-name=disk,writable=on |
| 239 | |
| 240 | Export a qcow2 image file ``disk.qcow2`` as a vhost-user-blk device over UNIX |
| 241 | domain socket ``vhost-user-blk.sock``:: |
| 242 | |
| 243 | $ qemu-storage-daemon \ |
| 244 | --blockdev driver=file,node-name=file,filename=disk.qcow2 \ |
| 245 | --blockdev driver=qcow2,node-name=qcow2,file=file \ |
| 246 | --export type=vhost-user-blk,id=export,addr.type=unix,addr.path=vhost-user-blk.sock,node-name=qcow2 |
| 247 | |
| 248 | Export a qcow2 image file ``disk.qcow2`` via FUSE on itself, so the disk image |
| 249 | file will then appear as a raw image:: |
| 250 | |
| 251 | $ qemu-storage-daemon \ |
| 252 | --blockdev driver=file,node-name=file,filename=disk.qcow2 \ |
| 253 | --blockdev driver=qcow2,node-name=qcow2,file=file \ |
| 254 | --export type=fuse,id=export,node-name=qcow2,mountpoint=disk.qcow2,writable=on |
| 255 | |
| 256 | See also |
| 257 | -------- |
| 258 | |
| 259 | :manpage:`qemu(1)`, :manpage:`qemu-block-drivers(7)`, :manpage:`qemu-storage-daemon-qmp-ref(7)` |