| 1 | ============================== |
| 2 | VMWare PVSCSI Device Interface |
| 3 | ============================== |
| 4 | |
| 5 | .. |
| 6 | Created by Dmitry Fleytman (dmitry@daynix.com), Daynix Computing LTD. |
| 7 | |
| 8 | This document describes the VMWare PVSCSI device interface specification, |
| 9 | based on the source code of the PVSCSI Linux driver from kernel 3.0.4. |
| 10 | |
| 11 | Overview |
| 12 | ======== |
| 13 | |
| 14 | The interface is based on a memory area shared between hypervisor and VM. |
| 15 | The memory area is obtained by driver as a device IO memory resource of |
| 16 | ``PVSCSI_MEM_SPACE_SIZE`` length. |
| 17 | The shared memory consists of a registers area and a rings area. |
| 18 | The registers area is used to raise hypervisor interrupts and issue device |
| 19 | commands. The rings area is used to transfer data descriptors and SCSI |
| 20 | commands from VM to hypervisor and to transfer messages produced by |
| 21 | hypervisor to VM. Data itself is transferred via virtual scatter-gather DMA. |
| 22 | |
| 23 | PVSCSI Device Registers |
| 24 | ======================= |
| 25 | |
| 26 | The length of the registers area is 1 page |
| 27 | (``PVSCSI_MEM_SPACE_COMMAND_NUM_PAGES``). The structure of the |
| 28 | registers area is described by the ``PVSCSIRegOffset`` enum. There |
| 29 | are registers to issue device commands (with optional short data), |
| 30 | issue device interrupts, and control interrupt masking. |
| 31 | |
| 32 | PVSCSI Device Rings |
| 33 | =================== |
| 34 | |
| 35 | There are three rings in shared memory: |
| 36 | |
| 37 | Request ring (``struct PVSCSIRingReqDesc *req_ring``) |
| 38 | ring for OS to device requests |
| 39 | |
| 40 | Completion ring (``struct PVSCSIRingCmpDesc *cmp_ring``) |
| 41 | ring for device request completions |
| 42 | |
| 43 | Message ring (``struct PVSCSIRingMsgDesc *msg_ring``) |
| 44 | ring for messages from device. This ring is optional and the |
| 45 | guest might not configure it. |
| 46 | |
| 47 | There is a control area (``struct PVSCSIRingsState *rings_state``) |
| 48 | used to control rings operation. |
| 49 | |
| 50 | PVSCSI Device to Host Interrupts |
| 51 | ================================ |
| 52 | |
| 53 | The following interrupt types are supported by the PVSCSI device: |
| 54 | |
| 55 | Completion interrupts (completion ring notifications): |
| 56 | |
| 57 | - ``PVSCSI_INTR_CMPL_0`` |
| 58 | - ``PVSCSI_INTR_CMPL_1`` |
| 59 | |
| 60 | Message interrupts (message ring notifications): |
| 61 | |
| 62 | - ``PVSCSI_INTR_MSG_0`` |
| 63 | - ``PVSCSI_INTR_MSG_1`` |
| 64 | |
| 65 | Interrupts are controlled via the ``PVSCSI_REG_OFFSET_INTR_MASK`` |
| 66 | register. If a bit is set it means the interrupt is enabled, and if |
| 67 | it is clear then the interrupt is disabled. |
| 68 | |
| 69 | The interrupt modes supported are legacy, MSI and MSI-X. |
| 70 | In the case of legacy interrupts, the ``PVSCSI_REG_OFFSET_INTR_STATUS`` |
| 71 | register is used to check which interrupt has arrived. Interrupts are |
| 72 | acknowledged when the corresponding bit is written to the interrupt |
| 73 | status register. |
| 74 | |
| 75 | PVSCSI Device Operation Sequences |
| 76 | ================================= |
| 77 | |
| 78 | Startup sequence |
| 79 | ---------------- |
| 80 | |
| 81 | a. Issue ``PVSCSI_CMD_ADAPTER_RESET`` command |
| 82 | b. Windows driver reads interrupt status register here |
| 83 | c. Issue ``PVSCSI_CMD_SETUP_MSG_RING`` command with no additional data, |
| 84 | check status and disable device messages if error returned |
| 85 | (Omitted if device messages disabled by driver configuration) |
| 86 | d. Issue ``PVSCSI_CMD_SETUP_RINGS`` command, provide rings configuration |
| 87 | as ``struct PVSCSICmdDescSetupRings`` |
| 88 | e. Issue ``PVSCSI_CMD_SETUP_MSG_RING`` command again, provide |
| 89 | rings configuration as ``struct PVSCSICmdDescSetupMsgRing`` |
| 90 | f. Unmask completion and message (if device messages enabled) interrupts |
| 91 | |
| 92 | Shutdown sequence |
| 93 | ----------------- |
| 94 | |
| 95 | a. Mask interrupts |
| 96 | b. Flush request ring using ``PVSCSI_REG_OFFSET_KICK_NON_RW_IO`` |
| 97 | c. Issue ``PVSCSI_CMD_ADAPTER_RESET`` command |
| 98 | |
| 99 | Send request |
| 100 | ------------ |
| 101 | |
| 102 | a. Fill next free request ring descriptor |
| 103 | b. Issue ``PVSCSI_REG_OFFSET_KICK_RW_IO`` for R/W operations |
| 104 | or ``PVSCSI_REG_OFFSET_KICK_NON_RW_IO`` for other operations |
| 105 | |
| 106 | Abort command |
| 107 | ------------- |
| 108 | |
| 109 | a. Issue ``PVSCSI_CMD_ABORT_CMD`` command |
| 110 | |
| 111 | Request completion processing |
| 112 | ----------------------------- |
| 113 | |
| 114 | a. Upon completion interrupt arrival process completion |
| 115 | and message (if enabled) rings |