| 1 | 'nitro-enclave' virtual machine (``nitro-enclave``) |
| 2 | =================================================== |
| 3 | |
| 4 | ``nitro-enclave`` is a machine type which emulates an *AWS nitro enclave* |
| 5 | virtual machine. `AWS nitro enclaves`_ is an Amazon EC2 feature that allows |
| 6 | creating isolated execution environments, called enclaves, from Amazon EC2 |
| 7 | instances which are used for processing highly sensitive data. Enclaves have |
| 8 | no persistent storage and no external networking. The enclave VMs are based |
| 9 | on Firecracker microvm with a vhost-vsock device for communication with the |
| 10 | parent EC2 instance that spawned it and a Nitro Secure Module (NSM) device |
| 11 | for cryptographic attestation. The parent instance VM always has CID 3 while |
| 12 | the enclave VM gets a dynamic CID. Enclaves use an EIF (`Enclave Image Format`_) |
| 13 | file which contains the necessary kernel, cmdline and ramdisk(s) to boot. |
| 14 | |
| 15 | In QEMU, ``nitro-enclave`` is a machine type based on ``microvm`` similar to how |
| 16 | AWS nitro enclaves look like a `Firecracker`_ microvm. This is useful for |
| 17 | local testing of EIF files using QEMU instead of running real AWS Nitro Enclaves |
| 18 | which can be difficult for debugging due to its roots in security. The vsock |
| 19 | device emulation is done using vhost-user-vsock which means another process that |
| 20 | can do the userspace emulation, like `vhost-device-vsock`_ from rust-vmm crate, |
| 21 | must be run alongside nitro-enclave for the vsock communication to work. |
| 22 | |
| 23 | ``libcbor`` and ``gnutls`` are required dependencies for nitro-enclave machine |
| 24 | support to be added when building QEMU from source. |
| 25 | |
| 26 | .. _AWS nitro enclaves: https://docs.aws.amazon.com/enclaves/latest/user/nitro-enclave.html |
| 27 | .. _Enclave Image Format: https://github.com/aws/aws-nitro-enclaves-image-format |
| 28 | .. _vhost-device-vsock: https://github.com/rust-vmm/vhost-device/tree/main/vhost-device-vsock |
| 29 | .. _Firecracker: https://firecracker-microvm.github.io |
| 30 | |
| 31 | Using the nitro-enclave machine type |
| 32 | ------------------------------------ |
| 33 | |
| 34 | Machine-specific options |
| 35 | ~~~~~~~~~~~~~~~~~~~~~~~~ |
| 36 | |
| 37 | It supports the following machine-specific options: |
| 38 | |
| 39 | - nitro-enclave.vsock=string (required) (Id of the chardev from '-chardev' option that vhost-user-vsock device will use) |
| 40 | - nitro-enclave.id=string (optional) (Set enclave identifier) |
| 41 | - nitro-enclave.parent-role=string (optional) (Set parent instance IAM role ARN) |
| 42 | - nitro-enclave.parent-id=string (optional) (Set parent instance identifier) |
| 43 | |
| 44 | |
| 45 | Running a nitro-enclave VM |
| 46 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 47 | |
| 48 | First, run `vhost-device-vsock`__ (or a similar tool that supports vhost-user-vsock). |
| 49 | The forward-cid option below with value 1 forwards all connections from the enclave |
| 50 | VM to the host machine and the forward-listen (port numbers separated by '+') is used |
| 51 | for forwarding connections from the host machine to the enclave VM:: |
| 52 | |
| 53 | $ vhost-device-vsock \ |
| 54 | --vm guest-cid=4,forward-cid=1,forward-listen=9001+9002,socket=/tmp/vhost4.socket |
| 55 | |
| 56 | __ https://github.com/rust-vmm/vhost-device/tree/main/vhost-device-vsock#using-the-vsock-backend |
| 57 | |
| 58 | Now run the necessary applications on the host machine so that the nitro-enclave VM |
| 59 | applications' vsock communication works. For example, the nitro-enclave VM's init |
| 60 | process connects to CID 3 and sends a single byte hello heartbeat (0xB7) to let the |
| 61 | parent VM know that it booted expecting a heartbeat (0xB7) response. So you must run |
| 62 | a AF_VSOCK server on the host machine that listens on port 9000 and sends the heartbeat |
| 63 | after it receives the heartbeat for enclave VM to boot successfully. You should run all |
| 64 | the applications on the host machine that would typically be running in the parent EC2 |
| 65 | VM for successful communication with the enclave VM. |
| 66 | |
| 67 | Then run the nitro-enclave VM using the following command where ``hello.eif`` is |
| 68 | an EIF file you would use to spawn a real AWS nitro enclave virtual machine:: |
| 69 | |
| 70 | $ qemu-system-x86_64 -M nitro-enclave,vsock=c,id=hello-world \ |
| 71 | -kernel hello-world.eif -nographic -m 4G --enable-kvm -cpu host \ |
| 72 | -chardev socket,id=c,path=/tmp/vhost4.socket |
| 73 | |
| 74 | In this example, the nitro-enclave VM has CID 4. If there are applications that |
| 75 | connect to the enclave VM, run them on the host machine after enclave VM starts. |
| 76 | You need to modify the applications to connect to CID 1 (instead of the enclave |
| 77 | VM's CID) and use the forward-listen (e.g., 9001+9002) option of vhost-device-vsock |
| 78 | to forward the ports they connect to. |