| 1 | AWS Nitro Enclaves |
| 2 | ================== |
| 3 | |
| 4 | `AWS Nitro Enclaves <https://aws.amazon.com/ec2/nitro/nitro-enclaves/>`_ |
| 5 | are isolated compute environments that run alongside EC2 instances. |
| 6 | They are created by partitioning CPU and memory resources from a parent |
| 7 | instance and launching a signed Enclave Image Format (EIF) file inside |
| 8 | a confidential VM managed by the Nitro Hypervisor. |
| 9 | |
| 10 | QEMU supports launching Nitro Enclaves on EC2 instances that have |
| 11 | enclave support enabled, using the ``nitro`` accelerator and the |
| 12 | ``nitro`` machine type. |
| 13 | |
| 14 | Prerequisites |
| 15 | ------------- |
| 16 | |
| 17 | * An EC2 instance with Nitro Enclaves enabled |
| 18 | * The ``nitro_enclaves`` kernel module loaded (provides ``/dev/nitro_enclaves``) |
| 19 | * CPU cores allocated to the Nitro Enclaves pool via ``nitro-enclaves-allocator`` |
| 20 | * Huge pages allocated for Nitro Enclaves via ``nitro-enclaves-allocator`` |
| 21 | |
| 22 | Quick Start |
| 23 | ----------- |
| 24 | |
| 25 | Launch a Nitro Enclave from a pre-built EIF file:: |
| 26 | |
| 27 | $ qemu-system-x86_64 -accel nitro,debug-mode=on -M nitro -nographic \ |
| 28 | -smp 2 -m 512M -kernel enclave.eif |
| 29 | |
| 30 | Launch an enclave from individual kernel and initrd files:: |
| 31 | |
| 32 | $ qemu-system-x86_64 -accel nitro,debug-mode=on -M nitro -nographic \ |
| 33 | -smp 2 -m 512M -kernel vmlinuz -initrd initrd.cpio \ |
| 34 | -append "console=ttyS0" |
| 35 | |
| 36 | The same commands work with ``qemu-system-aarch64`` on Graviton based EC2 |
| 37 | instances. |
| 38 | |
| 39 | Accelerator |
| 40 | ----------- |
| 41 | |
| 42 | The ``nitro`` accelerator (``-accel nitro``) drives the |
| 43 | ``/dev/nitro_enclaves`` device to create and manage a Nitro Enclave. |
| 44 | It handles: |
| 45 | |
| 46 | * Creating the enclave VM slot |
| 47 | * Donating memory regions (must be huge page backed) |
| 48 | * Adding vCPUs (must be full physical cores) |
| 49 | * Starting the enclave |
| 50 | * Notifying vsock bus devices of the enclave CID |
| 51 | |
| 52 | Accelerator options: |
| 53 | |
| 54 | ``debug-mode=on|off`` |
| 55 | Enable debug mode. When enabled, the Nitro Hypervisor exposes the |
| 56 | enclave's serial console output via a vsock port that the machine |
| 57 | model automatically connects to. In debug mode, PCR values are zero. |
| 58 | Default is ``off``. |
| 59 | |
| 60 | Machine |
| 61 | ------- |
| 62 | |
| 63 | The ``nitro`` machine (``-M nitro``) is a minimal, architecture-independent |
| 64 | machine that provides only what a Nitro Enclave needs: |
| 65 | |
| 66 | * RAM (huge page backed via memfd) |
| 67 | * vCPUs (defaults to ``host`` CPU type) |
| 68 | * A Nitro vsock bus with: |
| 69 | |
| 70 | - A heartbeat device (vsock server on port 9000) |
| 71 | - A serial console bridge (vsock client, debug mode only) |
| 72 | |
| 73 | Communication to the Nitro Enclave is limited to virtio-vsock. The Enclave |
| 74 | is allocated a CID at launch at which it is reachable. A specific CID can |
| 75 | be requested with ``-accel nitro,enclave-cid=<N>`` (0 lets the hypervisor |
| 76 | choose). The assigned CID is readable from the vsock bridge device:: |
| 77 | |
| 78 | (qemu) qom-get /machine/peripheral/nitro-vsock enclave-cid |
| 79 | |
| 80 | EIF Image Format |
| 81 | ^^^^^^^^^^^^^^^^ |
| 82 | |
| 83 | Nitro Enclaves boot from EIF (Enclave Image Format) files. When |
| 84 | ``-kernel`` points to an EIF file (detected by the ``.eif`` magic |
| 85 | bytes), it is loaded directly into guest memory. |
| 86 | |
| 87 | When ``-kernel`` points to a regular kernel image (e.g. a bzImage or |
| 88 | Image), the machine automatically assembles a minimal EIF on the fly |
| 89 | from ``-kernel``, ``-initrd``, and ``-append``. This allows standard |
| 90 | direct kernel boot without external EIF tooling. |
| 91 | |
| 92 | CPU Requirements |
| 93 | ^^^^^^^^^^^^^^^^ |
| 94 | |
| 95 | Nitro Enclaves require full physical CPU cores. On hyperthreaded |
| 96 | systems, this means ``-smp`` must be a multiple of the threads per |
| 97 | core (typically 2). |
| 98 | |
| 99 | Nitro Enclaves can only consume cores that are donated to the Nitro Enclave |
| 100 | CPU pool. You can configure the CPU pool using the ``nitro-enclaves-allocator`` |
| 101 | tool or manually by writing to the nitro_enclaves cpu pool parameter. To |
| 102 | allocate vCPUs 1, 2 and 3, you can call:: |
| 103 | |
| 104 | $ echo 1,2,3 | sudo tee /sys/module/nitro_enclaves/parameters/ne_cpus |
| 105 | |
| 106 | Beware that on x86-64 systems, hyperthread siblings are not consecutive |
| 107 | and must be added in pairs to the pool. Consult tools like ``lstopo`` |
| 108 | or ``lscpu`` for details about your instance's CPU topology. |
| 109 | |
| 110 | Memory Requirements |
| 111 | ^^^^^^^^^^^^^^^^^^^ |
| 112 | |
| 113 | Enclave memory must be huge page backed. The machine automatically |
| 114 | creates a memfd memory backend with huge pages enabled. To make the |
| 115 | huge page allocation work, ensure that huge pages are reserved in |
| 116 | the system. To reserve 1 GiB of memory on a 4 KiB PAGE_SIZE system, |
| 117 | you can call:: |
| 118 | |
| 119 | $ echo 512 | sudo tee /proc/sys/vm/nr_hugepages |
| 120 | |
| 121 | Emulated Nitro Enclaves |
| 122 | ----------------------- |
| 123 | |
| 124 | In addition to the native Nitro Enclaves invocation, you can also use |
| 125 | the emulated nitro-enclave machine target (see :doc:`i386/nitro-enclave`) |
| 126 | which implements the x86 Nitro Enclave device model. While -M nitro |
| 127 | delegates virtual machine device emulation to the Nitro Hypervisor, -M |
| 128 | nitro-enclave implements all devices itself, which means it also works |
| 129 | on non-EC2 instances. |
| 130 | |
| 131 | If you require NSM based attestation backed by valid AWS certificates, |
| 132 | you must use -M nitro. The -M nitro-enclave model does not provide |
| 133 | you with an AWS signed attestation document. |