@samitouri / QOSamiQemu
Code Commits Issues Pulls Sessions CI/CD
master
master
rst 165 lines 5.7 KB
Raw
1 ==================
2 QATzip Compression
3 ==================
4 In scenarios with limited network bandwidth, the ``QATzip`` solution can help
5 users save a lot of host CPU resources by accelerating compression and
6 decompression through the Intel QuickAssist Technology(``QAT``) hardware.
7
8
9 The following test was conducted using 8 multifd channels and 10Gbps network
10 bandwidth. The results show that, compared to zstd, ``QATzip`` significantly
11 saves CPU resources on the sender and reduces migration time. Compared to the
12 uncompressed solution, ``QATzip`` greatly improves the dirty page processing
13 capability, indicated by the Pages per Second metric, and also reduces the
14 total migration time.
15
16 ::
17
18 VM Configuration: 16 vCPU and 64G memory
19 VM Workload: all vCPUs are idle and 54G memory is filled with Silesia data.
20 QAT Devices: 4
21 |-----------|--------|---------|----------|----------|------|------|
22 |8 Channels |Total |down |throughput|pages per | send | recv |
23 | |time(ms)|time(ms) |(mbps) |second | cpu %| cpu% |
24 |-----------|--------|---------|----------|----------|------|------|
25 |qatzip | 16630| 28| 10467| 2940235| 160| 360|
26 |-----------|--------|---------|----------|----------|------|------|
27 |zstd | 20165| 24| 8579| 2391465| 810| 340|
28 |-----------|--------|---------|----------|----------|------|------|
29 |none | 46063| 40| 10848| 330240| 45| 85|
30 |-----------|--------|---------|----------|----------|------|------|
31
32
33 QATzip Compression Framework
34 ============================
35
36 ``QATzip`` is a user space library which builds on top of the Intel QuickAssist
37 Technology to provide extended accelerated compression and decompression
38 services.
39
40 For more ``QATzip`` introduction, please refer to `QATzip Introduction
41 <https://github.com/intel/QATzip?tab=readme-ov-file#introductionl>`_
42
43 ::
44
45 +----------------+
46 | MultiFd Thread |
47 +-------+--------+
48 |
49 | compress/decompress
50 +-------+--------+
51 | QATzip library |
52 +-------+--------+
53 |
54 +-------+--------+
55 | QAT library |
56 +-------+--------+
57 | user space
58 --------+---------------------
59 | kernel space
60 +------+-------+
61 | QAT Driver |
62 +------+-------+
63 |
64 +------+-------+
65 | QAT Devices |
66 +--------------+
67
68
69 QATzip Installation
70 -------------------
71
72 The ``QATzip`` installation package has been integrated into some Linux
73 distributions and can be installed directly. For example, the Ubuntu Server
74 24.04 LTS system can be installed using below command
75
76 .. code-block:: shell
77
78 #apt search qatzip
79 libqatzip-dev/noble 1.2.0-0ubuntu3 amd64
80 Intel QuickAssist user space library development files
81
82 libqatzip3/noble 1.2.0-0ubuntu3 amd64
83 Intel QuickAssist user space library
84
85 qatzip/noble,now 1.2.0-0ubuntu3 amd64 [installed]
86 Compression user-space tool for Intel QuickAssist Technology
87
88 #sudo apt install libqatzip-dev libqatzip3 qatzip
89
90 If your system does not support the ``QATzip`` installation package, you can
91 use the source code to build and install, please refer to `QATzip source code installation
92 <https://github.com/intel/QATzip?tab=readme-ov-file#build-intel-quickassist-technology-driver>`_
93
94 QAT Hardware Deployment
95 -----------------------
96
97 ``QAT`` supports physical functions(PFs) and virtual functions(VFs) for
98 deployment, and users can configure ``QAT`` resources for migration according
99 to actual needs. For more details about ``QAT`` deployment, please refer to
100 `Intel QuickAssist Technology Documentation
101 <https://intel.github.io/quickassist/index.html>`_
102
103 For more ``QAT`` hardware introduction, please refer to `intel-quick-assist-technology-overview
104 <https://www.intel.com/content/www/us/en/architecture-and-technology/intel-quick-assist-technology-overview.html>`_
105
106 How To Use QATzip Compression
107 =============================
108
109 1 - Install ``QATzip`` library
110
111 2 - Build ``QEMU`` with ``--enable-qatzip`` parameter
112
113 E.g. configure --target-list=x86_64-softmmu --enable-kvm ``--enable-qatzip``
114
115 3 - Set ``migrate_set_parameter multifd-compression qatzip``
116
117 4 - Set ``migrate_set_parameter multifd-qatzip-level comp_level``, the default
118 comp_level value is 1, and it supports levels from 1 to 9
119
120 QAT Memory Requirements
121 =======================
122
123 The user needs to reserve system memory for the QAT memory management to
124 allocate DMA memory. The size of the reserved system memory depends on the
125 number of devices used for migration and the number of multifd channels.
126
127 Because memory usage depends on QAT configuration, please refer to `QAT Memory
128 Driver Queries
129 <https://intel.github.io/quickassist/PG/infrastructure_debugability.html?highlight=memory>`_
130 for memory usage calculation.
131
132 .. list-table:: An example of a PF used for migration
133 :header-rows: 1
134
135 * - Number of channels
136 - Sender memory usage
137 - Receiver memory usage
138 * - 2
139 - 10M
140 - 10M
141 * - 4
142 - 12M
143 - 14M
144 * - 8
145 - 16M
146 - 20M
147
148 How To Choose Between QATzip and QPL
149 ====================================
150 Starting from 4th Gen Intel Xeon Scalable processors, codenamed Sapphire Rapids
151 processor(``SPR``), multiple built-in accelerators are supported including
152 ``QAT`` and ``IAA``. The former can accelerate ``QATzip`` and the latter is
153 used to accelerate ``QPL``.
154
155 Here are some suggestions:
156
157 1 - If the live migration scenario is limited by network bandwidth and ``QAT``
158 hardware resources exceed ``IAA``, use the ``QATzip`` method, which can save a
159 lot of host CPU resources for compression.
160
161 2 - If the system cannot support shared virtual memory (SVM) technology, use
162 the ``QATzip`` method because ``QPL`` performance is not good without SVM
163 support.
164
165 3 - For other scenarios, use the ``QPL`` method first.