A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 17:00 UTC
[Doc] Add oneDNN installation to CPU backend documentation (#8467)
Isotr0py committed
Sep 14, 2024 at 02:06 UTC
f57092c00b53d6da887f2b8071af332d42ccb6d4
1 file changed
+14
docs/source/getting_started/cpu-installation.rst
+14
@@ -59,6 +59,20 @@ Build from source
59
$ pip install wheel packaging ninja "setuptools>=49.4.0" numpy
60
$ pip install -v -r requirements-cpu.txt --extra-index-url https://download.pytorch.org/whl/cpu
61
62
+- Third, build and install oneDNN library from source:
63
+
64
+.. code-block:: console
65
+
66
+ $ git clone -b rls-v3.5 https://github.com/oneapi-src/oneDNN.git
67
+ $ cmake -B ./oneDNN/build -S ./oneDNN -G Ninja -DONEDNN_LIBRARY_TYPE=STATIC \
68
+ -DONEDNN_BUILD_DOC=OFF \
69
+ -DONEDNN_BUILD_EXAMPLES=OFF \
70
+ -DONEDNN_BUILD_TESTS=OFF \
71
+ -DONEDNN_BUILD_GRAPH=OFF \
72
+ -DONEDNN_ENABLE_WORKLOAD=INFERENCE \
73
+ -DONEDNN_ENABLE_PRIMITIVE=MATMUL
74
+ $ cmake --build ./oneDNN/build --target install --config Release
75
+
76
- Finally, build and install vLLM CPU backend:
77
78
.. code-block:: console