A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Doc]: improve CPU(x86) build-wheel-from-source section (#25617)
Signed-off-by: Kosseila (CloudThrill) <klouddude@gmail.com>
Clouddude committed
Sep 26, 2025 at 13:26 UTC
b761df963c2032144468a99bcb39a11e73e16ca4
1 file changed
+75
-2
docs/getting_started/installation/cpu/x86.inc.md
+75
-2
@@ -20,7 +20,80 @@ vLLM supports basic model inferencing and serving on x86 CPU platform, with data
20
# --8<-- [end:pre-built-wheels]
21
# --8<-- [start:build-wheel-from-source]
22
23
---8<-- "docs/getting_started/installation/cpu/build.inc.md"
23
+Install recommended compiler. We recommend to use `gcc/g++ >= 12.3.0` as the default compiler to avoid potential problems. For example, on Ubuntu 22.4, you can run:
24
+
25
+```bash
26
+sudo apt-get update -y
27
+sudo apt-get install -y gcc-12 g++-12 libnuma-dev python3-dev
28
+sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 10 --slave /usr/bin/g++ g++ /usr/bin/g++-12
29
+```
30
+
31
+Clone the vLLM project:
32
+
33
+```bash
34
+git clone https://github.com/vllm-project/vllm.git vllm_source
35
+cd vllm_source
36
+```
37
+
38
+Install the required dependencies:
39
+
40
+```bash
41
+uv pip install -r requirements/cpu-build.txt --torch-backend cpu
42
+uv pip install -r requirements/cpu.txt --torch-backend cpu
43
+```
44
+
45
+??? console "pip"
46
+ ```bash
47
+ pip install --upgrade pip
48
+ pip install -v -r requirements/cpu-build.txt --extra-index-url https://download.pytorch.org/whl/cpu
49
+ pip install -v -r requirements/cpu.txt --extra-index-url https://download.pytorch.org/whl/cpu
50
+ ```
51
+
52
+Build and install vLLM:
53
+
54
+```bash
55
+VLLM_TARGET_DEVICE=cpu uv pip install . --no-build-isolation
56
+```
57
+
58
+If you want to develop vLLM, install it in editable mode instead.
59
+
60
+```bash
61
+VLLM_TARGET_DEVICE=cpu uv pip install -e . --no-build-isolation
62
+```
63
+
64
+Optionally, build a portable wheel which you can then install elsewhere:
65
+
66
+```bash
67
+VLLM_TARGET_DEVICE=cpu uv build --wheel
68
+```
69
+
70
+```bash
71
+uv pip install dist/*.whl
72
+```
73
+
74
+??? console "pip"
75
+ ```bash
76
+ VLLM_TARGET_DEVICE=cpu python -m build --wheel --no-isolation
77
+ ```
78
+
79
+ ```bash
80
+ pip install dist/*.whl
81
+ ```
82
+
83
+!!! example "Troubleshooting"
84
+ - **NumPy ≥2.0 error**: Downgrade using `pip install "numpy<2.0"`.
85
+ - **CMake picks up CUDA**: Add `CMAKE_DISABLE_FIND_PACKAGE_CUDA=ON` to prevent CUDA detection during CPU builds, even if CUDA is installed.
86
+ - `AMD` requies at least 4th gen processors (Zen 4/Genoa) or higher to support [AVX512](https://www.phoronix.com/review/amd-zen4-avx512) to run vLLM on CPU.
87
+ - If you receive an error such as: `Could not find a version that satisfies the requirement torch==X.Y.Z+cpu+cpu`, consider updating [pyproject.toml](https://github.com/vllm-project/vllm/blob/main/pyproject.toml) to help pip resolve the dependency.
88
+ ```toml title="pyproject.toml"
89
+ [build-system]
90
+ requires = [
91
+ "cmake>=3.26.1",
92
+ ...
93
+ "torch==X.Y.Z+cpu" # <-------
94
+ ]
95
+ ```
96
+ - If you are building vLLM from source and not using the pre-built images, remember to set `LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libtcmalloc_minimal.so.4:$LD_PRELOAD"` on x86 machines before running vLLM.
97
98
# --8<-- [end:build-wheel-from-source]
99
# --8<-- [start:pre-built-images]
@@ -57,4 +130,4 @@ docker run --rm \
130
131
# --8<-- [end:build-image-from-source]
132
# --8<-- [start:extra-information]
60
-# --8<-- [end:extra-information]
133
+# --8<-- [end:extra-information]
\ No newline at end of file