A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Docs] Move build.inc into arm.inc (#26862)
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
Michael Yao committed
Oct 15, 2025 at 11:35 UTC
c43ca8259effad1735f0cf8821204247b6ac70ea
2 files changed
+40
-45
docs/getting_started/installation/cpu/arm.inc.md
+40
-1
@@ -23,7 +23,46 @@ ARM CPU backend currently supports Float32, FP16 and BFloat16 datatypes.
23
# --8<-- [end:pre-built-wheels]
24
# --8<-- [start:build-wheel-from-source]
25
26
---8<-- "docs/getting_started/installation/cpu/build.inc.md:extra-information"
26
+First, install the recommended compiler. We recommend using `gcc/g++ >= 12.3.0` as the default compiler to avoid potential problems. For example, on Ubuntu 22.4, you can run:
27
+
28
+```bash
29
+sudo apt-get update -y
30
+sudo apt-get install -y --no-install-recommends ccache git curl wget ca-certificates gcc-12 g++-12 libtcmalloc-minimal4 libnuma-dev ffmpeg libsm6 libxext6 libgl1 jq lsof
31
+sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 10 --slave /usr/bin/g++ g++ /usr/bin/g++-12
32
+```
33
+
34
+Second, clone the vLLM project:
35
+
36
+```bash
37
+git clone https://github.com/vllm-project/vllm.git vllm_source
38
+cd vllm_source
39
+```
40
+
41
+Third, install required dependencies:
42
+
43
+```bash
44
+uv pip install -r requirements/cpu-build.txt --torch-backend cpu
45
+uv pip install -r requirements/cpu.txt --torch-backend cpu
46
+```
47
+
48
+??? console "pip"
49
+ ```bash
50
+ pip install --upgrade pip
51
+ pip install -v -r requirements/cpu-build.txt --extra-index-url https://download.pytorch.org/whl/cpu
52
+ pip install -v -r requirements/cpu.txt --extra-index-url https://download.pytorch.org/whl/cpu
53
+ ```
54
+
55
+Finally, build and install vLLM:
56
+
57
+```bash
58
+VLLM_TARGET_DEVICE=cpu uv pip install . --no-build-isolation
59
+```
60
+
61
+If you want to develop vLLM, install it in editable mode instead.
62
+
63
+```bash
64
+VLLM_TARGET_DEVICE=cpu uv pip install -e . --no-build-isolation
65
+```
66
67
Testing has been conducted on AWS Graviton3 instances for compatibility.
68
docs/getting_started/installation/cpu/build.inc.md
deleted
-44
@@ -1,44 +0,0 @@
1
-# --8<-- [start:extra-information]
2
-
3
-First, install the recommended compiler. We recommend using `gcc/g++ >= 12.3.0` as the default compiler to avoid potential problems. For example, on Ubuntu 22.4, you can run:
4
-
5
-```bash
6
-sudo apt-get update -y
7
-sudo apt-get install -y --no-install-recommends ccache git curl wget ca-certificates gcc-12 g++-12 libtcmalloc-minimal4 libnuma-dev ffmpeg libsm6 libxext6 libgl1 jq lsof
8
-sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 10 --slave /usr/bin/g++ g++ /usr/bin/g++-12
9
-```
10
-
11
-Second, clone the vLLM project:
12
-
13
-```bash
14
-git clone https://github.com/vllm-project/vllm.git vllm_source
15
-cd vllm_source
16
-```
17
-
18
-Third, install required dependencies:
19
-
20
-```bash
21
-uv pip install -r requirements/cpu-build.txt --torch-backend cpu
22
-uv pip install -r requirements/cpu.txt --torch-backend cpu
23
-```
24
-
25
-??? console "pip"
26
- ```bash
27
- pip install --upgrade pip
28
- pip install -v -r requirements/cpu-build.txt --extra-index-url https://download.pytorch.org/whl/cpu
29
- pip install -v -r requirements/cpu.txt --extra-index-url https://download.pytorch.org/whl/cpu
30
- ```
31
-
32
-Finally, build and install vLLM:
33
-
34
-```bash
35
-VLLM_TARGET_DEVICE=cpu python setup.py install
36
-```
37
-
38
-If you want to develop vLLM, install it in editable mode instead.
39
-
40
-```bash
41
-VLLM_TARGET_DEVICE=cpu python setup.py develop
42
-```
43
-
44
-# --8<-- [end:extra-information]