A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 22:35 UTC
[Docs] use `uv` in CPU installation docs (#22089)
Signed-off-by: David Xia <david@davidxia.com>
David Xia committed
Aug 1, 2025 at 10:55 UTC
97608dc276c292d9217eb6d334d969c5e89913c6
3 files changed
+48
-31
docs/getting_started/installation/cpu/apple.inc.md
+6
-6
@@ -1,6 +1,6 @@
1
# --8<-- [start:installation]
2
3
-vLLM has experimental support for macOS with Apple silicon. For now, users shall build from the source vLLM to natively run on macOS.
3
+vLLM has experimental support for macOS with Apple silicon. For now, users must build from source to natively run on macOS.
4
5
Currently the CPU implementation for macOS supports FP32 and FP16 datatypes.
6
@@ -23,20 +23,20 @@ Currently the CPU implementation for macOS supports FP32 and FP16 datatypes.
23
# --8<-- [end:pre-built-wheels]
24
# --8<-- [start:build-wheel-from-source]
25
26
-After installation of XCode and the Command Line Tools, which include Apple Clang, execute the following commands to build and install vLLM from the source.
26
+After installation of XCode and the Command Line Tools, which include Apple Clang, execute the following commands to build and install vLLM from source.
27
28
```bash
29
git clone https://github.com/vllm-project/vllm.git
30
cd vllm
31
-pip install -r requirements/cpu.txt
32
-pip install -e .
31
+uv pip install -r requirements/cpu.txt
32
+uv pip install -e .
33
```
34
35
!!! note
36
- On macOS the `VLLM_TARGET_DEVICE` is automatically set to `cpu`, which currently is the only supported device.
36
+ On macOS the `VLLM_TARGET_DEVICE` is automatically set to `cpu`, which is currently the only supported device.
37
38
!!! example "Troubleshooting"
39
- If the build has error like the following snippet where standard C++ headers cannot be found, try to remove and reinstall your
39
+ If the build fails with errors like the following where standard C++ headers cannot be found, try to remove and reinstall your
40
[Command Line Tools for Xcode](https://developer.apple.com/download/all/).
41
42
```text
docs/getting_started/installation/cpu/build.inc.md
+14
-8
@@ -1,4 +1,4 @@
1
-First, install recommended compiler. We recommend to use `gcc/g++ >= 12.3.0` as the default compiler to avoid potential problems. For example, on Ubuntu 22.4, you can run:
1
+First, install the recommended compiler. We recommend using `gcc/g++ >= 12.3.0` as the default compiler to avoid potential problems. For example, on Ubuntu 22.4, you can run:
2
3
```bash
4
sudo apt-get update -y
@@ -6,28 +6,34 @@ sudo apt-get install -y --no-install-recommends ccache git curl wget ca-certific
6
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 10 --slave /usr/bin/g++ g++ /usr/bin/g++-12
7
```
8
9
-Second, clone vLLM project:
9
+Second, clone the vLLM project:
10
11
```bash
12
git clone https://github.com/vllm-project/vllm.git vllm_source
13
cd vllm_source
14
```
15
16
-Third, install Python packages for vLLM CPU backend building:
16
+Third, install required dependencies:
17
18
```bash
19
-pip install --upgrade pip
20
-pip install -v -r requirements/cpu-build.txt --extra-index-url https://download.pytorch.org/whl/cpu
21
-pip install -v -r requirements/cpu.txt --extra-index-url https://download.pytorch.org/whl/cpu
19
+uv pip install -r requirements/cpu-build.txt --torch-backend auto
20
+uv pip install -r requirements/cpu.txt --torch-backend auto
21
```
22
24
-Finally, build and install vLLM CPU backend:
23
+??? console "pip"
24
+ ```bash
25
+ pip install --upgrade pip
26
+ pip install -v -r requirements/cpu-build.txt --extra-index-url https://download.pytorch.org/whl/cpu
27
+ pip install -v -r requirements/cpu.txt --extra-index-url https://download.pytorch.org/whl/cpu
28
+ ```
29
+
30
+Finally, build and install vLLM:
31
32
```bash
33
VLLM_TARGET_DEVICE=cpu python setup.py install
34
```
35
30
-If you want to develop vllm, install it in editable mode instead.
36
+If you want to develop vLLM, install it in editable mode instead.
37
38
```bash
39
VLLM_TARGET_DEVICE=cpu python setup.py develop
docs/getting_started/installation/cpu/s390x.inc.md
+28
-17
@@ -1,6 +1,6 @@
1
# --8<-- [start:installation]
2
3
-vLLM has experimental support for s390x architecture on IBM Z platform. For now, users shall build from the vLLM source to natively run on IBM Z platform.
3
+vLLM has experimental support for s390x architecture on IBM Z platform. For now, users must build from source to natively run on IBM Z platform.
4
5
Currently the CPU implementation for s390x architecture supports FP32 datatype only.
6
@@ -40,21 +40,32 @@ curl https://sh.rustup.rs -sSf | sh -s -- -y && \
40
. "$HOME/.cargo/env"
41
```
42
43
-Execute the following commands to build and install vLLM from the source.
43
+Execute the following commands to build and install vLLM from source.
44
45
!!! tip
46
- Please build the following dependencies, `torchvision`, `pyarrow` from the source before building vLLM.
46
+ Please build the following dependencies, `torchvision`, `pyarrow` from source before building vLLM.
47
48
```bash
49
sed -i '/^torch/d' requirements-build.txt # remove torch from requirements-build.txt since we use nightly builds
50
- pip install -v \
51
- --extra-index-url https://download.pytorch.org/whl/nightly/cpu \
50
+ uv pip install -v \
51
+ --torch-backend auto \
52
-r requirements-build.txt \
53
-r requirements-cpu.txt \
54
VLLM_TARGET_DEVICE=cpu python setup.py bdist_wheel && \
55
- pip install dist/*.whl
55
+ uv pip install dist/*.whl
56
```
57
58
+??? console "pip"
59
+ ```bash
60
+ sed -i '/^torch/d' requirements-build.txt # remove torch from requirements-build.txt since we use nightly builds
61
+ pip install -v \
62
+ --extra-index-url https://download.pytorch.org/whl/nightly/cpu \
63
+ -r requirements-build.txt \
64
+ -r requirements-cpu.txt \
65
+ VLLM_TARGET_DEVICE=cpu python setup.py bdist_wheel && \
66
+ pip install dist/*.whl
67
+ ```
68
+
69
# --8<-- [end:build-wheel-from-source]
70
# --8<-- [start:pre-built-images]
71
@@ -63,19 +74,19 @@ Execute the following commands to build and install vLLM from the source.
74
75
```bash
76
docker build -f docker/Dockerfile.s390x \
66
- --tag vllm-cpu-env .
77
+ --tag vllm-cpu-env .
78
68
-# Launching OpenAI server
79
+# Launch OpenAI server
80
docker run --rm \
70
- --privileged=true \
71
- --shm-size=4g \
72
- -p 8000:8000 \
73
- -e VLLM_CPU_KVCACHE_SPACE=<KV cache space> \
74
- -e VLLM_CPU_OMP_THREADS_BIND=<CPU cores for inference> \
75
- vllm-cpu-env \
76
- --model=meta-llama/Llama-3.2-1B-Instruct \
77
- --dtype=float \
78
- other vLLM OpenAI server arguments
81
+ --privileged true \
82
+ --shm-size 4g \
83
+ -p 8000:8000 \
84
+ -e VLLM_CPU_KVCACHE_SPACE=<KV cache space> \
85
+ -e VLLM_CPU_OMP_THREADS_BIND=<CPU cores for inference> \
86
+ vllm-cpu-env \
87
+ --model meta-llama/Llama-3.2-1B-Instruct \
88
+ --dtype float \
89
+ other vLLM OpenAI server arguments
90
```
91
92
# --8<-- [end:build-image-from-source]