A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 03:30 UTC
[Doc] Fix some MkDocs snippets used in the installation docs (#20572)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Harry Mellor committed
Jul 7, 2025 at 15:44 UTC
1ad69e8375e841095c2f682299be487fd9b8f47e
8 files changed
+10
-26
docs/getting_started/installation/cpu/apple.inc.md
-3
@@ -54,9 +54,6 @@ If the build has error like the following snippet where standard C++ headers can
54
```
55
56
# --8<-- [end:build-wheel-from-source]
57
-# --8<-- [start:set-up-using-docker]
58
-
59
-# --8<-- [end:set-up-using-docker]
57
# --8<-- [start:pre-built-images]
58
59
# --8<-- [end:pre-built-images]
docs/getting_started/installation/cpu/arm.inc.md
-3
@@ -28,9 +28,6 @@ ARM CPU backend currently supports Float32, FP16 and BFloat16 datatypes.
28
Testing has been conducted on AWS Graviton3 instances for compatibility.
29
30
# --8<-- [end:build-wheel-from-source]
31
-# --8<-- [start:set-up-using-docker]
32
-
33
-# --8<-- [end:set-up-using-docker]
31
# --8<-- [start:pre-built-images]
32
33
# --8<-- [end:pre-built-images]
docs/getting_started/installation/cpu/s390x.inc.md
-3
@@ -56,9 +56,6 @@ Execute the following commands to build and install vLLM from the source.
56
```
57
58
# --8<-- [end:build-wheel-from-source]
59
-# --8<-- [start:set-up-using-docker]
60
-
61
-# --8<-- [end:set-up-using-docker]
59
# --8<-- [start:pre-built-images]
60
61
# --8<-- [end:pre-built-images]
docs/getting_started/installation/cpu/x86.inc.md
-3
@@ -31,9 +31,6 @@ vLLM initially supports basic model inferencing and serving on x86 CPU platform,
31
- If you want to force enable AVX512_BF16 for the cross-compilation, please set environment variable `VLLM_CPU_AVX512BF16=1` before the building.
32
33
# --8<-- [end:build-wheel-from-source]
34
-# --8<-- [start:set-up-using-docker]
35
-
36
-# --8<-- [end:set-up-using-docker]
34
# --8<-- [start:pre-built-images]
35
36
See [https://gallery.ecr.aws/q9t5s3a7/vllm-cpu-release-repo](https://gallery.ecr.aws/q9t5s3a7/vllm-cpu-release-repo)
docs/getting_started/installation/gpu.md
+2
-2
@@ -46,11 +46,11 @@ vLLM is a Python library that supports the following GPU variants. Select your G
46
47
=== "AMD ROCm"
48
49
- There is no extra information on creating a new Python environment for this device.
49
+ --8<-- "docs/getting_started/installation/gpu/rocm.inc.md:set-up-using-python"
50
51
=== "Intel XPU"
52
53
- There is no extra information on creating a new Python environment for this device.
53
+ --8<-- "docs/getting_started/installation/gpu/xpu.inc.md:set-up-using-python"
54
55
### Pre-built wheels
56
docs/getting_started/installation/gpu/cuda.inc.md
-4
@@ -232,9 +232,6 @@ pip install -e .
232
```
233
234
# --8<-- [end:build-wheel-from-source]
235
-# --8<-- [start:set-up-using-docker]
236
-
237
-# --8<-- [end:set-up-using-docker]
235
# --8<-- [start:pre-built-images]
236
237
See [deployment-docker-pre-built-image][deployment-docker-pre-built-image] for instructions on using the official Docker image.
@@ -261,4 +258,3 @@ See [deployment-docker-build-image-from-source][deployment-docker-build-image-fr
258
See [feature-x-hardware][feature-x-hardware] compatibility matrix for feature support information.
259
260
# --8<-- [end:supported-features]
264
-# --8<-- [end:extra-information]
docs/getting_started/installation/gpu/rocm.inc.md
+6
-4
@@ -2,6 +2,9 @@
2
3
vLLM supports AMD GPUs with ROCm 6.3.
4
5
+!!! tip
6
+ [Docker](#set-up-using-docker) is the recommended way to use vLLM on ROCm.
7
+
8
!!! warning
9
There are no pre-built wheels for this device, so you must either use the pre-built Docker image or build vLLM from source.
10
@@ -14,6 +17,8 @@ vLLM supports AMD GPUs with ROCm 6.3.
17
# --8<-- [end:requirements]
18
# --8<-- [start:set-up-using-python]
19
20
+There is no extra information on creating a new Python environment for this device.
21
+
22
# --8<-- [end:set-up-using-python]
23
# --8<-- [start:pre-built-wheels]
24
@@ -123,9 +128,7 @@ Currently, there are no pre-built ROCm wheels.
128
- For MI300x (gfx942) users, to achieve optimal performance, please refer to [MI300x tuning guide](https://rocm.docs.amd.com/en/latest/how-to/tuning-guides/mi300x/index.html) for performance optimization and tuning tips on system and workflow level.
129
For vLLM, please refer to [vLLM performance optimization](https://rocm.docs.amd.com/en/latest/how-to/tuning-guides/mi300x/workload.html#vllm-performance-optimization).
130
126
-## Set up using Docker (Recommended)
127
-
128
-# --8<-- [end:set-up-using-docker]
131
+# --8<-- [end:build-wheel-from-source]
132
# --8<-- [start:pre-built-images]
133
134
The [AMD Infinity hub for vLLM](https://hub.docker.com/r/rocm/vllm/tags) offers a prebuilt, optimized
@@ -227,4 +230,3 @@ Where the `<path/to/model>` is the location where the model is stored, for examp
230
See [feature-x-hardware][feature-x-hardware] compatibility matrix for feature support information.
231
232
# --8<-- [end:supported-features]
230
-# --8<-- [end:extra-information]
docs/getting_started/installation/gpu/xpu.inc.md
+2
-4
@@ -14,6 +14,8 @@ vLLM initially supports basic model inference and serving on Intel GPU platform.
14
# --8<-- [end:requirements]
15
# --8<-- [start:set-up-using-python]
16
17
+There is no extra information on creating a new Python environment for this device.
18
+
19
# --8<-- [end:set-up-using-python]
20
# --8<-- [start:pre-built-wheels]
21
@@ -43,9 +45,6 @@ VLLM_TARGET_DEVICE=xpu python setup.py install
45
type is supported on Intel Data Center GPU, not supported on Intel Arc GPU yet.
46
47
# --8<-- [end:build-wheel-from-source]
46
-# --8<-- [start:set-up-using-docker]
47
-
48
-# --8<-- [end:set-up-using-docker]
48
# --8<-- [start:pre-built-images]
49
50
Currently, there are no pre-built XPU images.
@@ -86,4 +85,3 @@ By default, a ray instance will be launched automatically if no existing one is
85
XPU platform uses **torch-ccl** for torch<2.8 and **xccl** for torch>=2.8 as distributed backend, since torch 2.8 supports **xccl** as built-in backend for XPU.
86
87
# --8<-- [end:distributed-backend]
89
-# --8<-- [end:extra-information]