@tduncan / Nvidia-Optimization-Stack-VLLM
Code Commits Issues Pulls Cloud Agent CI/CD
main
main
md 169 lines 4.07 KB
Rendered Raw
1 ---
2 toc_depth: 3
3 ---
4
5 # GPU
6
7 vLLM is a Python library that supports the following GPU variants. Select your GPU type to see vendor specific instructions:
8
9 === "NVIDIA CUDA"
10
11 --8<-- "docs/getting_started/installation/gpu.cuda.inc.md:installation"
12
13 === "AMD ROCm"
14
15 --8<-- "docs/getting_started/installation/gpu.rocm.inc.md:installation"
16
17 === "Intel XPU"
18
19 --8<-- "docs/getting_started/installation/gpu.xpu.inc.md:installation"
20
21 === "Apple Silicon"
22
23 --8<-- "docs/getting_started/installation/gpu.apple.inc.md:installation"
24
25 ## Requirements
26
27 - OS: Linux
28 - Python: 3.10 -- 3.13
29
30 !!! note
31 vLLM does not support Windows natively. To run vLLM on Windows, you can use the Windows Subsystem for Linux (WSL) with a compatible Linux distribution, or use some community-maintained forks, e.g. [https://github.com/SystemPanic/vllm-windows](https://github.com/SystemPanic/vllm-windows).
32
33 === "NVIDIA CUDA"
34
35 --8<-- "docs/getting_started/installation/gpu.cuda.inc.md:requirements"
36
37 === "AMD ROCm"
38
39 --8<-- "docs/getting_started/installation/gpu.rocm.inc.md:requirements"
40
41 === "Intel XPU"
42
43 --8<-- "docs/getting_started/installation/gpu.xpu.inc.md:requirements"
44
45 === "Apple Silicon"
46
47 --8<-- "docs/getting_started/installation/gpu.apple.inc.md:requirements"
48
49 ## Set up using Python
50
51 ### Create a new Python environment
52
53 --8<-- "docs/getting_started/installation/python_env_setup.inc.md"
54
55 === "NVIDIA CUDA"
56
57 --8<-- "docs/getting_started/installation/gpu.cuda.inc.md:set-up-using-python"
58
59 === "AMD ROCm"
60
61 --8<-- "docs/getting_started/installation/gpu.rocm.inc.md:set-up-using-python"
62
63 === "Intel XPU"
64
65 --8<-- "docs/getting_started/installation/gpu.xpu.inc.md:set-up-using-python"
66
67 === "Apple Silicon"
68
69 --8<-- "docs/getting_started/installation/gpu.apple.inc.md:set-up-using-python"
70
71 ### Pre-built wheels {#pre-built-wheels}
72
73 === "NVIDIA CUDA"
74
75 --8<-- "docs/getting_started/installation/gpu.cuda.inc.md:pre-built-wheels"
76
77 === "AMD ROCm"
78
79 --8<-- "docs/getting_started/installation/gpu.rocm.inc.md:pre-built-wheels"
80
81 === "Intel XPU"
82
83 --8<-- "docs/getting_started/installation/gpu.xpu.inc.md:pre-built-wheels"
84
85 === "Apple Silicon"
86
87 --8<-- "docs/getting_started/installation/gpu.apple.inc.md:pre-built-wheels"
88
89 ### Build wheel from source
90
91 === "NVIDIA CUDA"
92
93 --8<-- "docs/getting_started/installation/gpu.cuda.inc.md:build-wheel-from-source"
94
95 === "AMD ROCm"
96
97 --8<-- "docs/getting_started/installation/gpu.rocm.inc.md:build-wheel-from-source"
98
99 === "Intel XPU"
100
101 --8<-- "docs/getting_started/installation/gpu.xpu.inc.md:build-wheel-from-source"
102
103 === "Apple Silicon"
104
105 --8<-- "docs/getting_started/installation/gpu.apple.inc.md:build-wheel-from-source"
106
107 ## Set up using Docker
108
109 ### Pre-built images
110
111 --8<-- [start:pre-built-images]
112
113 === "NVIDIA CUDA"
114
115 --8<-- "docs/getting_started/installation/gpu.cuda.inc.md:pre-built-images"
116
117 === "AMD ROCm"
118
119 --8<-- "docs/getting_started/installation/gpu.rocm.inc.md:pre-built-images"
120
121 === "Intel XPU"
122
123 --8<-- "docs/getting_started/installation/gpu.xpu.inc.md:pre-built-images"
124
125 === "Apple Silicon"
126
127 --8<-- "docs/getting_started/installation/gpu.apple.inc.md:pre-built-images"
128
129 --8<-- [end:pre-built-images]
130
131 ### Build image from source
132
133 --8<-- [start:build-image-from-source]
134
135 === "NVIDIA CUDA"
136
137 --8<-- "docs/getting_started/installation/gpu.cuda.inc.md:build-image-from-source"
138
139 === "AMD ROCm"
140
141 --8<-- "docs/getting_started/installation/gpu.rocm.inc.md:build-image-from-source"
142
143 === "Intel XPU"
144
145 --8<-- "docs/getting_started/installation/gpu.xpu.inc.md:build-image-from-source"
146
147 === "Apple Silicon"
148
149 --8<-- "docs/getting_started/installation/gpu.apple.inc.md:build-image-from-source"
150
151 --8<-- [end:build-image-from-source]
152
153 ## Supported features
154
155 === "NVIDIA CUDA"
156
157 --8<-- "docs/getting_started/installation/gpu.cuda.inc.md:supported-features"
158
159 === "AMD ROCm"
160
161 --8<-- "docs/getting_started/installation/gpu.rocm.inc.md:supported-features"
162
163 === "Intel XPU"
164
165 --8<-- "docs/getting_started/installation/gpu.xpu.inc.md:supported-features"
166
167 === "Apple Silicon"
168
169 --8<-- "docs/getting_started/installation/gpu.apple.inc.md:supported-features"