A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 07:15 UTC
[Docs] Use `--torch-backend=auto` for editable install docs (#39511)
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Michael Goin committed
Apr 11, 2026 at 00:27 UTC
7f0b8f20205c340f2a4eed88fcf8b8df40f011f2
1 file changed
+6
-6
docs/getting_started/installation/gpu.cuda.inc.md
+6
-6
@@ -1,12 +1,12 @@
1
<!-- markdownlint-disable MD041 MD051 -->
2
--8<-- [start:installation]
3
4
-vLLM contains pre-compiled C++ and CUDA (12.8) binaries.
4
+vLLM contains pre-compiled C++ and CUDA (12.9) binaries.
5
6
--8<-- [end:installation]
7
--8<-- [start:requirements]
8
9
-- GPU: compute capability 7.0 or higher (e.g., V100, T4, RTX20xx, A100, L4, H100, etc.)
9
+- GPU: compute capability 7.5 or higher (e.g., T4, RTX20xx, A100, L4, H100, B200, etc.)
10
11
--8<-- [end:requirements]
12
--8<-- [start:set-up-using-python]
@@ -31,7 +31,7 @@ uv pip install vllm --torch-backend=auto
31
pip install vllm --extra-index-url https://download.pytorch.org/whl/cu129
32
```
33
34
-We recommend leveraging `uv` to [automatically select the appropriate PyTorch index at runtime](https://docs.astral.sh/uv/guides/integration/pytorch/#automatic-backend-selection) by inspecting the installed CUDA driver version via `--torch-backend=auto` (or `UV_TORCH_BACKEND=auto`). To select a specific backend (e.g., `cu128`), set `--torch-backend=cu128` (or `UV_TORCH_BACKEND=cu128`). If this doesn't work, try running `uv self update` to update `uv` first.
34
+We recommend leveraging `uv` to [automatically select the appropriate PyTorch index at runtime](https://docs.astral.sh/uv/guides/integration/pytorch/#automatic-backend-selection) by inspecting the installed CUDA driver version via `--torch-backend=auto` (or `UV_TORCH_BACKEND=auto`). To select a specific backend (e.g., `cu130`), set `--torch-backend=cu130` (or `UV_TORCH_BACKEND=cu130`). If this doesn't work, try running `uv self update` to update `uv` first.
35
36
!!! note
37
NVIDIA Blackwell GPUs (B200, GB200) require a minimum of CUDA 12.8, so make sure you are installing PyTorch wheels with at least that version. PyTorch itself offers a [dedicated interface](https://pytorch.org/get-started/locally/) to determine the appropriate pip command to run for a given target configuration.
@@ -93,7 +93,7 @@ If you only need to change Python code, you can build and install vLLM without c
93
```bash
94
git clone https://github.com/vllm-project/vllm.git
95
cd vllm
96
-VLLM_USE_PRECOMPILED=1 uv pip install --editable .
96
+VLLM_USE_PRECOMPILED=1 uv pip install --editable . --torch-backend=auto
97
```
98
99
This command will do the following:
@@ -134,7 +134,7 @@ If you want to modify C++ or CUDA code, you'll need to build vLLM from source. T
134
```bash
135
git clone https://github.com/vllm-project/vllm.git
136
cd vllm
137
-uv pip install -e .
137
+uv pip install -e . --torch-backend=auto
138
```
139
140
!!! tip
@@ -185,7 +185,7 @@ To achieve this, you can set the environment variable VLLM_CUTLASS_SRC_DIR to po
185
```bash
186
git clone https://github.com/vllm-project/vllm.git
187
cd vllm
188
-VLLM_CUTLASS_SRC_DIR=/path/to/cutlass uv pip install -e .
188
+VLLM_CUTLASS_SRC_DIR=/path/to/cutlass uv pip install -e . --torch-backend=auto
189
```
190
191
##### Troubleshooting