A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Docs] Update instructions for how to using existing torch binary (#24892)
Signed-off-by: Richard Zou <zou3519@gmail.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Richard Zou committed
Sep 15, 2025 at 22:25 UTC
e1279ef00fe2e21350600e17afe4164f7fdc46b5
1 file changed
+13
-1
docs/getting_started/installation/gpu/cuda.inc.md
+13
-1
@@ -165,7 +165,19 @@ There are scenarios where the PyTorch dependency cannot be easily installed with
165
- Building vLLM with PyTorch nightly or a custom PyTorch build.
166
- Building vLLM with aarch64 and CUDA (GH200), where the PyTorch wheels are not available on PyPI. Currently, only the PyTorch nightly has wheels for aarch64 with CUDA. You can run `uv pip install --index-url https://download.pytorch.org/whl/nightly/cu128 torch torchvision torchaudio` to [install PyTorch nightly](https://pytorch.org/get-started/locally/) and then build vLLM on top of it.
167
168
-To build vLLM using an existing PyTorch installation, it is recommended to use `uv`, because it has [a unique mechanism](https://docs.astral.sh/uv/concepts/projects/config/#disabling-build-isolation) for disabling build isolation for specific packages and vLLM leverages this mechanism to specify `torch` as the package to disable build isolation.
168
+To build vLLM using an existing PyTorch installation:
169
+
170
+```bash
171
+# install PyTorch first, either from PyPI or from source
172
+git clone https://github.com/vllm-project/vllm.git
173
+cd vllm
174
+python use_existing_torch.py
175
+uv pip install -r requirements/build.txt
176
+uv pip install --no-build-isolation -e .
177
+```
178
+
179
+Alternatively: if you are exclusively using `uv` to create and manage virtual environments, it has [a unique mechanism](https://docs.astral.sh/uv/concepts/projects/config/#disabling-build-isolation)
180
+for disabling build isolation for specific packages. vLLM can leverage this mechanism to specify `torch` as the package to disable build isolation for:
181
182
```bash
183
# install PyTorch first, either from PyPI or from source