A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 23:45 UTC
[Docs] Fix the outdated URL for installing from vLLM binaries (#21523)
Signed-off-by: Kay Yan <kay.yan@daocloud.io> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Kay Yan committed
Jul 29, 2025 at 19:56 UTC
2470419119aa5bc2734b4b8972bbfa348ccdc8b1
2 files changed
+5
-6
docs/contributing/ci/update_pytorch_version.md
+1
-2
@@ -57,8 +57,7 @@ cc the PyTorch release team to initiate discussion on how to address them.
57
58
## Update CUDA version
59
60
-The PyTorch release matrix includes both stable and experimental [CUDA versions](https://github.com/pytorch/pytorch/blob/main/RELEASE.md#release-compatibility-matrix). Due to limitations, only the latest stable CUDA version (for example,
61
-`torch2.7.0+cu12.6`) is uploaded to PyPI. However, vLLM may require a different CUDA version,
60
+The PyTorch release matrix includes both stable and experimental [CUDA versions](https://github.com/pytorch/pytorch/blob/main/RELEASE.md#release-compatibility-matrix). Due to limitations, only the latest stable CUDA version (for example, torch `2.7.1+cu126`) is uploaded to PyPI. However, vLLM may require a different CUDA version,
61
such as 12.8 for Blackwell support.
62
This complicates the process as we cannot use the out-of-the-box
63
`pip install torch torchvision torchaudio` command. The solution is to use
docs/getting_started/installation/gpu/cuda.inc.md
+4
-4
@@ -38,10 +38,10 @@ We recommend leveraging `uv` to [automatically select the appropriate PyTorch in
38
As of now, vLLM's binaries are compiled with CUDA 12.8 and public PyTorch release versions by default. We also provide vLLM binaries compiled with CUDA 12.6, 11.8, and public PyTorch release versions:
39
40
```bash
41
-# Install vLLM with CUDA 11.8.
42
-export VLLM_VERSION=0.6.1.post1
43
-export PYTHON_VERSION=312
44
-uv pip install https://github.com/vllm-project/vllm/releases/download/v${VLLM_VERSION}/vllm-${VLLM_VERSION}+cu118-cp${PYTHON_VERSION}-cp${PYTHON_VERSION}-manylinux1_x86_64.whl --extra-index-url https://download.pytorch.org/whl/cu118
41
+# Install vLLM with a specific CUDA version (e.g., 11.8 or 12.6).
42
+export VLLM_VERSION=$(curl -s https://api.github.com/repos/vllm-project/vllm/releases/latest | jq -r .tag_name | sed 's/^v//')
43
+export CUDA_VERSION=118 # or 126
44
+uv pip install https://github.com/vllm-project/vllm/releases/download/v${VLLM_VERSION}/vllm-${VLLM_VERSION}+cu${CUDA_VERSION}-cp38-abi3-manylinux1_x86_64.whl --extra-index-url https://download.pytorch.org/whl/cu${CUDA_VERSION}
45
```
46
47
[](){ #install-the-latest-code }