A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[DOCs] Update ROCm installation docs section (#24691)
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Gregory Shtrasberg committed
Sep 11, 2025 at 23:02 UTC
6a50eaa0d3858dd19a142746fce7181813c5d632
1 file changed
+15
-25
docs/getting_started/installation/gpu/rocm.inc.md
+15
-25
@@ -1,6 +1,6 @@
1
# --8<-- [start:installation]
2
3
-vLLM supports AMD GPUs with ROCm 6.3.
3
+vLLM supports AMD GPUs with ROCm 6.3 or above.
4
5
!!! tip
6
[Docker](#set-up-using-docker) is the recommended way to use vLLM on ROCm.
@@ -11,8 +11,9 @@ vLLM supports AMD GPUs with ROCm 6.3.
11
# --8<-- [end:installation]
12
# --8<-- [start:requirements]
13
14
-- GPU: MI200s (gfx90a), MI300 (gfx942), Radeon RX 7900 series (gfx1100/1101), Radeon RX 9000 series (gfx1200/1201)
15
-- ROCm 6.3
14
+- GPU: MI200s (gfx90a), MI300 (gfx942), MI350 (gfx950), Radeon RX 7900 series (gfx1100/1101), Radeon RX 9000 series (gfx1200/1201)
15
+- ROCm 6.3 or above
16
+ - MI350 requires ROCm 7.0 or above
17
18
# --8<-- [end:requirements]
19
# --8<-- [start:set-up-using-python]
@@ -32,35 +33,35 @@ Currently, there are no pre-built ROCm wheels.
33
- [ROCm](https://rocm.docs.amd.com/en/latest/deploy/linux/index.html)
34
- [PyTorch](https://pytorch.org/)
35
35
- For installing PyTorch, you can start from a fresh docker image, e.g, `rocm/pytorch:rocm6.3_ubuntu24.04_py3.12_pytorch_release_2.4.0`, `rocm/pytorch-nightly`. If you are using docker image, you can skip to Step 3.
36
+ For installing PyTorch, you can start from a fresh docker image, e.g, `rocm/pytorch:rocm6.4.3_ubuntu24.04_py3.12_pytorch_release_2.6.0`, `rocm/pytorch-nightly`. If you are using docker image, you can skip to Step 3.
37
38
Alternatively, you can install PyTorch using PyTorch wheels. You can check PyTorch installation guide in PyTorch [Getting Started](https://pytorch.org/get-started/locally/). Example:
39
40
```bash
41
# Install PyTorch
42
pip uninstall torch -y
42
- pip install --no-cache-dir --pre torch --index-url https://download.pytorch.org/whl/nightly/rocm6.3
43
+ pip install --no-cache-dir torch torchvision --index-url https://download.pytorch.org/whl/rocm6.4
44
```
45
45
-1. Install [Triton flash attention for ROCm](https://github.com/ROCm/triton)
46
+1. Install [Triton for ROCm](https://github.com/triton-lang/triton)
47
47
- Install ROCm's Triton flash attention (the default triton-mlir branch) following the instructions from [ROCm/triton](https://github.com/ROCm/triton/blob/triton-mlir/README.md)
48
+ Install ROCm's Triton (the default triton-mlir branch) following the instructions from [ROCm/triton](https://github.com/ROCm/triton/blob/triton-mlir/README.md)
49
50
```bash
51
python3 -m pip install ninja cmake wheel pybind11
52
pip uninstall -y triton
52
- git clone https://github.com/OpenAI/triton.git
53
+ git clone https://github.com/triton-lang/triton.git
54
cd triton
55
git checkout e5be006
55
- cd python
56
- pip3 install .
56
+ if [ ! -f setup.py ]; then cd python; fi
57
+ python3 setup.py install
58
cd ../..
59
```
60
61
!!! note
62
If you see HTTP issue related to downloading packages during building triton, please try again as the HTTP error is intermittent.
63
63
-2. Optionally, if you choose to use CK flash attention, you can install [flash attention for ROCm](https://github.com/ROCm/flash-attention)
64
+2. Optionally, if you choose to use CK flash attention, you can install [flash attention for ROCm](https://github.com/Dao-AILab/flash-attention)
65
66
Install ROCm's flash attention (v2.7.2) following the instructions from [ROCm/flash-attention](https://github.com/ROCm/flash-attention#amd-rocm-support)
67
Alternatively, wheels intended for vLLM use can be accessed under the releases.
@@ -68,9 +69,9 @@ Currently, there are no pre-built ROCm wheels.
69
For example, for ROCm 6.3, suppose your gfx arch is `gfx90a`. To get your gfx architecture, run `rocminfo |grep gfx`.
70
71
```bash
71
- git clone https://github.com/ROCm/flash-attention.git
72
+ git clone https://github.com/Dao-AILab/flash-attention.git
73
cd flash-attention
73
- git checkout b7d29fb
74
+ git checkout 1a7f4dfa
75
git submodule update --init
76
GPU_ARCHS="gfx90a" python3 setup.py install
77
cd ..
@@ -194,16 +195,6 @@ To build vllm on ROCm 6.3 for MI200 and MI300 series, you can use the default:
195
DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.rocm -t vllm-rocm .
196
```
197
197
-To build vllm on ROCm 6.3 for Radeon RX7900 series (gfx1100), you should pick the alternative base image:
198
-
199
-```bash
200
-DOCKER_BUILDKIT=1 docker build \
201
- --build-arg BASE_IMAGE="rocm/vllm-dev:navi_base" \
202
- -f docker/Dockerfile.rocm \
203
- -t vllm-rocm \
204
- .
205
-```
206
-
198
To run the above docker image `vllm-rocm`, use the below command:
199
200
??? console "Command"
@@ -218,8 +209,7 @@ To run the above docker image `vllm-rocm`, use the below command:
209
--device /dev/kfd \
210
--device /dev/dri \
211
-v <path/to/model>:/app/model \
221
- vllm-rocm \
222
- bash
212
+ vllm-rocm
213
```
214
215
Where the `<path/to/model>` is the location where the model is stored, for example, the weights for llama2 or llama3 models.