A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 21:00 UTC
[ROCm][Documentation] update quickstart and installation to include rocm nightly docker tips (#38367)
Signed-off-by: Hongxia Yang <hongxiay.yang@amd.com> Co-authored-by: Hongxia Yang <hongxiay.yang@amd.com>
Hongxia Yang committed
Mar 27, 2026 at 19:20 UTC
83a4df049dbb003965b9ca57a9b2cecd812613a1
2 files changed
+16
-22
docs/getting_started/installation/gpu.rocm.inc.md
+12
-21
@@ -172,8 +172,11 @@ uv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.15.0/rocm700
172
--8<-- [end:build-wheel-from-source]
173
--8<-- [start:pre-built-images]
174
175
-vLLM offers an official Docker image for deployment.
176
-The image can be used to run OpenAI compatible server and is available on Docker Hub as [vllm/vllm-openai-rocm](https://hub.docker.com/r/vllm/vllm-openai-rocm/tags).
175
+vLLM offers official Docker images for deployment.
176
+The images can be used to run OpenAI compatible server and are available on Docker Hub as [vllm/vllm-openai-rocm](https://hub.docker.com/r/vllm/vllm-openai-rocm/tags).
177
+
178
+- `vllm/vllm-openai-rocm:latest` — stable release
179
+- `vllm/vllm-openai-rocm:nightly` — preview build from the latest development branch, use this if you want the latest features and fixes
180
181
```bash
182
docker run --rm \
@@ -186,30 +189,18 @@ docker run --rm \
189
--env "HF_TOKEN=$HF_TOKEN" \
190
-p 8000:8000 \
191
--ipc=host \
189
- vllm/vllm-openai-rocm:latest \
192
+ vllm/vllm-openai-rocm:<tag> \
193
--model Qwen/Qwen3-0.6B
194
```
195
193
-#### Use AMD's Docker Images
196
+#### Use AMD's Docker Images (Deprecated)
197
195
-Prior to January 20th, 2026 when the official docker images are available on [upstream vLLM docker hub](https://hub.docker.com/v2/repositories/vllm/vllm-openai-rocm/tags/), the [AMD Infinity hub for vLLM](https://hub.docker.com/r/rocm/vllm/tags) offers a prebuilt, optimized
196
-docker image designed for validating inference performance on the AMD Instinct MI300X™ accelerator.
197
-AMD also offers nightly prebuilt docker image from [Docker Hub](https://hub.docker.com/r/rocm/vllm-dev), which has vLLM and all its dependencies installed. The entrypoint of this docker image is `/bin/bash` (different from the vLLM's Official Docker Image).
198
+!!! warning "Deprecated"
199
+ AMD's Docker images (`rocm/vllm` and `rocm/vllm-dev`) are deprecated in favor of the official vLLM Docker images above (`vllm/vllm-openai-rocm`). Please migrate to the official images.
200
199
-```bash
200
-docker pull rocm/vllm-dev:nightly # to get the latest image
201
-docker run -it --rm \
202
---network=host \
203
---group-add=video \
204
---ipc=host \
205
---cap-add=SYS_PTRACE \
206
---security-opt seccomp=unconfined \
207
---device /dev/kfd \
208
---device /dev/dri \
209
--v <path/to/your/models>:/app/models \
210
--e HF_HOME="/app/models" \
211
-rocm/vllm-dev:nightly
212
-```
201
+Prior to January 20th, 2026 when the official docker images became available on [upstream vLLM docker hub](https://hub.docker.com/v2/repositories/vllm/vllm-openai-rocm/tags/), the [AMD Infinity hub for vLLM](https://hub.docker.com/r/rocm/vllm/tags) offered a prebuilt, optimized
202
+docker image designed for validating inference performance on the AMD Instinct MI300X™ accelerator.
203
+AMD also offered nightly prebuilt docker image from [Docker Hub](https://hub.docker.com/r/rocm/vllm-dev), which has vLLM and all its dependencies installed. The entrypoint of this docker image is `/bin/bash` (different from the vLLM's Official Docker Image).
204
205
!!! tip
206
Please check [LLM inference performance validation on AMD Instinct MI300X](https://rocm.docs.amd.com/en/latest/how-to/performance-validation/mi300x/vllm-benchmark.html)
docs/getting_started/quickstart.md
+4
-1
@@ -56,9 +56,12 @@ This guide will help you quickly get started with vLLM to perform:
56
!!! note
57
It currently supports Python 3.12, ROCm 7.0 and `glibc >= 2.35`.
58
59
- !!! note
59
+ !!! note
60
Note that, previously, docker images were published using AMD's docker release pipeline and were located `rocm/vllm-dev`. This is being deprecated by using vLLM's docker release pipeline.
61
62
+ !!! tip
63
+ A nightly Docker image is also available as [vllm/vllm-openai-rocm:nightly](https://hub.docker.com/r/vllm/vllm-openai-rocm/tags) for testing the latest development builds.
64
+
65
=== "Google TPU"
66
67
To run vLLM on Google TPUs, you need to install the `vllm-tpu` package.