A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 11, 12:30 UTC
[Doc] Add instructions for building docker image on GB300 with CUDA13 (#30414)
Signed-off-by: Qidong Su <soodoshll@gmail.com>
Qidong Su committed
Dec 13, 2025 at 16:56 UTC
24429d592450a2b00d7df894288913a320a257e0
1 file changed
+20
-1
docs/deployment/docker.md
+20
-1
@@ -82,7 +82,7 @@ DOCKER_BUILDKIT=1 docker build . \
82
83
## Building for Arm64/aarch64
84
85
-A docker container can be built for aarch64 systems such as the Nvidia Grace-Hopper. At time of this writing, this should be considered **experimental**. Using the flag `--platform "linux/arm64"` will attempt to build for arm64.
85
+A docker container can be built for aarch64 systems such as the Nvidia Grace-Hopper and Grace-Blackwell. Using the flag `--platform "linux/arm64"` will build for arm64.
86
87
!!! note
88
Multiple modules must be compiled, so this process can take a while. Recommend using `--build-arg max_jobs=` & `--build-arg nvcc_threads=`
@@ -104,6 +104,25 @@ A docker container can be built for aarch64 systems such as the Nvidia Grace-Hop
104
--build-arg RUN_WHEEL_CHECK=false
105
```
106
107
+For (G)B300, we recommend using CUDA 13, as shown in the following command.
108
+
109
+??? console "Command"
110
+
111
+ ```bash
112
+ DOCKER_BUILDKIT=1 docker build \
113
+ --build-arg CUDA_VERSION=13.0.1 \
114
+ --build-arg BUILD_BASE_IMAGE=nvidia/cuda:13.0.1-devel-ubuntu22.04 \
115
+ --build-arg max_jobs=256 \
116
+ --build-arg nvcc_threads=2 \
117
+ --build-arg RUN_WHEEL_CHECK=false \
118
+ --build-arg torch_cuda_arch_list='9.0 10.0+PTX' \
119
+ --platform "linux/arm64" \
120
+ --tag vllm/vllm-gb300-openai:latest \
121
+ --target vllm-openai \
122
+ -f docker/Dockerfile \
123
+ .
124
+ ```
125
+
126
!!! note
127
If you are building the `linux/arm64` image on a non-ARM host (e.g., an x86_64 machine), you need to ensure your system is set up for cross-compilation using QEMU. This allows your host machine to emulate ARM64 execution.
128