[CI/Build][Hardware][NVIDIA] Add public CUDA 13.4 Rubin build path (#54640)
Signed-off-by: Meenakshi Venkataraman <meenakshiv@nvidia.com> Signed-off-by: meena-at-work <80416898+meena-at-work@users.noreply.github.com> Signed-off-by: Shang Wang <shangw@nvidia.com> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com> Co-authored-by: Shang Wang <shangw@nvidia.com> Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
meena-at-work committed
Sep 8, 2026 at 21:53 UTC
385dce36bcee42309924a5ece951a96db3dce7f2
3 files changed
+63
-33
docker/Dockerfile
+28
-8
@@ -797,6 +797,9 @@ RUN CUDA_VERSION_DASH=$(echo $CUDA_VERSION | cut -d. -f1,2 | tr '.' '-') && \
797
cuda-nvcc-${CUDA_VERSION_DASH} \
798
cuda-cudart-${CUDA_VERSION_DASH} \
799
cuda-nvrtc-${CUDA_VERSION_DASH} \
800
+ # FlashInfer's runtime JIT-compiled kernels (e.g. trtllm_fmha_kernel_launcher.cu)
801
+ # #include <nvrtc.h>, which only the -dev package ships.
802
+ cuda-nvrtc-dev-${CUDA_VERSION_DASH} \
803
cuda-cuobjdump-${CUDA_VERSION_DASH} \
804
libcurand-dev-${CUDA_VERSION_DASH} \
805
libcublas-dev-${CUDA_VERSION_DASH} \
@@ -806,10 +809,21 @@ RUN CUDA_VERSION_DASH=$(echo $CUDA_VERSION | cut -d. -f1,2 | tr '.' '-') && \
809
numactl && \
810
rm -rf /var/lib/apt/lists/*
811
812
+# Ensures nvcc/ptxas (installed above, needed at runtime for FlashInfer/
813
+# DeepGEMM/EP kernel JIT compilation) are reachable on PATH. Harmless no-op
814
+# if already set by FINAL_BASE_IMAGE.
815
+ENV CUDA_HOME=/usr/local/cuda
816
+ENV PATH="${CUDA_HOME}/bin:${PATH}"
817
+ENV LD_LIBRARY_PATH="${CUDA_HOME}/lib64:${LD_LIBRARY_PATH}"
818
+
819
# Apply the CUDA 13.4-only libcurand compatibility link described in the build
820
# stage. CUDA 13.5 uses its native libcurand.so.11. DeepEP expects NCCL directly
821
# under its configured /usr prefix, while Ubuntu installs it in a multiarch
812
-# subdirectory.
822
+# subdirectory - but that symlink is only possible when NCCL is already
823
+# present (e.g. nvcr.io/nvidia/cuda-dl-base bundles it; a bare-CUDA image
824
+# doesn't, and gets it later via the standard nvidia-nccl-cu* pip package,
825
+# UV_OVERRIDE below). Check reality (whether libnccl.so.2 is actually found)
826
+# and tolerate not finding it instead of failing the build.
827
RUN if [ "${INSTALL_RUBIN_PRERELEASE}" = "true" ]; then \
828
CUDA_MAJOR_MINOR=$(echo "${CUDA_VERSION}" | cut -d. -f1,2) \
829
&& if [ "${CUDA_MAJOR_MINOR}" = "13.4" ]; then \
@@ -827,10 +841,13 @@ RUN if [ "${INSTALL_RUBIN_PRERELEASE}" = "true" ]; then \
841
echo "Unsupported Rubin CUDA version: ${CUDA_MAJOR_MINOR}" >&2; \
842
exit 1; \
843
fi \
830
- && NCCL_LIBRARY=$(find /usr/lib -name libnccl.so.2 -print -quit) \
831
- && test -n "${NCCL_LIBRARY}" \
832
- && ln -sf "${NCCL_LIBRARY}" /usr/lib/libnccl.so.2 \
833
- && ln -sf libnccl.so.2 /usr/lib/libnccl.so \
844
+ && if NCCL_LIBRARY=$(find /usr/lib -name libnccl.so.2 -print -quit) \
845
+ && [ -n "${NCCL_LIBRARY}" ]; then \
846
+ if [ "${NCCL_LIBRARY}" != "/usr/lib/libnccl.so.2" ]; then \
847
+ ln -sf "${NCCL_LIBRARY}" /usr/lib/libnccl.so.2; \
848
+ fi \
849
+ && ln -sf libnccl.so.2 /usr/lib/libnccl.so; \
850
+ fi \
851
&& ldconfig; \
852
fi
853
@@ -961,10 +978,13 @@ ARG GDRCOPY_CUDA_VERSION=12.8
978
ARG GDRCOPY_OS_VERSION=Ubuntu24_04
979
ARG TARGETPLATFORM
980
COPY tools/install_gdrcopy.sh /tmp/install_gdrcopy.sh
981
+# Some Rubin FINAL_BASE_IMAGE choices (internal Rubin CUDA images,
982
+# nvcr.io/nvidia/cuda-dl-base) already provide GDRCopy; others don't, so it
983
+# falls through to the normal install below. Check reality (ldconfig) rather
984
+# than inferring from which FINAL_BASE_IMAGE flag is set.
985
RUN set -eux; \
965
- if [ "${INSTALL_RUBIN_PRERELEASE}" = "true" ]; then \
966
- # The Rubin CUDA image already provides GDRCopy. \
967
- ldconfig -p | grep -q 'libgdrapi\.so'; \
986
+ if [ "${INSTALL_RUBIN_PRERELEASE}" = "true" ] && ldconfig -p | grep -q 'libgdrapi\.so'; then \
987
+ true; \
988
else \
989
case "${TARGETPLATFORM}" in \
990
linux/arm64) UUARCH="aarch64" ;; \
docs/getting_started/installation/gpu.cuda.inc.md
+19
-25
@@ -425,12 +425,7 @@ For (G)B300, we recommend using CUDA 13, as shown in the following command.
425
426
#### [Preview] Building vLLM's Docker Image from Source for NVIDIA Rubin GPU Architecture
427
428
-At this time, the Rubin preview is available only to NVIDIA engineers and
429
-partners with access to NVIDIA's internal prerequisites. Set
430
-`INSTALL_RUBIN_PRERELEASE=true` to enable the Rubin build path. It requires a
431
-`requirements/rubin-prerelease.txt` file containing pointers to NVIDIA's
432
-internal prerelease packages, plus NVIDIA-internal CUDA 13.4 or CUDA 13.5
433
-development images for `BUILD_BASE_IMAGE` and `FINAL_BASE_IMAGE`.
428
+Set `INSTALL_RUBIN_PRERELEASE=true` to enable the Rubin build path.
429
430
Triton must currently be installed from source for Rubin compatibility.
431
Specify its repository with `TRITON_INSTALL_FROM_SOURCE_REPO`; an empty
@@ -444,54 +439,53 @@ BuildKit does not automatically invalidate cached layers when a mutable Git
439
ref changes. Use `--no-cache-filter extensions-build` to refresh an empty,
440
branch, or tag revision.
441
447
-Use `ARCH=arm64` on VR200 or `ARCH=amd64` on R100:
442
+For `FINAL_BASE_IMAGE`, use the public, multi-arch
443
+`nvcr.io/nvidia/cuda-dl-base:26.08-cuda13.4-devel-ubuntu24.04` image.
444
+For `BUILD_BASE_IMAGE`, use:
445
449
-??? console "CUDA 13.4 command"
446
+- `pytorch/manylinux2_28-builder:cuda13.4` for x86_64 CPUs.
447
+- `pytorch/manylinuxaarch64-builder:cuda13.4` for ARM64/AArch64 CPUs.
448
+
449
+??? console "ARM64/AArch64 build command"
450
451
```bash
452
- ARCH=arm64
452
docker buildx build --progress=plain --load \
453
--file docker/Dockerfile \
454
--target vllm-openai \
456
- --platform "linux/${ARCH}" \
457
- --tag "vllm/vllm-rubin-openai:prerelease-cu134-${ARCH}" \
455
+ --platform "linux/arm64" \
456
+ --tag "vllm/vllm-rubin-openai:prerelease-cu134-public-arm64" \
457
--build-arg max_jobs="$(nproc)" \
458
--build-arg nvcc_threads=2 \
459
--build-arg RUN_WHEEL_CHECK=false \
460
--build-arg INSTALL_RUBIN_PRERELEASE=true \
461
--build-arg TRITON_INSTALL_FROM_SOURCE_REPO=https://github.com/triton-lang/triton.git \
462
--build-arg TRITON_INSTALL_FROM_SOURCE_REVISION=3f6e41132b5edf639bfb872ad73d4688765e08b8 \
464
- --build-arg CUDA_VERSION=13.4.1 \
465
- --build-arg BUILD_BASE_IMAGE="<cuda-13.4-manylinux-devel-image-${ARCH}>" \
466
- --build-arg FINAL_BASE_IMAGE="<cuda-13.4-ubuntu24.04-devel-image-${ARCH}>" \
463
+ --build-arg CUDA_VERSION=13.4 \
464
+ --build-arg BUILD_BASE_IMAGE="pytorch/manylinuxaarch64-builder:cuda13.4" \
465
+ --build-arg FINAL_BASE_IMAGE="nvcr.io/nvidia/cuda-dl-base:26.08-cuda13.4-devel-ubuntu24.04" \
466
.
467
```
468
470
-??? console "CUDA 13.5 command"
469
+??? console "x86_64 build command"
470
471
```bash
473
- ARCH=arm64
472
docker buildx build --progress=plain --load \
473
--file docker/Dockerfile \
474
--target vllm-openai \
477
- --platform "linux/${ARCH}" \
478
- --tag "vllm/vllm-rubin-openai:prerelease-cu135-${ARCH}" \
475
+ --platform "linux/amd64" \
476
+ --tag "vllm/vllm-rubin-openai:prerelease-cu134-public-amd64" \
477
--build-arg max_jobs="$(nproc)" \
478
--build-arg nvcc_threads=2 \
479
--build-arg RUN_WHEEL_CHECK=false \
480
--build-arg INSTALL_RUBIN_PRERELEASE=true \
481
--build-arg TRITON_INSTALL_FROM_SOURCE_REPO=https://github.com/triton-lang/triton.git \
482
--build-arg TRITON_INSTALL_FROM_SOURCE_REVISION=3f6e41132b5edf639bfb872ad73d4688765e08b8 \
485
- --build-arg CUDA_VERSION=13.5.0 \
486
- --build-arg BUILD_BASE_IMAGE="<cuda-13.5-manylinux-devel-image-${ARCH}>" \
487
- --build-arg FINAL_BASE_IMAGE="<cuda-13.5-ubuntu24.04-devel-image-${ARCH}>" \
483
+ --build-arg CUDA_VERSION=13.4 \
484
+ --build-arg BUILD_BASE_IMAGE="pytorch/manylinux2_28-builder:cuda13.4" \
485
+ --build-arg FINAL_BASE_IMAGE="nvcr.io/nvidia/cuda-dl-base:26.08-cuda13.4-devel-ubuntu24.04" \
486
.
487
```
488
491
-CUDA 13.5 compiled-mode qualification passed with NVIDIA driver versions
492
-615.62.03 and 620.05. Driver 610.47.04 failed the tested compiled CUDA 13.5
493
-kernel. Validate the exact driver version on the target system.
494
-
489
!!! note
490
Keep the default explicit `torch_cuda_arch_list`. GPU-less BuildKit builds
491
cannot inspect the host GPU. R100 and VR200 report compute capability 10.7,
requirements/rubin-prerelease.txt
new
+16
@@ -0,0 +1,16 @@
1
+# Public CUDA 13.4 prerelease PyTorch nightly pins for Rubin bring-up.
2
+# Installed when INSTALL_RUBIN_PRERELEASE=true (see docker/Dockerfile).
3
+# TODO(rubin-bringup): delete once Rubin APIs ship in regular PyTorch wheels
4
+# (vllm-project/vllm#49735).
5
+--pre
6
+--extra-index-url https://download.pytorch.org/whl/nightly/cu134
7
+
8
+torch==2.15.0.dev20260903+cu134
9
+torchvision==0.30.0.dev20260903+cu134
10
+torchaudio==2.11.0.dev20260903+cu134
11
+
12
+# docker/Dockerfile strips these two from requirements/cuda.txt whenever
13
+# INSTALL_RUBIN_PRERELEASE=true, so they must be pinned here instead to
14
+# still get installed.
15
+nvidia-cutlass-dsl[cu13]==4.8.0.dev0
16
+flashinfer-python==0.6.18