A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 4, 03:45 UTC
[CI][ROCm] Ship RIXL with `vllm/vllm-openai-rocm` (#41634)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Simon Danielsson committed
May 8, 2026 at 09:05 UTC
f9b9bf3bbb36d84336bc88e077ee98b39246ee9a
2 files changed
+9
-1
docker/Dockerfile.rocm
+8
@@ -507,6 +507,10 @@ RUN --mount=type=bind,from=export_vllm,src=/,target=/install \
507
&& pip uninstall -y vllm \
508
&& uv pip install --system *.whl
509
510
+# Install RIXL wheel
511
+RUN --mount=type=bind,from=build_rixl,src=/app/install,target=/rixl_install \
512
+ uv pip install --system /rixl_install/*.whl
513
+
514
ARG COMMON_WORKDIR
515
ARG BASE_IMAGE
516
ARG NIC_BACKEND
@@ -517,6 +521,10 @@ COPY --from=export_vllm /benchmarks ${COMMON_WORKDIR}/vllm/benchmarks
521
COPY --from=export_vllm /examples ${COMMON_WORKDIR}/vllm/examples
522
COPY --from=export_vllm /docker ${COMMON_WORKDIR}/vllm/docker
523
524
+# Use legacy IPC mode for HSA to avoid GPU memory pinning issues with UCX rocm_ipc
525
+# See: https://github.com/ROCm/rocm-libraries/issues/6266
526
+ENV HSA_ENABLE_IPC_MODE_LEGACY=1
527
+
528
ENV TOKENIZERS_PARALLELISM=false
529
530
# ENV that can improve safe tensor loading, and end-to-end time
docs/features/nixl_connector_usage.md
+1
-1
@@ -13,7 +13,7 @@ Install the NIXL library: `uv pip install nixl`, as a quick start on Nvidia plat
13
- Refer to [NIXL official repository](https://github.com/ai-dynamo/nixl) for more installation instructions
14
- The specified required NIXL version can be found in [requirements/kv_connectors.txt](../../requirements/kv_connectors.txt) and other relevant config files
15
16
-For ROCm platform, the [base ROCm docker file](../../docker/Dockerfile.rocm_base) includes RIXL and ucx already.
16
+For ROCm platform, the [ROCm docker file](../../docker/Dockerfile.rocm) includes RIXL and ucx already.
17
18
- Refer to [RIXL official repository](https://github.com/rocm/rixl) for more information
19
- The supportive libraries for RIXL can be found in [requirements/kv_connectors_rocm.txt](../../requirements/kv_connectors_rocm.txt)