A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 11, 15:00 UTC
[Docs][CPU backend] Add pre-built Arm CPU Docker images (#30491)
Signed-off-by: Ioana Ghiban <ioana.ghiban@arm.com>
ioana ghiban committed
Dec 11, 2025 at 23:03 UTC
3efdc3feaef01d45fb54650163da480bdf2f0ce4
1 file changed
+17
-1
docs/getting_started/installation/cpu.arm.inc.md
+17
-1
@@ -100,7 +100,23 @@ Testing has been conducted on AWS Graviton3 instances for compatibility.
100
# --8<-- [end:build-wheel-from-source]
101
# --8<-- [start:pre-built-images]
102
103
-Currently, there are no pre-built Arm CPU images.
103
+See [Using Docker](../../deployment/docker.md) for instructions on using the official Docker image.
104
+
105
+Stable vLLM Docker images are being pre-built for Arm from version 0.12.0. Available image tags are here: [https://gallery.ecr.aws/q9t5s3a7/vllm-arm64-cpu-release-repo](https://gallery.ecr.aws/q9t5s3a7/vllm-arm64-cpu-release-repo).
106
+Please replace `<version>` in the command below with a specific version string (e.g., `0.12.0`).
107
+
108
+```bash
109
+docker pull public.ecr.aws/q9t5s3a7/vllm-arm64-cpu-release-repo:v<version>
110
+```
111
+
112
+You can also access the latest code with Docker images. These are not intended for production use and are meant for CI and testing only. They will expire after several days.
113
+
114
+The latest code can contain bugs and may not be stable. Please use it with caution.
115
+
116
+```bash
117
+export VLLM_COMMIT=6299628d326f429eba78736acb44e76749b281f5 # use full commit hash from the main branch
118
+docker pull public.ecr.aws/q9t5s3a7/vllm-ci-postmerge-repo:${VLLM_COMMIT}-arm64-cpu
119
+```
120
121
# --8<-- [end:pre-built-images]
122
# --8<-- [start:build-image-from-source]