A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 10, 22:45 UTC
add tip for VLLM_USE_PRECOMPILED arg to reduce docker build time (#31385)
Signed-off-by: yiting.jiang <yiting.jiang@daocloud.io>
yitingdc committed
Dec 28, 2025 at 11:19 UTC
b326598e976f3916ed00586ba8ae8606883950fd
1 file changed
+9
docs/deployment/docker.md
+9
@@ -80,6 +80,15 @@ DOCKER_BUILDKIT=1 docker build . \
80
If you are using Podman instead of Docker, you might need to disable SELinux labeling by
81
adding `--security-opt label=disable` when running `podman build` command to avoid certain [existing issues](https://github.com/containers/buildah/discussions/4184).
82
83
+!!! note
84
+ If you have not changed any C++ or CUDA kernel code, you can use precompiled wheels to significantly reduce Docker build time.
85
+
86
+ * **Enable the feature** by adding the build argument: `--build-arg VLLM_USE_PRECOMPILED="1"`.
87
+ * **How it works**: By default, vLLM automatically finds the correct wheels from our [Nightly Builds](https://docs.vllm.ai/en/latest/contributing/ci/nightly_builds/) by using the merge-base commit with the upstream `main` branch.
88
+ * **Override commit**: To use wheels from a specific commit, provide the `--build-arg VLLM_PRECOMPILED_WHEEL_COMMIT=<commit_hash>` argument.
89
+
90
+ For a detailed explanation, refer to the documentation on 'Set up using Python-only build (without compilation)' part in [Build wheel from source](https://docs.vllm.ai/en/latest/contributing/ci/nightly_builds.html#precompiled-wheels-usage), these args are similar.
91
+
92
## Building for Arm64/aarch64
93
94
A docker container can be built for aarch64 systems such as the Nvidia Grace-Hopper and Grace-Blackwell. Using the flag `--platform "linux/arm64"` will build for arm64.