[Doc] Add compile cache volume example to the Docker deployment page (#49782)
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Nils Matteson committed
Jul 25, 2026 at 23:24 UTC
2e860de498a229018d8795afa34417f43b813d94
1 file changed
+20
docs/deployment/docker.md
+20
@@ -8,6 +8,26 @@ toc_depth: 2
8
9
--8<-- "docs/getting_started/installation/gpu.md:pre-built-images"
10
11
+## Persist the compile cache across containers
12
+
13
+Mounting the Hugging Face cache keeps model weights across containers, but each
14
+new container still starts with an empty `VLLM_CACHE_ROOT` (default
15
+`~/.cache/vllm`) and recompiles the model's `torch.compile` artifacts. Mount a
16
+named volume at that path to reuse the inductor, Triton, and AOT artifacts from
17
+the second container onward:
18
+
19
+```bash
20
+docker run --rm --gpus all \
21
+ -v ~/.cache/huggingface:/root/.cache/huggingface \
22
+ -v vllm-cache:/root/.cache/vllm \
23
+ -p 8000:8000 \
24
+ vllm/vllm-openai:latest \
25
+ meta-llama/Llama-3.1-8B-Instruct
26
+```
27
+
28
+See [Faster Startup](../configuration/optimization.md#faster-startup) for the
29
+mechanism and for what invalidates the cache.
30
+
31
## Run as a non-root user
32
33
The CUDA `vllm/vllm-openai` image runs as root by default for backward