A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 22:15 UTC
[Doc] Dockerfile instructions for optional dependencies and dev transformers (#13699)
Cyrus Leung committed
Feb 22, 2025 at 22:04 UTC
8354f6640c85d6aadb0a439dad4eaf288f512780
1 file changed
+30
docs/source/deployment/docker.md
+30
@@ -27,6 +27,36 @@ container to access the host's shared memory. vLLM uses PyTorch, which uses shar
27
memory to share data between processes under the hood, particularly for tensor parallel inference.
28
:::
29
30
+:::{note}
31
+Optional dependencies are not included in order to avoid licensing issues (e.g. <gh-issue:8030>).
32
+
33
+If you need to use those dependencies (having accepted the license terms),
34
+create a custom Dockerfile on top of the base image with an extra layer that installs them:
35
+
36
+```Dockerfile
37
+FROM vllm/vllm-openai:v0.7.3
38
+
39
+# e.g. install the `audio` and `video` optional dependencies
40
+# NOTE: Make sure the version of vLLM matches the base image!
41
+RUN uv pip install --system vllm[audio,video]==0.7.3
42
+```
43
+
44
+:::
45
+
46
+:::{tip}
47
+Some new models may only be available on the main branch of [HF Transformers](https://github.com/huggingface/transformers).
48
+
49
+To use the development version of `transformers`, create a custom Dockerfile on top of the base image
50
+with an extra layer that installs their code from source:
51
+
52
+```Dockerfile
53
+FROM vllm/vllm-openai:latest
54
+
55
+RUN uv pip install --system git+https://github.com/huggingface/transformers.git
56
+```
57
+
58
+:::
59
+
60
(deployment-docker-build-image-from-source)=
61
62
## Building vLLM's Docker Image from Source