A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 01:45 UTC
[Doc] add visualization for multi-stage dockerfile (#4456)
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com> Co-authored-by: Roger Wang <ywang@roblox.com>
Prashant Gupta committed
Apr 30, 2024 at 10:41 UTC
b31a1fb63c98fa1c64666aaae15579439af60d95
4 files changed
+55
Dockerfile
+4
@@ -1,6 +1,10 @@
1
# The vLLM Dockerfile is used to construct vLLM image that can be directly used
2
# to run the OpenAI compatible server.
3
4
+# Please update any changes made here to
5
+# docs/source/dev/dockerfile/dockerfile.rst and
6
+# docs/source/assets/dev/dockerfile-stages-dependency.png
7
+
8
#################### BASE BUILD IMAGE ####################
9
# prepare basic build environment
10
FROM nvidia/cuda:12.1.0-devel-ubuntu22.04 AS dev
docs/source/assets/dev/dockerfile-stages-dependency.png
Binary files /dev/null and b/docs/source/assets/dev/dockerfile-stages-dependency.png differ
docs/source/dev/dockerfile/dockerfile.rst
new
+50
@@ -0,0 +1,50 @@
1
+Dockerfile
2
+====================
3
+
4
+See `here <https://github.com/vllm-project/vllm/blob/main/Dockerfile>`_ for the main Dockerfile to construct
5
+the image for running an OpenAI compatible server with vLLM.
6
+
7
+- Below is a visual representation of the multi-stage Dockerfile. The build graph contains the following nodes:
8
+
9
+ - All build stages
10
+ - The default build target (highlighted in grey)
11
+ - External images (with dashed borders)
12
+
13
+ The edges of the build graph represent:
14
+
15
+ - FROM ... dependencies (with a solid line and a full arrow head)
16
+ - COPY --from=... dependencies (with a dashed line and an empty arrow head)
17
+ - RUN --mount=(.*)from=... dependencies (with a dotted line and an empty diamond arrow head)
18
+
19
+ .. figure:: ../../assets/dev/dockerfile-stages-dependency.png
20
+ :alt: query
21
+ :width: 100%
22
+ :align: center
23
+
24
+ Made using: https://github.com/patrickhoefler/dockerfilegraph
25
+
26
+ Commands to regenerate the build graph (make sure to run it **from the `root` directory of the vLLM repository** where the dockerfile is present):
27
+
28
+ .. code:: bash
29
+
30
+ dockerfilegraph -o png --legend --dpi 200 --max-label-length 50 --filename Dockerfile
31
+
32
+ or in case you want to run it directly with the docker image:
33
+
34
+ .. code:: bash
35
+
36
+ docker run \
37
+ --rm \
38
+ --user "$(id -u):$(id -g)" \
39
+ --workdir /workspace \
40
+ --volume "$(pwd)":/workspace \
41
+ ghcr.io/patrickhoefler/dockerfilegraph:alpine \
42
+ --output png \
43
+ --dpi 200 \
44
+ --max-label-length 50 \
45
+ --filename Dockerfile \
46
+ --legend
47
+
48
+ (To run it for a different file, you can pass in a different argument to the flag `--filename`.)
49
+
50
+
\ No newline at end of file
docs/source/index.rst
+1
@@ -102,6 +102,7 @@ Documentation
102
dev/sampling_params
103
dev/engine/engine_index
104
dev/kernel/paged_attention
105
+ dev/dockerfile/dockerfile
106
107
Indices and tables
108
==================