@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / b31a1fb63c
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 01:45 UTC

[Doc] add visualization for multi-stage dockerfile (#4456)

Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com> Co-authored-by: Roger Wang <ywang@roblox.com>

Prashant Gupta committed Apr 30, 2024 at 10:41 UTC b31a1fb63c98fa1c64666aaae15579439af60d95
4 files changed +55
Dockerfile
+4
@@ -1,6 +1,10 @@
1 # The vLLM Dockerfile is used to construct vLLM image that can be directly used
2 # to run the OpenAI compatible server.
3
4 +# Please update any changes made here to
5 +# docs/source/dev/dockerfile/dockerfile.rst and
6 +# docs/source/assets/dev/dockerfile-stages-dependency.png
7 +
8 #################### BASE BUILD IMAGE ####################
9 # prepare basic build environment
10 FROM nvidia/cuda:12.1.0-devel-ubuntu22.04 AS dev
docs/source/assets/dev/dockerfile-stages-dependency.png
Binary files /dev/null and b/docs/source/assets/dev/dockerfile-stages-dependency.png differ
docs/source/dev/dockerfile/dockerfile.rst new
+50
@@ -0,0 +1,50 @@
1 +Dockerfile
2 +====================
3 +
4 +See `here <https://github.com/vllm-project/vllm/blob/main/Dockerfile>`_ for the main Dockerfile to construct
5 +the image for running an OpenAI compatible server with vLLM.
6 +
7 +- Below is a visual representation of the multi-stage Dockerfile. The build graph contains the following nodes:
8 +
9 + - All build stages
10 + - The default build target (highlighted in grey)
11 + - External images (with dashed borders)
12 +
13 + The edges of the build graph represent:
14 +
15 + - FROM ... dependencies (with a solid line and a full arrow head)
16 + - COPY --from=... dependencies (with a dashed line and an empty arrow head)
17 + - RUN --mount=(.*)from=... dependencies (with a dotted line and an empty diamond arrow head)
18 +
19 + .. figure:: ../../assets/dev/dockerfile-stages-dependency.png
20 + :alt: query
21 + :width: 100%
22 + :align: center
23 +
24 + Made using: https://github.com/patrickhoefler/dockerfilegraph
25 +
26 + Commands to regenerate the build graph (make sure to run it **from the `root` directory of the vLLM repository** where the dockerfile is present):
27 +
28 + .. code:: bash
29 +
30 + dockerfilegraph -o png --legend --dpi 200 --max-label-length 50 --filename Dockerfile
31 +
32 + or in case you want to run it directly with the docker image:
33 +
34 + .. code:: bash
35 +
36 + docker run \
37 + --rm \
38 + --user "$(id -u):$(id -g)" \
39 + --workdir /workspace \
40 + --volume "$(pwd)":/workspace \
41 + ghcr.io/patrickhoefler/dockerfilegraph:alpine \
42 + --output png \
43 + --dpi 200 \
44 + --max-label-length 50 \
45 + --filename Dockerfile \
46 + --legend
47 +
48 + (To run it for a different file, you can pass in a different argument to the flag `--filename`.)
49 +
50 +
\ No newline at end of file
docs/source/index.rst
+1
@@ -102,6 +102,7 @@ Documentation
102 dev/sampling_params
103 dev/engine/engine_index
104 dev/kernel/paged_attention
105 + dev/dockerfile/dockerfile
106
107 Indices and tables
108 ==================