@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / e83db9e7e3
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 23:15 UTC

[Doc] Update docker references (#5614)

Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>

Rafael Vasquez committed Jun 19, 2024 at 18:01 UTC e83db9e7e3d776cd9b059a49024f3950ef579b41
2 files changed +13 -14
docs/source/dev/dockerfile/dockerfile.rst
+10 -10
@@ -2,19 +2,19 @@ Dockerfile
2 ====================
3
4 See `here <https://github.com/vllm-project/vllm/blob/main/Dockerfile>`_ for the main Dockerfile to construct
5 -the image for running an OpenAI compatible server with vLLM.
5 +the image for running an OpenAI compatible server with vLLM. More information about deploying with Docker can be found `here <https://docs.vllm.ai/en/stable/serving/deploying_with_docker.html>`_.
6
7 -- Below is a visual representation of the multi-stage Dockerfile. The build graph contains the following nodes:
7 +Below is a visual representation of the multi-stage Dockerfile. The build graph contains the following nodes:
8
9 - - All build stages
10 - - The default build target (highlighted in grey)
11 - - External images (with dashed borders)
9 +- All build stages
10 +- The default build target (highlighted in grey)
11 +- External images (with dashed borders)
12
13 - The edges of the build graph represent:
14 -
15 - - FROM ... dependencies (with a solid line and a full arrow head)
16 - - COPY --from=... dependencies (with a dashed line and an empty arrow head)
17 - - RUN --mount=(.*)from=... dependencies (with a dotted line and an empty diamond arrow head)
13 +The edges of the build graph represent:
14 +
15 +- FROM ... dependencies (with a solid line and a full arrow head)
16 +- COPY --from=... dependencies (with a dashed line and an empty arrow head)
17 +- RUN --mount=(.*)from=... dependencies (with a dotted line and an empty diamond arrow head)
18
19 .. figure:: ../../assets/dev/dockerfile-stages-dependency.png
20 :alt: query
docs/source/serving/deploying_with_docker.rst
+3 -4
@@ -3,9 +3,8 @@
3 Deploying with Docker
4 ============================
5
6 -vLLM offers official docker image for deployment.
7 -The image can be used to run OpenAI compatible server.
8 -The image is available on Docker Hub as `vllm/vllm-openai <https://hub.docker.com/r/vllm/vllm-openai/tags>`_.
6 +vLLM offers an official Docker image for deployment.
7 +The image can be used to run OpenAI compatible server and is available on Docker Hub as `vllm/vllm-openai <https://hub.docker.com/r/vllm/vllm-openai/tags>`_.
8
9 .. code-block:: console
10
@@ -25,7 +24,7 @@ The image is available on Docker Hub as `vllm/vllm-openai <https://hub.docker.co
24 memory to share data between processes under the hood, particularly for tensor parallel inference.
25
26
28 -You can build and run vLLM from source via the provided dockerfile. To build vLLM:
27 +You can build and run vLLM from source via the provided `Dockerfile <https://github.com/vllm-project/vllm/blob/main/Dockerfile>`_. To build vLLM:
28
29 .. code-block:: console
30