A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
[Doc] Update docker references (#5614)
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Rafael Vasquez committed
Jun 19, 2024 at 18:01 UTC
e83db9e7e3d776cd9b059a49024f3950ef579b41
2 files changed
+13
-14
docs/source/dev/dockerfile/dockerfile.rst
+10
-10
@@ -2,19 +2,19 @@ Dockerfile
2
====================
3
4
See `here <https://github.com/vllm-project/vllm/blob/main/Dockerfile>`_ for the main Dockerfile to construct
5
-the image for running an OpenAI compatible server with vLLM.
5
+the image for running an OpenAI compatible server with vLLM. More information about deploying with Docker can be found `here <https://docs.vllm.ai/en/stable/serving/deploying_with_docker.html>`_.
6
7
-- Below is a visual representation of the multi-stage Dockerfile. The build graph contains the following nodes:
7
+Below is a visual representation of the multi-stage Dockerfile. The build graph contains the following nodes:
8
9
- - All build stages
10
- - The default build target (highlighted in grey)
11
- - External images (with dashed borders)
9
+- All build stages
10
+- The default build target (highlighted in grey)
11
+- External images (with dashed borders)
12
13
- The edges of the build graph represent:
14
-
15
- - FROM ... dependencies (with a solid line and a full arrow head)
16
- - COPY --from=... dependencies (with a dashed line and an empty arrow head)
17
- - RUN --mount=(.*)from=... dependencies (with a dotted line and an empty diamond arrow head)
13
+The edges of the build graph represent:
14
+
15
+- FROM ... dependencies (with a solid line and a full arrow head)
16
+- COPY --from=... dependencies (with a dashed line and an empty arrow head)
17
+- RUN --mount=(.*)from=... dependencies (with a dotted line and an empty diamond arrow head)
18
19
.. figure:: ../../assets/dev/dockerfile-stages-dependency.png
20
:alt: query
docs/source/serving/deploying_with_docker.rst
+3
-4
@@ -3,9 +3,8 @@
3
Deploying with Docker
4
============================
5
6
-vLLM offers official docker image for deployment.
7
-The image can be used to run OpenAI compatible server.
8
-The image is available on Docker Hub as `vllm/vllm-openai <https://hub.docker.com/r/vllm/vllm-openai/tags>`_.
6
+vLLM offers an official Docker image for deployment.
7
+The image can be used to run OpenAI compatible server and is available on Docker Hub as `vllm/vllm-openai <https://hub.docker.com/r/vllm/vllm-openai/tags>`_.
8
9
.. code-block:: console
10
@@ -25,7 +24,7 @@ The image is available on Docker Hub as `vllm/vllm-openai <https://hub.docker.co
24
memory to share data between processes under the hood, particularly for tensor parallel inference.
25
26
28
-You can build and run vLLM from source via the provided dockerfile. To build vLLM:
27
+You can build and run vLLM from source via the provided `Dockerfile <https://github.com/vllm-project/vllm/blob/main/Dockerfile>`_. To build vLLM:
28
29
.. code-block:: console
30