A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 20:45 UTC
[DOC] - Add docker image to Cerebrium Integration (#6510)
milo157 committed
Jul 17, 2024 at 13:22 UTC
a38524f338a100bb0f0121c3aa24e8c4a9160d51
1 file changed
+3
docs/source/serving/deploying_with_cerebrium.rst
+3
@@ -28,6 +28,9 @@ Next, to install the required packages, add the following to your cerebrium.toml
28
29
.. code-block:: toml
30
31
+ [cerebrium.deployment]
32
+ docker_base_image_url = "nvidia/cuda:12.1.1-runtime-ubuntu22.04"
33
+
34
[cerebrium.dependencies.pip]
35
vllm = "latest"
36