@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / a38524f338
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 20:45 UTC

[DOC] - Add docker image to Cerebrium Integration (#6510)

milo157 committed Jul 17, 2024 at 13:22 UTC a38524f338a100bb0f0121c3aa24e8c4a9160d51
1 file changed +3
docs/source/serving/deploying_with_cerebrium.rst
+3
@@ -28,6 +28,9 @@ Next, to install the required packages, add the following to your cerebrium.toml
28
29 .. code-block:: toml
30
31 + [cerebrium.deployment]
32 + docker_base_image_url = "nvidia/cuda:12.1.1-runtime-ubuntu22.04"
33 +
34 [cerebrium.dependencies.pip]
35 vllm = "latest"
36