A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 19:45 UTC
[Doc] More neutral K8s deployment guide (#14084)
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
Yuan Tang committed
Mar 14, 2025 at 19:12 UTC
54a8804455a14234ba246f7cbaf29fb5e8587d64
1 file changed
+10
-8
docs/source/deployment/k8s.md
+10
-8
@@ -4,17 +4,19 @@
4
5
Deploying vLLM on Kubernetes is a scalable and efficient way to serve machine learning models. This guide walks you through deploying vLLM using native Kubernetes.
6
7
---------
8
-
9
-Alternatively, you can also deploy Kubernetes using [helm chart](https://docs.vllm.ai/en/latest/deployment/frameworks/helm.html). There are also open-source projects available to make your deployment even smoother.
10
-
11
-* [vLLM production-stack](https://github.com/vllm-project/production-stack): Born out of a Berkeley-UChicago collaboration, vLLM production stack is a project that contains latest research and community effort, while still delivering production-level stability and performance. Checkout the [documentation page](https://docs.vllm.ai/en/latest/deployment/integrations/production-stack.html) for more details and examples.
12
-
13
---------
7
+Alternatively, you can deploy vLLM to Kubernetes using any of the following:
8
+* [Helm](frameworks/helm.md)
9
+* [InftyAI/llmaz](integrations/llmaz.md)
10
+* [KServe](integrations/kserve.md)
11
+* [kubernetes-sigs/lws](frameworks/lws.md)
12
+* [meta-llama/llama-stack](integrations/llamastack.md)
13
+* [substratusai/kubeai](integrations/kubeai.md)
14
+* [vllm-project/aibrix](https://github.com/vllm-project/aibrix)
15
+* [vllm-project/production-stack](integrations/production-stack.md)
16
17
## Pre-requisite
18
17
-Ensure that you have a running Kubernetes environment with GPU (you can follow [this tutorial](https://github.com/vllm-project/production-stack/blob/main/tutorials/00-install-kubernetes-env.md) to install a Kubernetes environment on a bare-metal GPU machine).
19
+Ensure that you have a running [Kubernetes cluster with GPUs](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/).
20
21
## Deployment using native K8s
22