A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[Doc] add KAITO to integrations (#25521)
Signed-off-by: "Abhishek Sheth" <absheth@microsoft.com>
abhisheksheth28 committed
Oct 6, 2025 at 02:30 UTC
77c95f72f78e4edefc75cfe42c974e85836ea8f5
2 files changed
+6
docs/deployment/integrations/kaito.md
new
+5
@@ -0,0 +1,5 @@
1
+# KAITO
2
+
3
+[KAITO](https://kaito-project.github.io/kaito/docs/) is a Kubernetes operator that supports deploying and serving LLMs with vLLM. It offers managing large models via container images with built-in OpenAI-compatible inference, auto-provisioning GPU nodes and curated model presets.
4
+
5
+Please refer to [quick start](https://kaito-project.github.io/kaito/docs/quick-start) for more details.
docs/deployment/k8s.md
+1
@@ -12,6 +12,7 @@ Alternatively, you can deploy vLLM to Kubernetes using any of the following:
12
13
- [Helm](frameworks/helm.md)
14
- [InftyAI/llmaz](integrations/llmaz.md)
15
+- [KAITO](integrations/kaito.md)
16
- [KServe](integrations/kserve.md)
17
- [KubeRay](integrations/kuberay.md)
18
- [kubernetes-sigs/lws](frameworks/lws.md)