A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 22:15 UTC
Add llmaz as another integration (#13643)
Signed-off-by: kerthcet <kerthcet@gmail.com>
Kante Yin committed
Feb 21, 2025 at 11:52 UTC
44c33f01f309099066d358b1f51f0d0a703b46ca
2 files changed
+8
docs/source/deployment/integrations/index.md
+1
@@ -6,4 +6,5 @@
6
kserve
7
kubeai
8
llamastack
9
+llmaz
10
:::
docs/source/deployment/integrations/llmaz.md
new
+7
@@ -0,0 +1,7 @@
1
+(deployment-llmaz)=
2
+
3
+# llmaz
4
+
5
+[llmaz](https://github.com/InftyAI/llmaz) is an easy-to-use and advanced inference platform for large language models on Kubernetes, aimed for production use. It uses vLLM as the default model serving backend.
6
+
7
+Please refer to the [Quick Start](https://github.com/InftyAI/llmaz?tab=readme-ov-file#quick-start) for more details.