@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / f72297562f
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 08:00 UTC

Add news for the vllm+skypilot example (#314)

Zhanghao Wu committed Jun 29, 2023 at 12:32 UTC f72297562f0afb13d762eec4dfd76af90f4c6fe2
1 file changed +1 -1
README.md
+1 -1
@@ -17,7 +17,7 @@ Easy, fast, and cheap LLM serving for everyone
17 ---
18
19 *Latest News* 🔥
20 -
20 +- [2023/06] Serving vLLM On any Cloud with SkyPilot. Check out a 1-click [example](https://github.com/skypilot-org/skypilot/blob/master/llm/vllm) to start the vLLM demo, and the [blog post](https://blog.skypilot.co/serving-llm-24x-faster-on-the-cloud-with-vllm-and-skypilot/) for the story behind vLLM development on the clouds.
21 - [2023/06] We officially released vLLM! FastChat-vLLM integration has powered [LMSYS Vicuna and Chatbot Arena](https://chat.lmsys.org) since mid-April. Check out our [blog post](https://vllm.ai).
22
23 ---