@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 9eed4d1f3e
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 06:45 UTC

Update README.md (#1292)

Zhuohan Li committed Oct 8, 2023 at 23:15 UTC 9eed4d1f3e012d52059ae28a956b38f34890376a
9 files changed +1 -39
README.md
+1 -39
@@ -16,16 +16,8 @@ Easy, fast, and cheap LLM serving for everyone
16
17 ---
18
19 -**The First vLLM Bay Area Meetup (Oct 5th 6pm-8pm PT)**
20 -
21 -We are excited to invite you to the first vLLM meetup!
22 -The vLLM team will share recent updates and roadmap.
23 -We will also have vLLM users and contributors coming up to the stage to share their experiences.
24 -Please register [here](https://lu.ma/first-vllm-meetup) and join us!
25 -
26 ----
27 -
19 *Latest News* 🔥
20 +- [2023/10] We hosted [the first vLLM meetup](https://lu.ma/first-vllm-meetup) in SF! Please find the meetup slides [here](https://docs.google.com/presentation/d/1QL-XPFXiFpDBh86DbEegFXBXFXjix4v032GhShbKf3s/edit?usp=sharing).
21 - [2023/09] We created our [Discord server](https://discord.gg/jz7wjKhh6g)! Join us to discuss vLLM and LLM serving! We will also post the latest announcements and updates there.
22 - [2023/09] We released our [PagedAttention paper](https://arxiv.org/abs/2309.06180) on arXiv!
23 - [2023/08] We would like to express our sincere gratitude to [Andreessen Horowitz](https://a16z.com/2023/08/30/supporting-the-open-source-ai-community/) (a16z) for providing a generous grant to support the open-source development and research of vLLM.
@@ -82,36 +74,6 @@ Visit our [documentation](https://vllm.readthedocs.io/en/latest/) to get started
74 - [Quickstart](https://vllm.readthedocs.io/en/latest/getting_started/quickstart.html)
75 - [Supported Models](https://vllm.readthedocs.io/en/latest/models/supported_models.html)
76
85 -## Performance
86 -
87 -vLLM outperforms Hugging Face Transformers (HF) by up to 24x and Text Generation Inference (TGI) by up to 3.5x, in terms of throughput.
88 -For details, check out our [blog post](https://vllm.ai).
89 -
90 -<p align="center">
91 - <picture>
92 - <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a10g_n1_dark.png">
93 - <img src="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a10g_n1_light.png" width="45%">
94 - </picture>
95 - <picture>
96 - <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a100_n1_dark.png">
97 - <img src="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a100_n1_light.png" width="45%">
98 - </picture>
99 - <br>
100 - <em> Serving throughput when each request asks for 1 output completion. </em>
101 -</p>
102 -
103 -<p align="center">
104 - <picture>
105 - <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a10g_n3_dark.png">
106 - <img src="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a10g_n3_light.png" width="45%">
107 - </picture>
108 - <picture>
109 - <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a100_n3_dark.png">
110 - <img src="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a100_n3_light.png" width="45%">
111 - </picture> <br>
112 - <em> Serving throughput when each request asks for 3 output completions. </em>
113 -</p>
114 -
77 ## Contributing
78
79 We welcome and value any contributions and collaborations.
docs/source/assets/figures/perf_a100_n1_dark.png
Binary files a/docs/source/assets/figures/perf_a100_n1_dark.png and /dev/null differ
docs/source/assets/figures/perf_a100_n1_light.png
Binary files a/docs/source/assets/figures/perf_a100_n1_light.png and /dev/null differ
docs/source/assets/figures/perf_a100_n3_dark.png
Binary files a/docs/source/assets/figures/perf_a100_n3_dark.png and /dev/null differ
docs/source/assets/figures/perf_a100_n3_light.png
Binary files a/docs/source/assets/figures/perf_a100_n3_light.png and /dev/null differ
docs/source/assets/figures/perf_a10g_n1_dark.png
Binary files a/docs/source/assets/figures/perf_a10g_n1_dark.png and /dev/null differ
docs/source/assets/figures/perf_a10g_n1_light.png
Binary files a/docs/source/assets/figures/perf_a10g_n1_light.png and /dev/null differ
docs/source/assets/figures/perf_a10g_n3_dark.png
Binary files a/docs/source/assets/figures/perf_a10g_n3_dark.png and /dev/null differ
docs/source/assets/figures/perf_a10g_n3_light.png
Binary files a/docs/source/assets/figures/perf_a10g_n3_light.png and /dev/null differ