A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
[Community] Add vLLM Discord server (#1086)
Zhuohan Li committed
Sep 18, 2023 at 12:23 UTC
c1026311b59446d1ada5f950ddbdbe0bb21943b0
1 file changed
+3
-2
README.md
+3
-2
@@ -10,13 +10,14 @@ Easy, fast, and cheap LLM serving for everyone
10
</h3>
11
12
<p align="center">
13
-| <a href="https://vllm.readthedocs.io/en/latest/"><b>Documentation</b></a> | <a href="https://vllm.ai"><b>Blog</b></a> | <a href="https://arxiv.org/abs/2309.06180"><b>Paper</b></a> | <a href="https://github.com/vllm-project/vllm/discussions"><b>Discussions</b></a> |
13
+| <a href="https://vllm.readthedocs.io/en/latest/"><b>Documentation</b></a> | <a href="https://vllm.ai"><b>Blog</b></a> | <a href="https://arxiv.org/abs/2309.06180"><b>Paper</b></a> | <a href="https://discord.gg/jz7wjKhh6g"><b>Discord</b></a> |
14
15
</p>
16
17
---
18
19
*Latest News* 🔥
20
+- [2023/09] We created our [Discord server](https://discord.gg/jz7wjKhh6g)! Join us to discuss vLLM and LLM serving! We will also post the latest announcements and updates there.
21
- [2023/09] We released our [PagedAttention paper](https://arxiv.org/abs/2309.06180) on arXiv!
22
- [2023/08] We would like to express our sincere gratitude to [Andreessen Horowitz](https://a16z.com/2023/08/30/supporting-the-open-source-ai-community/) (a16z) for providing a generous grant to support the open-source development and research of vLLM.
23
- [2023/07] Added support for LLaMA-2! You can run and serve 7B/13B/70B LLaMA-2s on vLLM with a single command!
@@ -111,7 +112,7 @@ Please check out [CONTRIBUTING.md](./CONTRIBUTING.md) for how to get involved.
112
If you use vLLM for your research, please cite our [paper](https://arxiv.org/abs/2309.06180):
113
```bibtex
114
@inproceedings{kwon2023efficient,
114
- title={Efficient Memory Management for Large Language Model Serving with PagedAttention},
115
+ title={Efficient Memory Management for Large Language Model Serving with PagedAttention},
116
author={Woosuk Kwon and Zhuohan Li and Siyuan Zhuang and Ying Sheng and Lianmin Zheng and Cody Hao Yu and Joseph E. Gonzalez and Hao Zhang and Ion Stoica},
117
booktitle={Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles},
118
year={2023}