A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 08:00 UTC
[Minor] Fix URLs (#166)
Woosuk Kwon committed
Jun 19, 2023 at 22:57 UTC
794e578de0f9657e169c713908a5fb8d25046b13
3 files changed
+8
-8
README.md
+6
-6
@@ -10,7 +10,7 @@ Easy, fast, and cheap LLM serving for everyone
10
</h3>
11
12
<p align="center">
13
-| <a href="https://vllm.readthedocs.io/en/latest/"><b>Documentation</b></a> | <a href=""><b>Blog</b></a> |
13
+| <a href="https://vllm.readthedocs.io/en/latest/"><b>Documentation</b></a> | <a href="https://vllm.ai"><b>Blog</b></a> | <a href="https://github.com/vllm-project/vllm/discussions"><b>Discussions</b></a> |
14
15
</p>
16
@@ -18,7 +18,7 @@ Easy, fast, and cheap LLM serving for everyone
18
19
*Latest News* 🔥
20
21
-- [2023/06] We officially released vLLM! vLLM has powered [LMSYS Vicuna and Chatbot Arena](https://chat.lmsys.org) since mid April. Check out our [blog post]().
21
+- [2023/06] We officially released vLLM! vLLM has powered [LMSYS Vicuna and Chatbot Arena](https://chat.lmsys.org) since mid April. Check out our [blog post](https://vllm.ai).
22
23
---
24
@@ -62,7 +62,7 @@ Visit our [documentation](https://vllm.readthedocs.io/en/latest/) to get started
62
## Performance
63
64
vLLM outperforms HuggingFace Transformers (HF) by up to 24x and Text Generation Inference (TGI) by up to 3.5x, in terms of throughput.
65
-For details, check out our [blog post]().
65
+For details, check out our [blog post](https://vllm.ai).
66
67
<p align="center">
68
<picture>
@@ -70,7 +70,7 @@ For details, check out our [blog post]().
70
<img src="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a10g_n1_light.png" width="45%">
71
</picture>
72
<picture>
73
- <source media="(prefers-color-scheme: dark)" srcset="./docs/source/assets/figures/perf_a100_n1_dark.png">
73
+ <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a100_n1_dark.png">
74
<img src="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a100_n1_light.png" width="45%">
75
</picture>
76
<br>
@@ -79,11 +79,11 @@ For details, check out our [blog post]().
79
80
<p align="center">
81
<picture>
82
- <source media="(prefers-color-scheme: dark)" srcset="./docs/source/assets/figures/perf_a10g_n3_dark.png">
82
+ <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a10g_n3_dark.png">
83
<img src="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a10g_n3_light.png" width="45%">
84
</picture>
85
<picture>
86
- <source media="(prefers-color-scheme: dark)" srcset="./docs/source/assets/figures/perf_a100_n3_dark.png">
86
+ <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a100_n3_dark.png">
87
<img src="https://raw.githubusercontent.com/vllm-project/vllm/main/docs/source/assets/figures/perf_a100_n3_light.png" width="45%">
88
</picture> <br>
89
<em> Serving throughput when each request asks for 3 output completions. </em>
docs/source/index.rst
+1
-1
@@ -40,7 +40,7 @@ vLLM is flexible and easy to use with:
40
* Streaming outputs
41
* OpenAI-compatible API server
42
43
-For more information, please refer to our `blog post <>`_.
43
+For more information, please refer to our `blog post <https://vllm.ai>`_.
44
45
46
Documentation
docs/source/models/adding_model.rst
+1
-1
@@ -18,7 +18,7 @@ This document provides a high-level guide on integrating a `HuggingFace Transfor
18
0. Fork the vLLM repository
19
--------------------------------
20
21
-Start by forking our `GitHub <https://github.com/vllm-project/vllm/issues>`_ repository and then :ref:`build it from source <build_from_source>`.
21
+Start by forking our `GitHub <https://github.com/vllm-project/vllm/>`_ repository and then :ref:`build it from source <build_from_source>`.
22
This gives you the ability to modify the codebase and test your model.
23
24