A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
Fix typo in README.md (#1033)
Ikko Eltociear Ashimine committed
Sep 14, 2023 at 04:55 UTC
3272d7a0b715a79059ebfcf8959d1ac0488ad18c
1 file changed
+3
-3
README.md
+3
-3
@@ -35,13 +35,13 @@ vLLM is fast with:
35
36
vLLM is flexible and easy to use with:
37
38
-- Seamless integration with popular HuggingFace models
38
+- Seamless integration with popular Hugging Face models
39
- High-throughput serving with various decoding algorithms, including *parallel sampling*, *beam search*, and more
40
- Tensor parallelism support for distributed inference
41
- Streaming outputs
42
- OpenAI-compatible API server
43
44
-vLLM seamlessly supports many Huggingface models, including the following architectures:
44
+vLLM seamlessly supports many Hugging Face models, including the following architectures:
45
46
- Aquila (`BAAI/Aquila-7B`, `BAAI/AquilaChat-7B`, etc.)
47
- Baichuan (`baichuan-inc/Baichuan-7B`, `baichuan-inc/Baichuan-13B-Chat`, etc.)
@@ -72,7 +72,7 @@ Visit our [documentation](https://vllm.readthedocs.io/en/latest/) to get started
72
73
## Performance
74
75
-vLLM outperforms HuggingFace Transformers (HF) by up to 24x and Text Generation Inference (TGI) by up to 3.5x, in terms of throughput.
75
+vLLM outperforms Hugging Face Transformers (HF) by up to 24x and Text Generation Inference (TGI) by up to 3.5x, in terms of throughput.
76
For details, check out our [blog post](https://vllm.ai).
77
78
<p align="center">