A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 22:00 UTC
[doc] update pipeline parallel in readme (#6347)
youkaichao committed
Jul 11, 2024 at 11:38 UTC
2d23b42d9255f724f955a1cf91ed78c983854737
2 files changed
+2
-2
README.md
+1
-1
@@ -56,7 +56,7 @@ vLLM is flexible and easy to use with:
56
57
- Seamless integration with popular Hugging Face models
58
- High-throughput serving with various decoding algorithms, including *parallel sampling*, *beam search*, and more
59
-- Tensor parallelism support for distributed inference
59
+- Tensor parallelism and pipieline parallelism support for distributed inference
60
- Streaming outputs
61
- OpenAI-compatible API server
62
- Support NVIDIA GPUs, AMD CPUs and GPUs, Intel CPUs and GPUs, PowerPC CPUs
docs/source/index.rst
+1
-1
@@ -38,7 +38,7 @@ vLLM is flexible and easy to use with:
38
39
* Seamless integration with popular HuggingFace models
40
* High-throughput serving with various decoding algorithms, including *parallel sampling*, *beam search*, and more
41
-* Tensor parallelism support for distributed inference
41
+* Tensor parallelism and pipieline parallelism support for distributed inference
42
* Streaming outputs
43
* OpenAI-compatible API server
44
* Support NVIDIA GPUs and AMD GPUs