@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 2d23b42d92
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 22:00 UTC

[doc] update pipeline parallel in readme (#6347)

youkaichao committed Jul 11, 2024 at 11:38 UTC 2d23b42d9255f724f955a1cf91ed78c983854737
2 files changed +2 -2
README.md
+1 -1
@@ -56,7 +56,7 @@ vLLM is flexible and easy to use with:
56
57 - Seamless integration with popular Hugging Face models
58 - High-throughput serving with various decoding algorithms, including *parallel sampling*, *beam search*, and more
59 -- Tensor parallelism support for distributed inference
59 +- Tensor parallelism and pipieline parallelism support for distributed inference
60 - Streaming outputs
61 - OpenAI-compatible API server
62 - Support NVIDIA GPUs, AMD CPUs and GPUs, Intel CPUs and GPUs, PowerPC CPUs
docs/source/index.rst
+1 -1
@@ -38,7 +38,7 @@ vLLM is flexible and easy to use with:
38
39 * Seamless integration with popular HuggingFace models
40 * High-throughput serving with various decoding algorithms, including *parallel sampling*, *beam search*, and more
41 -* Tensor parallelism support for distributed inference
41 +* Tensor parallelism and pipieline parallelism support for distributed inference
42 * Streaming outputs
43 * OpenAI-compatible API server
44 * Support NVIDIA GPUs and AMD GPUs