@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / c0d9a98d0c
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[Doc] Include performance benchmark in README (#9135)

Kuntai Du committed Oct 7, 2024 at 15:04 UTC c0d9a98d0c7182b73c2e7f88508e690a186bf0e3
1 file changed +1 -1
README.md
+1 -1
@@ -42,7 +42,7 @@ vLLM is fast with:
42 - Speculative decoding
43 - Chunked prefill
44
45 -**Performance benchmark**: We include a [performance benchmark](https://buildkite.com/vllm/performance-benchmark/builds/4068) that compares the performance of vLLM against other LLM serving engines ([TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM), [text-generation-inference](https://github.com/huggingface/text-generation-inference) and [lmdeploy](https://github.com/InternLM/lmdeploy)).
45 +**Performance benchmark**: We include a performance benchmark at the end of [our blog post](https://blog.vllm.ai/2024/09/05/perf-update.html). It compares the performance of vLLM against other LLM serving engines ([TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM), [SGLang](https://github.com/sgl-project/sglang) and [LMDeploy](https://github.com/InternLM/lmdeploy)). The implementation is under [nightly-benchmarks folder](.buildkite/nightly-benchmarks/) and you can [reproduce](https://github.com/vllm-project/vllm/issues/8176) this benchmark using our one-click runnable script.
46
47 vLLM is flexible and easy to use with:
48