A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Plugin] Make plugin group clear (#26757)
wangxiyuan committed about 1 year ago
[Bugfix][FE]: Always include usage with `--enable-force-include-usage ` (#20983)
Max Wittig committed about 1 year ago
[CI/Build] Use 127.0.0.1 instead of localhost in utils (#26750)
Ye (Charlotte) Qi committed about 1 year ago
[Model][Bugfix]fix ernie45 load failed due to ernie45 eplb code (#26684)
CSWYF3634076 committed about 1 year ago
scheduler.py: Update the name of the default scheduler. (#26758)
Ryan Li committed about 1 year ago
[ci] Adding the test-amd.yaml for test definitions for the AMD backend. (alternative PR) (#26718)
Alexei-V-Ivanov-AMD committed about 1 year ago
[CI] [ROCm] Automate CC list for ROCm related issue (#26753)
vllmellm committed about 1 year ago
Fix lora tests failure in TPU CI due to the removal of LoRA bias (#26723)
XiongfeiWei committed about 1 year ago
[Docs] Add a start tag to build.inc.md (#26747)
Michael Yao committed about 1 year ago
Add support for the /rerank endpoint in vllm bench serve (#26602)
Maximilien de Bayser committed about 1 year ago
[Feature][Quantization] auto_round format add support for regex (#24024)
Heng Guo committed about 1 year ago
[torch.compile] Unwrap fused_marlin_moe custom op (#26739)
Varun Sundar Rabindranath committed about 1 year ago
[build][torch.compile] upgrade depyf version (#26702)
youkaichao committed about 1 year ago
[Easy] Fix env type check errors from VLLM_DEBUG_LOG_API_SERVER_RESPONSE (#26742)
Jialin Ouyang committed about 1 year ago
[compile] Enable sequence parallelism for full cuda graph without specifying compile sizes (#26681)
Angela Yi committed about 1 year ago
[UX] Replace VLLM_ALL2ALL_BACKEND with --all2all-backend (#26732)
Michael Goin committed about 1 year ago
[Misc][DP] support customized aggregated logger for dp (#24354)
Lucia Fang committed about 1 year ago
Add tests for chunked prefill and prefix cache with causal pooling models (#26526)
Maximilien de Bayser committed about 1 year ago
[ResponseAPI] Further polish message serialization and unit tests (#26728)
Jialin Ouyang committed about 1 year ago
Pruning kernel Core Tests (#26727)
Fardin Hoque committed about 1 year ago