A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[CI/Build] Update causal-conv1d installation (#27529)
Cyrus Leung committed about 1 year ago
Revert "[CI/Build] Use CPU for mm processing test on CI (#27522)" (#27531)
Cyrus Leung committed about 1 year ago
[Chore]:Extract math and argparse utilities to separate modules (#27188)
Yeshwanth N committed about 1 year ago
[Doc] Fix links to GH projects (#27530)
Cyrus Leung committed about 1 year ago
[Doc] Remove Molmo warning (#27527)
Cyrus Leung committed about 1 year ago
[bugfix]fix empty prompts for async-engine mode in benchmark throughput (#27494)
Lucia Fang committed about 1 year ago
[Bugfix] Fix processor initialization for model from modelscope instead of HF (#27461)
rongfu.leng committed about 1 year ago
JartX committed about 1 year ago
[CI/Build] Use CPU for mm processing test on CI (#27522)
Isotr0py committed about 1 year ago
[CI/Build] Refactor processing tests (#27470)
Cyrus Leung committed about 1 year ago
[Attention] Add missing kv cache scale setup (#27490)
Matthew Bonanni committed about 1 year ago
[Misc] Simplify max tokens in multimodal registry (#27500)
Cyrus Leung committed about 1 year ago
[Core][Hybrid allocator + kv connector 1/n] Enable hybrid allocator + KV cache connector (#25712)
Kuntai Du committed about 1 year ago
Revert "[Misc] Remove use of CUDA_VISIBLE_DEVICES for device selectio… (#27502)
Zhuohan Li committed about 1 year ago
[CI] Add tests for cudagraph (#27391)
Jiangyun Zhu committed about 1 year ago
[KVConnector] Migrate the LMCache integration code to be vLLM native (#25542)
Yihua Cheng committed about 1 year ago
[Misc][DP] Guard mxfp4 implementation selection (#27484)
Varun Sundar Rabindranath committed about 1 year ago
[Log] Optimize Startup Log (#26740)
Wentao Ye committed about 1 year ago
[Distributed] Basic set of configuration for large EP deployment on GB200 (#27328)
Pengchao Wang committed about 1 year ago
[Perf][Async Scheduling] Remove CPU->GPU sync in dummy_run (#27455)
Lehua Ding committed about 1 year ago