A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Core] Relax the LoRA max rank (#26461)
Jee Jee Li committed about 1 year ago
[CI/Build] Fix model nightly tests (#26466)
Cyrus Leung committed about 1 year ago
[Hybrid]: Decouple Kernel Block Size from KV Page Size (#24486)
Zhiyuan Li committed about 1 year ago
[Core][KVConnector] Propagate all tokens on resumed preemptions (#24926)
Qier Li committed about 1 year ago
[MM][Doc] Add documentation for configurable mm profiling (#26200)
Wenlong Wang committed about 1 year ago
[Hardware][AMD] Enable FlexAttention backend on ROCm (#26439)
Matt committed about 1 year ago
[Bugfix] Incorrect another MM data format in vllm bench throughput (#26462)
Huy Do committed about 1 year ago
[Bugfix] Catch and log invalid token ids in detokenizer #2 (#26445)
Nick Hill committed about 1 year ago
[Minor] Change warning->warning_once in preprocess (#26455)
Zhuohan Li committed about 1 year ago
[Bugfix] Set the minimum python version for gpt-oss (#26392)
Jee Jee Li committed about 1 year ago
[Misc] Redact ray runtime env before logging (#26302)
Rui Qiao committed about 1 year ago
Separate MLAAttention class from Attention (#25103)
Naveenraj Kamalakannan committed about 1 year ago
[Attention] Register FLASHMLA_SPARSE (#26441)
Matthew Bonanni committed about 1 year ago
[Kernels] Modular kernel refactor (#24812)
bnellnm committed about 1 year ago
[Bugfix] Respect min_tokens in scheduler stop check (#26317)
Elaine Zhao committed about 1 year ago
[Models] Improve iteration over layers (#26425)
Lukas Geiger committed about 1 year ago
[Kernel] Centralize platform kernel import in `current_platform.import_kernels` (#26286)
Nicolò Lucchesi committed about 1 year ago
Morrison Turnansky committed about 1 year ago
elvischenv committed about 1 year ago
[Attention] Implement universal BACKEND_MAP (#25900)
Matthew Bonanni committed about 1 year ago