A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 13, 08:15 UTC
main
[Frontend] split append tool output (#28333)
Andrew Xia committed about 1 year ago
[n-gen] DO NOT repeatedly return finished child requests (#28591)
Jialin Ouyang committed about 1 year ago
Rename clashing method names for vLLM model protocol (#27583)
Harry Mellor committed about 1 year ago
[Docs] Add some details about what the MoE block needs for the Transformers backend (#28588)
Harry Mellor committed about 1 year ago
[BugFix] Fix `mm_encoder_attn_backend` arg type checking (#28599)
Nick Hill committed about 1 year ago
[Bugfix] Fix SM100 gpt-oss regression due to faulty attn sink support (#28561)
Michael Goin committed about 1 year ago
[platform] Move get_cu_count to utils (#27005)
wangxiyuan committed about 1 year ago
[ROCm][Bugfix] Revert removing setuptools version restriction (#28592)
Gregory Shtrasberg committed about 1 year ago
[Docs] Update meetups.md description (#28583)
Michael Goin committed about 1 year ago
Skip models that cannot currently init on Transformers v5 (#28471)
Harry Mellor committed about 1 year ago
[MoE][Kernel][Perf] Improve Shared Expert Stream Overlap (#28406)
Alexander Matveev committed about 1 year ago
[Core][AMD] Migrate fully transparent sleep mode to ROCm platform (#12695)
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 committed about 1 year ago
[Bugfix] Adjust Marlin CUDA arch selection to 8.0+PTX;9.0+PTX (#28294)
Michael Goin committed about 1 year ago
[TPU] Support GCS path in VLLM_TORCH_PROFILER_DIR (#28487)
QiliangCui committed about 1 year ago
vllmellm committed about 1 year ago
[Performance][B200] Fix deepgemm prologue (#27897)
Varun Sundar Rabindranath committed about 1 year ago
[Misc]Fix typo in llm_engine.py (#28584)
Wei Wei committed about 1 year ago
[BugFix] Priority scheduling and spec tokens preemption (#28558)
Andy Lo committed about 1 year ago
[KV connector][WIP] KV cache proxy based on LMCache multi-process mode (#27902)
Yihua Cheng committed about 1 year ago
[CI] Skip "Multi-Modal Models Test (Extended) 3" test that's broken in current Transformers (#28559)
Harry Mellor committed about 1 year ago