A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Bugfix] Fix idefics3 `tie_word_embeddings` (#25454)
Isotr0py committed about 1 year ago
[Test]: Hermes tool parser stream output error in Qwen3 case (#25203)
Andreas Hartel committed about 1 year ago
[Misc] Move DP for ViT code inside model executor dir (#25459)
Cyrus Leung committed about 1 year ago
[Frontend] Add a new xml-based tool parser for qwen3-coder (#25028)
Zhikaiiii committed about 1 year ago
Handle triton kernel import exception (#25319)
Ming Yang committed about 1 year ago
[Model] Enable DP for ViT in Qwen2-VL (#25445)
Cyrus Leung committed about 1 year ago
[NIXL][OOT platform] support nixl_connector with oot platform and other nixl_backend (#25121)
Chendi.Xue committed about 1 year ago
[DP/EP][GPTOSS] Use triton matmul-ogs kernels for GPTOSS DP/EP (#24588)
Varun Sundar Rabindranath committed about 1 year ago
[Docs] Fix griffe warnings in vllm/lora/ops (#25369)
Michael Yao committed about 1 year ago
[Bugfix] Remove contiguous output req for context parallel MLA (#25414)
Michael Goin committed about 1 year ago
[benchmarks]allow skip ready check for bench serve (#25420)
Lucia Fang committed about 1 year ago
[XPU] Fix `compile_size` is `None` case. (#25433)
Kunshang Ji committed about 1 year ago
[feat] Support MRoPE + YaRN (#25384)
JJJYmmm committed about 1 year ago
[Bug] Fix Long Context OOM Issue (#25290)
Wentao Ye committed about 1 year ago
[V0 deprecation] Remove `_set_default_args_v0` function (#25409)
Isotr0py committed about 1 year ago
[V0 deprecation] Remove platform v1 controling interface (#25410)
Isotr0py committed about 1 year ago
[Perf] Apply torch.compile for `per_block_cast_to_fp8` (#24611)
Wentao Ye committed about 1 year ago
[Performance] Remove input pads in cutlass_mla and optimize v_proj output handling (#25184)
Alexander Matveev committed about 1 year ago
[BugFix] [DP/EP] Fix slow execution when BS <= DP (#25407)
Matthew Bonanni committed about 1 year ago
[Bugfix] Fix missing `clear_connector_metadata` (#25397)
Nicolò Lucchesi committed about 1 year ago