A high-throughput and memory-efficient inference and serving engine for LLMs
main
[XPU] Route weightless RMSNorm to _C dispatch (#47121)
Yintong Lu committed 3 months ago
sungsoo ha committed 3 months ago
[Bugfix][Multimodal] Include media IO config in MM cache hash (#49975)
Guan-Ming Chiu committed 3 months ago
[KV Offload] Move CPUOffloadingSpec onto SharedOffloadRegion (#50094)
Chang Guo committed 3 months ago
[CI/Perf] Fix malformed serving benchmark config (#43538)
Minh Vu committed 3 months ago
[Rust Frontend] Add --limit-mm-per-prompt support (#49604)
Maria Guevara committed 3 months ago
[CI] Fix MXFP8 MOE backend selection tests on gfx942 (#50222)
fxmarty-amd committed 3 months ago
[EC Connector] Add has_pending_push_work (#49582)
omerpaz95 committed 3 months ago
[XPU] upgrade to torch 2.13 (#48677)
Yan Ma committed 3 months ago
[Misc][Minimax-M3]add default video_processor (#50092)
rongfu.leng committed 3 months ago
[BugFix] eagle draft max position embeddings (#49343)
Jared Wen committed 3 months ago
[Model] Support Qwen3.5 text-only dense and MoE models (#50210)
Perkz Zheng committed 3 months ago
[Model] Add Kimi K3 support: Python frontend [2/2] (#50093)
Bugen Zhao committed 3 months ago
[Frontend] Reuse prefill token ids on the decode chat path for disaggregated serving (#48145)
Seiji Eicher committed 3 months ago
[CPU] Fix FP8 attention scratchpad sizing (#50194)
Tianmu Li committed 3 months ago
[CI] Allow PR comment acknowledgements (#50211)
Kevin H. Luu committed 3 months ago
[XPU][UT][CI] add xpu config to run gpt-oss accuracy in ut and ci (#48703)
zofia committed 3 months ago
[Model] Add Kimi K3 support: model files and kernels [1/N] (#50089)
Jiangyun Zhu committed 3 months ago
fused_moe: add VLLM_TRITON_USE_TD tensor-descriptor path (#42436)
Artur Fierka committed 3 months ago
[Test] dynamic_shapes_compilation (#49974)
Jared Wen committed 3 months ago