A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix][EPD][Model Runner V2] Skip gather mm embeddings for encoder only instance (#51222)
Tianyu Guo committed 3 months ago
[Feat] Support thinking_token_budget in Model Runner V2 (#46727)
Chauncey committed 3 months ago
[CI] Re-enable FI autotune in GSM8K config for Qwen3.5-35B-A3B (#51293)
Artem Perevedentsev committed 3 months ago
fix pre-commit broken (#51341)
Kunshang Ji committed 3 months ago
[Kernel] Support Nvfp4 Cutedsl Moe Swiglu-oai and Relu2(non-gated) Activation (#47106)
Bi Tiekai committed 3 months ago
[V1] Copy NaN-in-logits counts to host asynchronously (#51304)
Nick Hill committed 3 months ago
[CI] Exclude KV-connector subtree from broad source dependencies (#51046)
Nicolò Lucchesi committed 3 months ago
[Quantization] Share online weight scales across TP (#49764)
Matej Sirovatka committed 3 months ago
[Model Runner V2] Fix -1 placeholder draft token ids in rejection sam… (#50939)
Giancarlo Delfin committed 3 months ago
docs(governance): refresh committers list, add TSC note, update project leads (#51300)
Simon Mo committed 3 months ago
[ModelRunner v2] Enable decoder token-wise pooling (#50931)
Taneem Ibrahim committed 3 months ago
fix: resolve silent request skipping in PRIORITY scheduling (#49206)
Tejas committed 3 months ago
[Misc] Upgrade fastsafetensors version, fix metadata is null (#50827)
rongfu.leng committed 3 months ago
[ModelRunner V2] Minor indexing optimizations (#51210)
Nick Hill committed 3 months ago
[Bugfix] Fix packed KV block zeroing stride (#50276)
wangxian001 committed 3 months ago
[Mypy Fix] Mypy fix for "vllm/model_executor/models/[aA][bB]" (#48977)
Wentao Ye committed 3 months ago
[ROCm][MLA] Use asm decode for non-divisor small head counts (#50578)
vanshbhatia-amd committed 3 months ago
[Frontend] Watch frontend processes during engine startup (#43417)
Bugen Zhao committed 3 months ago
[ROCm][CI] Update AITER AR+RMS e2e fusion counts for final-norm coverage (#51273)
Divakar Verma committed 3 months ago
[Bugfix] Keep mamba align prefill chunks block-aligned past last_cache_position (#51113)
Yifan Qiao committed 3 months ago