A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Kernel][DSV4.1] Fuse MXFP8 wo_b GEMM with sequence-parallel reduce-scatter (#57428)
Canlin Guo committed about 1 month ago
[Rust Frontend] Separate multimodal instrumentation from request timing (#58084)
Bugen Zhao committed about 1 month ago
[Kernel] Add opt-in load-time MXFP4 dequantization (#50814)
yinfengLiu committed about 1 month ago
[ROCm][Perf] Avoid extra reshape kernel in Qwen GDN output norm (#47842)
Mikko Tukiainen committed about 1 month ago
[Rust Frontend] Build full-output grammars from initialized reasoning parsers (#57340)
Bugen Zhao committed about 1 month ago
[Bugfix][Qwen2.5-VL] Honor video fps for temporal M-RoPE (#47736)
Ting SUN committed about 1 month ago
[Bugfix][Frontend] Validate mixed prompt embedding mask lengths (#57006)
shaohuaxi committed about 1 month ago
[Security] Reject min_tokens that exceeds the filled max_tokens default (#57731)
Juan Pérez de Algaba committed about 1 month ago
[Rust Frontend] Introduce parser-owned output grammar interfaces (#55269)
Bugen Zhao committed about 1 month ago
[Frontend][Rust] Add mm-processor benchmark for Rust frontend (#51922)
karthik committed about 1 month ago
[Spec Decode] Enable async scheduling for DFlash (#58065)
Jeff (Junze) Ma committed about 1 month ago
[ROCm][Bugfix] Explicitly reject FSE=1 with DPA+ETP deployment for DeepSeek-V4 (#57919)
Shanshan Shen committed about 1 month ago
[RL][Sleep] Retain frozen weights across level-2 sleep (#57891)
aoshen02 committed about 1 month ago
[Bugfix][Spec Decode] Cap DFlash/DSpark profiling query batch (#56448)
Yicong Wang committed about 1 month ago
[Rust Frontend] Add MiMo V2.5 parser support (#57933)
Bugen Zhao committed about 1 month ago
[Fast Start] Cache the MTP draft model in a separate daemon group (#57312)
siyu committed about 1 month ago
[Docker] Expose bundled vllm-rs on PATH (#57606)
Alec committed about 1 month ago
Chaojun Zhang committed about 1 month ago
[CI] Add pre-commit check that new tests are tethered to Buildkite jobs (#54867)
Turner Jabbour committed about 1 month ago
[Bugfix][GDN] Fix stateless first-chunk classification (#51565)
cherry77-cloud committed about 1 month ago