A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Model] Add support for Nanbeige4.2 (transformers backend) (#56071)
zql committed about 1 month ago
[PCP][DCP] Declare FlashMLASparse MTP support at CP interleave > 1 (#56722)
Lucas Wilkinson committed about 1 month ago
[Bugfix] Scale KV page size for hidden states extraction with TP (#50894)
Orestis Zambounis committed about 1 month ago
[Spec Decode] Fix Qwen3 DSpark d2t requirement for padded-vocab drafts (#55133)
Orestis Zambounis committed about 1 month ago
[Docs] Move russellb to emeritus committer (#56467)
Russell Bryant committed about 1 month ago
[LoRA][Refactor] Unify multimodal LoRA token count hooks (#55071)
linitra24 committed about 1 month ago
[Bugfix] MiniCPM-V 4.6: fix ViT self-attn qkv weight loading (#49417)
arkohut committed about 1 month ago
[Bugfix] Trim stale consequence claims from unannotated-eagle warning (#56791)
Jiangyun Zhu committed about 1 month ago
[Bugfix][Models] Fix OpenPangu sleep mode with static sinks (#53696)
Ronald committed about 1 month ago
[ROCm][Connector] SWA+HMA-support in MoRI-IO connector (Gemma4) (#53721)
Simon Danielsson committed about 1 month ago
[Qwen3.8-Flash-Next] Fuse PLE residual and QSA output gate (#55309)
Canlin Guo committed about 1 month ago
[Bugfix][KV Offload] Reuse in-flight async lookup probes (#55823)
AlexHuang committed about 1 month ago
[CI][CPU] Add speculative-decoding coverage to CPU CI (#54934)
Ganesh R committed about 1 month ago
[ROCm][Perf] Enable CSA multi-stream overlap for DeepSeek-V4 (#51794)
Shanshan Shen committed about 1 month ago
[CI][Test] Mock CPU backend block sizes in kv_connector unit conftest (#56671)
Xiaozhen Bi committed about 1 month ago
Reid committed about 1 month ago
[Bugfix][NixlPush] Guard _remote_agents read in _do_send_reg_notif (X1) (#56317)
Nicolò Lucchesi committed about 1 month ago
[Security] Cap Qwen-VL video sampling knobs (#56729)
Juan Pérez de Algaba committed about 1 month ago
[Bugfix][EPD] Preserve media processing options in encoder requests (#56786)
jiangkuaixue123 committed about 1 month ago
[Frontend] Replace `VLLM_ENABLE_SCALE_OUT_ENDPOINTS` with `--enable-scale-out` (#55176)
Martin Hickey committed about 1 month ago