A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 2, 04:45 UTC
main
[Bugfix] Fix Gemma4 startup crash with recent transformers multimodal processor (#44232)
Luciano Martins committed 5 months ago
[Bugfix] Sync block_size from EngineCore to frontend for hybrid Mamba… (#42967)
gruner committed 5 months ago
[Frontend] Consolidate dev entrypoints. (#44170)
wang.yuqi committed 5 months ago
Fix DFlash prefix cache corruption due to missing lookahead block (#42971)
Shreyas Kulkarni committed 5 months ago
[XPU][Mamba] Triton-based selective scan forward op for XPU (#43421)
Marceli Fylcek committed 5 months ago
[KV Offload] Add `on_schedule_end()` hook to separate step lifecycle from event draining (#44206)
Ronen Schaffer committed 5 months ago
[HARDWARE][POWER] Enable SHM communicator support for PowerPC (#43754)
Rukhaiya2004 committed 5 months ago
[Multimodal] Automatically select registered video loader for VLM (#44126)
Isotr0py committed 5 months ago
[Parser] Migrate `ResponsesParser` to unified `Parser` interface (#42977)
alberto committed 5 months ago
[EC Connector] Non blocking EC Connector lookup (#41627)
omerpaz95 committed 5 months ago
[Core][Refactor]: thread `scheduler_block_size` into KVCacheManager and KVCacheCoordinator (#44165)
Yifan Qiao committed 5 months ago
[Rust Frontend] add --enable-request-id-headers flag support. (#43883)
Maria Guevara committed 5 months ago
[kv_offload] Add `@override` decorators to subclass method implementations (#44177)
Ronen Schaffer committed 5 months ago
[CPU][Perf] Enable fused kernels for GDN's gated delta rules (#43534)
Fadi Arafeh committed 5 months ago
[Perf] use triton moe backend on hopper by default (#44220)
Jiangyun Zhu committed 5 months ago
[Refactor] Unify reasoning + tool-call parsing behind Parser.parse() (#44267)
Flora Feng committed 5 months ago
[Model Runner V2] Use actual batch max_seq_len for attn metadata (#43991)
zhrrr committed 5 months ago
[Model Runner V2] Support zeroing freshly allocated KV blocks for hybrid + fp8 KVCache (#43990)
zhrrr committed 5 months ago
[Bugfix] Convert Gemma4-MM ViT linear layers to vllm native impl (#43798)
Isotr0py committed 5 months ago
[MM][CG] Profile encoder CUDA graph pool memory (#41714)
JooHo Lee committed 5 months ago