A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Kernel][Helion] Disable warp specialization in rms_norm_per_block_quant B200 configs (#48797)
Shangdi Yu committed 3 months ago
[Bugfix][V1/V2] Fix prompt_logprobs to respect logprobs_mode (#47680)
aoshen02 committed 3 months ago
[CI] Fix macOS wheel release annotation context (#48771)
Kevin H. Luu committed 3 months ago
[Hardware][GPU] Profiler config additional to increase it scope and annotation details (#37524)
devalshahamd committed 3 months ago
[Perf] Optimize dsv4 routing using specialized kernel, 2.94% E2E TPOT improvement (#48660)
Wentao Ye committed 3 months ago
[Loader] Improve InstantTensor loading (#46868)
Michael Goin committed 3 months ago
limeward committed 3 months ago
[Attention] Allow selecting a different attention backend per KV-cache group (#48012)
Nicolò Lucchesi committed 3 months ago
[Bugfix][GLM4V] Fix video dummy profiling and memory usage (#48729)
labAxiaoming committed 3 months ago
[ROCm][Perf][DSV4] Improve sparse decode reduction occupancy on gfx950 (#48788)
Fangzhou Ai committed 3 months ago
mosya415 committed 3 months ago
[Refactor] Remove deepseek dead code (#48780)
Wentao Ye committed 3 months ago
Fix: Restore data_parallel_size > 1 for use_sequence_parallel_moe (#48849)
passtoor-agi committed 3 months ago
[Performance] Use CuTe-DSL for FlashInfer MXFP4 quantization (#48417)
JooHo Lee committed 3 months ago
[Bugfix][KV Offloading] Offload last block at request finish and prevent reuse race (#48596)
AlexHuang committed 3 months ago
[XPU] Bump vllm_xpu_kernels to v0.1.11.1 (#48942)
Artur Fierka committed 3 months ago
Yejing Lai committed 3 months ago
[XPU][UT]fix _POSSIBLE_KERNELS error on XPU (#47516)
Yejing Lai committed 3 months ago
[XPU] support HND layout (#47975)
liuzhenwei committed 3 months ago
[Bugfix] Enable FlashAttention MLA prefill for Mistral Small 4 head dims (#48855)
Julien Denize committed 3 months ago