A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Perf][Frontend] Vectorize Cohere binary embedding bit-packing (#52277)
Fangchen Li committed 2 months ago
[XPU]bump up vllm_xpu_kernels to 0.1.13.2 (#52138)
Kunshang Ji committed 2 months ago
[PP][XPU]Overlap async-scheduling PP sampled-token broadcast with compute (#51650)
YiSheng5 committed 2 months ago
[CI] Increase extended generation test timeout (#52252)
Lucas Wilkinson committed 2 months ago
[CPU] Fold the MXFP4 block scale in 2 instructions instead of 4 (#51583)
ccaadaro committed 2 months ago
[XPU][CI/Release][3/N] Add xpu wheel release to release pipeline (#52108)
Kunshang Ji committed 2 months ago
[Bugfix][CPU][RISC-V] Fix build: make FP32Vec copy constructors non-explicit (#51099)
velonica0 committed 2 months ago
Add Muse Glimmer model support (#51655)
Tiezhen WANG committed 2 months ago
Detect ROCm wheel variant from environment for precompiled wheels. (#49365)
Aarushi Jain committed 2 months ago
[Bugfix][NIXL] Include transfer mode (push/pull) in the compatibility hash (#50620)
Tzu-Ling Kan committed 2 months ago
[Bugfix][Refactor] Keep Qwen3Next layer boundaries sequence parallel (#50685)
kzwrime committed 2 months ago
[Kernel][Perf] Add fused CUDA post-conv MTP decode kernel for Qwen3.5 GDN (#51674)
Jie Fang committed 2 months ago
[UT] fix device of test_outputs.py (#52237)
Qiming Zhang committed 2 months ago
[Kernel] Add B12X dense linear backends (#52016)
Luke Alonso committed 2 months ago
[XPU] [Bugfix] process ragged weights in xpu linear backend (#52118)
zofia committed 2 months ago
[Bugfix] Reject negative token ids as out-of-vocabulary (#51795)
Junhao Shen committed 2 months ago
[Kernel] Gemma-4 FA4 FP8 Kernel (#48666)
Jhao-Ting Chen committed 2 months ago
[Bugfix] Reject NUL byte in structured_outputs.regex (#51796)
Junhao Shen committed 2 months ago
[Model Runner V2][Spec Decode] Add KV cache support for multi-layer MTP (#50062)
Giancarlo Delfin committed 2 months ago
bastefaniak committed 2 months ago