A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Rust Frontend] Reduce copy in auxiliary frame resolution (#54148)
Bugen Zhao committed about 2 months ago
[XPU]bump up vllm_xpu_kernels to 0.1.14.1 (#54203)
Xinyu Chen committed about 2 months ago
[Bugfix] Restore multimodal support on the plain "vllm" throughput backend (#52168)
Michał Ganczarenko committed about 2 months ago
[ROCm][CI] Warm up the RLHF dev server before the pause/resume timing checks (#53594)
stefankoncarevic committed about 2 months ago
[Bugfix][CI/Build] Fail closed when selected precompiled CUDA variant is unavailable (#52545)
Jung Jiyu committed about 2 months ago
[Kimi-K3][Bugfix] Fix low-latency GEMM fallback initialization (#54167)
Thien Tran committed about 2 months ago
[ROCm][CI] Keep startup profiling from aborting when free memory grows (#53591)
stefankoncarevic committed about 2 months ago
[Test] Fix mock_current_vllm_config missing kernel_config in test_dflash2 (#54132)
Qiming Zhang committed about 2 months ago
[CPU][MLA] Fix prefill backend selection so MLA runs end-to-end on CPU (#51471)
maobaolong committed about 2 months ago
Count store offers, not lookups, for CPU offload store_threshold (#52227)
Oxana Korzh committed about 2 months ago
[Bugfix][KV Cache] Prevent negative external block allocation (#52707)
haic0 committed about 2 months ago
Roy Wang committed about 2 months ago
Remove wrongly added e2e test (#54099)
Elvir Crnčević committed about 2 months ago
[Misc] Separate adaptive verification config validation (#54108)
Nick Hill committed about 2 months ago
[Bugfix] Remove race in fused groupwise RMSNorm quantization (#54111)
Misha Goin committed about 2 months ago
[Kimi-K3] Merge MLA gate into QKV-A projection (#54015)
Thien Tran committed about 2 months ago
fix(build): correct preprocessor guard for GDN decode to fix Ampere c… (#52743)
prakharPant committed about 2 months ago
[Bugfix] Revert renderer warmup overlap to avoid fork deadlock (#54023)
Kevin H. Luu committed about 2 months ago
[Rust Frontend][gRPC] Enforce LoRA path validation across transports (#53756)
Connor Carpenter committed about 2 months ago
[Bugfix][Model] Fix K3 DSpark config for 96-head drafts (#54005)
qizixi committed about 2 months ago