A high-throughput and memory-efficient inference and serving engine for LLMs
main
Add Humming indexed-MoE regression test (#50468)
Michael Goin committed 3 months ago
[Quantization] Honor `--linear-backend` for ModelOpt W4A16 (#50273)
Netanel Haber committed 3 months ago
[compile] Fix fake kernel return dtype (#50444)
Richard Zou committed 3 months ago
[ROCm] Pass pointers to FlyDSL MoE kernels (#50378)
Andreas Karatzas committed 3 months ago
[CI] Initialize fused gated RMSNorm weights (#50377)
Andreas Karatzas committed 3 months ago
[Bugfix] Shut down private Tensorizer engines (#49840)
Andreas Karatzas committed 3 months ago
[Frontend] Preserve bare Inkling text in Python and Rust parsers (#50403)
Bugen Zhao committed 3 months ago
[ROCm][DSV4] B-preshuffle the attention fp8 projections (#46720)
Mehmet Cagri committed 3 months ago
[Build] Fix CUDA release wheel builds (#50243)
Kevin H. Luu committed 3 months ago
[Bugfix][MoE] Write Humming results to the supplied output buffer (#50338)
Netanel Haber committed 3 months ago
[DSv4 Perf] Remove redundant full kernel for dsv4, 1.88x kernel performance improvement (#50298)
Wentao Ye committed 3 months ago
Wentao Ye committed 3 months ago
[XPU][CI] skip kimi-k3 test (#50447)
Kunshang Ji committed 3 months ago
[FlexAttention] Avoid encoder block-mask compile explosion (#50339)
Andreas Karatzas committed 3 months ago
[rl] Stateful Trainer Send: IPC [2/N] (#48981)
Aaron Hao committed 3 months ago
[Compilation]Fuse Transformers Residual Add + RMSNorm (#48757)
BadrBasowid committed 3 months ago
[PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947)
Julien Denize committed 3 months ago
[ROCm]Migrating Deepseek V3.2 to vllm/models/deepseek_v32/ (#47207)
sroberts-amd committed 3 months ago
[MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570)
Martin Hickey committed 3 months ago
docs(security): document Ray cluster trust model and env var propagation (#50397)
Juan Pérez de Algaba committed 3 months ago