A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Tests][Multimodal] Cover scoped processor kwargs precedence (#59399)
apex-mochen committed 21 days ago
[Quantization] Add per-token NVFP4 CuTe-DSL MoE backend (#50030)
Matej Sirovatka committed 21 days ago
[UX] Tag torch.compile log lines with the component being compiled (#48133)
Francesco Fusco committed 21 days ago
[Bugfix][Model][CPU] Fix Mamba2 quantized in_proj weight and scale loading for TP >1 (#58083)
Akash kaothalkar committed 21 days ago
[ROCm][Perf] Allocate the pinned PLE prefetch buffer lazily (#58797)
Mathew Odden committed 21 days ago
[ROCm][CI] Move Basic Models (Other) to MI355 (#59400)
djramic committed 21 days ago
[ROCm][MoE] Support MiMo-V2.6 MXFP4 on gfx942 (#58262)
yhcheong (vllmellm) committed 21 days ago
[CI] Skip the IPC weight-checker test on non-CUDA platforms (#59398)
aoshen02 committed 21 days ago
[Frontend][RL] Track HTTP weight operation outcomes and concurrency (#55781)
Ronald committed 21 days ago
[Hardware][PowerPC] Prioritize bfloat16 for auto dtype on PowerPC (#58528)
Akash kaothalkar committed 21 days ago
[Model] Make the BERT/RoBERTa embedding class a class attribute (#59348)
Avishek Goswami committed 21 days ago
[ROCm][Perf][GLM-5.3-Flash] Stride-aware decode KDA (#57979)
Simon Danielsson committed 21 days ago
[K2 Horizon] fold partial-RoPE permutation into q/k (and norm) weights (#55335)
Alexandra Sidorova committed 21 days ago
[Model] Extend device-side mm normalization to GLM4V/GLM5Next (#55389)
cjackal committed 21 days ago
[ROCm][DSv4.1][Perf] Use the shared prefill chunk plan in the ROCm sparse prefill (#58539)
Zheng Gong committed 21 days ago
[Rust Frontend] Replay roundtrip output grammars through XGrammar (#59143)
Bugen Zhao committed 21 days ago
[ROCm][DSv4][Perf] Use the shared prefill chunk plan in the ROCm sparse prefill (#58405)
Zheng Gong committed 21 days ago
[CPU][Perf] Add vectorized Sampler Kernel (#53913)
Rehan Khan committed 21 days ago
[ROCm][Perf] Add opt-in a4w4 (FP4 activation) MoE for DeepSeek V4.1 on AITER (#58819)
Fangzhou Ai committed 21 days ago
[Frontend][RL] Align sleep-mode API responses and operation metrics (#52864)
Ronald committed 22 days ago