A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix][Pooling] Handle multimodal cache misses without crashing (#58975)
Chaoying committed 19 days ago
[Bugfix][Multimodal] Fix reference counting of the SHM processor cache (#59695)
tenotek committed 19 days ago
[Bugfix][Quark] Pass grouped-routing arguments to OCP MX monolithic kernels (#59752)
aoshen02 committed 19 days ago
[Perf] Keep non-speculative GDN decode on the standard path (#59735)
DimensionSTP committed 19 days ago
[Model] Add Qwen4Exp to the Qwen GDN Triton warmup (#56742)
SeongJun Lee committed 19 days ago
[Docs] Fix stale W4A16 NVFP4 default kernel in ModelOpt docs (#59135)
Rajeev Warrier committed 19 days ago
[CI][Kimi-K3] Test prefix cache reuse with KV offload, P/D and DCP (#59229)
Jiangyun Zhu committed 19 days ago
[Bugfix][Quantization] Detect NVFP4 in ModelOpt mixed-precision checkpoints (#56050)
Stefano Castagnetta committed 19 days ago
[Test] Re-enable Ovis2.5 and Ovis2.6-MoE vLLM tests on Transformers v5 (#59763)
Abhay Shukla committed 19 days ago
[XPU][Kernel] Tune Triton W8A8 block-FP8 GEMM for Intel B70 (#56063)
pmanczak committed 19 days ago
[CI] Replace shellcheck-suppressed patterns flagged in #52572 with clean equivalents (#53341)
Turner Jabbour committed 19 days ago
[Model] Migrate GPT-NeoX, Phi, Seed-OSS and Jais2 to the Transformers modeling backend (#59701)
Harry Mellor committed 19 days ago
[Misc] Remove code paths for Transformers < 5.16.1 (#59762)
Harry Mellor committed 19 days ago
[Bugfix][LoRA] Fall back for high-rank MoE LoRA (#55161)
xiaoyu-xyz committed 19 days ago
[ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes (#58167)
Simon Danielsson committed 19 days ago
[ROCm][Perf][GLM-5.3-Flash] Remove redundant copy after ragged sparse MLA (#58569)
Simon Danielsson committed 19 days ago
[Bugfix][GLM-5.3] Size the image encoder cache from the exact token ceiling (#59565)
Yufeng He committed 19 days ago
Farhad Al-Amin committed 19 days ago
[Feature] Offload KV-init runtime state on sleep (#59158)
aoshen02 committed 19 days ago
[Security] Accept zero-sum DeepSeek-OCR pixel tensors (#59417)
Juan Pérez de Algaba committed 19 days ago