A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Misc] Clean up "swap_space" (#48549)
wangxiyuan committed 4 months ago
[Bugfix] Skip minimax_m3 tool parser tests when Rust extension is absent (#48523)
Matt Woodson committed 4 months ago
[ROCm][MiniMax-M3][Spec Decode] Support speculative decode with AITER sparse PA (#47984)
Tan Pin Siang committed 4 months ago
[Perf][ROCm] Fix GDN KKT warmup regression on RDNA by avoiding fp32 tl.dot (#45000)
nemanjaudovic committed 4 months ago
[ROCm][Kernel] Add HybridW4A16LinearKernel: Triton prefill + HIP skinny decode (#40977)
Matthias Gehre committed 4 months ago
[ROCm][MiniMax-M2] Dispatch fused QK-norm + AllReduce via AITER (#44849)
akii96 committed 4 months ago
up FI fp8 moe topk to 32 (#44462)
Dan Blanaru committed 4 months ago
[1/N] Add dense MHA path for sparse MLA short sequences (#47327)
Matthew Bonanni committed 4 months ago
[Quantization][INC][ARK] Support INT2 XPU WOQ Linear (#47521)
Zhenzhong Xu committed 4 months ago
add pad-aware swiglu limit kernel (#48287)
gnovack committed 4 months ago
[BugFix] Correct OTEL span start time for Dynamo compilation (#40698)
emricksini-h committed 4 months ago
[Frontend] Expose logprob_token_ids on Python OpenAI endpoints (#43463)
langzhao-netizen committed 4 months ago
[Feat] Add Support for BertForMaskedLM to vLLM (#48463)
Snehlata committed 4 months ago
[BugFix] Initialize model_config for Qwen3-VL MoE (#44863)
wenpengw-nv committed 4 months ago
[Core] Preserve Marconi caching with selective hybrid cache retention (#47782)
Nick Hill committed 4 months ago
Added sliding window attention support for qwen-eagle3 architecture (#47568)
shanjiaz committed 4 months ago
Add DCP + Eagle support for Tokenspeed MLA backends (#48180)
Pavani Majety committed 4 months ago
[ROCm][CI] Unblock `AMD: Language Models Test (Extended Pooling)` (#48513)
Micah Williamson committed 4 months ago
akii96 committed 4 months ago
[Doc] Add DeepseekV32ForCausalLM to supported_models.md (#48293)
Gavin Morris committed 4 months ago