A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Perf] Accumulate Conformer attention scores with baddbmm (#55062)
Sun committed about 2 months ago
[Bugfix] Retain vocab embeddings during replacement (#55083)
Taneem Ibrahim committed about 2 months ago
[Bugfix] Account for PCP in multi-node world size validation (#55111)
Simon committed about 2 months ago
[Model Runner V2][Spec Decode] Skip DP sync for all speculator uniform decodes (#54856)
Giancarlo Delfin committed about 2 months ago
[Bugfix][Model] Enable torch.compile for StableLM (#54969)
djramic committed about 2 months ago
[Kernel] Warm up Qwen GDN gated RMSNorm (#54251)
Zupeng Wang committed about 2 months ago
[CI][AMD] Avoid expandable segments in LoRA TP tests (#55094)
Andreas Karatzas committed about 2 months ago
[CI] Bump Transformers version to 5.16.1 (#53905)
Harry Mellor committed about 2 months ago
[ROCm][CI] Add MiniMax reduce RMS kernel coverage (#55057)
Andreas Karatzas committed about 2 months ago
[Model] Add K2-Horizon model support (#55063)
Bowen Tan committed about 2 months ago
[Bugfix] Wait for offload keys before storing chunks (#52923)
chuanzhisongshu committed about 2 months ago
[Bugfix][Core] Wait for the previous PP tensor sends before the next forward pass (#54962)
Derek Weitzel committed about 2 months ago
Use server-generated keys for late-interaction query caches (#51445)
Clinton Thomas committed about 2 months ago
[Bugfix][Multimodal] Handle prefix-covered items in SHM worker cache (#54994)
waizuichougou committed about 2 months ago
[CI] Fix DeepSeek-V4 registry platform guard (#55042)
Taneem Ibrahim committed about 2 months ago
[Bugfix][MoE] Preserve unquantized weight storage on ROCm (#46009)
Zhaodong Bing committed about 2 months ago
[ROCm][Installation] Add mooncake package to image using public wheels (#55002)
Giuseppe Grossi committed about 2 months ago
[CI] Force HTTP/1.1 for runtime Git installs (#55044)
Kevin H. Luu committed about 2 months ago
[Skills] Add kernel benchmark sanity references (#54995)
Woosuk Kwon committed about 2 months ago
[Perf][Rust Frontend] Coalesce decoded chunks per engine update (#55012)
Bugen Zhao committed about 2 months ago