A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix][HiSparse] Adopt GPU prefix copies after the hit's allocation (#59282)
Matthew Bonanni committed 21 days ago
[Perf][MiniMax-M3] Triton indexer: decode grid retune + SM12.0 split-K (#56151)
Venky committed 21 days ago
[ROCm][CI] Relax simple-nemotron-h-8b GSM8K threshold on ROCm (#59477)
djramic committed 21 days ago
[CI] Add supports_multimodal_inputs to test_executor_replace's mock config (#59476)
Matthew Bonanni committed 21 days ago
[XPU][CI] Skip test_core_engine_actor_manager.py on Intel CI (#59338)
Chaojun Zhang committed 21 days ago
[ROCm][CI] Remove duplicate MI355 DPX jobs (#59256)
Aarushi Jain committed 21 days ago
[ROCm]Transpose compressed-tensors MoE weights on device (#59253)
Aarushi Jain committed 21 days ago
[ROCm][Kimi-K3][Perf] Fuse MLA decode KV-cache write and Q-prep via AITER (#57640)
Rita Brugarolas committed 21 days ago
[CI][ROCm] Drop the duplicate OAI Triton MoE run from FP8 MoE Kernels (#59237)
stefankoncarevic committed 21 days ago
[Docs] Add @gau-nernst to CODEOWNERS and committers (#59369)
Thien Tran committed 21 days ago
[watermarking] add context deduplication support to speculative decoding (#56807)
Raphaël Rialland committed 21 days ago
[Frontend] Port chat_parsing core from Transformers (#58602)
Yoni Gozlan committed 21 days ago
[ROcm][BugFix][The Rock] Update The Rock dockerfile to most recent Triton 3.8 (#59287)
rasmith committed 21 days ago
[Bugfix][HiSparse] Preserve host prefix publication after request completion (#59007)
Yueh-Ting (eop) Chen committed 21 days ago
[Docs] Add @wzhao18 to NVIDIA integration and kv offloading code owners (#59456)
Wei Zhao committed 21 days ago
Add support for unquantized ngram in CT format (#59431)
Eldar Kurtić committed 21 days ago
[ROCm][BugFix][The Rock] Fix mori build for the rock (#59372)
rasmith committed 21 days ago
[Security] Fix chat template resource-exhaustion DoS (GHSA-4hhp-h66f-… (#50300)
Juan Pérez de Algaba committed 21 days ago
Disable mypy `arg-type` and `assignment` checks in tests (#59428)
Harry Mellor committed 21 days ago
[BugFix][Multimodal] Pick worst-case DeepSeek-V4 VL dummy image size (#59271) (#59373)
Yida Weng committed 21 days ago