A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Pooling] Preserve BERT-family heads for raw logits (#57664)
Taneem Ibrahim committed 26 days ago
[AuxOutput] Only require Model Runner V2 on GPU platform (#58205)
Linkun committed 26 days ago
[Bugfix] Don't drop the rest of the allocator config when toggling expandable segments (#57982)
Oxana Korzh committed 26 days ago
[watermarking] golden tests for backwards compatibility (#56809)
Raphaël Rialland committed 26 days ago
[Bugfix] Default missing detail for Responses API input images (#57241)
yzong-rh committed 26 days ago
[Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams (#56067)
Cheng Rui committed 26 days ago
[Bugfix][Frontend] Document 404 response for `/generative_scoring` (#58788)
Nick Hill committed 26 days ago
[Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop (#58551)
Denis Ermakov committed 26 days ago
[Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 (#55957)
Yousaf committed 26 days ago
[ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision (#50605)
Rohan Potdar committed 26 days ago
[ROCm][CI] Run the MLA attention+quant fusion test on ROCm (#58717)
stefankoncarevic committed 26 days ago
[Bugfix][CI] Report subprocess test skips as skips, not passes (#58701)
stefankoncarevic committed 26 days ago
[CI] [MRV2] Restore MRV2 pp dp coverage (#57735)
Taneem Ibrahim committed 26 days ago
[ROCm][CI] Expand single-GPU coverage on MI355 DPX (#57599)
Sheral Kumar committed 26 days ago
[Perf] DiffusionGemma: one-pass sampler statistics kernel (#58226)
Matt Mastracci committed 26 days ago
[ROCm][CI] Test AMD DeepSeek V4 MoE routing against a PyTorch reference (#58740)
stefankoncarevic committed 26 days ago
[ROCm][CI] Add quantized MoE serving test for gfx950 (#58748)
stefankoncarevic committed 26 days ago
[ROCm][CI] Cover the AITER MQA logits dispatch on gfx950 (#58724)
stefankoncarevic committed 26 days ago
[ROCm][Kimi-K3] Optimize low-concurrency speculative KDA (#58045)
jiacao-amd committed 26 days ago
[Bugfix][KV Connector] Reap expired NIXL leases behind a heartbeated head (#58292)
GokayAI committed 26 days ago