A high-throughput and memory-efficient inference and serving engine for LLMs
main
[XPU][Tests] Enable test_per_token_group_quant_int8 on XPU (#55681)
pmanczak committed about 1 month ago
[Bugfix][Core] Retire Mamba states across null gaps (#55450)
lucamotz committed about 1 month ago
[ROCm][CI] Accept `base-v2-preview` images during content hash lookup for ROCm base images (#56356)
Micah Williamson committed about 1 month ago
[Docs] Add Nebius Serverless AI deployment guide (#56414)
Alexander committed about 1 month ago
[Attention] Extend XQA decode support on SM90 (#50439)
Andrii Skliar committed about 1 month ago
[Bugfix] Fix FlashInfer KV sharing with omitted K/V (#55864)
Zhicheng Liu committed about 1 month ago
[Frontend][last/N] Move all non-OpenAI content out of the OpenAI folder. (#56369)
wang.yuqi committed about 1 month ago
[Bugfix][MM] Fix swapped H/W in dummy video profiling inputs (#56385)
Hung Hoang (黄兴) committed about 1 month ago
[Perf][ROCm] Add AITER custom AG/RS (DP only) (#48247)
Simon Danielsson committed about 1 month ago
[PCP][Spec Decode] Adds PCP support for single-module MTP and replicated DSpark. (#56107)
Lucas Wilkinson committed about 1 month ago
[EC] Automatically enable embedding inputs on EC/KV consumers (#56379)
Tianyu Guo committed about 1 month ago
[Bugfix][LoRA] Use stored rsLoRA scaling factor in MoE expert packing (#55548)
Kushal committed about 1 month ago
[Bugfix][Mooncake] Fix heterogeneous PP transfer completion (#56033)
Yicong Wang committed about 1 month ago
[Feature][SimpleCPU] Load fine-grained hybrid prefix hits (#54736)
YukioZzz committed about 1 month ago
[ROCm][Bugfix] Route GLM-5.3-Flash MTP through ragged sparse MLA (#55239)
James E T Smith committed about 1 month ago
[Bugfix][Core] Stop zero-progress preemption cascades for deferred KV frees (#49675)
焦恺文 committed about 1 month ago
[XPU] Add forward_xpu to Mixer2RMSNormGated and FusedRMSNormGated (#54968)
Marceli Fylcek committed about 1 month ago
[CI/Build] Pin HyperCLOVAX V2 test model revision (#56335)
Tyler Michael Smith committed about 1 month ago
[Bugfix] Set stop_sequence explicitly in streaming message_delta event (#55325)
Colin McNamara committed about 1 month ago
[ROCm][Bugfix][Perf] Tune multi-stream shared experts use; wvSplitKrc fixes (#56098)
Matt committed about 1 month ago