A high-throughput and memory-efficient inference and serving engine for LLMs
main
[ROCm][CI] Stabilize ModernBERT token-classification parity against Hugging Face (#44040)
Andreas Karatzas committed 5 months ago
[WIP][XPU] upgrade torch-xpu to 2.12 (#42262)
Kunshang Ji committed 5 months ago
Remove `raw_inputs` from transformers backend (#39425)
Raushan Turganbay committed 5 months ago
[Perf] fuse qk rmsnorm rope gate for qwen3.5 (#44176)
Jiangyun Zhu committed 5 months ago
[Bugfix] Fix minimax_qk_norm_fusion (#44983)
Jee Jee Li committed 5 months ago
[Cohere] Cohere2 moe parser fix (#44907)
Terrence Zhao committed 5 months ago
[Cohere] Fix Cohere2MoE weight loading when using Transformers ≥5.10 (#44747)
Terrence Zhao committed 5 months ago
[PD][Bugfix] Fix KV Cache sharing with HMA (#44629)
Nicolò Lucchesi committed 5 months ago
[Rust Frontend] Support Kimi K2 tool call IDs (#44901)
Maria Guevara committed 5 months ago
[XPU][CI] Refine docker image build and pull/create lock mechanism in Intel GPU CI (#44481)
xiangdong committed 5 months ago
[Rust Frontend] Add /tokenize and /detokenize endpoints (#44222)
TanNgocDo committed 5 months ago
[ROCm][Compile] Fuse AR + RMSNorm + per-group FP8 quant (+ DSv3.2 indexer fan-out) (#42864)
Markus Hartikainen committed 5 months ago
[KV Events] Switch event structs from array to map encoding (#42892)
Sage committed 5 months ago
Qiuyang Yue committed 5 months ago
[Rust Frontend] Support API key authentication (#44321)
Chao-Ju Chen committed 5 months ago
[CI/Docs] Remove stale disagg prefill links (#44918)
Mohammad Miadh Angkad committed 5 months ago
[DSV4][XPU] Add MHC fused_post_pre support (#44144)
Ma Jian committed 5 months ago
[ROCm][CI] Stabilize sleep-mode memory release (#43022)
Andreas Karatzas committed 5 months ago
[Bugfix][CI] Fix `test_offloading_connector.py::test_fs_tiering_offloading` (#44903)
Nicolò Lucchesi committed 5 months ago
[CI] Reorganize entrypoints CI (#44947)
wang.yuqi committed 5 months ago