A high-throughput and memory-efficient inference and serving engine for LLMs
main
[CI][XPU] Reject CUDA-IPC weight cache on non-CUDA/ROCm platforms (#56010)
Qiming Zhang committed about 1 month ago
[ROCm] [BugFix] Fix AITER MXFP4 ASM-GEMM crash on unfused shared experts (#55213)
Colin Z committed about 1 month ago
[Frontend][EPD] Use JSON arrays for multimodal metadata (#56090)
Tianyu Guo committed about 1 month ago
[CI][ROCm][Disagg] Add GLM-5.2-FP8 to MoRIIO model catalog (#53885)
Avinash Paul committed about 1 month ago
[feature] Watermarked generation and detection (Gumbel-max algorithm) (#54053)
Raphaël Rialland committed about 1 month ago
[Refactor] Extract maybe_run_omni() and defer CLI imports past omni early-return (#56103)
zvier committed about 1 month ago
[CI][XPU] skip test_models_text on XPU (#56038)
Qiming Zhang committed about 1 month ago
Chaojun Zhang committed about 1 month ago
[Security][Rust Frontend] Normalize HTTP method labels in metrics (#56058)
Juan Pérez de Algaba committed about 1 month ago
[Rust Frontend] Strongly type wire dtypes and multimodal modalities (#56174)
Bugen Zhao committed about 1 month ago
[Rust Frontend] Resolve unified and split parser selections consistently (#56018)
Bugen Zhao committed about 1 month ago
[XPU] update triton-xpu 3.8.0 shim layer (#56014)
Yan Ma committed about 1 month ago
[Bugfix][Parser] Seed-OSS turn-boundary tokens + boundary-fallback tests (#54264)
Mikhail Podvitskii committed about 1 month ago
[Cohere] Bound remaining request priorities to the MessagePack int64 range (#56146)
Taneem Ibrahim committed about 1 month ago
ykamiset committed about 1 month ago
[ROCm][CI] Split MI300 Distributed Compile by graph partition mode (#53602)
Aarushi Jain committed about 1 month ago
[ROCm][Bugfix] Support shared KV prefill in AITER attention (#55887)
Andreas Karatzas committed about 1 month ago
[Bugfix] Fix Step-3.5 reasoning parser for structured outputs (#53174)
yzong-rh committed about 1 month ago
[Perf] Fix TRTLLM ragged prefill perf regression (#55499)
Wei Zhao committed about 1 month ago
wuhangxian committed about 1 month ago