A high-throughput and memory-efficient inference and serving engine for LLMs
main
[ROCm][CI] Add MI355 TP2 AR-RMS and B200 fusion mirrors (#58284)
Andreas Karatzas committed 23 days ago
[ROCm][CI] Mirror split kernel groups on MI355 and fix exposed tests (#58654)
Andreas Karatzas committed 23 days ago
[CI][ROCm] Mirror large-model GSM8K evaluations on MI355 (#58683)
Andreas Karatzas committed 23 days ago
[MM][Mistral3] Image preprocessing optimization (#57531)
Jakub Zakrzewski committed 23 days ago
burak committed 23 days ago
[ROCm][DSv4.1] Paged MXFP4 sparse indexer on aiter's MQA-logits kernel (#58671)
Mehmet Cagri committed 23 days ago
[Rust Frontend] Feed parser tests and benchmarks attributed input (#59048)
Bugen Zhao committed 23 days ago
[Perf][Spec Decode] Enable fused multi-step draft decode for FlashInfer trtllm-gen (#58371)
Bowen Wang committed 23 days ago
[Rust Frontend] Relax schema-aware tool argument conversion (#59004)
Bugen Zhao committed 23 days ago
[Bugfix][Engram] Keep THP tables private when resolving shared memory (#59068)
Ren Yuzhou committed 23 days ago
[Model] Engine based plamo3 parser (#56575)
Shinichi Hemmi committed 23 days ago
[Bugfix][Frontend] Preserve caller parameters in offline pooling (#58842)
shaohuaxi committed 23 days ago
[Bugfix] Detect the CUDA toolkit the way FlashInfer does in has_flashinfer() (#58360)
Bowen Wang committed 23 days ago
fix: return loaded parameters in Aria load_weights (#57476)
debanshd committed 23 days ago
[Bugfix][Multimodal] Fix flat/scoped mm_processor_kwargs merge and resolution (#56372)
Daniel M. García-Ocaña Hernández committed 23 days ago
[Model] Extend device-side mm normalization to Qwen3VL/Qwen3.5/Qwen4Next (#51289)
cjackal committed 23 days ago
[Multimodal] Remove processsor fallback for fused input norm params resolve (#59073)
Isotr0py committed 23 days ago
[CI/Build] Skip the snapshot runtime on CUDA 12.x images (#59118)
Kevin H. Luu committed 23 days ago
[MRV2] Support stock torch.compile mode (#59079)
Cheese Cake committed 23 days ago
[Bugfix][FT] Pass stateless process-group timeouts explicitly (#58611)
Tyler Michael Smith committed 23 days ago