A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Bugfix] Revert "[ROCm][Perf] Replace torch.topk in DSA candidate block selection" (#59125)
Fangzhou Ai committed 23 days ago
[Bugfix][Frontend] Validate reused prompt token ids; render messages for media and echo (#55771)
Shijie Lyu committed 23 days ago
[ROCm][DSv4.1][Perf] Run the delayed mHC seams through aiter's fused Triton kernel (#58655)
Muhammad Ahmed committed 23 days ago
[XPU] Preserve non-contiguous strides when pinning CPU tensors for UVA view (#54874)
Chaojun Zhang committed 23 days ago
[ROCm][MLA] Enable sparse MLA Gluon kernel from Aiter (#53492)
Mehmet Cagri committed 23 days ago
[CI] Bound the CRCR report by build age, and stop gating it on a job (#59066)
Andrey Talman committed 23 days ago
[torch.compile] Canonicalize functionalized split slices for fusion pa… (#57299)
Laith Sakka committed 23 days ago
[Elastic EP] Support Model Runner V2 (#53934)
Almog Tavor committed 23 days ago
[Bugfix][Frontend] Keep in-flight requests on the same DP engine (#59017)
Benedikt Falk committed 23 days ago
[Bugfix][ROCm] Drop -1 sentinels when building the ragged sparse-MLA indices (#58058)
Mehmet Cagri committed 23 days ago
[Bugfix] Fix moe_wna16 w13 zero-point shard split for 8-bit asym GPTQ MoE (#58950)
Spandan Tiwari committed 23 days ago
rasmith committed 23 days ago
[Minimax M3] Enable fp8 indexer cache on triton indexer for non-SM100 architectures. (#59081)
rmhaskar committed 23 days ago
[XPU][CI] Make `test_mamba_prefix_cache` block-size agnostic (#58269)
Fanli Lin committed 23 days ago
[XPU] skip test_online_quantization_loads_real_weights (#58987)
Yan Ma committed 23 days ago
[XPU] use rms_norm xpu kernel for context-key normalization (#58896)
Yan Ma committed 23 days ago
[CI] Drop test-group rules that no longer match the tree (#58895)
Abhinandan committed 23 days ago
[Perf][Spec Decode] Avoid triton recompiles in the acceptance estimator (#57107)
Giancarlo Delfin committed 23 days ago
[Bugfix][Scheduler] Refresh max tokens for streaming continuations (#57676)
0xsensei committed 23 days ago
[ROCm][CI] Add missing test coverage for upstream parity (#50519)
Andreas Karatzas committed 23 days ago