[XPU] Support block fp8 moe by fallback to TritonExpert on XPU (#36458)

Kunshang Ji committed 8 months ago

feat: add RISC-V support for CPU backend (v2) (#36578)

typer-J committed 8 months ago

feat(spec_decode): fuse EAGLE step slot mapping and metadata updates (#33503)

Sladyn committed 8 months ago

[Doc] Fix duplicate words in comments (#36713)

Hongbin Guo committed 8 months ago

[Misc][Attention] Clean up unused method in `CPU_ATTN` (#36673)

Matthew Bonanni committed 8 months ago

[Fix] Use torch.empty for output in attention+quant fusion (#31785)

elvischenv committed 8 months ago

[Perf] Optimize scheduler overhead for PD disaggregation, around 5% E2E perf improvement (#35781)

Wentao Ye committed 8 months ago

[Model] Add support for moonshotai/Kimi-Audio-7B-Instruct (#36127)

tunglinwood committed 8 months ago

Add tuned H100 MoE configs for LFM2 8B and 24B (#36699)

tianshu-Michael-yu committed 8 months ago

[ci] Bound nvidia-cudnn-frontend version (#36719)

Kevin H. Luu committed 8 months ago

[DSV3.2][MTP] Optimize Indexer MTP handling (#36723)

Benjamin Chislett committed 8 months ago

[Bugfix] Surface exceptions from non-blocking execute_model in UniProcExecutor to avoid DP deadlocks (#35194)

fangyuchu committed 8 months ago

fix bugs when token_classify & classify run concurrently (#36614)

Augusto Yao committed 8 months ago

[openapi server] log exception in exception handler(2/N) (#36201)

Ning Xie committed 8 months ago

[Bugfix] Fix DeepSeek V3.2 OOM during CG memory profiling (#36691)

Matthew Bonanni committed 8 months ago

[Bug] Fix TRTLLM Block FP8 MoE Monolithic (#36296)

Wei Zhao committed 8 months ago

[ROCm][CI] Making some tests optional to reduce workload (#36090)

Andreas Karatzas committed 8 months ago

[Minor] Enhance error message for TRTLLM decode uniformity check (#36609)

Woosuk Kwon committed 8 months ago

[Model Runner V2] Add initial CI tests (#36041)

Nick Hill committed 8 months ago

[Core] Simplify core kv-cache blocks initialization logic (#36521)

Nick Hill committed 8 months ago