[Bugfix][Kernel] Correct FlashInfer CUTLASS MoE tuning token bound (#46838)

Hiki committed 4 months ago

[ROCm][MLA] Fuse MLA q/kv RMSNorm + FP8 per-token quant in the FP8 attention path (#44977)

xaguilar-amd committed 4 months ago

[BugFix][Spec Decode] Compact shared topk indices buffer after first MTP draft step (#47238)

Giancarlo Delfin committed 4 months ago

[UX] Include NVTX in cuda.txt (#47319)

Jee Jee Li committed 4 months ago

[Misc] Use functions instead of PTX for the PDL instruction (#46984)

Jee Jee Li committed 4 months ago

[CI][Bugfix] Rerun test_engine_log_metrics_ray on Ray GCS startup timeout (#47208)

peizhang56 committed 4 months ago

[Spec Decode] DSpark speculators checkpoint support (#47093)

Michael Goin committed 4 months ago

[Docker] Remove unused Dockerfile.nightly_torch (#47338)

Andrey Talman committed 4 months ago

[CI] Remove torch_nightly mirror tags (superseded by TORCH_NIGHTLY full-nightly build) (#47342)

Andrey Talman committed 4 months ago

[MoE] Plumb gemm1_alpha/beta/clamp_limit into TRT-LLM FP8 MoE (#45723)

Yongye Zhu committed 4 months ago

[DSV4] Better MXFP8 quantization kernel (#47229)

Yongye Zhu committed 4 months ago

[Bugfix] Don't read KV cache past `seq_len` in triton paged attn kernels (#47305)

Nick Hill committed 4 months ago

[ModelRunner V2] Warmup cross-attn properly in encoder-decoder case (#47308)

Nick Hill committed 4 months ago

[ROCm][P/D] MoRIIO toy proxy: support JSON Content-Type for OpenAI clients. (#46482)

Chaitanya Sri Krishna Lolla committed 4 months ago

[Bugfix] Prevent padding placeholders from reaching embeddings (#47029)

Yiliu Dong committed 4 months ago

[Bench][BugFix] Fix empty decoder prompt for Cohere ASR in throughput benchmark (#47135)

Michał Ganczarenko committed 4 months ago

[Spec Decode] DSpark (#46995)

Benjamin Chislett committed 4 months ago

[Bugfix] Fix GraniteMoeShared weight loading broken by #41184 (#47031)

Michał Ganczarenko committed 4 months ago

[Rust Frontend] Split engine core DTOs into separate modules (#47265)

Bugen Zhao committed 4 months ago

[xpu][lora]: Align LoRA implementation with Punica GPU: fix _apply_expand rank mismatch, add_inputs hardcode, and MoE EP (#45368)

Chaojun Zhang committed 4 months ago