[ModelBash][DSV3] Add TRTLLM DSV3 Router GEMM kernel (6% B1 Speedup) (#34302)

Robert Shaw committed 9 months ago

[XPU] allow TORCH_SDPA/TRITON_ATTN as XPU vit Backend (#35010)

Yan Ma committed 9 months ago

[ROCm][CI] Fix spec decode profile assertion and logprob test determinism (#35043)

Andreas Karatzas committed 9 months ago

[CLEANING] Remove unused disable_by_batch_size from SpeculativeConfig (#35060)

Vincent Gimenes committed 9 months ago

[Refactor] Remove dead private func `_fp8_perm` and `_extract_mask_for_item` (#35068)

Wentao Ye committed 9 months ago

Fix pipeline parallel with embed scaling in the Transformers modelling backend (#35094)

Harry Mellor committed 9 months ago

[Feature] Lazy import for the "mistral" tokenizer module. (#34651)

Neil Schemenauer committed 9 months ago

fix: Apply embedding_multiplier to inputs_embeds (#34813)

Gabe Goodhart committed 9 months ago

[BugFix]: Fix local mypy issues (#34739)

Martin Hickey committed 9 months ago

[Bugfix] Fix kernel benchmark (#33752)

Jee Jee Li committed 9 months ago

[Refactor] Simplify dummy data generation (#35025)

Cyrus Leung committed 9 months ago

[Model Runner V2] Remove propose_draft method (#35070)

Woosuk Kwon committed 9 months ago

[Model Runner V2][Minor] Remove redundant `do_spec_decode` field (#35039)

Nick Hill committed 9 months ago

[Spec Decode] Reduce TP communication for speculative decoding draft token generation (#34049)

qizixi committed 9 months ago

[Model] Add NVFP4 quantization support for Step3.5-Flash (#34478)

tacos8me committed 9 months ago

[Bug] Refactor max_num_batched_tokens to account for drafting (#34898)

Benjamin Chislett committed 9 months ago

[Spec Decode] Defer clearing KV connector metadata for EAGLE3 speculative decode + prefill / decode disagg setup (#34529)

qizixi committed 9 months ago

[ROCm][CI] Fix realtime test timeouts caused by aiter JIT compilation delays (#35052)

Andreas Karatzas committed 9 months ago

[ROCm][CI] Fix flaky embedding chat test by using tolerance-based comparison (#35050)

Andreas Karatzas committed 9 months ago

[Bugfix] Fix Qwen3/Qwen3.5 Reasoning Parser (#34779)

Roger Wang committed 9 months ago