[ROCm][MLA] enable fp8 MLA decode on ROCm (#28032)

gbyu-amd committed 12 months ago

[Perf][Deepseek] optimize gather_and_maybe_dequant_cache kernel's perf for extremely long sequence (#28029)

Pleaplusone committed 12 months ago

[Perf] Disable DeepGEMM MoE by default when TP=8 is used (#29346)

Michael Goin committed 12 months ago

Scheduled removal of `ParallelConfig`'s direct child EPLB fields (#29324)

Harry Mellor committed 12 months ago

[BugFix] Fix duplicate id tool-call race condition (#29355)

Nick Hill committed 12 months ago

[XPU] upgrade torch & ipex 2.9 on XPU platform (#29307)

Kunshang Ji committed 12 months ago

[Spec Decode] Add support for EAGLE3 heads that do not use_aux_hidden_states (#27688)

Hanjie Qiu committed 12 months ago

[UX] Raise error for attn backend of batch invariant (#29348)

Wentao Ye committed 12 months ago

[Bugfix] Only use triton_kernels for MXFP4 on SM90 and SM100 (#29339)

Michael Goin committed 12 months ago

[Hybrid Allocator] Better layer padding strategy for gpt-oss eagle (#29303)

Chen Zhang committed 12 months ago

Remove VLLM_SKIP_WARMUP tip (#29331)

Tyler Michael Smith committed 12 months ago

[Tests] Verify gpt_oss package is installed in harmony tests (#29336)

Nick Hill committed 12 months ago

[Model Runner V2] Simplify Eagle bookkeeping with num_rejected (#29347)

Woosuk Kwon committed 12 months ago

[XPU]fix Kimi-VL-A3B-thinking on xpu (#29309)

Yan Ma committed 12 months ago

[CI/Build] Moves to cuda-base runtime image while retaining minimal JIT dependencies (#29270)

Benjamin Bartels committed 12 months ago

[Model Runner V2] Add minor clarification comments for Eagle (#29332)

Woosuk Kwon committed 12 months ago

[Multimodal][Qwen3 Omni] Make Qwen3 Omni work with audio-in-video inputs in V1 engine. (#27721)

Chenheli Hua committed 12 months ago

[MoE][Refactor] Make select_experts a non-static method (#29067)

bnellnm committed 12 months ago

[Model Runner V2] Change Numba AoT to JIT (#29328)

Woosuk Kwon committed 12 months ago

[Model Runner V2] Implement Single-step Eagle 1 (#29300)

Woosuk Kwon committed 12 months ago