[ROCm] Add skinny gemm bias support for dtypes fp16,bf16,fp8 (#24988)

Hashem Hashemi committed about 1 year ago

[Core] Ensure LoRA linear respect the base_layer's tp_size and tp_rank (#25487)

Jee Jee Li committed about 1 year ago

[Spec Decode][CI] Add e2e test for `examples/spec_decode.py` and prevent breaking Acceptance Length (#24531)

Ekagra Ranjan committed about 1 year ago

[Misc] Reduce initialization time of auto_tune (#23682)

Weida Hong committed about 1 year ago

[V1][Kernel] Add triton implementation for `reshape_and_cache_flash` (#24503)

Burkhard Ringlein committed about 1 year ago

Use macro guard CUDA functions for back compatibility in grouped_topk_kernel.cu (#25346)

Ming Yang committed about 1 year ago

[Log] Optimize kv cache memory log from Bytes to GiB (#25204)

Wentao Ye committed about 1 year ago

[BugFix] Fix UB in per_token_group_quant.cu (#24913)

rivos-shreeasish committed about 1 year ago

[Kernels] Support blocked fp8 quantization for compressed tensors MoE (#25219)

bnellnm committed about 1 year ago

Add backward compatibility for `GuidedDecodingParams` (#25422)

Harry Mellor committed about 1 year ago

[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845)

Lucas Wilkinson committed about 1 year ago

[V1] Remove V0 code paths for Hybrid models (#25400)

Thomas Parnell committed about 1 year ago

[UX] Change kv-cache-memory log level to debug (#25479)

Michael Goin committed about 1 year ago

[XPU] Fix MOE DP accuracy issue on XPU (#25465)

Fanli Lin committed about 1 year ago

[Docs] NixlConnector quickstart guide (#24249)

Peter Pan committed about 1 year ago

[P/D] Support NIXL connector to disconnect during a clean shutdown (#24423)

Chauncey committed about 1 year ago

[BugFix] Register expert_map as named buffer for wake_up and sleep (#25458)

Joel committed about 1 year ago

[CI/Build] Fix disabled v1 attention backend selection test (#25471)

Isotr0py committed about 1 year ago

[docs] Benchmark Serving Incorrect Arg (#25474)

vllmellm committed about 1 year ago

[Core] Optimize LoRA weight loading (#25403)

Jee Jee Li committed about 1 year ago