[Speculative Decoding] Propagate norm_output and fc_norm config for Eagle3 speculators (#46488)

Orestis Zambounis committed 4 months ago

[Kernel] Enable PDL for per_token_group_quant_8bit_kernel (#46508)

Jee Jee Li committed 4 months ago

[Frontend] Port seed_oss to the streaming parser engine as a Qwen3 subclass (#46314)

Maxwill Lin committed 4 months ago

[Bugfix][MooncakeStore] track resumed requests via scheduler's resumed_req_ids (#46595)

Yifan Qiao committed 4 months ago

[Kernel][MoE] Allow FlashInfer MXINT4 MoE for gated SiLU (#46518)

Kaihang Jiang committed 4 months ago

[Bugfix] Normalize slashes in Helion GPU names (#46101)

cyq committed 4 months ago

[Bugfix] Support -1 (invalid/non-local) slots in topk_ids for Triton MoE (#46408)

Woosuk Kwon committed 4 months ago

[Bugfix] Support non-power-of-2 top_k in legacy triton_kernels routing (#46406)

Woosuk Kwon committed 4 months ago

[ROCm] Begin Deprecation Window for CUDA_VISIBLE_DEVICES on ROCm (#46636)

Micah Williamson committed 4 months ago

[Frontend] Fix Kimi K2 tool call IDs for required tool choice (#46344)

Chauncey committed 4 months ago

[Feature] Support DCP with FP8 KV cache in MLA decode path (#44044)

shivampr committed 4 months ago

Chore: Fix minor doc sentence, grammar, quote errors (#40469)

Ashwin Phadke committed 4 months ago

[Bugfix][Model Runner V2][Spec Decode] Fix int32 offset overflow in sampler kernels (#46560)

JessieWei committed 4 months ago

Humming support for 2/3/5/6/7-bit pack-quantized weight-only inference (#46389)

HDCharles committed 4 months ago

Run DeepSeek-V2-Lite prefetch-offload eval eager on ROCm (#46386)

Aarushi Jain committed 4 months ago

[KV Offload] Gate packed HMA KV cache on cross-layer config (#46252)

Lucas Wilkinson committed 4 months ago

[ROCm] Fix OOB During Model Warmup With `ROCM_ATTN` and MRV2 (#46548)

Micah Williamson committed 4 months ago

[Bug] Fix `IndentationError: expected an indented block after 'with' statement` (#46627)

Wentao Ye committed 4 months ago

[KV Offload] Use background thread for mmap / cpu_tensors pinning (#45850)

Yiwei Hu committed 4 months ago

[KV Offloading] Replace `bool|None` lookup return with LookupResult enum (#46363)

Ronen Schaffer committed 4 months ago