[ROCm] Use quant_dtype in per_token_quant instead of hardcoded FP8 (#39121)

Andrew Barnes committed 6 months ago

[Feature]: IndexCache support for DSA models (#37735)

Chauncey committed 6 months ago

Replace shape_invariants with simpler apprach in dynamic_arg_dims utilizing shape_id property. (#36194)

Laith Sakka committed 6 months ago

Fix timeout when using LoRA adapters with Nemotron Super (#40916)

danisereb committed 6 months ago

[CI/Build] Enable FP8 on NVIDIA Thor (#39712)

Cyrus Leung committed 6 months ago

better logging for large uncachable items (#41145)

Avshalom Manevich committed 6 months ago

[Models] Cohere MoE (#40817)

Terrence Zhao committed 6 months ago

[Bugfix] Report compile time for in-memory cache hit path (#41023)

Frederik Gossen committed 6 months ago

[Model Runner v2] Fix block table IMA issue (#40648)

Wentao Ye committed 6 months ago

[Perf] Optimize `AllPool.forward` by slicing first, 51% faster in the method level benchmark (#41163)

Wentao Ye committed 6 months ago

[Perf] Enable FlashInfer top-k/top-p sampler by default (#40376)

Artem Perevedentsev committed 6 months ago

[Bugfix][CPU] Backport PT cpp codegen indirect_assert scalar-mask fix (#40973)

Lalithnarayan C committed 6 months ago

hf_name argument for vllm bench throughput CLI (#41012)

pmaybank committed 6 months ago

[Feat] CPU fp8 attn for AMX/AVX-512 (#39445)

Tianmu Li committed 6 months ago

[Bugfix] Fix repeated DSv4 RoPE cache initialization (#41148)

Jee Jee Li committed 6 months ago

[Frontend] Add `defer_loading` and `tool_reference` support for Anthropic and OpenAI APIs (#40190)

Jared Wen committed 6 months ago

[DSV4] Support `max` reasoning effort (#40982)

Bugen Zhao committed 6 months ago

[KV Offload] Tighten `keys` type from `Iterable` to `Sequence` in `OffloadingManager` (#41200)

Ronen Schaffer committed 6 months ago

Fix PP in Gemma4 (#40786)

Rohit Kumar Singh committed 6 months ago

[Bugfix] BailingMoeV2.5: rotate full qk_rope_head_dim in MLA RoPE (#41185)

Jiangyun Zhu committed 6 months ago