[Fix]Load kv-cache dtype from hf_quant_config.json automatically (fix for reverted PR) (#30785)

danielafrimi committed 11 months ago

[compile] Ignore VLLM_FORCE_AOT_LOAD from cache factors (#30809)

Zhengxu Chen committed 11 months ago

[compile] Disable aot when eager backend is used. (#30810)

Zhengxu Chen committed 11 months ago

[UX] Make `vllm bench serve` discover model by default and use --input-len (#30816)

Michael Goin committed 11 months ago

[Kernels][FI] Skip trtllm attention when num_kv_heads=1 (#30842)

Ye (Charlotte) Qi committed 11 months ago

[Doc][ResponsesAPI] add documentation (#30840)

Andrew Xia committed 11 months ago

[NIXL][Bugfix] Fix NIXL/RDMA registration failure over CuMemAllocator (#29569)

Sheng Lin committed 11 months ago

CustomOp: grouped topk (#29575)

Xinyu Chen committed 11 months ago

[Mamba] Removed disable cascade attn in MambaModelConfig (#30712)

Asaf Joseph Gardin committed 11 months ago

[Bugfix][Frontend] Prevent IndexError in MiniMax M2 tool parser during streaming extraction (#30555)

Robin committed 11 months ago

[XPU] fix broken fp8 online quantization for XPU platform (#30831)

Yan Ma committed 11 months ago

[Bugfix][CPU] Fix CPU backend ROPE dispatch for VL models (#30829)

Li, Jiang committed 11 months ago

Update note comment for flashinfer attention warmup (#30711)

Michael Goin committed 11 months ago

bump up compressed tensors version to 0.13.0 (#30799)

shanjiaz committed 11 months ago

[CI/Build] Fix compatibility between #30244 and #30396 (#30787)

Cyrus Leung committed 11 months ago

[CPU] Add action to automatically label CPU related PRs (#30678)

Fadi Arafeh committed 11 months ago

Update model-hosting-container-standards to 0.1.10 (#30815)

Michael Goin committed 11 months ago

Fix nemotron_nas intermediate_size computation (#30795)

Grzegorz K. Karch committed 11 months ago

[docker] Allow kv_connectors install to fail on arm64 (#30806)

Amr Mahdi committed 11 months ago

[MM] Pass FA version in ViT Attn (#30756)

Nicolò Lucchesi committed 11 months ago