[P/D] Introduce Mooncake Transfer Engine as kv_connector (#24718)

dtc committed 11 months ago

[CI/Build][AMD] Skip test on test_hybrid_attention_mamba_tensor_shapes on ROCm, requires FLASHINFER (#29995)

rasmith committed 11 months ago

[Core] Remove forced None assignment for deprecated PassConfig flags (#29994)

Arpit Khandelwal committed 11 months ago

Add DeepSeek-V3.2 tool parser. (#29848)

Xu Wenqing committed 11 months ago

[ROCm] add fallback for aiter fp8 decode mla (#30005)

Ye (Charlotte) Qi committed 11 months ago

[Core] Fix standalone runs of test_reset_prefix_cache_e2e (#29899)

Mark McLoughlin committed 11 months ago

[Misc] Move functions into `PoolingMetadata` (#30027)

Cyrus Leung committed 11 months ago

[CI][AMD] Match Main CI Behavior By Skipping test_eplb_spec_decode In AMD CI (#30006)

Micah Williamson committed 11 months ago

[ROCm] [Bugfix] `compute_attn_mask_seqlen` for qwen3 omni (#29974)

TJian committed 11 months ago

[Rocm][CI] Fix test_speculator_eagle3 by skipping the CompressedTensorw4a16 Model (#30001)

Charlie Fu committed 11 months ago

[docs] Remove _total from counter metrics names (#30028)

CYJiang committed 11 months ago

[Frontend] refactor harmony utils output message parsing (#29820)

daniel-salib committed 11 months ago

[Bugfix] fixed deepseekv32 tool calling error (#30025)

Chauncey committed 11 months ago

[Misc] Various cleanups for MM input processing (#29970)

Cyrus Leung committed 11 months ago

Fix error while downloading dependencies for CPU backend (#29797)

Jianwei Mao committed 11 months ago

[KVConnector] remove unused code (the model aware kv ops class) (#29709)

Kuntai Du committed 11 months ago

[Frontend] Fixes anthropic /v1/messages streaming not containing input_tokens on first chunk (#29971)

Benjamin Bartels committed 11 months ago

[Bugfix][CPU] Fix CPU KV cache fallback memory allocation (#29604)

gausah01 committed 11 months ago

[Bugfix] Fix adapter_enabled IMA (#29977)

Jee Jee Li committed 11 months ago

[Misc] Add docker build env for Ascend NPU (#30015)

Li Wang committed 11 months ago