[Config] Unify indexer cache dtype under attention_config.indexer_kv_dtype (#52550)

Yongye Zhu committed 2 months ago

[ROCm][CI] Enable ViT CUDA graph tests on AMD gfx950 GPUs (#52256)

Shanshan Shen committed 2 months ago

[Kimi-K3][Perf] Update FlashKDA for automatic K2 V-split (#52458)

kiroxu committed 2 months ago

[Bugfix][Mooncake] Reference GPU blocks for in-flight store jobs and key the store ledger by store_job_id (#52372)

Guanyi Chen committed 2 months ago

[Bugfix][V1][Multimodal] Ignore stale same-step encoder cache evictions (#52482)

Tianyu Guo committed 2 months ago

[CI/Build] Avoid duplicate runner startup for multimodal test (#52417)

Isotr0py committed 2 months ago

[CI] Fit small KV-offload evals within shared memory (#52496)

Taneem Ibrahim committed 2 months ago

[ModelRunner v2] Support Transformers pooling model (#52425)

Taneem Ibrahim committed 2 months ago

[ROCm][DSV4][Perf] Optimize Triton sparse-MLA decode on gfx950 (#52212)

Fangzhou Ai committed 2 months ago

[Bugfix][Attention] Temporarily disable FA4 head-dim 256 (#52050)

Taneem Ibrahim committed 2 months ago

[Refactor] Remove dead code for quantization (#52221)

Wentao Ye committed 2 months ago

[ROCm][Perf] gfx942: use FlyDSL fp8 MQA logits kernel (ROCm/aiter#3913) (#49544)

akii96 committed 2 months ago

[Perf][DSV4] Optimize global top-k index kernel with compile-time constants (#51967)

Chauncey committed 2 months ago

[Core] Add CuMemAllocator.discard() for tag-selective GPU memory release (#52514)

Dakai An committed 2 months ago

[Bugfix][ROCm] Skip FP8 MLA prefill PS-metadata build for chunked-context batches (#52356)

Shantipriya Parida committed 2 months ago

[Perf][DSV4] Optimize sparse top-k metadata kernels for higher prefill throughput (#52084)

Chauncey committed 2 months ago

[Bugfix][Spec Decode][Structured Output] DSpark: fix the grammar bitmask mapping when the draft budget is zero (#52436)

oops-oom committed 2 months ago

[Bugfix][Multimodal] Keep Gemma 4 video frame counts on CPU (#52441)

Chauncey committed 2 months ago

[Bugfix][Model Runner V2][Spec Decode] Fix off-by-one in bad_words draft-prefix matching (#52311)

Jyan-R committed 2 months ago

[Bugfix][Anthropic] Return 4xx for client-caused errors in /v1/messages (#52246)

Do_it_now_! committed 2 months ago