[Bugfix][KV Offload] Namespace auto cache dtype by effective dtype (#49438)

Jonguk Cheong committed 3 months ago

[Bugfix] Fix handling 5D KV cache in kv_postprocess_layout_on_receive (#47791)

Daniel Socek committed 3 months ago

[KV Offload] Fix num_tokens_after_batch for different termination types (#49285)

AlexHuang committed 3 months ago

[Bugfix] Reject contradictory custom-op directives (#49134)

Taneem Ibrahim committed 3 months ago

[Bugfix][KV Offload] Bound unaligned SWA loads by physical GPU blocks (#49052)

coltonottley committed 3 months ago

[KVOffload][P2P] Generic P2P secondary tier: peer lookup and serving via ParentManager (#48021)

liranschour committed 3 months ago

[Model] Support top_k and top_p sampling for DiffusionGemma (#45429)

Guan-Ming Chiu committed 3 months ago

[Perf] Isolate MM preprocessing on its own executor (#49524)

Guan-Ming Chiu committed 3 months ago

[CI] Fix speech correctness check rejecting improved WER (#49853)

Taneem Ibrahim committed 3 months ago

[Perf] DeepSeek-OCR-2 TTFT Optimize (#49531)

RED committed 3 months ago

[Doc] Add compile cache volume example to the Docker deployment page (#49782)

Nils Matteson committed 3 months ago

[KV Offload] Deduplicate replicated MLA KV in the shared CPU region (#48906)

Chang Guo committed 3 months ago

[Bugfix][KV Offload] Namespace persistent cache by model runner (#49440)

Jonguk Cheong committed 3 months ago

[Bugfix] Respect declared attention contract for ColQwen3.5 retrievers (#49372)

Athrael Soju committed 3 months ago

[Bugfix][KVConnector] Disable cross-layer KV blocks for per-token-head quant (#49226)

achyuthan.s committed 3 months ago

[XPU][CI] add heterogeneous TP UT (#49651)

liuzhenwei committed 3 months ago

[CI] Compute speech WER directly with jiwer (#49773)

Chang Guo committed 3 months ago

[Kernel] TD operand loads for batched MoE GEMM (moe_mmk) on XPU (#46340)

Lena Onyshchenko committed 3 months ago

[Bugfix][KV Offloading] Defer request finalization until final store (#49671)

Rui Yin committed 3 months ago

[UX] DCP Topology Validation (#49777)

Taneem Ibrahim committed 3 months ago