[Mooncake] Skip KV lookup for non-reachable SWA blocks (#45444)

Wei Zhao committed 5 months ago

fix(anthropic): preserve inline system message position for prefix caching (#44602)

MrFan committed 5 months ago

[Bugfix][Frontend] Fix Anthropic count_tokens decorator order driving server load negative (#44725)

Ting SUN committed 5 months ago

[XPU] Update nixl to v0.10.1 in Dockerfile (#40287)

liuzhenwei committed 5 months ago

Revert "[Kernel] Add PDL support for DeepGEMM kernel" (#45999)

Micah Williamson committed 5 months ago

[Test][KV Connector] Add request_finished fence population tests for offloading scheduler (#45679)

AlexHuang committed 5 months ago

[KV Offloading] Remove dummy worker-side stats from OffloadingConnector (#45905)

AlexHuang committed 5 months ago

[Refactor] Remove dead cutlass mxfp8 code (#44681)

Wentao Ye committed 5 months ago

[Fix][KV offload] Defer `on_request_finished` until in-flight transfers drain (#45823)

Ronen Schaffer committed 5 months ago

[Kernel] Add PDL support for DeepGEMM kernel (#42996)

Jee Jee Li committed 5 months ago

[SimpleCPUOffloadConnector]: Add support for reset_cache() (#39726)

Jonathan Chen committed 5 months ago

[CPUOffloading] Guard CPU eviction check (#45757)

Varun Sundar Rabindranath committed 5 months ago

[Rust Frontend] Validate tokenized bad_words vocabulary range (#45876)

Reid committed 5 months ago

[XPU] Fix FP8 block-scaled scheme selection on non-CUDA platforms (#43958)

Yejing Lai committed 5 months ago

[Rust Frontend][Perf] O(n) argument scan in tool parser (#45826)

Bugen Zhao committed 5 months ago

[Bugfix] Complete one-shot fused all-reduce PDL at end to avoid NaN (#45448)

Ace Eldeib committed 5 months ago

[CI/Build][Bugfix] Fix SD LoRA (#45941)

Jee Jee Li committed 5 months ago

[XPU][CI] add model runner v2 into CI (#44650)

liuzhenwei committed 5 months ago

Revert "[DSV4 Perf] Optimize dsv4 cudagraph by reducing `eager_break_during_capture`" (#45309) (#45972)

Woosuk Kwon committed 5 months ago

[Bugfix] Fix NixlConnector handshake block_len validation for GQA-replicated KV heads (#45879)

Bryan Shan committed 5 months ago