[Bug] Fix batch invariant test issue, bs=1 with `max_seq_num = 1` (#39320)

Wentao Ye committed 7 months ago

[Speculative Decoding] Add DFlash speculators config parsing (#38300)

zhanqiuhu committed 7 months ago

[Bugfix][NIXL] Fix `_logical_to_kernel_block_ids` conversion for non-mamba models (#39724)

zhanqiuhu committed 7 months ago

[CI] Only build release Docker images when NIGHTLY=1 (#39882)

Kevin H. Luu committed 7 months ago

[Mooncake] Fix mixed MLA+Eagle block-size validation (#39596)

Zhewen Li committed 7 months ago

Bug/test eagle dp v2 (#39838)

Monishver committed 7 months ago

[Test] Fix @create_new_process_for_each_test("fork") in interactive shell pipeline (#29130)

Mark McLoughlin committed 7 months ago

[BugFix] KeyError on scope["method"] for realtime api websocket in AuthenticationMiddleware (#36934)

daniebrill committed 7 months ago

[KVConnector][LMCache] Propagate cache_salt through MP connector for per-user cache isolation (#39837)

Roy Huang committed 7 months ago

[SpecDecode][Benchmark] Add SPEED-bench support to benchmarking CLI (#36029)

Talor Abramovich committed 7 months ago

[XPU] properly handle q_descale on XPU as quant query input not supported (#39676)

Yan Ma committed 7 months ago

FIX: support language_model.backbone naming in NemotronH Nano VL quantization config (#39901)

danielafrimi committed 7 months ago

[BugFix][Graph] fix: handle empty sym_shape_indices in PiecewiseBackend. (#39395)

Chauncey committed 7 months ago

[XPU][MXFP4] add mxfp4 quant op for XPU (#39857)

zofia committed 7 months ago

[Metrics] Add request_id to FinishedRequestStats to enable correlation between metrics and requests (#39710)

Csrayz committed 7 months ago

[Quantization][Autoround][CPU] Add W4A16 Support (#38192)

Zhenzhong Xu committed 7 months ago

fix online fp8 for MiniCPM models (#39862)

Yan Ma committed 7 months ago

[kv_offload+HMA][3/N]: Remove block_size from KVEvents (#36644)

Or Ozeri committed 7 months ago

[Bugfix] Fix tool_calls Iterable consumed when debug logging is enabled (#34844)

Wojciech Wais committed 7 months ago

[CI][NIXL] Fix PD CI breakage: pin nixl-cu{12,13} versions (#39851)

zhanqiuhu committed 7 months ago