[benchmark] Port benchmark request sent optimization to benchmark_serving (#21209)

Jialin Ouyang committed over 1 year ago

[Core] Optimize update checks in LogitsProcessor (#21245)

Jialin Ouyang committed over 1 year ago

[Misc] Remove deprecated args in v0.10 (#21349)

Kebe committed over 1 year ago

[Bugfix] Fix eviction cached blocked logic (#21357)

Simon Mo committed over 1 year ago

Add arcee model (#21296)

Raghav Ravishankar committed over 1 year ago

[Feature][eplb] add verify ep or tp or dp (#21102)

rongfu.leng committed over 1 year ago

Update fp4 quantize API (#21327)

Shu Wang committed over 1 year ago

[Bug] DeepGemm: Fix Cuda Init Error (#21312)

Wentao Ye committed over 1 year ago

[Misc] DeepEPHighThroughtput - Enable Inductor pass (#21311)

Varun Sundar Rabindranath committed over 1 year ago

Fix kv_cache_dtype handling for out-of-tree HPU plugin (#21302)

Konrad Zawora committed over 1 year ago

[Refactor] Fix Compile Warning #1444-D (#21208)

Wentao Ye committed over 1 year ago

[V1] [Hybrid] Add new test to verify that hybrid views into KVCacheTensor are compatible (#21300)

Thomas Parnell committed over 1 year ago

[Core] Minimize number of dict lookup in _maybe_evict_cached_block (#21281)

Jialin Ouyang committed over 1 year ago

Revert "[Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) (#21334)

Ming Yang committed over 1 year ago

[Intel GPU] Ray Compiled Graph avoid NCCL for Intel GPU (#21338)

Ratnam Parikh committed over 1 year ago

[Doc] Fix CPU doc format (#21316)

Li, Jiang committed over 1 year ago

[XPU] Enable external_launcher to serve as an executor via torchrun (#21021)

Chaojun Zhang committed over 1 year ago

[v1][sampler] Inplace logprobs comparison to get the token rank (#21283)

Lu Fang committed over 1 year ago

[perf] Speed up align sum kernels (#21079)

Himanshu Jaju committed over 1 year ago

Fix bad lm-eval fork (#21318)

Michael Goin committed over 1 year ago