Fix torch version check for SM100 mxfp4 (#22535)

zifeitong committed over 1 year ago

[CI][Nixl] Check kv cache layout during handshake (#22745)

Nicolò Lucchesi committed over 1 year ago

[Kernel][AMD] Avoid D2H copy and cumsum kernel (#22683)

Xiaozhu Meng committed over 1 year ago

[Chore] Update CODEOWNERS to include @yewentao256 for CUDA kernels, attention backends, quantization, and related tests (#22741)

Wentao Ye committed over 1 year ago

Add more test scenario for tensor schema (#22733)

TeeKen Lau committed over 1 year ago

Add: `SupportsEagle3` interface for explicit EAGLE3 support (#22642)

Rahul Tuli committed over 1 year ago

[Benchmark] Fix terminal colors in benchmark_serving_multi_turn (python 3.12) (#22730)

Daniel Serebrenik committed over 1 year ago

[Bugfix][CI] Fix `test_remote_decode_lifecycle.py::test_short_prompt_lifecycle` (#22727)

Nicolò Lucchesi committed over 1 year ago

Fix Llama4 FlashInfer FP4 MoE issues (#22511)

Po-Han Huang (NVIDIA) committed over 1 year ago

[CI Failure] fix tests/entrypoints/openai/test_skip_tokenizer.py (#22708)

wang.yuqi committed over 1 year ago

Officially support SmolLM3 using the Transformers backend (#22665)

Harry Mellor committed over 1 year ago

[BugFix][Nixl][PD] Fix heterogenous TP (#22663)

Nicolò Lucchesi committed over 1 year ago

[Docs] Improve docs navigation (#22720)

Harry Mellor committed over 1 year ago

[gpt-oss] Enable gpt-oss on ampere (#22714)

Yongye Zhu committed over 1 year ago

[Misc] remove GH discussions link (#22722)

Jee Jee Li committed over 1 year ago

[LMCache][Example] Align the PYTHONHASHSEED for prefillers and decoders for KV chunks hashing (#21161)

zejunchen-zejun committed over 1 year ago

[Bugfix] Mamba2 SSD varlen bug fix initstates decay, improve test, assert chunk pwr 2 (#21783)

RishiAstra committed over 1 year ago

Fix: AWQ Marlin get_quant_method does not recognize "modules_to_not_convert" (#21888)

Jun-Howie committed over 1 year ago

[Bugfix] Fix erroneous randomly generated cases in bad word testing (#22170)

phantomlei committed over 1 year ago

[V0] Correct CUDA Graph capture for encoder-decoder models (#22630)

Sugar-zsg committed over 1 year ago