[V1][Metrics] Handle preemptions (#13169)

Mark McLoughlin committed over 1 year ago

[ROCm][V1] Update reshape_and_cache to properly work with CUDA graph padding (#13922)

Sage Moore committed over 1 year ago

[misc] Rename Ray ADAG to Compiled Graph (#13928)

Rui Qiao committed over 1 year ago

Use CUDA 12.4 as default for release and nightly wheels (#12098)

Michael Goin committed over 1 year ago

[ROCm][Quantization][Kernel] Use FP8 FNUZ when OCP flag is 0 or undefined (#13851)

ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 committed over 1 year ago

[Kernel] FlashMLA integration (#13747)

Lucas Wilkinson committed over 1 year ago

[BugFix] Make FP8 Linear compatible with torch.compile (#13918)

Woosuk Kwon committed over 1 year ago

[Bugfix] Backend option to disable xgrammar any_whitespace (#12744)

Wallas Henrique committed over 1 year ago

[Misc] fixed qwen_vl_utils parameter error (#13906)

Chauncey committed over 1 year ago

[TPU] use torch2.6 with whl package (#13860)

Chenyaaang committed over 1 year ago

[Bugfix] Update expected token counts for Ultravox tests (#13895)

Cyrus Leung committed over 1 year ago

[Bugfix] Do not crash V0 engine on input errors (#13101)

Joe Runde committed over 1 year ago

[Misc]: Add support for goodput on guided benchmarking + TPOT calculation refactor (#13736)

Brayden Zhong committed over 1 year ago

[Bugfix] Handle None parameters in Mistral function calls. (#13786)

Florian Greinacher committed over 1 year ago

Add comments on accessing `kv_cache` and `attn_metadata` (#13887)

Harry Mellor committed over 1 year ago

[Bugfix] Add test example for Ultravox v0.5 (#13890)

Cyrus Leung committed over 1 year ago

[Misc] Fix input processing for Ultravox (#13871)

Roger Wang committed over 1 year ago

[Misc] Improve LoRA spelling (#13831)

Jee Jee Li committed over 1 year ago

[v0][Core] Use xgrammar shared context to avoid copy overhead for offline engine (#13837)

Seth Kimmel committed over 1 year ago

[ROCm] Disable chunked prefill/prefix caching when running MLA on non-cuda platforms (#13844)

Sage Moore committed over 1 year ago