[Bugfix][AMD] Update torch_bindings so that scaled_fp4_quant isn't build on ROCm (#13235)

Sage Moore committed almost 2 years ago

[V1][PP] Fix memory profiling in PP (#13315)

Woosuk Kwon committed almost 2 years ago

[V1][Sampler] Don't apply temp for greedy-only (#13311)

Nick Hill committed almost 2 years ago

[V1][CI] Fix failed v1-test because of min_p (#13316)

Woosuk Kwon committed almost 2 years ago

[V1][Core] min_p sampling support (#13191)

Aoyu committed almost 2 years ago

[Core] Reduce TTFT with concurrent partial prefills (#10235)

Joe Runde committed almost 2 years ago

[Quant][Perf] Use moe_wna16 kernel by default for MoEs with many experts (#13236)

Michael Goin committed almost 2 years ago

[Hardware][Gaudi][Bugfix] Fix error for guided decoding (#12317)

Yu-Zhou committed almost 2 years ago

[Core] choice-based structured output with xgrammar (#12632)

Russell Bryant committed almost 2 years ago

Support logit_bias in v1 Sampler (#13079)

Lu Fang committed almost 2 years ago

[V1] Simplify GPUModelRunner._update_states check (#13265)

Nick Hill committed almost 2 years ago

[VLM] Keep track of whether prompt replacements have been applied (#13215)

Cyrus Leung committed almost 2 years ago

[Misc] Log time consumption of sleep and wake-up (#13115)

Jun Duan committed almost 2 years ago

[Bugfix] Fix missing parentheses (#13263)

Xu Song committed almost 2 years ago

[Frontend] Optionally remove memory buffer used for uploading to URLs in run_batch (#12927)

Pooya Davoodi committed almost 2 years ago

[WIP] TPU V1 Support Refactored (#13049)

Alexander Matveev committed almost 2 years ago

[Bugfix][V1] GPUModelRunner._update_states should return True when there is a finished request in batch (#13126)

Kero Liang committed almost 2 years ago

[ROCm][V1] Add intial ROCm support to V1 (#12790)

Sage Moore committed almost 2 years ago

[V1] LoRA - Enable Serving Usecase (#12883)

Varun Sundar Rabindranath committed almost 2 years ago

Expand MLA to support most types of quantization (#13181)

Michael Goin committed almost 2 years ago