[V1] Eagerly remove finished requests from the batch (#14388)

Nick Hill committed over 1 year ago

[v1] torch.compile integration explanation (#14437)

youkaichao committed over 1 year ago

[Misc] Add Phi4-MM example (#14343)

Jee Jee Li committed over 1 year ago

[Kernel] optimize performance of gptq marlin kernel when n is small (#14138)

Jinzhen Lin committed over 1 year ago

[Benchmarks] Make detokenization optional in benchmark scripts (#11697)

Jeremy Arnold committed over 1 year ago

[Doc] Update prefix_caching.md to match the example image (#14420)

York-RDWang committed over 1 year ago

[V1][Core] Support for Structured Outputs (#12388)

Aaron Pham committed over 1 year ago

Use the optimized block sizes after tuning the kernel. (#14329)

iefgnoix committed over 1 year ago

Fix missing `kv_caches` and `attn_metadata` in `OpenVINOCausalLM` (#14271)

Harry Mellor committed over 1 year ago

[BUGFIX] Skip tokenization support for throughput benchmark (#12712)

Aleksandr Malyshev committed over 1 year ago

[Misc] Set default value of seed to None (#14274)

மனோஜ்குமார் பழனிச்சாமி committed over 1 year ago

[Bugfix] Clean up multi-modal processors (#14417)

Cyrus Leung committed over 1 year ago

[Bugfix] Further clean up LoRA test (#14422)

Jee Jee Li committed over 1 year ago

correct wrong markdown syntax (#14414)

Peng Li committed over 1 year ago

[GH] Auto-apply multi-modality label to relevant PRs (#14402)

Cyrus Leung committed over 1 year ago

OpenVINO: added CPU-like conditions (#14338)

Ilya Lavrenov committed over 1 year ago

[Build] Add nightly wheel fallback when latest commit wheel unavailable (#14358)

Isotr0py committed over 1 year ago

[Bugfix] Fix JambaForCausalLM LoRA (#14370)

Jee Jee Li committed over 1 year ago

[BugFix] Illegal Memory Access in the blockwise cutlass fp8 GEMMs (#14396)

Lucas Wilkinson committed over 1 year ago

[FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390)

Luka Govedič committed over 1 year ago