[Bugfix] Fix the fp8_mqa_logits dim mismatch (#32652)

Chauncey committed 10 months ago

[3/N] Initialize MM components in context managers (I-L) (#32650)

Cyrus Leung committed 10 months ago

[Core] Cleanup shm based object store on engine shutdown (#32429)

Walter Beller-Morales committed 10 months ago

[2/N] Initialize MM components in context managers (E-H) (#32641)

Cyrus Leung committed 10 months ago

[Refactor] Make FP8 Linear Ops use kernel abstraction (#27814)

vllmellm committed 10 months ago

[Model Runner V2] Skip kernel launch for penalties & logit_bias (#32634)

Woosuk Kwon committed 10 months ago

[1/N] Initialize MM components in context managers (A-D) (#32632)

Cyrus Leung committed 10 months ago

[Model] Use context managers for encoder- and LM-only mode (#32605)

Cyrus Leung committed 10 months ago

[Model Runner V2] Decouple temperature from penalties (#32629)

Woosuk Kwon committed 10 months ago

[Model Runner V2] Refactor get_cudagraph_and_dp_padding (#32625)

Woosuk Kwon committed 10 months ago

[Feat] allow inplace loading lora (#31326)

Jackmin801 committed 10 months ago

[Model Runner V2] Initialized communication buffer for DP (#32624)

Woosuk Kwon committed 10 months ago

[Attention][MLA] Make FLASHINFER_MLA the default MLA backend on Blackwell, and TRTLLM the default prefill (#32615)

Matthew Bonanni committed 10 months ago

[Model Runner V2] Refactor `dummy_run` (#32533)

Woosuk Kwon committed 10 months ago

feat: spec decode with draft models (#24322)

Tomas Ruiz committed 10 months ago

docs: prefix caching seems quite outdated (#28784)

lon committed 10 months ago

[BugFix] Fix TRT-LLM NVFP4 DP/EP (#32349)

jiahanc committed 10 months ago

[CI] Add Helion as an optional dependency (#32482)

Yanan Cao committed 10 months ago

[BUGFIX] Fix `test_mla_backends.py`. Scale MLA projection weights to prevent numerical instability (#32529)

Vadim Gimpelson committed 10 months ago

[CI][amd] Revert NIXL connector change to avoid crash (#32570)

qli88 committed 10 months ago