[Model Runner V2] Change bookkeeping logic in preparation for spec decoding (#29194)

Woosuk Kwon committed 12 months ago

[Bugfix] Use lazy string reference for DeepseekV3Config in config registry (#28958)

Luke committed 12 months ago

[ROCm][CI] Fix "Cannot re-initialize CUDA in forked subprocess" in test_pynccl.py (#29119)

Micah Williamson committed 12 months ago

[CI/Build][AMD] Add check for flash_att_varlen_func to test_tree_attention.py (#29252)

rasmith committed 12 months ago

[CI/Build][AMD] Skip test_multi_shared_storage_connector_consistency in test_multi_connector.py due to hipErrorLaunchFailure when calling .cpu() (#29253)

rasmith committed 12 months ago

[Model Runner V2] Minor fix for cudagraph_utils (#29256)

Woosuk Kwon committed 12 months ago

[Doc] Update more docs with respect to V1 (#29188)

Cyrus Leung committed 12 months ago

docs: fixes distributed executor backend config for multi-node vllm (#29173)

Michael Act committed 12 months ago

[Fix] Add SM check to flashinfer MOE backend (#29144)

jiahanc committed 12 months ago

Patch DeepEP when building docker image with CUDA 13 (#29154)

Qidong Su committed 12 months ago

[Chore] Update batch invariant code owner (#29246)

Wentao Ye committed 12 months ago

Refactor: Move CUDA graph dispatch logic earlier (#27382)

Yizhou committed 12 months ago

[BugFix] Fix chunked prompt logprobs + preemption (#29071)

Nick Hill committed 12 months ago

[CI/build] Removes source compilation from runtime image (#26966)

Benjamin Bartels committed 12 months ago

[Bugfix] Use HF config fields as fallback when loading Mistral config (#29239)

Cyrus Leung committed 12 months ago

[perf][cpu] Accelerate paged attention GEMMs (QK, PV) on Arm CPUs with NEON (#29193)

Fadi Arafeh committed 12 months ago

chore: add RTX_PRO_6000 GLM4.6-FP8 kernel tuning (#29240)

Federico committed 12 months ago

Fix EVS crash when using `video_embeds` inputs in Qwen2.5-VL (#29232)

ZiTian Zhao committed 12 months ago

[Misc] Fix pre-commit (#29238)

Cyrus Leung committed 12 months ago

[BugFix] Fix returned logprobs with spec decode + prefill chunking (#29216)

Nick Hill committed 12 months ago