[Docs] Add Expert Parallelism Initial Documentation (#21373)

Simon Mo committed over 1 year ago

[Docs][minor] Fix broken gh-file link in distributed serving docs (#21543)

Ricardo Decal committed over 1 year ago

[P/D] Support CPU Transfer in NixlConnector (#18293)

Juncheng Gu committed over 1 year ago

[P/D] Move FakeNixlWrapper to test dir (#21328)

Rui Qiao committed over 1 year ago

[XPU] Conditionally import CUDA-specific passes to avoid import errors on xpu platform (#21036)

Chaojun Zhang committed over 1 year ago

Update flashinfer CUTLASS MoE Kernel (#21408)

Shu Wang committed over 1 year ago

[Bug] Fix Compressed Tensor NVFP4 `cutlass_fp4_group_mm` illegal memory access (#21465)

Wentao Ye committed over 1 year ago

[Docs] Rewrite Distributed Inference and Serving guide (#20593)

Ricardo Decal committed over 1 year ago

[Docs] Update Tensorizer usage documentation (#21190)

Sanger Steel committed over 1 year ago

[Fix] Update mamba_ssm to 2.2.5 (#21421)

elvischenv committed over 1 year ago

[Bugfix] Fix CUDA arch flags for MoE permute (#21426)

Ming Yang committed over 1 year ago

[Model] Officially support Emu3 with Transformers backend (#21319)

Harry Mellor committed over 1 year ago

[Attention] Optimize FlashInfer MetadataBuilder Build call (#21137)

Lucas Wilkinson committed over 1 year ago

bump `flashinfer` to `v0.2.8` (#21385)

cjackal committed over 1 year ago

[Feat] Allow custom naming of vLLM processes (#21445)

Chauncey committed over 1 year ago

[Misc] Improve comment for DPEngineCoreActor._set_cuda_visible_devices() (#21501)

Rui Qiao committed over 1 year ago

Replace `--expand-tools-even-if-tool-choice-none` with `--exclude-tools-when-tool-choice-none` for v0.10.0 (#20544)

Shintarou Okada committed over 1 year ago

remove GLM-4.5 quantization wrong Code (#21435)

Yuxuan Zhang committed over 1 year ago

[Core] Support model loader plugins (#21067)

22quinn committed over 1 year ago

[Misc] Fix duplicate FusedMoEConfig debug messages (#21455)

Nick Hill committed over 1 year ago