A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 10, 12:45 UTC
main
[Fix] Enable mm_processor_cache with vision LoRA (#31927)
prashanth058 committed 10 months ago
[BugFix] Fix spec decoding edge case bugs (#31944)
Nick Hill committed 10 months ago
[grpc] Support gRPC server entrypoint (#30190)
Chang Su committed 10 months ago
[chore] Update FA commit (#30460)
Lucas Wilkinson committed 10 months ago
[platform] add dp_metadata arg to set_additional_forward_context (#31942)
Ronald committed 10 months ago
[Model] Enable LoRA support for tower and connector in DotsOCR (#31825)
ShaanveerS committed 10 months ago
[ROCm][CI] v1 cpu offloading attention backend fix (#31833)
Andreas Karatzas committed 10 months ago
[Doc] Add Claude code usage example (#31188)
Michael Goin committed 10 months ago
[CI][BugFix][AMD] Actually skip tests marked @pytest.mark.skip_v1 (#31873)
rasmith committed 10 months ago
[ROCm][CI] Add rocm support for run-multi-node-test.sh (#31922)
Charlie Fu committed 10 months ago
[ROCm][CI] Fix attention backend test flakiness from uninitialized KV cache memory (#31928)
Andreas Karatzas committed 10 months ago
[ROCm][LoRA] Fix MoE accuracy regression by preserving float32 router weight scaling (#31931)
Andreas Karatzas committed 10 months ago
[BugFix] Fix flakiness in test_eagle_dp for PyTorch 2.10 (#31915)
Richard Zou committed 10 months ago
[MoE Refactor][16/N] Apply Refactor to NVFP4 (#31692)
Robert Shaw committed 10 months ago
[CI] Skip Qwen-VL in multimodal processing tests due to flaky external dependency (#31932)
Andreas Karatzas committed 10 months ago
fix(rocm): add early return in get_flash_attn_version for ROCm (#31286)
Rabi Mishra committed 10 months ago
feat(moe): Add is_act_and_mul=False support for Triton MoE kernels (#31645)
Rabi Mishra committed 10 months ago
[0/N][Attention] Fix miscellaneous pre-commit issues (#31924)
Matthew Bonanni committed 10 months ago
[MoE Refactor][15/N] Apply Refactor to Fp8 (#31415)
Robert Shaw committed 10 months ago
Add back missing DeepEP LL params (#31911)
Elvir Crnčević committed 10 months ago