A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 22:15 UTC
main
[Model Runner V2] Enable forcing a specific acceptance rate during rejection sampling (#38045)
Giancarlo Delfin committed 7 months ago
Fix multi-node allreduce fusion (#38136)
Wei Zhao committed 7 months ago
[Bugfix][ROCm] Fix lru_cache on paged_mqa_logits_module (#37547)
Stig-Arne Grönroos committed 7 months ago
[ROCm] [Bugfix] [Release] Fix nightly rocm release pipeline (#38263)
TJian committed 7 months ago
[ROCm][CI] Override PYTORCH_ROCM_ARCH with detected GPU arch in test containers (#38165)
Andreas Karatzas committed 7 months ago
[ROCm][CI] Add uv pip compile workflow for rocm-test.txt lockfile (#37930)
Andreas Karatzas committed 7 months ago
[ROCM][Bugfix] Use correct stride in cp_mha_gather_cache_kernel for hybrid model (#37228) (#37228)
jennyyyyzhen committed 7 months ago
[ROCm][CI] Add LM Eval Qwen3.5 Models test for MI355 (#38155)
Andreas Karatzas committed 7 months ago
[CI] Fix conch kernel crash on 3D input by reshaping to 2D before GEMM (#38178)
Andreas Karatzas committed 7 months ago
[Mamba][APC] Add test case to compare apc outputs (#34977)
Divakar Verma committed 7 months ago
[Releases] [ROCm] Enable Nightly Docker Image and Wheel Releases for ROCm (#37283)
TJian committed 7 months ago
[ROCm][CI] Fix AITER state leak in shared_fused_moe_routed_transform test (#38137)
Andreas Karatzas committed 7 months ago
[Bugfix] Restore CUDA graph persistent buffers for FP8 FlashMLA decode (#35175)
haosdent committed 7 months ago
[Bugfix][Minor] Fix potential NameError in mamba backend selector and misc typos (#35886)
Chuan (Richard) Li committed 7 months ago
[CI] Add batch invariant test for b200 (#38014)
Wentao Ye committed 7 months ago
zhang-prog committed 7 months ago
Revert "[MoE Kernel] Flashinfer nvfp4 cutedsl moe kernel integration" (#38050) (#38169)
Zhewen Li committed 7 months ago
[Renderer] Consolidate factory methods (#38218)
Cyrus Leung committed 7 months ago
[bug-fix] GLM OCR Patch Merger context_dim (#37962)
Jared Wen committed 7 months ago
[CI] Reorganize scoring tests (#38207)
wang.yuqi committed 7 months ago