A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 2, 07:15 UTC
main
[Bugfix] fix wrong partial_rotary_factor calculation for bailing_moe model. (#43770)
zzt committed 5 months ago
[Rust Frontend] Add InternLM2 tool parser (#43481)
Will.hou committed 5 months ago
[Frontend] Resettle generative scoring entrypoint. (#44153)
wang.yuqi committed 5 months ago
fix: glm5.1 pp model loading (#42944)
Uranus committed 5 months ago
[CPU][RISC-V] Add missing RVV cpu_types helpers for WNA16 (#42730)
wcy committed 5 months ago
[Docs] Replace broken video url in examples (#44159)
Isotr0py committed 5 months ago
[BugFix] Fix `_has_module` to verify native deps via trial import (#44035)
Jeffrey Wang committed 5 months ago
[MRV2] Remove Eagle's dedicated CUDA graph pool (#44078)
Lucas Wilkinson committed 5 months ago
Umut Polat committed 5 months ago
docs: fix MLA attention docstring examples (#44118)
nightcityblade committed 5 months ago
[CI/Build] Enable Step3p7ForConditionalGeneration testing (#43956)
Jee Jee Li committed 5 months ago
[CPU][Zen] Route W8A8 and W4A16 linear inference through zentorch on AMD Zen CPUs (#41813)
Aakar Dwivedi committed 5 months ago
[MRV2] Support breakable CUDA graph (#44050)
Woosuk Kwon committed 5 months ago
Wentao Ye committed 5 months ago
[Governance] Add @BugenZhao as Rust frontend code owner (#44047)
Bugen Zhao committed 5 months ago
[ROCm] Add attention sink support to AITer flash attention backend (#43817)
Xiaoran committed 5 months ago
[Bugfix] Fix RMSNorm kernels to multiply in weight's native dtype (#42379)
Lanze Liu committed 5 months ago
[BugFix][Platform] Fix import vllm.platforms.rocm error on non-CUDA test_gpt_oss.py (#43571)
Liangliang Ma committed 5 months ago
[ROCm] cmake: support PYTORCH_FOUND_HIP for torch 2.13 native HIP language support (#43881)
nemanjaudovic committed 5 months ago
[ROCm][CI] Fix failure in the Phi3V pooling test (#44028)
Andreas Karatzas committed 5 months ago