A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 3, 22:45 UTC
main
[CI] Trigger LoRA test when changing MoE code. (#42196)
Jee Jee Li committed 6 months ago
[Docs] Fix broken local links (#42160)
Ethan Feng committed 6 months ago
Add @zyongye to CODEOWNERS (#42200)
Yongye Zhu committed 6 months ago
Fix mypy failure on main (#42197)
Mohammad Miadh Angkad committed 6 months ago
[Frontend][Bugfix] Abort ASR engine requests on cancellation (#41266)
abdulrahman-cohere committed 6 months ago
[ROCm][CI] Fix NIXL spec-decode acceptance startup and diagnostics (#41313)
Andreas Karatzas committed 6 months ago
Dao007forever committed 6 months ago
[ROCm][CI] Stabilize ROCm shutdown and distributed compile CI (#41573)
Andreas Karatzas committed 6 months ago
[Bugfix] Fix DeepSeek v4 topk numerical issue for unaligned max-model-len (#42169)
Wei Zhao committed 6 months ago
docs: clarify Gemma 4 assistant speculative decoding (#42180)
Abhishek Gupta committed 6 months ago
Use CU_MEMCPY_SRC_ACCESS_ORDER_ANY for batch KV cache swaps (#39306)
Itay Etelis committed 6 months ago
Handle optional bool-or-string CLI args in get_kwargs (#40951)
Christian Van committed 6 months ago
[CI/Build] Use modelscope's international site for regression test (#42176)
Isotr0py committed 6 months ago
[Bugfix] Fix SP pass for multimodal models and PP+SP residual handling (#33322)
Wang Xingran committed 6 months ago
Fix: Nemotron 3 rescue whitespace-only final_content, not just None (#41846)
Nave Assaf committed 6 months ago
Add NVFP4 all-gather GEMM fusion for AsyncTP (#41882)
baonudesifeizhai committed 6 months ago
[Bugfix] Skip routed-experts hot path when disabled (#42148)
aoshen02 committed 6 months ago
Wentao Ye committed 6 months ago
[Refactor] Nixl util using lazy init (#41392)
Wentao Ye committed 6 months ago
[Quantization] Add ModelOpt NVFP4 W4A16 (4-bit weights, fp16/bf16 activations) support (#41769)
Juhi Mittal committed 6 months ago