A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 4, 03:45 UTC
main
[Compressed Tensors] Allow configs with non-explicit ignores (#41965)
Kyle Sayers committed 6 months ago
Laguna xs dflash support (#41880)
MeganEFlynn committed 6 months ago
[ROCm] Fix AITER AR+RMSNorm no-residual fusion (#41972)
akii96 committed 6 months ago
[ROCm][DeepSeek] Enable V3.2 TP4 AITER MLA (#41835)
akii96 committed 6 months ago
[Core] Replace routing replay with device cache and async D2H pipeline (#39917)
TomerBN-Nvidia committed 6 months ago
[Refactor] Consolidate required/named tool_choice streaming into DelegatingParser (#41876)
Flora Feng committed 6 months ago
[Misc] Delay EPLB Nixl import until needed (#41805)
Nicolò Lucchesi committed 6 months ago
[Model Runner V2] support qwen35 / mamba hybrid model (#35520)
zhrrr committed 6 months ago
Fix spec decode benchmark metrics (#41916)
noobHappylife committed 6 months ago
tokenizer: Add fastokens support (#41741)
AlonKejzman committed 6 months ago
[ROCm][CI] Avoid duplicate ROCm AITER norm-quant patterns (#41534)
Andreas Karatzas committed 6 months ago
[XPU] Implement out-of-place all-reduce functionality (#41808)
Chaojun Zhang committed 6 months ago
[XPU] Fix lora bugs & enable UTs under tests/lora (#38206)
Chaojun Zhang committed 6 months ago
[Bugfix] Fix GLM4-MoE weight loading for NVFP4 quantized checkpoints (#41755)
s-yanev committed 6 months ago
[MM][Gemma4] Use video profiling hints in encoder budget (#41837)
SeongJun Lee committed 6 months ago
[CPU] Bump up to the latest CPU kernels (#41924)
Li, Jiang committed 6 months ago
[Model] Use AutoWeightsLoader for AXK1 (#41901)
wenyili committed 6 months ago
[Bugfix] Fix FusedMoEWithLoRA has no attribute `runner` (#41889)
Jee Jee Li committed 6 months ago
[CI][Arm] skip e2e model tests if HF_TOKEN is not set (#41919)
Fadi Arafeh committed 6 months ago
Mehdi Ghanimifard committed 6 months ago