A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
Disable FlashInfer sampler by default (#26859)
Michael Goin committed about 1 year ago
Chendi.Xue committed about 1 year ago
[Bugfix]fix Qwen3 xml tool parser (#26345)
Zhikaiiii committed about 1 year ago
[Core][Easy] Use envs.__getattr__ for all Unify to environment variable access (#26810)
Jialin Ouyang committed about 1 year ago
[torch.compile] Fix tests for torch==2.9 inductor partition (#26116)
Luka Govedič committed about 1 year ago
[WideEP][P/D] Add usage stats for DP+EP and KV Connector (#26836)
Tyler Michael Smith committed about 1 year ago
[frontend][gptoss] Add per turn stats into Harmony Context (#25061)
Ye Hu committed about 1 year ago
[CI Failure] Fix torchao dep failure for Quantization Test (#26824)
Michael Goin committed about 1 year ago
[Core] Streamline some structured output related code (#26737)
Nick Hill committed about 1 year ago
remove attn output view kernel (#26680)
Boyuan Feng committed about 1 year ago
[Minor] Group async_scheduling related fields in model runner init (#26736)
Nick Hill committed about 1 year ago
fix: response_format for completion (#23212)
Nan Qin committed about 1 year ago
Added MoE configs for llama 4, H200 device with tp=4/8 tuning (#26837)
Dhruvil Bhatt committed about 1 year ago
[Perf] Cache vllm.env.__getattr__ result to avoid recomputation (#26146)
Jialin Ouyang committed about 1 year ago
Notice for deprecation of AutoAWQ (#26820)
HDCharles committed about 1 year ago
[Core] Reuse empty block lists whenever possible in KVCacheBlocks to mitigate GC costs (#24964)
Jialin Ouyang committed about 1 year ago
[Attention][Spec Decode] FlashMLA spec decode support (#26541)
Matthew Bonanni committed about 1 year ago
llama4_vision_rope: add HIP override to accept (q, k) and avoid (positions, q, k) mismatch (#26790)
Huamin Li committed about 1 year ago
[CI Failure] Fix tests with missing TinyLlama-1.1B-Chat-v1.0-FP8-e2e (#26816)
Michael Goin committed about 1 year ago
[Kernel][MoE] Add MoE tunings for GLM 4.6-FP8 and GLM 4.5 Air on NVidia B200 (#26818)
Ze'ev Klapow committed about 1 year ago