A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 4, 16:15 UTC
main
Fix Nano Nemotron VL static image inputs (#40724)
milesial committed 6 months ago
[Bugfix] Fix IMA in DSA + MTP (#40772)
Woosuk Kwon committed 6 months ago
[Model] Gemma4: add bidirectional vision attention for sliding layers with window guard (#40534)
Luciano Martins committed 6 months ago
[XPU] Enable torch.compile for XPU GDN attention (#39466)
Yuwen Zhou committed 6 months ago
Support only half types for concat_mla_q kernel (#37892)
Xin Yang committed 6 months ago
[XPU][CI] Fix Docker cleanup races on Intel CI runners (#40761)
xiangdong committed 6 months ago
[MoE] Move cutlass moe to fused_moe/experts/ (#40574)
Jackmin801 committed 6 months ago
[Deprecate] Deprecate LLM.reward offline api, use LLM.encode instead. (#40688)
wang.yuqi committed 6 months ago
[Bugfix] Avoid mutating `chat_template_kwargs` in `HYV3ReasoningParser` initialization (#40713)
Bugen Zhao committed 6 months ago
[Platform] Fix RISC-V platform detection (lscpu parsing + non-NUMA meminfo) (#40427)
lyd1992 committed 6 months ago
[Feature] Avoid eager import of the "mistral_common" package. (#40043)
Neil Schemenauer committed 6 months ago
[Doc] fix capitalization consistency in README (vLLM, Hugging Face) (#40729)
Vinayak Kumar committed 6 months ago
[MRV2] Ensure warmup covers prefill path (#40746)
Nick Hill committed 6 months ago
[BUG]: fix HF tokenizer concurrent borrow in tool parsers (#40059)
yzong-rh committed 6 months ago
[Misc] Added curl retries in install_python_libraries.sh (#36700)
Dmitry Tokarev committed 6 months ago
[Bugfix] Treat <tool_call> as implicit reasoning end in Qwen3 parser (#35687)
Doug Campos committed 6 months ago
[Feat] Unified Synthetic Acceptance Rate for V1 and V2 (#40662)
Benjamin Chislett committed 6 months ago
[Core] Avoid seq_lens_cpu GPU->CPU sync (#40654)
Nick Hill committed 6 months ago
[EPLB] Remove asyncio infrastructure from Async EPLB (#40730)
Sage Moore committed 6 months ago
[ROCm] Cast score correction bias tensor during model construction for DeepSeek/Kimi-K2 (#39999)
Hemanth Acharya committed 6 months ago