A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 8, 11:15 UTC
main
[Docs] Reduce time spent generating API docs (#34255)
Harry Mellor committed 9 months ago
Patch protobuf for CVE-2026-0994 (#34253)
Seiji Eicher committed 9 months ago
[Frontend] Exploit tokenizers "new stream" in FastIncrementalDetokenizer (#34217)
Nick Hill committed 9 months ago
[Doc] Update Marlin support matrix for Turing (#34319)
Tianqi Ren committed 9 months ago
[Misc] Bump `fastsafetensors` version for latest fixes (#34273)
Nick Hill committed 9 months ago
[torch.compile] Enable AR+rms fusion by default available for `-O2` (#34299)
Luka Govedič committed 9 months ago
[Chore] Move `BaseRenderer` to `base.py` (#34308)
Cyrus Leung committed 9 months ago
[XPU][9/N] clean up existing ipex code/doc (#34111)
Kunshang Ji committed 9 months ago
[model] support FunASR model (#33247)
AllenDou committed 9 months ago
[CPU] Enable FP16 (Half dtype) support for s390x (#34116)
R3hankhan committed 9 months ago
[Bugfix] Fix weight naming in Qwen3.5 (#34313)
Roger Wang committed 9 months ago
[Bugfix] Fix fused MoE IMA (sans chunking) by using int64 for strides (#34279)
Tyler Michael Smith committed 9 months ago
[ModelBash][DSR1 NVFp4] Removed Bf16 Bias Cast (#34298)
Robert Shaw committed 9 months ago
Threshold fix wvSplitk for occasional CI fails (#34013)
Hashem Hashemi committed 9 months ago
[Bugfix] Fix benchmark_moe.py inplace assertion with torch >= 2.9 (#34149)
Matthias Gehre committed 9 months ago
[Plugin] Simplify IO Processor Plugin interface (#34236)
Cyrus Leung committed 9 months ago
[XPU][7/N] enable xpu fp8 moe (#34202)
zofia committed 9 months ago
[Kernel] Apply 256bit LDG/STG To Activation Kernels (#33022)
Дзержи́нский committed 9 months ago
[Bugfix][DeepSeek-V3.2] fix fp8 kvcache type cast (#33884)
Kebe committed 9 months ago
[Misc] Clean up validation logic in input processor (#34144)
Cyrus Leung committed 9 months ago