A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 11:00 UTC
main
Michael Goin committed 7 months ago
[Bugfix][Frontend] Fix Gemma4 streaming HTML duplication after tool calls (#38909)
yoke committed 7 months ago
[Parser] Migrate response api streaming to unified parser (#38755)
Flora Feng committed 7 months ago
[Bugfix] Fix Qwen3 tool parser for Responses API tools (#38848)
Flora Feng committed 7 months ago
[Docs] Add Phi-4-reasoning-vision to supported models + examples (#39232)
Varun Sundar Rabindranath committed 7 months ago
[release 2.11] Update to torch 2.11 (#34644)
Andrey Talman committed 7 months ago
[CI] Add reasoning parser tests to CI (#37025)
Flora Feng committed 7 months ago
docs: clarify SMT and OMP acronyms in CpuPlatform (#39085)
Md. Mekayel Anik committed 7 months ago
[Model Runner V2] Fuse probabilistic rejection sample kernels (#38496)
Giancarlo Delfin committed 7 months ago
[XPU] add xpu backend implementation of mxfp8 quant (#38682)
zofia committed 7 months ago
[Attention][V0 Deprecation] Deprecate accept output buffer (#39125)
Lucas Wilkinson committed 7 months ago
[Bugfix] Fix extract_hidden_states crash with quantized KV cache dtype (#39160)
Yubo Wang committed 7 months ago
[Parser] Pass request.tools to tool parser (#38860)
Flora Feng committed 7 months ago
[CI][Bugfix][AMD][ Ensure weights created when using emulating OCP MXFP4 (#36993)
rasmith committed 7 months ago
[XPU] Quick fix for TritonMLA to remove cuda hardcode (#39088)
Chendi.Xue committed 7 months ago
[Bugfix] Fix marlin nvfp4 rescaling (#37502)
Jinzhen Lin committed 7 months ago
[KVConnector] Support 3FS KVConnector (#37636)
ibifrost committed 7 months ago
kkyyxhll committed 7 months ago
[KVConnector]: prioritize external connector over internal registry (#38301)
maobaolong committed 7 months ago
fix(test): recompute Jina ColBERT rotary inv_freq cleared by transformers v5 weight loader (#39176)
Ilya Boytsov committed 7 months ago