A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 04:30 UTC
main
[Pixtral] Improve loading (#11040)
Patrick von Platen committed almost 2 years ago
[Frontend] Use request id from header (#10968)
Joe Runde committed almost 2 years ago
[Bugfix] Fix usage of `deprecated` decorator (#11025)
Cyrus Leung committed almost 2 years ago
[Model] Add has_weight to RMSNorm and re-enable weights loading tracker for Mamba (#10739)
Isotr0py committed almost 2 years ago
[Docs] Add dedicated tool calling page to docs (#10554)
Michael Goin committed almost 2 years ago
[V1] Do not store `None` in self.generators (#11038)
Woosuk Kwon committed almost 2 years ago
Build tpu image in release pipeline (#10936)
Richard Liu committed almost 2 years ago
[ROCm][bugfix] scpecilative decoding worker class (#11035)
Gregory Shtrasberg committed almost 2 years ago
[Neuron] Upgrade neuron to 2.20.2 (#11016)
xendo committed almost 2 years ago
[v1] fix use compile sizes (#11000)
youkaichao committed almost 2 years ago
[Bugfix][Hardware][Gaudi] Bump vllm_hpu_extension version (#11028)
Konrad Zawora committed almost 2 years ago
[Model] merged input processor for Phi-3-Vision models (#10977)
Isotr0py committed almost 2 years ago
[Misc][LoRA] Abstract PunicaWrapper (#10955)
Jee Jee Li committed almost 2 years ago
[V1] Further reduce CPU overheads in flash-attn (#10989)
Woosuk Kwon committed almost 2 years ago
[ci/build] Recompile CI dependencies list with Python 3.12 (#11013)
Kevin H. Luu committed almost 2 years ago
[V1] Input Batch Relocation (#10962)
Varun Sundar Rabindranath committed almost 2 years ago
[Platform] Move `async output` check to platform (#10768)
wangxiyuan committed almost 2 years ago
[Core] Require xgrammar >= 0.1.6 (#11021)
Russell Bryant committed almost 2 years ago
[V1] Fix Detokenizer loading in `AsyncLLM` (#10997)
Roger Wang committed almost 2 years ago
[torch.compile] add dynamo time tracking (#11005)
youkaichao committed almost 2 years ago