A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
main
[ci] Use sccache to build images (#5419)
Kevin H. Luu committed over 2 years ago
[Core][Distributed] code deduplication in tp&pp with coordinator(#5293)
youkaichao committed over 2 years ago
[Bugfix] Fix wrong multi_modal_input format for CPU runner (#5451)
Isotr0py committed over 2 years ago
[Frontend] Add "input speed" to tqdm postfix alongside output speed (#5425)
Michael Goin committed over 2 years ago
[Bugfix] Fix typo in scheduler.py (requeset -> request) (#5470)
Michael Goin committed over 2 years ago
[Doc] Update debug docs (#5438)
Cyrus Leung committed over 2 years ago
[misc] add hint for AttributeError (#5462)
youkaichao committed over 2 years ago
[Frontend] [Core] Support for sharded tensorized models (#4990)
Travis Johnson committed over 2 years ago
[Bugfix] TYPE_CHECKING for MultiModalData (#5444)
Arthur Kim committed over 2 years ago
[Kernel] Vectorized FP8 quantize kernel (#5396)
Cody Yu committed over 2 years ago
[ci] Add AMD, Neuron, Intel tests for AWS CI and turn off default soft fail for GPU tests (#5464)
Kevin H. Luu committed over 2 years ago
[Bugfix] Add device assertion to TorchSDPA (#5402)
Li, Jiang committed over 2 years ago
[Hardware] Initial TPU integration (#5292)
Woosuk Kwon committed over 2 years ago
[CI] Upgrade codespell version. (#5381)
SangBin Cho committed over 2 years ago
Simon Mo committed over 2 years ago
[CI/Build] Add `is_quant_method_supported` to control quantization test configurations (#5253)
Michael Goin committed over 2 years ago
[Doc] add common case for long waiting time (#5430)
youkaichao committed over 2 years ago
[Core][Doc] Default to multiprocessing for single-node distributed case (#5230)
Nick Hill committed over 2 years ago
[Core][Distributed] add same-node detection (#5369)
youkaichao committed over 2 years ago
[Frontend] Customizable RoPE theta (#5197)
sasha0552 committed over 2 years ago