A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 20:45 UTC
main
[Bugfix] Fix `vocab_size` field access in `llava_next.py` (#6624)
Jae-Won Chung committed over 2 years ago
[Misc] Add a wrapper for torch.inference_mode (#6618)
Woosuk Kwon committed over 2 years ago
[Model] Initial Support for Chameleon (#5770)
Roger Wang committed over 2 years ago
[Kernel][Core] Add AWQ support to the Marlin kernel (#6612)
Alexander Matveev committed over 2 years ago
[Model] Refactor and decouple phi3v image embedding (#6621)
Isotr0py committed over 2 years ago
[Misc] Remove abused noqa (#6619)
Woosuk Kwon committed over 2 years ago
sroy745 committed over 2 years ago
[Frontend] Move chat utils (#6602)
Cyrus Leung committed over 2 years ago
[ Bugfix ] Fix AutoFP8 fp8 marlin (#6609)
Robert Shaw committed over 2 years ago
[Misc] Fix input_scale typing in w8a8_utils.py (#6579)
Michael Goin committed over 2 years ago
[ Kernel ] Enable `fp8-marlin` for `fbgemm-fp8` models (#6606)
Robert Shaw committed over 2 years ago
Matt Wong committed over 2 years ago
[ Misc ] `fbgemm` checkpoints (#6559)
Robert Shaw committed over 2 years ago
[Misc] Consolidate and optimize logic for building padded tensors (#6541)
Cyrus Leung committed over 2 years ago
Pipeline Parallel: Guard for KeyErrors at request abort (#6587)
Travis Johnson committed over 2 years ago
[Core] Allow specifying custom Executor (#6557)
Antoni Baum committed over 2 years ago
[ Kernel ] FP8 Dynamic Per Token Quant - Add scale_ub (#6593)
Varun Sundar Rabindranath committed over 2 years ago
[build] add ib in image for out-of-the-box infiniband support (#6599)
youkaichao committed over 2 years ago
Murali Andoorveedu committed over 2 years ago
[ Kernel ] Enable Dynamic Per Token `fp8` (#6547)
Robert Shaw committed over 2 years ago