A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 20:45 UTC
main
[doc][distributed] fix doc argument order (#6691)
youkaichao committed over 2 years ago
[Docs] Announce llama3.1 support (#6688)
Woosuk Kwon committed over 2 years ago
Bump version to v0.5.3 (#6674)
Simon Mo committed over 2 years ago
support ignore patterns in model loader (#6673)
Simon Mo committed over 2 years ago
[VLM][Model] Support image input for Chameleon (#6633)
Roger Wang committed over 2 years ago
[misc] only tqdm for first rank (#6672)
youkaichao committed over 2 years ago
[Misc] Manage HTTP connections in one place (#6600)
Cyrus Leung committed over 2 years ago
[doc][distributed] doc for setting up multi-node environment (#6529)
youkaichao committed over 2 years ago
[Misc] Support FP8 kv cache scales from compressed-tensors (#6528)
Michael Goin committed over 2 years ago
add tqdm when loading checkpoint shards (#6569)
zhaotyer committed over 2 years ago
[misc] add start loading models for users information (#6670)
youkaichao committed over 2 years ago
[Misc] Enable chunked prefill by default for long context models (#6666)
Woosuk Kwon committed over 2 years ago
[Bugfix] Fix null `modules_to_not_convert` in FBGEMM Fp8 quantization (#6665)
Cheng Li committed over 2 years ago
[Core] Modulize prepare input and attention metadata builder (#6596)
Cody Yu committed over 2 years ago
[Misc] Remove deprecation warning for beam search (#6659)
Woosuk Kwon committed over 2 years ago
[ci][build] add back vim in docker (#6661)
youkaichao committed over 2 years ago
[Core] Support dynamically loading Lora adapter from HuggingFace (#6234)
Jiaxin Shan committed over 2 years ago
[ci] Use different sccache bucket for CUDA 11.8 wheel build (#6656)
Kevin H. Luu committed over 2 years ago
[Bugfix][Kernel] Use int64_t for indices in fp8 quant kernels (#6649)
Tyler Michael Smith committed over 2 years ago
[Frontend] Refactor prompt processing (#4028)
Cyrus Leung committed over 2 years ago