A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 19:30 UTC
main
[Model]Refactor MiniCPMV (#7020)
Jee Jee Li committed over 2 years ago
[core][misc] simply output processing with shortcut code path (#7117)
youkaichao committed over 2 years ago
[misc] add zmq in collect env (#7119)
youkaichao committed over 2 years ago
Support for guided decoding for offline LLM (#6878)
Yihuan Bu committed over 2 years ago
[Frontend] Warn if user `max_model_len` is greater than derived `max_model_len` (#7080)
Jeff Fialho committed over 2 years ago
[ci][test] finalize fork_new_process_for_each_test (#7114)
youkaichao committed over 2 years ago
[CI] Temporarily turn off H100 performance benchmark (#7104)
Kuntai Du committed over 2 years ago
[LoRA] ReplicatedLinear support LoRA (#7081)
Jee Jee Li committed over 2 years ago
[Bugfix] Fix block table for seqs that have prefix cache hits (#7018)
Zach Zheng committed over 2 years ago
[Model] Refactor and decouple weight loading logic for InternVL2 model (#7067)
Isotr0py committed over 2 years ago
[ci][distributed] disable ray dag tests (#7099)
youkaichao committed over 2 years ago
[ci][distributed] merge distributed test commands (#7097)
youkaichao committed over 2 years ago
[Frontend] Factor out chat message parsing (#7055)
Cyrus Leung committed over 2 years ago
[ci][distributed] shorten wait time if server hangs (#7098)
youkaichao committed over 2 years ago
[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883)
Robert Shaw committed over 2 years ago
[misc] add a flag to enable compile (#7092)
youkaichao committed over 2 years ago
[Misc] Revive to use loopback address for driver IP (#7091)
Rui Qiao committed over 2 years ago
[Core] Pipeline parallel with Ray ADAG (#6837)
Rui Qiao committed over 2 years ago
[Misc] Disambiguate quantized types via a new ScalarType (#6396)
Lucas Wilkinson committed over 2 years ago
[CI/Build] Add support for Python 3.12 (#7035)
Michael Goin committed over 2 years ago