A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 22:00 UTC
main
Benchmark: add H100 suite (#6047)
Simon Mo committed over 2 years ago
[ Misc ] Refactor Marlin Python Utilities (#6082)
Robert Shaw committed over 2 years ago
[BugFix] get_and_reset only when scheduler outputs are not empty (#6266)
Mor Zusman committed over 2 years ago
[Bugfix] GPTBigCodeForCausalLM: Remove lm_head from supported_lora_modules. (#6326)
Thomas Parnell committed over 2 years ago
[BugFix]: fix engine timeout due to request abort (#6255)
pushan committed over 2 years ago
[Misc] refactor(config): clean up unused code (#6320)
aniaan committed over 2 years ago
[Bugfix] Fix snapshot download in serving benchmark (#6318)
Roger Wang committed over 2 years ago
[BugFix]: set outlines pkg version (#6262)
Lim Xiang Yang committed over 2 years ago
[Doc] Update description of vLLM support for CPUs (#6003)
Jie Fu (傅杰) committed over 2 years ago
[Doc] Remove comments incorrectly copied from another project (#6286)
daquexian committed over 2 years ago
[Bugfix][Neuron] Fix soft prompt method error in NeuronExecutor (#6313)
Woosuk Kwon committed over 2 years ago
sroy745 committed over 2 years ago
[Bugfix] Fix OpenVINOExecutor abstractmethod error (#6296)
sangjune.park committed over 2 years ago
[CI/Build] Enable mypy typing for remaining folders (#6268)
Benjamin Muskalla committed over 2 years ago
[Bugfix] MLPSpeculator: Use ParallelLMHead in tie_weights=False case. (#6303)
Thomas Parnell committed over 2 years ago
[Bugfix] Support 2D input shape in MoE layer (#6287)
Woosuk Kwon committed over 2 years ago
[Doc] Guide for adding multi-modal plugins (#6205)
Cyrus Leung committed over 2 years ago
[Bugfix][TPU] Add prompt adapter methods to TPUExecutor (#6279)
Woosuk Kwon committed over 2 years ago
[core][distributed] zmq fallback for broadcasting large objects (#6183)
youkaichao committed over 2 years ago
[Speculative Decoding] Medusa Implementation with Top-1 proposer (#4978)
Abhinav Goyal committed over 2 years ago