A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 01:45 UTC
main
[Frontend] Add --log-level option to api server (#4377)
Norman Mu committed over 2 years ago
[Bugfix] Fix parameter name in `get_tokenizer` (#4107)
Cyrus Leung committed over 2 years ago
[ROCm][Hardware][AMD][Doc] Documentation update for ROCm (#4376)
Hongxia Yang committed over 2 years ago
[Core] Move function tracing setup to util function (#4352)
Nick Hill committed over 2 years ago
[Core] Add `shutdown()` method to `ExecutorBase` (#4349)
Nick Hill committed over 2 years ago
[Misc] Fix flash attention backend log (#4368)
Roy committed over 2 years ago
[Mypy] Typing lora folder (#4337)
SangBin Cho committed over 2 years ago
[Core]refactor aqlm quant ops (#4351)
Kunshang Ji committed over 2 years ago
[Doc] README Phi-3 name fix. (#4372)
Caio Mendes committed over 2 years ago
[CI/Build] Adding functionality to reset the node's GPUs before processing. (#4213)
Alexei-V-Ivanov-AMD committed over 2 years ago
[Bugfix][Model] Refactor OLMo model to support new HF format in transformers 4.40.0 (#4324)
Isotr0py committed over 2 years ago
[Core] Move ray_utils.py from `engine` to `executor` package (#4347)
Nick Hill committed over 2 years ago
[Model] Adds Phi-3 support (#4298)
Caio Mendes committed over 2 years ago
[Misc] Use public API in benchmark_throughput (#4300)
zifeitong committed over 2 years ago
[Doc] Add note for docker user (#4340)
youkaichao committed over 2 years ago
[Bugfix] Fix marlin kernel crash on H100 (#4218)
alexm-nm committed over 2 years ago
[Misc] Update ShareGPT Dataset Sampling in Serving Benchmark (#4279)
Roger Wang committed over 2 years ago
[Core][Distributed] use existing torch.cuda.device (#4318)
youkaichao committed over 2 years ago
[Misc] Reduce supported Punica dtypes (#4304)
Woosuk Kwon committed over 2 years ago
[CI][Build] change pynvml to nvidia-ml-py (#4302)
youkaichao committed over 2 years ago