A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 03:00 UTC
main
[Speculative decoding 4/9] Lookahead scheduling for speculative decoding (#3250)
Cade Daniel committed over 2 years ago
[Misc] Fix Benchmark TTFT Calculation for Chat Completions (#3768)
Roger Wang committed over 2 years ago
[Misc] Some minor simplifications to detokenization logic (#3670)
Nick Hill committed over 2 years ago
[Misc] Minor fixes in requirements.txt (#3769)
Woosuk Kwon committed over 2 years ago
[CI/Build] Make Marlin Tests Green (#3753)
Robert Shaw committed over 2 years ago
[Doc] Update installation doc (#3746)
youkaichao committed over 2 years ago
[Kernel] Layernorm performance optimization (#3662)
mawong-amd committed over 2 years ago
CMake build elf without PTX (#3739)
Simon Mo committed over 2 years ago
Fix build when nvtools is missing (#3698)
bnellnm committed over 2 years ago
[Core][Bugfix] cache len of tokenizer (#3741)
youkaichao committed over 2 years ago
[BugFix] Use consistent logger everywhere (#3738)
Nick Hill committed over 2 years ago
usage lib get version another way (#3735)
Simon Mo committed over 2 years ago
[ROCm][Bugfix] Fixed several bugs related to rccl path and attention selector logic (#3699)
Hongxia Yang committed over 2 years ago
bump version to v0.4.0 (#3712)
Simon Mo committed over 2 years ago
[Bugfix] Command-R Max Model Length (#3727)
Roger Wang committed over 2 years ago
[BugFix][Frontend] Fix completion logprobs=0 error (#3731)
Roy committed over 2 years ago
[BugFix] Fix tokenizer out of vocab size (#3685)
Roy committed over 2 years ago
Usage Stats Collection (#2852)
yhu422 committed over 2 years ago
add ccache to docker build image (#3704)
Simon Mo committed over 2 years ago
[Core][Test] move local_rank to the last arg with default value(#3711)
youkaichao committed over 2 years ago