A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 09:15 UTC
main
DecodingParams -> SamplingParams
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed almost 4 years ago
Woosuk Kwon committed almost 4 years ago
Woosuk Kwon committed almost 4 years ago
Woosuk Kwon committed almost 4 years ago
Woosuk Kwon committed almost 4 years ago
Woosuk Kwon committed almost 4 years ago
Woosuk Kwon committed almost 4 years ago
Woosuk Kwon committed almost 4 years ago
Fix a ref count bug in swapping
Woosuk Kwon committed almost 4 years ago