A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 09:15 UTC
main
Woosuk Kwon committed over 3 years ago
Reduce the number of states in scheduler
Woosuk Kwon committed over 3 years ago
Refactor and annotate types for attention
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Implement scheduler.step & Add a threshold for batch size
Woosuk Kwon committed over 3 years ago
decoding.py -> sampling_params.py
Woosuk Kwon committed over 3 years ago