A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 09:15 UTC
main
Implement presence and frequency penalties (#95)
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Avoid sorting waiting queue & Minor code cleaning (#93)
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Rename variables and methods (#91)
Woosuk Kwon committed over 3 years ago
Update sample prompts in `simple_server.py` (#89)
Woosuk Kwon committed over 3 years ago
Use slow tokenizer for LLaMA (#84)
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Refactor system architecture (#82)
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Specify python package dependencies in requirements.txt (#78)
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Replace FlashAttention with xformers (#70)
Woosuk Kwon committed over 3 years ago
Use dtype from model config & Add Dolly V2 (#63)
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Fix a bug in attention kernel (#68)
Woosuk Kwon committed over 3 years ago
Support bfloat16 data type (#54)
Woosuk Kwon committed over 3 years ago
Refactor attention kernels (#53)
Woosuk Kwon committed over 3 years ago
New weight loader without np copy (#52)
Zhuohan Li committed over 3 years ago
Add an option to launch cacheflow without ray (#51)
Zhuohan Li committed over 3 years ago