A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 09:15 UTC
main
Add activation registry (#126)
Woosuk Kwon committed over 3 years ago
OpenAI Compatible Frontend (#116)
Zhuohan Li committed over 3 years ago
Incrementally decode output tokens (#121)
Woosuk Kwon committed over 3 years ago
Print warnings/errors for large swap space (#123)
Woosuk Kwon committed over 3 years ago
Add contributing guideline and mypy config (#122)
Woosuk Kwon committed over 3 years ago
Fix latency benchmark script (#118)
Woosuk Kwon committed over 3 years ago
Add initial sphinx docs (#120)
Woosuk Kwon committed over 3 years ago
Introduce LLM class for offline inference (#115)
Woosuk Kwon committed over 3 years ago
Implement stop strings and best_of (#114)
Woosuk Kwon committed over 3 years ago
Refactor system architecture (#109)
Woosuk Kwon committed over 3 years ago
Remove unused parts in Megatron-LM code and add copyright notice (#110)
Zhuohan Li committed over 3 years ago
Fix timeout error in the FastAPI frontend (#34)
Zhuohan Li committed over 3 years ago
Use runtime profiling to replace manual memory analyzers (#81)
Zhuohan Li committed over 3 years ago
Use pytest format for unit tests (#107)
Woosuk Kwon committed over 3 years ago
Add docstrings to some modules and classes (#100)
Woosuk Kwon committed over 3 years ago
Add copyright headers to source files adapted from FT (#104)
Woosuk Kwon committed over 3 years ago
Add copyright headers for HF models (#103)
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago
Minor code cleaning for SamplingParams (#99)
Woosuk Kwon committed over 3 years ago
Woosuk Kwon committed over 3 years ago