A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
main
Add minimum capability requirement for AWQ (#1064)
Woosuk Kwon committed about 3 years ago
Convert before transpose (#1073)
Woosuk Kwon committed about 3 years ago
Fix get_max_num_running_seqs for waiting and swapped seq groups (#1068)
Zhuohan Li committed about 3 years ago
align llm_engine and async_engine. (#1081)
Roy committed about 3 years ago
added support for quantize on LLM module (#1080)
orellavie1212 committed about 3 years ago
[FIX] Don't initialize parameter by default (#1067)
Zhuohan Li committed about 3 years ago
Fix hanging when prompt exceeds limit (#1029)
陈序 committed about 3 years ago
Remove AsyncLLMEngine busy loop, shield background task (#1059)
Antoni Baum committed about 3 years ago
Implement AWQ quantization support for LLaMA (#1032)
Woosuk Kwon committed about 3 years ago
Abort when coroutine is cancelled (#1020)
Jerry Yang committed about 3 years ago
Fix warning message on LLaMA FastTokenizer (#1037)
Woosuk Kwon committed about 3 years ago
Only fail if logit_bias has actual values (#1045)
Lukas Kreussel committed about 3 years ago
Add pandas to requirements.txt (#1047)
Woosuk Kwon committed about 3 years ago
Fix detokenization leaving special tokens (#1044)
Antoni Baum committed about 3 years ago
Announce paper release (#1036)
Woosuk Kwon committed about 3 years ago
Zhuohan Li committed about 3 years ago
Add Model Revision Support (#1014)
Jasmond L committed about 3 years ago
Use TGI-like incremental detokenization (#984)
Antoni Baum committed about 3 years ago
Ikko Eltociear Ashimine committed about 3 years ago
Make `max_model_len` configurable (#972)
Antoni Baum committed about 3 years ago