A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
main
Implement PagedAttention V2 (#1348)
Woosuk Kwon committed about 3 years ago
Minor fix on AWQ kernel launch (#1356)
Woosuk Kwon committed about 3 years ago
Fix error message on `TORCH_CUDA_ARCH_LIST` (#1239)
Woosuk Kwon committed about 3 years ago
Fix the issue for AquilaChat2-* models (#1339)
Lu Wang committed about 3 years ago
Bump up transformers version & Remove MistralConfig (#1254)
Woosuk Kwon committed about 3 years ago
Improve detokenization performance (#1338)
Antoni Baum committed about 3 years ago
ldwang committed about 3 years ago
Add blacklist in model checkpoint (#1325)
Woosuk Kwon committed about 3 years ago
Add Mistral to quantization model list (#1278)
amaleshvemula committed about 3 years ago
Fix overflow in awq kernel (#1295)
CHU Tianxiang committed about 3 years ago
workaround of AWQ for Turing GPUs (#1252)
twaka committed about 3 years ago
change the timing of sorting logits (#1309)
yhlskt23 committed about 3 years ago
Fix `__repr__` of `SequenceOutputs` (#1311)
Wang Ran (汪然) committed about 3 years ago
lock torch version to 2.0.1 (#1290)
yanxiyue committed about 3 years ago
[Minor] Fix comment in mistral.py (#1303)
Zhuohan Li committed about 3 years ago
Zhuohan Li committed about 3 years ago
[FIX] Explain why the finished_reason of ignored sequences are length (#1289)
Zhuohan Li committed about 3 years ago
Move bfloat16 check to worker (#1259)
Antoni Baum committed about 3 years ago
API server support ipv4 / ipv6 dualstack (#1288)
Yunfeng Bai committed about 3 years ago
Use monotonic time where appropriate (#1249)
Antoni Baum committed about 3 years ago