A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
main
Fix logging so we actually get info level entries in the log. (#1494)
Adam Brusselback committed about 3 years ago
Refactor LLMEngine demo script for clarity and modularity (#1413)
iongpt committed about 3 years ago
Delay GPU->CPU sync in sampling (#1337)
Antoni Baum committed about 3 years ago
Woosuk Kwon committed about 3 years ago
Support repetition_penalty (#1424)
ljss committed about 3 years ago
fix: don't skip first special token. (#1497)
Ricardo Lu committed about 3 years ago
Add rope_scaling to Aquila model (#1457)
Qing committed about 3 years ago
chooper1 committed about 3 years ago
Pin pydantic dependency versions (#1429)
Thiago Salvatore committed about 3 years ago
remove useless statements (#1408)
Wang Ran (汪然) committed about 3 years ago
Light Lin committed about 3 years ago
[BugFix] Define `__eq__` in SequenceGroupOutputs (#1389)
Woosuk Kwon committed about 3 years ago
Wang Ran (汪然) committed about 3 years ago
Add Mistral 7B to `test_models` (#1366)
Woosuk Kwon committed about 3 years ago
Change scheduler & input tensor shape (#1381)
Woosuk Kwon committed about 3 years ago
Bump up the version to v0.2.1 (#1355)
Zhuohan Li committed about 3 years ago
Woosuk Kwon committed about 3 years ago
Fix PyTorch index URL in workflow (#1378)
Woosuk Kwon committed about 3 years ago
Fix PyTorch version to 2.0.1 in workflow (#1377)
Woosuk Kwon committed about 3 years ago
Implement prompt logprobs & Batched topk for computing logprobs (#1328)
Zhuohan Li committed about 3 years ago