A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 09:15 UTC
Add README
Woosuk Kwon committed
Feb 24, 2023 at 12:04 UTC
c84c708a1d13e5815376c2640cafeefaaf801827
1 file changed
+13
README.md
+13
@@ -1 +1,14 @@
1
# CacheFlow
2
+
3
+## Installation
4
+
5
+```bash
6
+pip install cmake torch transformers
7
+pip install -e .
8
+```
9
+
10
+## Run
11
+
12
+```bash
13
+python server.py
14
+```