@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / c84c708a1d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 09:15 UTC

Add README

Woosuk Kwon committed Feb 24, 2023 at 12:04 UTC c84c708a1d13e5815376c2640cafeefaaf801827
1 file changed +13
README.md
+13
@@ -1 +1,14 @@
1 # CacheFlow
2 +
3 +## Installation
4 +
5 +```bash
6 +pip install cmake torch transformers
7 +pip install -e .
8 +```
9 +
10 +## Run
11 +
12 +```bash
13 +python server.py
14 +```