A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 09:15 UTC
Add ninja to dependency (#21)
Woosuk Kwon committed
Apr 1, 2023 at 19:00 UTC
2c5cd0defe110cb1a5c699852f4b38284a2b86b4
1 file changed
+2
-1
README.md
+2
-1
@@ -6,7 +6,8 @@
6
pip install psutil numpy ray torch
7
pip install git+https://github.com/huggingface/transformers # Required for LLaMA.
8
pip install sentencepiece # Required for LlamaTokenizer.
9
-pip install flash-attn # This may take up to 20 mins.
9
+pip install ninja # To parallelize the compilation of flash-attn.
10
+pip install flash-attn # This may take up to 10 mins.
11
pip install -e .
12
```
13