@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 2c5cd0defe
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 09:15 UTC

Add ninja to dependency (#21)

Woosuk Kwon committed Apr 1, 2023 at 19:00 UTC 2c5cd0defe110cb1a5c699852f4b38284a2b86b4
1 file changed +2 -1
README.md
+2 -1
@@ -6,7 +6,8 @@
6 pip install psutil numpy ray torch
7 pip install git+https://github.com/huggingface/transformers # Required for LLaMA.
8 pip install sentencepiece # Required for LlamaTokenizer.
9 -pip install flash-attn # This may take up to 20 mins.
9 +pip install ninja # To parallelize the compilation of flash-attn.
10 +pip install flash-attn # This may take up to 10 mins.
11 pip install -e .
12 ```
13