A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 05:30 UTC
Update Ray version requirements (#2636)
Simon Mo committed
Jan 28, 2024 at 14:27 UTC
7d648418b8b1aadb90489ef18cff1763ffc82ed5
2 files changed
+2
-2
requirements-rocm.txt
+1
-1
@@ -2,7 +2,7 @@ ninja # For faster builds.
2
typing-extensions>=4.8.0
3
starlette
4
psutil
5
-ray >= 2.5.1
5
+ray >= 2.9
6
sentencepiece # Required for LLaMA tokenizer.
7
numpy
8
tokenizers>=0.15.0
requirements.txt
+1
-1
@@ -1,6 +1,6 @@
1
ninja # For faster builds.
2
psutil
3
-ray >= 2.5.1
3
+ray >= 2.9
4
sentencepiece # Required for LLaMA tokenizer.
5
numpy
6
torch == 2.1.2