A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 21:00 UTC
Moved numba from common requirements to cuda/rocm specific requirements (#14199)
Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>
Nishidha committed
Mar 5, 2025 at 09:55 UTC
5b143d33be7c6a273dc8f8186e72b2cd245a28e2
3 files changed
+5
-2
requirements-common.txt
-1
@@ -1,7 +1,6 @@
1
psutil
2
sentencepiece # Required for LLaMA tokenizer.
3
numpy < 2.0.0
4
-numba == 0.60.0 # v0.61 doesn't support Python 3.9. Required for N-gram speculative decoding.
4
requests >= 2.26.0
5
tqdm
6
blake3
requirements-cuda.txt
+2
@@ -1,6 +1,8 @@
1
# Common dependencies
2
-r requirements-common.txt
3
4
+numba == 0.60.0 # v0.61 doesn't support Python 3.9. Required for N-gram speculative decoding
5
+
6
# Dependencies for NVIDIA GPUs
7
ray[cgraph] >= 2.43.0 # Ray Compiled Graph, required for pipeline parallelism in V1.
8
torch == 2.5.1
requirements-rocm.txt
+3
-1
@@ -1,6 +1,8 @@
1
# Common dependencies
2
-r requirements-common.txt
3
4
+numba == 0.60.0 # v0.61 doesn't support Python 3.9. Required for N-gram speculative decoding
5
+
6
# Dependencies for AMD GPUs
7
awscli
8
boto3
@@ -11,4 +13,4 @@ peft
13
pytest-asyncio
14
tensorizer>=2.9.0
15
runai-model-streamer==0.11.0
14
-runai-model-streamer-s3==0.11.0
\ No newline at end of file
16
+runai-model-streamer-s3==0.11.0