A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 01:45 UTC
[CI][Build] change pynvml to nvidia-ml-py (#4302)
youkaichao committed
Apr 23, 2024 at 18:33 UTC
e4bf860a54a302ccb2d80489368d5df686e46923
1 file changed
+1
-1
requirements-cuda.txt
+1
-1
@@ -3,7 +3,7 @@
3
4
# Dependencies for NVIDIA GPUs
5
ray >= 2.9
6
-pynvml == 11.5.0
6
+nvidia-ml-py # for pynvml package
7
vllm-nccl-cu12>=2.18,<2.19 # for downloading nccl library
8
torch == 2.2.1
9
xformers == 0.0.25 # Requires PyTorch 2.2.1