A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 10:45 UTC
[OpenVINO] Use torch 2.4.0 and newer optimim version (#9121)
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Ilya Lavrenov committed
Oct 10, 2024 at 21:18 UTC
83ea5c72b9a287b65c9f7b95fbd868b3f613e6f5
1 file changed
+5
-5
requirements-openvino.txt
+5
-5
@@ -1,8 +1,8 @@
1
# Common dependencies
2
-r requirements-common.txt
3
4
-# OpenVINO dependencies
5
-torch >= 2.1.2
6
-openvino ~= 2024.4.0
7
-openvino-tokenizers[transformers] ~= 2024.4.0
8
-optimum-intel[openvino] >= 1.19.0
4
+torch == 2.4.0 # should be aligned with "common" vLLM torch version
5
+openvino >= 2024.4.0 # since 2024.4.0 both CPU and GPU support Paged Attention
6
+
7
+optimum @ git+https://github.com/huggingface/optimum.git@main # latest optimum is used to support latest transformers version
8
+optimum-intel[nncf] @ git+https://github.com/huggingface/optimum-intel.git@main # latest optimum-intel is used to support latest transformers version