@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 83ea5c72b9
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[OpenVINO] Use torch 2.4.0 and newer optimim version (#9121)

Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

Ilya Lavrenov committed Oct 10, 2024 at 21:18 UTC 83ea5c72b9a287b65c9f7b95fbd868b3f613e6f5
1 file changed +5 -5
requirements-openvino.txt
+5 -5
@@ -1,8 +1,8 @@
1 # Common dependencies
2 -r requirements-common.txt
3
4 -# OpenVINO dependencies
5 -torch >= 2.1.2
6 -openvino ~= 2024.4.0
7 -openvino-tokenizers[transformers] ~= 2024.4.0
8 -optimum-intel[openvino] >= 1.19.0
4 +torch == 2.4.0 # should be aligned with "common" vLLM torch version
5 +openvino >= 2024.4.0 # since 2024.4.0 both CPU and GPU support Paged Attention
6 +
7 +optimum @ git+https://github.com/huggingface/optimum.git@main # latest optimum is used to support latest transformers version
8 +optimum-intel[nncf] @ git+https://github.com/huggingface/optimum-intel.git@main # latest optimum-intel is used to support latest transformers version