@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / b0a1d667b0
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

Pin PyTorch & xformers versions (#2155)

Woosuk Kwon committed Dec 17, 2023 at 01:46 UTC b0a1d667b04dcce8e54201cb429a57155bf9670c
4 files changed +5 -5
.github/workflows/publish.yml
+1 -1
@@ -49,7 +49,7 @@ jobs:
49 matrix:
50 os: ['ubuntu-20.04']
51 python-version: ['3.8', '3.9', '3.10', '3.11']
52 - pytorch-version: ['2.1.1']
52 + pytorch-version: ['2.1.2'] # Must be the most recent version that meets requirements.txt.
53 cuda-version: ['11.8', '12.1']
54
55 steps:
pyproject.toml
+1 -1
@@ -4,7 +4,7 @@ requires = [
4 "ninja",
5 "packaging",
6 "setuptools >= 49.4.0",
7 - "torch >= 2.1.1",
7 + "torch == 2.1.2",
8 "wheel",
9 ]
10 build-backend = "setuptools.build_meta"
requirements-build.txt
+1 -1
@@ -2,5 +2,5 @@
2 ninja
3 packaging
4 setuptools>=49.4.0
5 -torch>=2.1.0
5 +torch==2.1.2
6 wheel
\ No newline at end of file
requirements.txt
+2 -2
@@ -5,9 +5,9 @@ pandas # Required for Ray data.
5 pyarrow # Required for Ray data.
6 sentencepiece # Required for LLaMA tokenizer.
7 numpy
8 -torch >= 2.1.1
8 +torch == 2.1.2
9 transformers >= 4.36.0 # Required for Mixtral.
10 -xformers >= 0.0.23 # Required for CUDA 12.1.
10 +xformers == 0.0.23 # Required for CUDA 12.1.
11 fastapi
12 uvicorn[standard]
13 pydantic == 1.10.13 # Required for OpenAI server.