A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 05:30 UTC
Remove python 3.10 requirement (#2040)
Woosuk Kwon committed
Dec 11, 2023 at 12:26 UTC
31d2ab4aff51c537dd4bc82451efbc194e0b8f2b
2 files changed
+2
-4
README.md
+1
-1
@@ -72,7 +72,7 @@ Install vLLM with pip or [from source](https://vllm.readthedocs.io/en/latest/get
72
```bash
73
pip install vllm
74
```
75
-**NOTE:** The Mixtral model additionally requires `megablocks` which can be installed with pip or [from source](https://github.com/stanford-futuredata/megablocks) on **Python 3.10**:
75
+**NOTE:** The Mixtral model additionally requires `megablocks` which can be installed with pip or [from source](https://github.com/stanford-futuredata/megablocks):
76
```bash
77
pip install megablocks
78
```
vllm/model_executor/models/mixtral.py
+1
-3
@@ -35,9 +35,7 @@ try:
35
import megablocks.ops as ops
36
except ImportError:
37
print(
38
- "MegaBlocks not found. Please install it by `pip install megablocks`. "
39
- "Note that MegaBlocks depends on mosaicml-turbo, which only supports "
40
- "Python 3.10 for now.")
38
+ "MegaBlocks not found. Please install it by `pip install megablocks`.")
39
try:
40
import stk
41
except ImportError: