@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 31d2ab4aff
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

Remove python 3.10 requirement (#2040)

Woosuk Kwon committed Dec 11, 2023 at 12:26 UTC 31d2ab4aff51c537dd4bc82451efbc194e0b8f2b
2 files changed +2 -4
README.md
+1 -1
@@ -72,7 +72,7 @@ Install vLLM with pip or [from source](https://vllm.readthedocs.io/en/latest/get
72 ```bash
73 pip install vllm
74 ```
75 -**NOTE:** The Mixtral model additionally requires `megablocks` which can be installed with pip or [from source](https://github.com/stanford-futuredata/megablocks) on **Python 3.10**:
75 +**NOTE:** The Mixtral model additionally requires `megablocks` which can be installed with pip or [from source](https://github.com/stanford-futuredata/megablocks):
76 ```bash
77 pip install megablocks
78 ```
vllm/model_executor/models/mixtral.py
+1 -3
@@ -35,9 +35,7 @@ try:
35 import megablocks.ops as ops
36 except ImportError:
37 print(
38 - "MegaBlocks not found. Please install it by `pip install megablocks`. "
39 - "Note that MegaBlocks depends on mosaicml-turbo, which only supports "
40 - "Python 3.10 for now.")
38 + "MegaBlocks not found. Please install it by `pip install megablocks`.")
39 try:
40 import stk
41 except ImportError: