A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 05:30 UTC
Update README.md to add megablocks requirement for mixtral (#2033)
Ram committed
Dec 12, 2023 at 01:07 UTC
2eaa81b23671c4f1936697c97d18c1636a8c495e
1 file changed
+4
README.md
+4
@@ -72,6 +72,10 @@ Install vLLM with pip or [from source](https://vllm.readthedocs.io/en/latest/get
72
```bash
73
pip install vllm
74
```
75
+**NOTE:** The Mixtral model additionally requires `megablocks` which can be installed with pip or [from source](https://github.com/stanford-futuredata/megablocks) on **Python 3.10**:
76
+```bash
77
+pip install megablocks
78
+```
79
80
## Getting Started
81