@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 2eaa81b236
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

Update README.md to add megablocks requirement for mixtral (#2033)

Ram committed Dec 12, 2023 at 01:07 UTC 2eaa81b23671c4f1936697c97d18c1636a8c495e
1 file changed +4
README.md
+4
@@ -72,6 +72,10 @@ Install vLLM with pip or [from source](https://vllm.readthedocs.io/en/latest/get
72 ```bash
73 pip install vllm
74 ```
75 +**NOTE:** The Mixtral model additionally requires `megablocks` which can be installed with pip or [from source](https://github.com/stanford-futuredata/megablocks) on **Python 3.10**:
76 +```bash
77 +pip install megablocks
78 +```
79
80 ## Getting Started
81