A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 05:30 UTC
[Docs] Update installation instructions to include CUDA 11.8 xFormers (#2246)
Shivam Thakkar committed
Dec 23, 2023 at 12:50 UTC
1db83e31a2468cae37f326a642c0a4c4edbb5e4f
1 file changed
+4
docs/source/getting_started/installation.rst
+4
@@ -42,6 +42,10 @@ You can install vLLM using pip:
42
$ pip uninstall torch -y
43
$ pip install torch --upgrade --index-url https://download.pytorch.org/whl/cu118
44
45
+ $ # Re-install xFormers with CUDA 11.8.
46
+ $ pip uninstall xformers -y
47
+ $ pip install --upgrade xformers --index-url https://download.pytorch.org/whl/cu118
48
+
49
50
.. _build_from_source:
51