A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 05:30 UTC
Update installation instruction for vLLM + CUDA 11.8 (#2086)
Woosuk Kwon committed
Dec 13, 2023 at 09:25 UTC
6565d9e33eb6f0a49d9824bff7ccc3f4fa5e4110
1 file changed
+4
-3
docs/source/getting_started/installation.rst
+4
-3
@@ -20,7 +20,7 @@ You can install vLLM using pip:
20
.. code-block:: console
21
22
$ # (Optional) Create a new conda environment.
23
- $ conda create -n myenv python=3.8 -y
23
+ $ conda create -n myenv python=3.9 -y
24
$ conda activate myenv
25
26
$ # Install vLLM with CUDA 12.1.
@@ -34,8 +34,9 @@ You can install vLLM using pip:
34
.. code-block:: console
35
36
$ # Install vLLM with CUDA 11.8.
37
- $ # Replace `cp310` with your Python version (e.g., `cp38`, `cp39`, `cp311`).
38
- $ pip install https://github.com/vllm-project/vllm/releases/download/v0.2.2/vllm-0.2.2+cu118-cp310-cp310-manylinux1_x86_64.whl
37
+ $ export VLLM_VERSION=0.2.4
38
+ $ export PYTHON_VERSION=39
39
+ $ pip install https://github.com/vllm-project/vllm/releases/download/v${VLLM_VERSION}/vllm-${VLLM_VERSION}+cu118-cp${PYTHON_VERSION}-cp${PYTHON_VERSION}-manylinux1_x86_64.whl
40
41
$ # Re-install PyTorch with CUDA 11.8.
42
$ pip uninstall torch -y