@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 6565d9e33e
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

Update installation instruction for vLLM + CUDA 11.8 (#2086)

Woosuk Kwon committed Dec 13, 2023 at 09:25 UTC 6565d9e33eb6f0a49d9824bff7ccc3f4fa5e4110
1 file changed +4 -3
docs/source/getting_started/installation.rst
+4 -3
@@ -20,7 +20,7 @@ You can install vLLM using pip:
20 .. code-block:: console
21
22 $ # (Optional) Create a new conda environment.
23 - $ conda create -n myenv python=3.8 -y
23 + $ conda create -n myenv python=3.9 -y
24 $ conda activate myenv
25
26 $ # Install vLLM with CUDA 12.1.
@@ -34,8 +34,9 @@ You can install vLLM using pip:
34 .. code-block:: console
35
36 $ # Install vLLM with CUDA 11.8.
37 - $ # Replace `cp310` with your Python version (e.g., `cp38`, `cp39`, `cp311`).
38 - $ pip install https://github.com/vllm-project/vllm/releases/download/v0.2.2/vllm-0.2.2+cu118-cp310-cp310-manylinux1_x86_64.whl
37 + $ export VLLM_VERSION=0.2.4
38 + $ export PYTHON_VERSION=39
39 + $ pip install https://github.com/vllm-project/vllm/releases/download/v${VLLM_VERSION}/vllm-${VLLM_VERSION}+cu118-cp${PYTHON_VERSION}-cp${PYTHON_VERSION}-manylinux1_x86_64.whl
40
41 $ # Re-install PyTorch with CUDA 11.8.
42 $ pip uninstall torch -y