A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 22:00 UTC
[doc][misc] bump up py version in installation doc (#6119)
youkaichao committed
Jul 3, 2024 at 15:52 UTC
966fe72141e8365721840b7ababfb78601c23ead
1 file changed
+2
-2
docs/source/getting_started/installation.rst
+2
-2
@@ -20,7 +20,7 @@ You can install vLLM using pip:
20
.. code-block:: console
21
22
$ # (Recommended) Create a new conda environment.
23
- $ conda create -n myenv python=3.9 -y
23
+ $ conda create -n myenv python=3.10 -y
24
$ conda activate myenv
25
26
$ # Install vLLM with CUDA 12.1.
@@ -35,7 +35,7 @@ You can install vLLM using pip:
35
36
$ # Install vLLM with CUDA 11.8.
37
$ export VLLM_VERSION=0.4.0
38
- $ export PYTHON_VERSION=39
38
+ $ export PYTHON_VERSION=310
39
$ pip install https://github.com/vllm-project/vllm/releases/download/v${VLLM_VERSION}/vllm-${VLLM_VERSION}+cu118-cp${PYTHON_VERSION}-cp${PYTHON_VERSION}-manylinux1_x86_64.whl --extra-index-url https://download.pytorch.org/whl/cu118
40
41
In order to be performant, vLLM has to compile many cuda kernels. The compilation unfortunately introduces binary incompatibility with other CUDA versions and PyTorch versions, even for the same PyTorch version with different building configurations.