@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / d9fa1c05ad
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 03:15 UTC

[doc] update how pip can install nightly wheels (#11806)

Signed-off-by: youkaichao <youkaichao@gmail.com>

youkaichao committed Jan 7, 2025 at 21:42 UTC d9fa1c05ad7149a43051a283d0cbeeb65bf6b4a3
1 file changed +6 -6
docs/source/getting_started/installation/gpu-cuda.md
+6 -6
@@ -66,19 +66,19 @@ LLM inference is a fast-evolving field, and the latest code may contain bug fixe
66 ### Install the latest code using `pip`
67
68 ```console
69 -$ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
69 +$ pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
70 ```
71
72 -If you want to access the wheels for previous commits (e.g. to bisect the behavior change, performance regression), you can specify the commit hash in the URL:
72 +`--pre` is required for `pip` to consider pre-released versions.
73 +
74 +If you want to access the wheels for previous commits (e.g. to bisect the behavior change, performance regression), due to the limitation of `pip`, you have to specify the full URL of the wheel file by embedding the commit hash in the URL:
75
76 ```console
77 $ export VLLM_COMMIT=33f460b17a54acb3b6cc0b03f4a17876cff5eafd # use full commit hash from the main branch
78 $ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/${VLLM_COMMIT}/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
79 ```
80
79 -Note that the wheels are built with Python 3.8 ABI (see [PEP 425](https://peps.python.org/pep-0425/) for more details about ABI), so **they are compatible with Python 3.8 and later**. The version string in the wheel file name (`1.0.0.dev`) is just a placeholder to have a unified URL for the wheels. The actual versions of wheels are contained in the wheel metadata. Although we don't support Python 3.8 any more (because PyTorch 2.5 dropped support for Python 3.8), the wheels are still built with Python 3.8 ABI to keep the same wheel name as before.
80 -
81 -Due to the limitation of `pip`, you have to specify the full URL of the wheel file.
81 +Note that the wheels are built with Python 3.8 ABI (see [PEP 425](https://peps.python.org/pep-0425/) for more details about ABI), so **they are compatible with Python 3.8 and later**. The version string in the wheel file name (`1.0.0.dev`) is just a placeholder to have a unified URL for the wheels, the actual versions of wheels are contained in the wheel metadata (the wheels listed in the extra index url have correct versions). Although we don't support Python 3.8 any more (because PyTorch 2.5 dropped support for Python 3.8), the wheels are still built with Python 3.8 ABI to keep the same wheel name as before.
82
83 ### Install the latest code using `uv`
84
@@ -126,7 +126,7 @@ $ cd vllm
126 $ VLLM_USE_PRECOMPILED=1 pip install --editable .
127 ```
128
129 -This will download the latest nightly wheel and use the compiled libraries from there in the install.
129 +This will download the latest nightly wheel from https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl and use the compiled libraries from there in the installation.
130
131 The `VLLM_PRECOMPILED_WHEEL_LOCATION` environment variable can be used instead of `VLLM_USE_PRECOMPILED` to specify a custom path or URL to the wheel file. For example, to use the [0.6.1.post1 PyPi wheel](https://pypi.org/project/vllm/#files):
132