A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 15:45 UTC
[misc] fix wheel name (#8919)
youkaichao committed
Sep 27, 2024 at 22:10 UTC
d86f6b2afb006ea4b4b14a49a58f64bf3b952de6
2 files changed
+15
-10
.buildkite/release-pipeline.yaml
+3
-2
@@ -8,8 +8,9 @@ steps:
8
- "docker run --rm -v $(pwd)/artifacts:/artifacts_host vllm-ci:build-image bash -c 'cp -r dist /artifacts_host && chmod -R a+rw /artifacts_host'"
9
# rename the files to change linux -> manylinux1
10
- "for f in artifacts/dist/*.whl; do mv -- \"$$f\" \"$${f/linux/manylinux1}\"; done"
11
- - "aws s3 cp --recursive artifacts/dist s3://vllm-wheels/$BUILDKITE_COMMIT/"
12
- - "aws s3 cp --recursive artifacts/dist s3://vllm-wheels/nightly/"
11
+ - "mv artifacts/dist/$(ls artifacts/dist) artifacts/dist/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl"
12
+ - "aws s3 cp artifacts/dist/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl s3://vllm-wheels/$BUILDKITE_COMMIT/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl"
13
+ - "aws s3 cp artifacts/dist/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl s3://vllm-wheels/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl"
14
env:
15
DOCKER_BUILDKIT: "1"
16
docs/source/getting_started/installation.rst
+12
-8
@@ -48,15 +48,20 @@ You can install vLLM using pip:
48
49
.. note::
50
51
- vLLM also publishes a subset of wheels (Python 3.10, 3.11 with CUDA 12) for every commit since v0.5.3. You can download them with the following command:
51
+ vLLM also publishes wheels for Linux running on x86 platform with cuda 12 for every commit since v0.5.3. You can download and install them with the following command:
52
53
.. code-block:: console
54
55
- $ export VLLM_VERSION=0.6.1.post1 # vLLM's main branch version is currently set to latest released tag
56
- $ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-${VLLM_VERSION}-cp38-abi3-manylinux1_x86_64.whl
57
- $ # You can also access a specific commit
58
- $ # export VLLM_COMMIT=...
59
- $ # pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/${VLLM_COMMIT}/vllm-${VLLM_VERSION}-cp38-abi3-manylinux1_x86_64.whl
55
+ $ export VLLM_COMMIT=33f460b17a54acb3b6cc0b03f4a17876cff5eafd # use full commit hash from the main branch
56
+ $ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/${VLLM_COMMIT}/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
57
+
58
+ You can also just download the latest wheel by running:
59
+
60
+ .. code-block:: console
61
+
62
+ $ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
63
+
64
+ Note that the wheels are built with Python 3.8 abi (see `PEP 425 <https://peps.python.org/pep-0425/>`_ for more details about abi), so they are compatible with Python 3.8 and later. The version string in the wheel file name (``1.0.0.dev``) is just a placeholder to have a unified URL for the wheels. The actual version of wheels is contained in the wheel metadata.
65
66
Build from source (without compilation)
67
---------------------------------------
@@ -67,8 +72,7 @@ The first step is to follow the previous instructions to install the latest vLLM
72
73
.. code-block:: console
74
70
- $ export VLLM_VERSION=0.6.1.post1
71
- $ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-${VLLM_VERSION}-cp38-abi3-manylinux1_x86_64.whl
75
+ $ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
76
77
After verifying that the installation is successful, we have a script for you to copy and link directories, so that you can edit the Python code directly:
78