@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / d86f6b2afb
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 15:45 UTC

[misc] fix wheel name (#8919)

youkaichao committed Sep 27, 2024 at 22:10 UTC d86f6b2afb006ea4b4b14a49a58f64bf3b952de6
2 files changed +15 -10
.buildkite/release-pipeline.yaml
+3 -2
@@ -8,8 +8,9 @@ steps:
8 - "docker run --rm -v $(pwd)/artifacts:/artifacts_host vllm-ci:build-image bash -c 'cp -r dist /artifacts_host && chmod -R a+rw /artifacts_host'"
9 # rename the files to change linux -> manylinux1
10 - "for f in artifacts/dist/*.whl; do mv -- \"$$f\" \"$${f/linux/manylinux1}\"; done"
11 - - "aws s3 cp --recursive artifacts/dist s3://vllm-wheels/$BUILDKITE_COMMIT/"
12 - - "aws s3 cp --recursive artifacts/dist s3://vllm-wheels/nightly/"
11 + - "mv artifacts/dist/$(ls artifacts/dist) artifacts/dist/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl"
12 + - "aws s3 cp artifacts/dist/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl s3://vllm-wheels/$BUILDKITE_COMMIT/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl"
13 + - "aws s3 cp artifacts/dist/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl s3://vllm-wheels/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl"
14 env:
15 DOCKER_BUILDKIT: "1"
16
docs/source/getting_started/installation.rst
+12 -8
@@ -48,15 +48,20 @@ You can install vLLM using pip:
48
49 .. note::
50
51 - vLLM also publishes a subset of wheels (Python 3.10, 3.11 with CUDA 12) for every commit since v0.5.3. You can download them with the following command:
51 + vLLM also publishes wheels for Linux running on x86 platform with cuda 12 for every commit since v0.5.3. You can download and install them with the following command:
52
53 .. code-block:: console
54
55 - $ export VLLM_VERSION=0.6.1.post1 # vLLM's main branch version is currently set to latest released tag
56 - $ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-${VLLM_VERSION}-cp38-abi3-manylinux1_x86_64.whl
57 - $ # You can also access a specific commit
58 - $ # export VLLM_COMMIT=...
59 - $ # pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/${VLLM_COMMIT}/vllm-${VLLM_VERSION}-cp38-abi3-manylinux1_x86_64.whl
55 + $ export VLLM_COMMIT=33f460b17a54acb3b6cc0b03f4a17876cff5eafd # use full commit hash from the main branch
56 + $ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/${VLLM_COMMIT}/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
57 +
58 + You can also just download the latest wheel by running:
59 +
60 + .. code-block:: console
61 +
62 + $ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
63 +
64 + Note that the wheels are built with Python 3.8 abi (see `PEP 425 <https://peps.python.org/pep-0425/>`_ for more details about abi), so they are compatible with Python 3.8 and later. The version string in the wheel file name (``1.0.0.dev``) is just a placeholder to have a unified URL for the wheels. The actual version of wheels is contained in the wheel metadata.
65
66 Build from source (without compilation)
67 ---------------------------------------
@@ -67,8 +72,7 @@ The first step is to follow the previous instructions to install the latest vLLM
72
73 .. code-block:: console
74
70 - $ export VLLM_VERSION=0.6.1.post1
71 - $ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-${VLLM_VERSION}-cp38-abi3-manylinux1_x86_64.whl
75 + $ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
76
77 After verifying that the installation is successful, we have a script for you to copy and link directories, so that you can edit the Python code directly:
78