@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 60c68df6d1
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 15, 23:30 UTC

[Build] Automatically use the wheel of the base commit with Python-only build (#13178)

Cody Yu committed Feb 12, 2025 at 23:10 UTC 60c68df6d1a4421480710c2303c8814724f90bb5
2 files changed +36 -7
docs/source/getting_started/installation/gpu/cuda.inc.md
+13 -3
@@ -89,12 +89,22 @@ cd vllm
89 VLLM_USE_PRECOMPILED=1 pip install --editable .
90 ```
91
92 -This will download the [latest nightly wheel](https://wheels.vllm.ai/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl) and use the compiled libraries from there in the installation.
92 +This command will do the following:
93 +1. Look for the current branch in your vLLM clone.
94 +2. Identify the corresponding base commit in the main branch.
95 +3. Download the pre-built wheel of the base commit.
96 +4. Use its compiled libraries in the installation.
97
94 -The `VLLM_PRECOMPILED_WHEEL_LOCATION` environment variable can be used instead of `VLLM_USE_PRECOMPILED` to specify a custom path or URL to the wheel file. For example, to use the [0.6.1.post1 PyPi wheel](https://pypi.org/project/vllm/#files):
98 +:::{note}
99 +1. If you change C++ or kernel code, you cannot use Python-only build; otherwise you will see an import error about library not found or undefined symbol.
100 +2. If you rebase your dev branch, it is recommended to uninstall vllm and re-run the above command to make sure your libraries are up to date.
101 +:::
102 +
103 +In case you see an error about wheel not found when running the above command, it might be because the commit you based on in the main branch was just merged and the wheel is being built. In this case, you can wait for around an hour to try again, or manually assign the previous commit in the installation using the `VLLM_PRECOMPILED_WHEEL_LOCATION` environment variable.
104
105 ```console
97 -export VLLM_PRECOMPILED_WHEEL_LOCATION=https://files.pythonhosted.org/packages/4a/4c/ee65ba33467a4c0de350ce29fbae39b9d0e7fcd887cc756fa993654d1228/vllm-0.6.3.post1-cp38-abi3-manylinux1_x86_64.whl
106 +export VLLM_COMMIT=72d9c316d3f6ede485146fe5aabd4e61dbc59069 # use full commit hash from the main branch
107 +export VLLM_PRECOMPILED_WHEEL_LOCATION=https://wheels.vllm.ai/${VLLM_COMMIT}/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
108 pip install --editable .
109 ```
110
setup.py
+23 -4
@@ -268,15 +268,34 @@ class cmake_build_ext(build_ext):
268
269 class repackage_wheel(build_ext):
270 """Extracts libraries and other files from an existing wheel."""
271 - default_wheel = "https://wheels.vllm.ai/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl"
271
273 - def run(self) -> None:
274 - wheel_location = os.getenv("VLLM_PRECOMPILED_WHEEL_LOCATION",
275 - self.default_wheel)
272 + def get_base_commit_in_main_branch(self) -> str:
273 + import subprocess
274 +
275 + try:
276 + current_branch = subprocess.check_output(
277 + ["git", "branch", "--show-current"]).decode("utf-8").strip()
278 +
279 + base_commit = subprocess.check_output(
280 + ["git", "merge-base", "main",
281 + current_branch]).decode("utf-8").strip()
282 + return base_commit
283 + except Exception as err:
284 + logger.warning(
285 + "Failed to get the base commit in the main branch. "
286 + "Using the nightly wheel. The libraries in this "
287 + "wheel may not be compatible with your dev branch: %s", err)
288 + return "nightly"
289
290 + def run(self) -> None:
291 assert _is_cuda(
292 ), "VLLM_USE_PRECOMPILED is only supported for CUDA builds"
293
294 + wheel_location = os.getenv("VLLM_PRECOMPILED_WHEEL_LOCATION", None)
295 + if wheel_location is None:
296 + base_commit = self.get_base_commit_in_main_branch()
297 + wheel_location = f"https://wheels.vllm.ai/{base_commit}/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl"
298 +
299 import zipfile
300
301 if os.path.isfile(wheel_location):