@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 931746bc6d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 05:30 UTC

Add documentation on how to do incremental builds (#2796)

Philipp Moritz committed Feb 7, 2024 at 14:42 UTC 931746bc6d7c1c0ab40b2c4f58b51b855f0b2c94
2 files changed +15
docs/source/getting_started/installation.rst
+10
@@ -67,3 +67,13 @@ You can also build and install vLLM from source:
67
68 $ # Use `--ipc=host` to make sure the shared memory is large enough.
69 $ docker run --gpus all -it --rm --ipc=host nvcr.io/nvidia/pytorch:23.10-py3
70 +
71 +.. note::
72 + If you are developing the C++ backend of vLLM, consider building vLLM with
73 +
74 + .. code-block:: console
75 +
76 + $ python setup.py develop
77 +
78 + since it will give you incremental builds. The downside is that this method
79 + is `deprecated by setuptools <https://github.com/pypa/setuptools/issues/917>`_.
setup.py
+5
@@ -15,6 +15,11 @@ from torch.utils.cpp_extension import BuildExtension, CUDAExtension, CUDA_HOME,
15
16 ROOT_DIR = os.path.dirname(__file__)
17
18 +# If you are developing the C++ backend of vLLM, consider building vLLM with
19 +# `python setup.py develop` since it will give you incremental builds.
20 +# The downside is that this method is deprecated, see
21 +# https://github.com/pypa/setuptools/issues/917
22 +
23 MAIN_CUDA_VERSION = "12.1"
24
25 # Supported NVIDIA GPU architectures.