@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 02b1988b9f
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 19:30 UTC

[Doc] building vLLM with VLLM_TARGET_DEVICE=empty (#7403)

tomeras91 committed Aug 12, 2024 at 00:38 UTC 02b1988b9f2081e29189804669fc9c4c5257c0e0
1 file changed +10
docs/source/getting_started/installation.rst
+10
@@ -68,6 +68,16 @@ You can also build and install vLLM from source:
68 $ cd vllm
69 $ pip install -e . # This may take 5-10 minutes.
70
71 +.. note::
72 +
73 + vLLM can fully run only on Linux, but you can still build it on other systems (for example, macOS). This build is only for development purposes, allowing for imports and a more convenient dev environment. The binaries will not be compiled and not work on non-Linux systems. You can create such a build with the following commands:
74 +
75 + .. code-block:: console
76 +
77 + $ export VLLM_TARGET_DEVICE=empty
78 + $ pip install -e .
79 +
80 +
81 .. tip::
82
83 Building from source requires quite a lot compilation. If you are building from source for multiple times, it is beneficial to cache the compilation results. For example, you can install `ccache <https://github.com/ccache/ccache>`_ via either `conda install ccache` or `apt install ccache` . As long as `which ccache` command can find the `ccache` binary, it will be used automatically by the build system. After the first build, the subsequent builds will be much faster.