A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 19:30 UTC
[Doc] building vLLM with VLLM_TARGET_DEVICE=empty (#7403)
tomeras91 committed
Aug 12, 2024 at 00:38 UTC
02b1988b9f2081e29189804669fc9c4c5257c0e0
1 file changed
+10
docs/source/getting_started/installation.rst
+10
@@ -68,6 +68,16 @@ You can also build and install vLLM from source:
68
$ cd vllm
69
$ pip install -e . # This may take 5-10 minutes.
70
71
+.. note::
72
+
73
+ vLLM can fully run only on Linux, but you can still build it on other systems (for example, macOS). This build is only for development purposes, allowing for imports and a more convenient dev environment. The binaries will not be compiled and not work on non-Linux systems. You can create such a build with the following commands:
74
+
75
+ .. code-block:: console
76
+
77
+ $ export VLLM_TARGET_DEVICE=empty
78
+ $ pip install -e .
79
+
80
+
81
.. tip::
82
83
Building from source requires quite a lot compilation. If you are building from source for multiple times, it is beneficial to cache the compilation results. For example, you can install `ccache <https://github.com/ccache/ccache>`_ via either `conda install ccache` or `apt install ccache` . As long as `which ccache` command can find the `ccache` binary, it will be used automatically by the build system. After the first build, the subsequent builds will be much faster.