A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 01:45 UTC
Unable to find Punica extension issue during source code installation (#4494)
Co-authored-by: Simon Mo <simon.mo@hey.com>
fuchen.ljl committed
May 1, 2024 at 08:42 UTC
ee37328da085af14f89ad1af8eb2c359ae2f46a1
1 file changed
+1
docs/source/getting_started/installation.rst
+1
@@ -53,6 +53,7 @@ You can also build and install vLLM from source:
53
54
$ git clone https://github.com/vllm-project/vllm.git
55
$ cd vllm
56
+ $ # export VLLM_INSTALL_PUNICA_KERNELS=1 # optionally build for multi-LoRA capability
57
$ pip install -e . # This may take 5-10 minutes.
58
59
.. tip::