@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / ee37328da0
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 01:45 UTC

Unable to find Punica extension issue during source code installation (#4494)

Co-authored-by: Simon Mo <simon.mo@hey.com>

fuchen.ljl committed May 1, 2024 at 08:42 UTC ee37328da085af14f89ad1af8eb2c359ae2f46a1
1 file changed +1
docs/source/getting_started/installation.rst
+1
@@ -53,6 +53,7 @@ You can also build and install vLLM from source:
53
54 $ git clone https://github.com/vllm-project/vllm.git
55 $ cd vllm
56 + $ # export VLLM_INSTALL_PUNICA_KERNELS=1 # optionally build for multi-LoRA capability
57 $ pip install -e . # This may take 5-10 minutes.
58
59 .. tip::