@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 8c00f9c15d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 23:15 UTC

[Docs][TPU] Add installation tip for TPU (#5761)

Woosuk Kwon committed Jun 21, 2024 at 23:09 UTC 8c00f9c15d13aed34b129b31c32a227be230e218
1 file changed +18
docs/source/getting_started/tpu-installation.rst
+18
@@ -73,3 +73,21 @@ Next, build vLLM from source. This will only take a few seconds:
73 .. code-block:: console
74
75 $ VLLM_TARGET_DEVICE="tpu" python setup.py develop
76 +
77 +
78 +.. tip::
79 +
80 + If you encounter the following error:
81 +
82 + .. code-block:: console
83 +
84 + from torch._C import * # noqa: F403
85 + ImportError: libopenblas.so.0: cannot open shared object file: No such file or directory
86 +
87 +
88 + You can install OpenBLAS with the following command:
89 +
90 + .. code-block:: console
91 +
92 + $ sudo apt-get install libopenblas-base libopenmpi-dev libomp-dev
93 +