A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
[Docs][TPU] Add installation tip for TPU (#5761)
Woosuk Kwon committed
Jun 21, 2024 at 23:09 UTC
8c00f9c15d13aed34b129b31c32a227be230e218
1 file changed
+18
docs/source/getting_started/tpu-installation.rst
+18
@@ -73,3 +73,21 @@ Next, build vLLM from source. This will only take a few seconds:
73
.. code-block:: console
74
75
$ VLLM_TARGET_DEVICE="tpu" python setup.py develop
76
+
77
+
78
+.. tip::
79
+
80
+ If you encounter the following error:
81
+
82
+ .. code-block:: console
83
+
84
+ from torch._C import * # noqa: F403
85
+ ImportError: libopenblas.so.0: cannot open shared object file: No such file or directory
86
+
87
+
88
+ You can install OpenBLAS with the following command:
89
+
90
+ .. code-block:: console
91
+
92
+ $ sudo apt-get install libopenblas-base libopenmpi-dev libomp-dev
93
+