A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 19:30 UTC
[Doc] add VLLM_TARGET_DEVICE=neuron to documentation for neuron (#6844)
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
omrishiv committed
Jul 26, 2024 at 20:20 UTC
3c3012398e4aecde9e40981d79a0576203158d24
1 file changed
+1
-1
docs/source/getting_started/neuron-installation.rst
+1
-1
@@ -131,6 +131,6 @@ Once neuronx-cc and transformers-neuronx packages are installed, we will be able
131
$ git clone https://github.com/vllm-project/vllm.git
132
$ cd vllm
133
$ pip install -U -r requirements-neuron.txt
134
- $ pip install .
134
+ $ VLLM_TARGET_DEVICE="neuron" pip install .
135
136
If neuron packages are detected correctly in the installation process, ``vllm-0.3.0+neuron212`` will be installed.