@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 3c3012398e
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 19:30 UTC

[Doc] add VLLM_TARGET_DEVICE=neuron to documentation for neuron (#6844)

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

omrishiv committed Jul 26, 2024 at 20:20 UTC 3c3012398e4aecde9e40981d79a0576203158d24
1 file changed +1 -1
docs/source/getting_started/neuron-installation.rst
+1 -1
@@ -131,6 +131,6 @@ Once neuronx-cc and transformers-neuronx packages are installed, we will be able
131 $ git clone https://github.com/vllm-project/vllm.git
132 $ cd vllm
133 $ pip install -U -r requirements-neuron.txt
134 - $ pip install .
134 + $ VLLM_TARGET_DEVICE="neuron" pip install .
135
136 If neuron packages are detected correctly in the installation process, ``vllm-0.3.0+neuron212`` will be installed.