A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 10:45 UTC
[Doc][Neuron] add note to neuron documentation about resolving triton issue (#9257)
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
omrishiv committed
Oct 10, 2024 at 16:36 UTC
f990bab2a4198c4de6b5b349d35fc74bf0f36f3e
1 file changed
+4
docs/source/getting_started/neuron-installation.rst
+4
@@ -27,6 +27,10 @@ Installation steps:
27
28
.. _build_from_source_neuron:
29
30
+.. note::
31
+
32
+ The currently supported version of Pytorch for Neuron installs `triton` version `2.1.0`. This is incompatible with vLLM >= 0.5.3. You may see an error `cannot import name 'default_dump_dir...`. To work around this, run a `pip install --upgrade triton==3.0.0` after installing the vLLM wheel.
33
+
34
Build from source
35
-----------------
36