@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / f990bab2a4
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[Doc][Neuron] add note to neuron documentation about resolving triton issue (#9257)

Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>

omrishiv committed Oct 10, 2024 at 16:36 UTC f990bab2a4198c4de6b5b349d35fc74bf0f36f3e
1 file changed +4
docs/source/getting_started/neuron-installation.rst
+4
@@ -27,6 +27,10 @@ Installation steps:
27
28 .. _build_from_source_neuron:
29
30 +.. note::
31 +
32 + The currently supported version of Pytorch for Neuron installs `triton` version `2.1.0`. This is incompatible with vLLM >= 0.5.3. You may see an error `cannot import name 'default_dump_dir...`. To work around this, run a `pip install --upgrade triton==3.0.0` after installing the vLLM wheel.
33 +
34 Build from source
35 -----------------
36