@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 056a68c7db
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 09:30 UTC

[XPU] avoid triton import for xpu (#9440)

Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

Yan Ma committed Oct 24, 2024 at 13:14 UTC 056a68c7dbaff03252d2f8c058d3fb700565ad1f
1 file changed +7 -5
vllm/triton_utils/importing.py
+7 -5
@@ -5,10 +5,12 @@ from vllm.platforms import current_platform
5
6 logger = init_logger(__name__)
7
8 -# neuron has too old torch
9 -HAS_TRITON = find_spec(
10 - "triton") is not None and not current_platform.is_neuron()
8 +HAS_TRITON = (
9 + find_spec("triton") is not None
10 + and not current_platform.is_xpu() # Not compatible
11 + and not current_platform.is_neuron() # neuron has too old torch
12 +)
13
14 if not HAS_TRITON:
13 - logger.info("Triton not installed; certain GPU-related functions"
14 - " will not be available.")
15 + logger.info("Triton not installed or not compatible; certain GPU-related"
16 + " functions will not be available.")