A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 14:45 UTC
[Intel][Triton] Support `round_int8` for Intel backend (#38825)
Signed-off-by: Mieszko Dziadowiec <mdziadowiec@habana.ai> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: Stefano Castagnetta <scastagnetta@nvidia.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Mieszko Dziadowiec committed
Apr 3, 2026 at 14:47 UTC
bf8b022e60abd3008cc9309d837ef2727e6c0acd
1 file changed
+7
vllm/model_executor/layers/quantization/utils/int8_utils.py
+7
@@ -88,6 +88,13 @@ if current_platform.is_rocm():
88
def round_int8(x):
89
return tl.extra.hip.libdevice.round(x).to(tl.int8)
90
91
+
92
+elif current_platform.is_xpu():
93
+
94
+ @triton.jit
95
+ def round_int8(x):
96
+ return tl.extra.intel.libdevice.round(x).to(tl.int8)
97
+
98
else:
99
100
@triton.jit