@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / e85250b1d1
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 05:45 UTC

[Hardware][Gaudi]add get_name method for HPUAttentionBackend (#10667)

Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>

Kunshang Ji committed Nov 27, 2024 at 14:49 UTC e85250b1d164c9975816fa7aaf591aa5abad577d
1 file changed +4
vllm/attention/backends/hpu_attn.py
+4
@@ -22,6 +22,10 @@ logger = init_logger(__name__)
22
23 class HPUAttentionBackend(AttentionBackend):
24
25 + @staticmethod
26 + def get_name() -> str:
27 + return "HPU_ATTN"
28 +
29 @staticmethod
30 def get_impl_cls() -> Type["HPUAttentionImpl"]:
31 return HPUAttentionImpl