@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 8108ac841d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 00:45 UTC

[Bugfix] Fix unsupported FA version check for Turing GPU (#12828)

Isotr0py committed Feb 7, 2025 at 01:18 UTC 8108ac841d66515b58252edc26ba63da6cf980e5
1 file changed +1 -1
vllm/attention/backends/utils.py
+1 -1
@@ -612,5 +612,5 @@ try:
612 return fa_version
613
614 VLLM_FLASH_ATTN_VERSION = flash_attn_version()
615 -except ImportError:
615 +except (ImportError, AssertionError):
616 VLLM_FLASH_ATTN_VERSION = None