@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 533c217792
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 00:30 UTC

Fix cutlass sm_90a vesrion in CMakeList

simon-mo committed May 31, 2024 at 02:13 UTC 533c2177925ba19934eab0095a50d0a783185e6b
1 file changed +1 -1
CMakeLists.txt
+1 -1
@@ -200,7 +200,7 @@ if(VLLM_GPU_LANG STREQUAL "CUDA")
200 # The CUTLASS kernels for Hopper require sm90a to be enabled.
201 # This is done via the below gencode option, BUT that creates kernels for both sm90 and sm90a.
202 # That adds an extra 17MB to compiled binary, so instead we selectively enable it.
203 - if(${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER 11)
203 + if(${CMAKE_CUDA_COMPILER_VERSION} VERSION_GREATER 12.0)
204 set_source_files_properties(
205 "csrc/quantization/cutlass_w8a8/scaled_mm_dq_c3x.cu"
206 PROPERTIES