@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 51c31bc10c
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 03:00 UTC

CMake build elf without PTX (#3739)

Simon Mo committed Mar 29, 2024 at 18:53 UTC 51c31bc10ca7c48b580cd58fcd741ba4d6db4447
1 file changed +4 -1
cmake/utils.cmake
+4 -1
@@ -240,9 +240,12 @@ macro(override_gpu_arches GPU_ARCHES GPU_LANG GPU_SUPPORTED_ARCHES)
240 endif()
241
242 if (_SM)
243 - set(_VIRT "")
243 + # -real suffix let CMake to only generate elf code for the kernels.
244 + # we want this, otherwise the added ptx (default) will increase binary size.
245 + set(_VIRT "-real")
246 set(_CODE_ARCH ${_SM})
247 else()
248 + # -virtual suffix let CMake to generate ptx code for the kernels.
249 set(_VIRT "-virtual")
250 set(_CODE_ARCH ${_CODE})
251 endif()