@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / da5a0b539d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 00:30 UTC

Remove marlin warning (#4918)

Alexander Matveev committed May 20, 2024 at 10:55 UTC da5a0b539d6a5fe0c0195513a797814d2c267540
1 file changed -4
csrc/quantization/gptq_marlin/gptq_marlin.cu
-4
@@ -1519,10 +1519,6 @@ exec_config_t determine_thread_config(int prob_m, int prob_n, int prob_k,
1519 }
1520 }
1521
1522 - printf("WARNING: Marlin kernel is reducing max_m_blocks due to small SM "
1523 - "GPU cache. This may "
1524 - "hurt performance. Consider upgrading your GPU.\n");
1525 -
1522 max_m_blocks--; // Process less M blocks per invocation to reduce cache
1523 // usage
1524 }