@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 9984605412
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 18:15 UTC

[AMD][CI/Build] Disambiguation of the function call for ROCm 6.2 headers compatibility (#7477)

Co-authored-by: Charlie Fu <Charlie.Fu@amd.com>

Gregory Shtrasberg committed Aug 21, 2024 at 19:47 UTC 9984605412de1171a72d955cfcb954725edd4d6f
1 file changed +1 -1
csrc/attention/attention_utils.cuh
+1 -1
@@ -34,7 +34,7 @@ inline __device__ float qk_dot_(const Vec (&q)[N], const Vec (&k)[N]) {
34 A_vec qk_vec = mul<A_vec, Vec, Vec>(q[0], k[0]);
35 #pragma unroll
36 for (int ii = 1; ii < N; ++ii) {
37 - qk_vec = fma(q[ii], k[ii], qk_vec);
37 + qk_vec = vllm::fma(q[ii], k[ii], qk_vec);
38 }
39
40 // Finalize the reduction across lanes.