@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 50704f52c4
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 20:45 UTC

[Bugfix][Kernel] Promote another index to int64_t (#6838)

Tyler Michael Smith committed Jul 26, 2024 at 14:41 UTC 50704f52c4643777fb0e5dc99f6c048dd9f54f2d
1 file changed +1 -1
csrc/quantization/fp8/common.cu
+1 -1
@@ -48,7 +48,7 @@ __global__ void segmented_max_reduction(float* __restrict__ scale,
48 const scalar_t* __restrict__ input,
49 int64_t num_elems) {
50 __shared__ float cache[1024];
51 - int i = blockDim.x * blockIdx.x + threadIdx.x;
51 + int64_t i = blockDim.x * blockIdx.x + threadIdx.x;
52
53 // First store maximum for all values processes by
54 // the current thread in cache[threadIdx.x]