A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 9, 23:00 UTC
[Bugfix] Fix Off-by-one error in _num_tokens_to_min_blocks calculation (#32603)
Signed-off-by: linhaifeng <1371675203@qq.com>
linhaifeng committed
Jan 21, 2026 at 00:13 UTC
7901109ea5c9794a5e7e01481343235e3a042971
1 file changed
+2
-2
tests/kernels/utils.py
+2
-2
@@ -609,7 +609,7 @@ def _num_tokens_to_min_blocks(num_tokens: int, block_size: int) -> int:
609
Compute the minimum number of blocks required to hold num_tokens tokens,
610
given block_size
611
"""
612
- return (num_tokens + block_size) // block_size
612
+ return (num_tokens + block_size - 1) // block_size
613
614
615
def make_empty_slot_mapping_tensor(device: torch.device | str):
@@ -694,7 +694,7 @@ def make_block_tables_slot_mapping(
694
For a sequence with num_tokens tokens the minimum number
695
of required KV cache blocks is
696
697
- num_blocks = (num_tokens + block_size) // block_size
697
+ num_blocks = (num_tokens + block_size - 1) // block_size
698
699
Then the minimum KV cache size in blocks is
700