@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 8eadcf0b90
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 23:15 UTC

[misc][typo] fix typo (#5620)

youkaichao committed Jun 17, 2024 at 20:54 UTC 8eadcf0b90f126cf9b23f9583a53b19b6b58fd87
1 file changed +1 -1
vllm/block.py
+1 -1
@@ -13,7 +13,7 @@ TokensBlock = List[int]
13
14
15 class BlockPool:
16 - """A pool of physical blocks.
16 + """A pool of logical blocks.
17 When requests come, we create a lot of logical blocks;
18 when requests are done, we destroy a lot of logical blocks.
19 It turns out that creating and destroying logical blocks can be expensive,