A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
[misc][typo] fix typo (#5620)
youkaichao committed
Jun 17, 2024 at 20:54 UTC
8eadcf0b90f126cf9b23f9583a53b19b6b58fd87
1 file changed
+1
-1
vllm/block.py
+1
-1
@@ -13,7 +13,7 @@ TokensBlock = List[int]
13
14
15
class BlockPool:
16
- """A pool of physical blocks.
16
+ """A pool of logical blocks.
17
When requests come, we create a lot of logical blocks;
18
when requests are done, we destroy a lot of logical blocks.
19
It turns out that creating and destroying logical blocks can be expensive,