A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 04:15 UTC
fix document error for value and v_vec illustration (#3421)
laneeee committed
Mar 16, 2024 at 07:06 UTC
8fa7357f2d3171e3d373be865c8f9520e538c415
3 files changed
+1
-1
docs/source/assets/kernel/v_vec.png
Binary files a/docs/source/assets/kernel/v_vec.png and b/docs/source/assets/kernel/v_vec.png differ
docs/source/assets/kernel/value.png
Binary files a/docs/source/assets/kernel/value.png and b/docs/source/assets/kernel/value.png differ
docs/source/dev/kernel/paged_attention.rst
+1
-1
@@ -447,7 +447,7 @@ Value
447
a whole block of value tokens. And each ``accs`` in each thread
448
contains 8 elements that accumulated at 8 different head positions.
449
For the thread 0, the ``accs`` variable will have 8 elements, which
450
- are 0th, 16th … 112th elements of a value head that are accumulated
450
+ are 0th, 32th … 224th elements of a value head that are accumulated
451
from all assigned 8 tokens.
452
453
LV