@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 8fa7357f2d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 04:15 UTC

fix document error for value and v_vec illustration (#3421)

laneeee committed Mar 16, 2024 at 07:06 UTC 8fa7357f2d3171e3d373be865c8f9520e538c415
3 files changed +1 -1
docs/source/assets/kernel/v_vec.png
Binary files a/docs/source/assets/kernel/v_vec.png and b/docs/source/assets/kernel/v_vec.png differ
docs/source/assets/kernel/value.png
Binary files a/docs/source/assets/kernel/value.png and b/docs/source/assets/kernel/value.png differ
docs/source/dev/kernel/paged_attention.rst
+1 -1
@@ -447,7 +447,7 @@ Value
447 a whole block of value tokens. And each ``accs`` in each thread
448 contains 8 elements that accumulated at 8 different head positions.
449 For the thread 0, the ``accs`` variable will have 8 elements, which
450 - are 0th, 16th … 112th elements of a value head that are accumulated
450 + are 0th, 32th … 224th elements of a value head that are accumulated
451 from all assigned 8 tokens.
452
453 LV