@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 2b0879bfc2
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 05:45 UTC

Super tiny little typo fix (#10633)

fzyzcjy committed Nov 25, 2024 at 21:08 UTC 2b0879bfc273a08d339b952890c4e88e77f0a014
1 file changed +1 -1
docs/source/quantization/fp8_e5m2_kvcache.rst
+1 -1
@@ -4,7 +4,7 @@ FP8 E5M2 KV Cache
4 ==================
5
6 The int8/int4 quantization scheme requires additional scale GPU memory storage, which reduces the expected GPU memory benefits.
7 -The FP8 data format retains 2~3 mantissa bits and can convert float/fp16/bflaot16 and fp8 to each other.
7 +The FP8 data format retains 2~3 mantissa bits and can convert float/fp16/bfloat16 and fp8 to each other.
8
9 Here is an example of how to enable this feature:
10