A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 05:45 UTC
Super tiny little typo fix (#10633)
fzyzcjy committed
Nov 25, 2024 at 21:08 UTC
2b0879bfc273a08d339b952890c4e88e77f0a014
1 file changed
+1
-1
docs/source/quantization/fp8_e5m2_kvcache.rst
+1
-1
@@ -4,7 +4,7 @@ FP8 E5M2 KV Cache
4
==================
5
6
The int8/int4 quantization scheme requires additional scale GPU memory storage, which reduces the expected GPU memory benefits.
7
-The FP8 data format retains 2~3 mantissa bits and can convert float/fp16/bflaot16 and fp8 to each other.
7
+The FP8 data format retains 2~3 mantissa bits and can convert float/fp16/bfloat16 and fp8 to each other.
8
9
Here is an example of how to enable this feature:
10