wassname/vllm
1
0 Fork 0
mirror of https://github.com/wassname/vllm.git synced 2026-10-01 12:42:39 +08:00
Code Issues Packages Projects Releases Wiki Activity

Super tiny little typo fix (#10633)

Browse Source
This commit is contained in:
fzyzcjy authored and GitHub committed 2024-11-25 13:08:30 +00:00
1 parent ed46f14321
commit 2b0879bfc2
1 file changed
+1 -1
Download Patch File Download Diff File Expand all files Collapse all files
docs/source/quantization/fp8_e5m2_kvcache.rst
+1 -1
View File
@@ -4,7 +4,7 @@ FP8 E5M2 KV Cache
==================
The int8/int4 quantization scheme requires additional scale GPU memory storage, which reduces the expected GPU memory benefits.
The FP8 data format retains 2~3 mantissa bits and can convert float/fp16/bflaot16 and fp8 to each other.
The FP8 data format retains 2~3 mantissa bits and can convert float/fp16/bfloat16 and fp8 to each other.
Here is an example of how to enable this feature: