A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 03:30 UTC
[Docs] Update EAGLE example (#20375)
Signed-off-by: NickLucche <nlucches@redhat.com>
Nicolò Lucchesi committed
Jul 3, 2025 at 02:13 UTC
3dd359147d81fd0788c18968130ecd6c55dac415
1 file changed
+1
docs/features/spec_decode.md
+1
@@ -201,6 +201,7 @@ an [EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency)](https
201
speculative_config={
202
"model": "yuhuili/EAGLE-LLaMA3-Instruct-8B",
203
"draft_tensor_parallel_size": 1,
204
+ "num_speculative_tokens": 2,
205
},
206
)
207