A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 12:15 UTC
[doc] miss result (#17589)
Signed-off-by: reidliu41 <reid201711@gmail.com> Co-authored-by: reidliu41 <reid201711@gmail.com>
Reid committed
May 2, 2025 at 22:04 UTC
3a500cd0b6161933ec7e71cca8b2dfc0982b6f81
1 file changed
+1
-1
docs/source/features/quantization/fp8.md
+1
-1
@@ -106,7 +106,7 @@ Load and run the model in `vllm`:
106
```python
107
from vllm import LLM
108
model = LLM("./Meta-Llama-3-8B-Instruct-FP8-Dynamic")
109
-model.generate("Hello my name is")
109
+result = model.generate("Hello my name is")
110
print(result[0].outputs[0].text)
111
```
112