A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 17:15 UTC
[Docs] Document v0 engine support in reasoning outputs (#15739)
Signed-off-by: Ce Gao <cegao@tensorchord.ai>
Ce Gao committed
Mar 29, 2025 at 11:46 UTC
762b424a528e025ef3d9b02828eb926c6dbddb2c
1 file changed
+8
-1
docs/source/features/reasoning_outputs.md
+8
-1
@@ -136,7 +136,14 @@ Remember to check whether the `reasoning_content` exists in the response before
136
137
## Structured output
138
139
-The reasoning content is also available in the structured output. The structured output engine like `xgrammar` will use the reasoning content to generate structured output.
139
+The reasoning content is also available in the structured output. The structured output engine like `xgrammar` will use the reasoning content to generate structured output. It is only supported in v0 engine now.
140
+
141
+```bash
142
+VLLM_USE_V1=0 vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B \
143
+ --enable-reasoning --reasoning-parser deepseek_r1
144
+```
145
+
146
+Please note that the `VLLM_USE_V1` environment variable must be set to `0` to use the v0 engine.
147
148
```python
149
from openai import OpenAI