A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 09:45 UTC
[Doc] Add doc to explain the usage of Qwen3 thinking (#18291)
Signed-off-by: WangErXiao <863579016@qq.com>
Robin committed
May 19, 2025 at 07:04 UTC
d1211f8794c28da64f80e55464b4ce6f97b2c0cb
1 file changed
+5
docs/source/features/reasoning_outputs.md
+5
@@ -19,6 +19,7 @@ vLLM currently supports the following reasoning models:
19
20
:::{note}
21
IBM Granite 3.2 reasoning is disabled by default; to enable it, you must also pass `thinking=True` in your `chat_template_kwargs`.
22
+The reasoning feature for the Qwen3 series is enabled by default. To disable it, you must pass `enable_thinking=False` in your `chat_template_kwargs`.
23
:::
24
25
## Quickstart
@@ -49,6 +50,8 @@ model = models.data[0].id
50
# Round 1
51
messages = [{"role": "user", "content": "9.11 and 9.8, which is greater?"}]
52
# For granite, add: `extra_body={"chat_template_kwargs": {"thinking": True}}`
53
+# For Qwen3 series, if you want to disable thinking in reasoning mode, add:
54
+# extra_body={"chat_template_kwargs": {"enable_thinking": False}}
55
response = client.chat.completions.create(model=model, messages=messages)
56
57
reasoning_content = response.choices[0].message.reasoning_content
@@ -104,6 +107,8 @@ model = models.data[0].id
107
108
messages = [{"role": "user", "content": "9.11 and 9.8, which is greater?"}]
109
# For granite, add: `extra_body={"chat_template_kwargs": {"thinking": True}}`
110
+# For Qwen3 series, if you want to disable thinking in reasoning mode, add:
111
+# extra_body={"chat_template_kwargs": {"enable_thinking": False}}
112
stream = client.chat.completions.create(model=model,
113
messages=messages,
114
stream=True)