A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 19:45 UTC
[Doc]add doc for Qwen models tool calling (#14478)
Signed-off-by: WangErXiao <863579016@qq.com>
Robin committed
Mar 8, 2025 at 14:58 UTC
7b6fd6e48666b60cb2eed53b2f0f7373e25cab8d
1 file changed
+9
docs/source/features/tool_calling.md
+9
@@ -209,6 +209,15 @@ AI21's Jamba-1.5 models are supported.
209
210
Flags: `--tool-call-parser jamba`
211
212
+### Qwen Models
213
+
214
+For Qwen2.5, the chat template in tokenizer_config.json has already included support for the Hermes-style tool use. Therefore, you can use the `hermes` parser to enable tool calls for Qwen models. For more detailed information, please refer to the official [Qwen documentation](https://qwen.readthedocs.io/en/latest/framework/function_call.html#vllm)
215
+
216
+* `Qwen/Qwen2.5-*`
217
+* `Qwen/QwQ-32B`
218
+
219
+Flags: `--tool-call-parser hermes`
220
+
221
### Models with Pythonic Tool Calls (`pythonic`)
222
223
A growing number of models output a python list to represent tool calls instead of using JSON. This has the advantage of inherently supporting parallel tool calls and removing ambiguity around the JSON schema required for tool calls. The `pythonic` tool parser can support such models.