A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 02:15 UTC
Update kimi-k2 tool calling docs, enable unit tests (#20821)
Signed-off-by: wangzhengtao <wangzhengtao@moonshot.cn> Co-authored-by: wangzhengtao <wangzhengtao@moonshot.cn> Co-authored-by: wangzhengtao <wangzhengtao@msh.team>
bigmoyan committed
Jul 12, 2025 at 04:16 UTC
5f0af36af555a3813b9d30983bd29c384b84b647
2 files changed
+8
-2
docs/features/tool_calling.md
+8
@@ -282,6 +282,14 @@ Supported models:
282
283
Flags: `--tool-call-parser deepseek_v3 --chat-template {see_above}`
284
285
+### Kimi-K2 Models (`kimi_k2`)
286
+
287
+Supported models:
288
+
289
+* `moonshotai/Kimi-K2-Instruct`
290
+
291
+Flags: `--tool-call-parser kimi_k2`
292
+
293
### Models with Pythonic Tool Calls (`pythonic`)
294
295
A growing number of models output a python list to represent tool calls instead of using JSON. This has the advantage of inherently supporting parallel tool calls and removing ambiguity around the JSON schema required for tool calls. The `pythonic` tool parser can support such models.
tests/tool_use/test_kimi_k2_tool_parser.py
-2
@@ -10,8 +10,6 @@ from vllm.entrypoints.openai.protocol import FunctionCall, ToolCall
10
from vllm.entrypoints.openai.tool_parsers import KimiK2ToolParser
11
from vllm.transformers_utils.tokenizer import get_tokenizer
12
13
-pytest.skip("skip kimi_k2 parser test", allow_module_level=True)
14
-
13
# Use a common model that is likely to be available
14
MODEL = "moonshotai/Kimi-K2-Instruct"
15