| 1 | # vllm/tool_parsers |
| 2 | |
| 3 | Every `--tool-call-parser` name resolves here. `ParserManager.get_parser` |
| 4 | composes the tool parser chosen by name with the reasoning parser into the |
| 5 | unified `Parser` the serving layer uses. The `ToolParser` interface itself is |
| 6 | legacy: it is kept so existing in-tree parsers and out-of-tree plugins keep |
| 7 | working, not as the base for new ones. |
| 8 | |
| 9 | ## Layout |
| 10 | |
| 11 | - `abstract_tool_parser.py` — `ToolParser` (`extract_tool_calls`, |
| 12 | `extract_tool_calls_streaming`, `adjust_request`) and `ToolParserManager`. |
| 13 | - `__init__.py` — `_TOOL_PARSERS_TO_REGISTER`, the name → (file, class) table |
| 14 | that lazily registers every built-in parser. |
| 15 | - `structural_tag_registry.py` — structural-tag builders used to constrain |
| 16 | tool-call output for `tool_choice="required"` / named tools. |
| 17 | - `<model>_tool_parser.py` — one file per family. Either a subclass of a |
| 18 | `ParserToolAdapter` generated from a `vllm/parser/<model>.py` engine grammar |
| 19 | (`mistral_tool_parser.py`, `kimi_k2_tool_parser.py`; the `_engine_` suffix |
| 20 | is not applied consistently), or a self-contained `ToolParser` subclass. |
| 21 | |
| 22 | ## Guidelines |
| 23 | |
| 24 | - Do not change the `ToolParser` interface in `abstract_tool_parser.py`. |
| 25 | Out-of-tree plugins loaded through `ToolParserManager.import_tool_parser` |
| 26 | subclass it. New capabilities go on `Parser` in `vllm/parser/`; if a legacy |
| 27 | parser needs the same capability, `DelegatingParser` supplies the default. |
| 28 | - For a new model, implement a unified `Parser` in `vllm/parser/` first, then |
| 29 | expose it here as a thin adapter subclass plus an entry in |
| 30 | `_TOOL_PARSERS_TO_REGISTER`. |
| 31 | - The `_TOOL_PARSERS_TO_REGISTER` key is what users pass on the command line; |
| 32 | treat renames as breaking. |
| 33 | - Keep adapters thin: registration, `adjust_request` tweaks, and class |
| 34 | attributes. Parsing logic belongs in `vllm/parser/`. |
| 35 | - Keep a model-specific fix inside that model's `<model>_tool_parser.py`, or |
| 36 | its grammar in `vllm/parser/<model>.py`, where possible. Touch |
| 37 | `abstract_tool_parser.py` or `structural_tag_registry.py` only for behavior |
| 38 | every parser needs, and say so in the PR. |
| 39 | - Set `structural_tag_model` only to a key in `SUPPORTED_STRUCTURAL_TAG_MODELS` |
| 40 | (an xgrammar builtin, or one registered with `register_vllm_structural_tag`). |
| 41 | It only takes effect under `VLLM_ENFORCE_STRICT_TOOL_CALLING`, where it also |
| 42 | disables the generic required/named handling for that parser. |
| 43 | - Comment only where the code alone would leave a reader with a question — |
| 44 | a request tweak in `adjust_request`, a marker handled unusually, a model |
| 45 | quirk being worked around. Keep it minimal and concise. |