| 1 | # extract_tools.py DOX |
| 2 | |
| 3 | ## Purpose |
| 4 | |
| 5 | - Own the `extract_tools.py` helper module. |
| 6 | - This module normalizes and repairs model-emitted tool-call JSON. |
| 7 | - Keep this file-level DOX profile synchronized with `extract_tools.py` because this directory is intentionally flat. |
| 8 | |
| 9 | ## Ownership |
| 10 | |
| 11 | - `extract_tools.py` owns the runtime implementation. |
| 12 | - `extract_tools.py.dox.md` owns durable notes about responsibilities, contracts, side effects, and verification for that implementation. |
| 13 | - Top-level functions: |
| 14 | - `json_parse_dirty(json: str) -> dict[str, Any] | None` |
| 15 | - `extract_tool_request(content: str) -> dict[str, Any] | None` |
| 16 | - `is_misformatted_tool_request(content: str) -> bool` |
| 17 | - `normalize_tool_request(tool_request: Any) -> tuple[str, dict]` |
| 18 | - `extract_json_root_string(content: str) -> str | None` |
| 19 | - `extract_json_root_strings(content: str) -> list[str]` |
| 20 | - `extract_json_object_string(content)` |
| 21 | - `extract_json_string(content)` |
| 22 | - `fix_json_string(json_string)` |
| 23 | |
| 24 | ## Runtime Contracts |
| 25 | |
| 26 | - Helper modules own reusable framework APIs and must preserve public callers unless all callers, tests, and docs are updated together. |
| 27 | - Update this file whenever public functions, classes, persistence behavior, path/security assumptions, side effects, or cross-module contracts change. |
| 28 | - Observed side-effect areas: settings/state persistence. |
| 29 | - Dirty parsing scans complete JSON object roots in prose and prefers the first object that normalizes as a valid tool request for permissive repair and legacy callers. |
| 30 | Normalization accepts canonical `tool_name`/`tool_args`, legacy `tool`/`args`, native `type="function"` `name`/`parameters`, and a single-item `actions` wrapper; malformed or multi-action wrappers are rejected. |
| 31 | - `extract_tool_request` is the execution boundary: it accepts a request only when the complete trimmed content is one valid tool object. Plain text, ordinary JSON, and tool-shaped JSON embedded in prose remain final text. |
| 32 | - `extract_tool_request` rejects content that does not have complete object boundaries before invoking the dirty root scanner. This keeps incomplete streaming prefixes cheap without changing which complete canonical tool objects are accepted. |
| 33 | - `is_misformatted_tool_request` identifies a tool request wrapped in a JSON code fence, concatenated complete roots containing tool intent, or a complete Agent Zero envelope that starts with `thoughts` and whose dirty parser has absorbed `headline`, `tool_name`, and `tool_args` into that list. It routes that output to the existing repair prompt without executing it. |
| 34 | - Streaming tool snapshots use `extract_tool_request`; the permissive root helpers remain available for repair and legacy callers, not tool execution. |
| 35 | - Root extraction ignores objects nested inside an open parent object, so streamed wrapper tools such as `parallel` cannot stop early on the first nested `tool_calls` item. |
| 36 | - Imported dependency areas include: `dirty_json`, `helpers.modules`, `re`, `regex`, `typing`. |
| 37 | |
| 38 | ## Key Concepts |
| 39 | |
| 40 | - Important called helpers/classes observed in the source: `extract_json_object_string`, `content.find`, `DirtyJson`, `content.rfind`, `regex.search`, `re.sub`, `json.strip`, `ValueError`, `tool_name.split`, `parser.parse`, `match.group`, `match.group.replace`, `DirtyJson.parse_string`. |
| 41 | - Keep request/response, tool, or helper semantics documented here at the same time as source changes. |
| 42 | |
| 43 | ## Work Guidance |
| 44 | |
| 45 | - Preserve public helper APIs used by core code and plugins unless every caller is updated. |
| 46 | - Keep path, auth, secret, persistence, network, and subprocess behavior explicit and bounded. |
| 47 | - Prefer adding cohesive helper functions here only when behavior is reused across modules. |
| 48 | |
| 49 | ## Verification |
| 50 | |
| 51 | - Run targeted tests for changed helper behavior; run security regressions for auth, filesystem, WebSocket, tunnel, upload, or secret-handling helpers. |
| 52 | - Related tests observed by source search: |
| 53 | - `tests/test_stream_tool_early_stop.py` |
| 54 | - `tests/test_tool_request_normalization.py` |
| 55 | |
| 56 | ## Child DOX Index |
| 57 | |
| 58 | No child DOX files. |