development
claude/code-feature-parity-q003hm
claude/elegant-carson-l1menh
claude/sigit-acp-local-chat-cx6380
claude/sigit-cloud-agent-expansion-reox0a
claude/tool-permission-system
claude/zen-feynman-0u78dk
development
feature/agent-tools-multiedit-glob-todos-remember
feature/background-commands
feature/commit-coauthor-attribution
feature/headless-mode
feature/init-command
feature/load-local-model-explicitly
feature/session-persistence-compaction
feature/sigit-code-cloud
feature/subagent-tool
feature/tool-permission-system
feature/tui-repo-tabs
feature/tui-tabs
main
release/v1.3.1
| 1 | --- |
| 2 | name: tool-calling |
| 3 | description: Implement or debug tool calling in siGit Code across the app, Onde Inference, and mistral.rs. Use when working on tool schemas, execution loops, model support, session cwd handling, or tool-call troubleshooting. |
| 4 | --- |
| 5 | |
| 6 | # Skill: Tool Calling in siGit Code |
| 7 | |
| 8 | ## Overview |
| 9 | |
| 10 | siGit Code supports **agentic tool calling** — the LLM invokes tools (read/write files, run commands, read websites) to operate on the user's codebase. This works in both **interactive TUI mode** and **ACP server mode** (Zed editor). |
| 11 | |
| 12 | Tool calling spans these layers: |
| 13 | |
| 14 | ``` |
| 15 | siGit (agent loop + tool execution) |
| 16 | → InferenceBackend (src/backend.rs — LocalBackend or OpenAiBackend/cloud) |
| 17 | → onde ChatEngine (tool-aware API) ── for LocalBackend |
| 18 | → mistral.rs (model inference + tool call parsing) |
| 19 | └ OpenAI-compatible HTTP endpoint ── for OpenAiBackend (siGit Code Cloud) |
| 20 | ``` |
| 21 | |
| 22 | The agent loop talks to an `InferenceBackend` trait object, not the engine |
| 23 | directly. `LocalBackend` wraps the on-device `ChatEngine`; `OpenAiBackend` calls |
| 24 | a remote OpenAI-compatible endpoint (the siGit Code Cloud tiers). Both implement |
| 25 | `send_message_with_tools` / `send_tool_results`, so the loop is identical. |
| 26 | |
| 27 | --- |
| 28 | |
| 29 | ## Model Requirement |
| 30 | |
| 31 | Tool calling needs a model mistral.rs has a tool-call parser for. The supported |
| 32 | set is the **Qwen 3 family** (`<tool_call>...</tool_call>` XML) plus **Qwen 2.5 |
| 33 | Coder 7B**. Plain Qwen 2.5 and the smaller Qwen 2.5 Coder variants do NOT support |
| 34 | tool calling. The authoritative list is `is_tool_calling()` in `src/models.rs`. |
| 35 | |
| 36 | | Model | Constructor | Size | Tool calling | |
| 37 | |-------|-----------|------|:---:| |
| 38 | | Qwen 3 14B (Q4_K_M) | `GgufModelConfig::qwen3_14b()` | ~9 GB | ✅ | |
| 39 | | Qwen 3 8B (Q4_K_M) | `GgufModelConfig::qwen3_8b()` | ~5 GB | ✅ | |
| 40 | | Qwen 3 4B (Q4_K_M) | `GgufModelConfig::qwen3_4b()` | ~2.7 GB | ✅ | |
| 41 | | Qwen 3 1.7B (Q4_K_M) | `GgufModelConfig::qwen3_1_7b()` | ~1.3 GB | ✅ | |
| 42 | | Qwen 2.5 Coder 7B | `GgufModelConfig::qwen25_coder_7b()` | ~5 GB | ✅ | |
| 43 | | Qwen 2.5 Coder 3B | `GgufModelConfig::qwen25_coder_3b()` | ~1.93 GB | ❌ | |
| 44 | | Qwen 2.5 Coder 1.5B | `GgufModelConfig::qwen25_coder_1_5b()` | ~941 MB | ❌ | |
| 45 | | Qwen 2.5 3B / 1.5B | `qwen25_3b()` / `qwen25_1_5b()` | ~1.93 GB / ~941 MB | ❌ | |
| 46 | |
| 47 | ### Default model |
| 48 | |
| 49 | There is **no hardcoded default model.** Startup uses the saved selection |
| 50 | (`setup::startup_model_selection`), then the first complete locally-cached model, |
| 51 | falling back to `GgufModelConfig::platform_default()` (Qwen 2.5 3B on macOS) when |
| 52 | nothing is cached. The TUI/ACP code in `main.rs` uses `qwen25_3b()` as that final |
| 53 | fallback. Users pick a tool-calling model via the `/models` picker. |
| 54 | |
| 55 | ### max_tokens |
| 56 | |
| 57 | `max_tokens_for()` in `src/models.rs` gives tool-calling models **4096** tokens |
| 58 | and non-tool models **512** (tool models need headroom because `<think>` blocks |
| 59 | eat the budget). The TUI startup load in `run_interactive` overrides this to |
| 60 | **8192**. Don't assume a single value. |
| 61 | |
| 62 | ### Why prefer 8B+ over 4B for editing |
| 63 | |
| 64 | 4B struggles with `edit_file`: it reads a file, then fails to reproduce the exact |
| 65 | `old_text` it just saw, spiralling into retry rounds that burn `max_tokens` on |
| 66 | `<think>` blocks and return nothing. 8B (or larger) lands edits far more reliably. |
| 67 | |
| 68 | ### bartowski GGUF naming convention |
| 69 | |
| 70 | bartowski's repos use the publisher name as a prefix with an underscore: |
| 71 | |
| 72 | | Constant | Value | |
| 73 | |----------|-------| |
| 74 | | `BARTOWSKI_QWEN3_8B_GGUF` | `"bartowski/Qwen_Qwen3-8B-GGUF"` | |
| 75 | | `QWEN3_8B_GGUF_FILE` | `"Qwen_Qwen3-8B-Q4_K_M.gguf"` | |
| 76 | | `BARTOWSKI_QWEN3_4B_GGUF` | `"bartowski/Qwen_Qwen3-4B-GGUF"` | |
| 77 | | `QWEN3_4B_GGUF_FILE` | `"Qwen_Qwen3-4B-Q4_K_M.gguf"` | |
| 78 | |
| 79 | These constants live in `onde/src/inference/models.rs`. |
| 80 | |
| 81 | --- |
| 82 | |
| 83 | ## Tools (9 total) |
| 84 | |
| 85 | Defined in `sigit/src/tools.rs` via `all_tools()`: |
| 86 | |
| 87 | | # | Tool | Parameters | Behavior | |
| 88 | |---|------|-----------|----------| |
| 89 | | 1 | `read_file` | `path` | Reads file contents, truncates at 10,000 chars | |
| 90 | | 2 | `create_directory` | `path` | Creates directory and all parents | |
| 91 | | 3 | `list_directory` | `path` | Lists entries with `[DIR]`/`[FILE]` prefix, dirs first | |
| 92 | | 4 | `search_files` | `pattern`, `path` (optional) | Recursive regex search, max 50 matches | |
| 93 | | 5 | `read_website` | `url` | Fetches HTTP/HTTPS, strips HTML, returns text | |
| 94 | | 6 | `create_file` | `path`, `content` | Creates new file (fails if exists) | |
| 95 | | 7 | `edit_file` | `path`, `old_text`, `new_text` | Find-and-replace (must match exactly once) | |
| 96 | | 8 | `delete_file` | `path` | Deletes file or empty directory | |
| 97 | | 9 | `run_command` | `command`, `cwd` (optional) | Shell command with 120s timeout | |
| 98 | |
| 99 | ### Async handling |
| 100 | |
| 101 | `execute_tool()` is `async`. Most tools run synchronously, except: |
| 102 | |
| 103 | - **`read_website`** — uses `tokio::task::spawn_blocking` because `reqwest::blocking::Client` panics inside a tokio runtime ("Cannot start a runtime from within a runtime") |
| 104 | |
| 105 | ### Tool gating by model |
| 106 | |
| 107 | In TUI mode, `run_inference_task()` takes a `tools_enabled: bool` parameter. When the picker item's `tool_calling` (from `models::is_tool_calling`) is `false`, an empty tool list is passed so the model doesn't receive tool schemas it can't use. |
| 108 | |
| 109 | In ACP mode, `handle_prompt` currently always passes the full tool set (`agent_tools_as_specs()`) regardless of the active model — there is no per-model gate on the ACP path. |
| 110 | |
| 111 | --- |
| 112 | |
| 113 | ## Architecture |
| 114 | |
| 115 | ### Layer 1: mistral.rs (model-level) |
| 116 | |
| 117 | - `RequestBuilder::set_tools(Vec<Tool>)` — attach tool definitions |
| 118 | - `RequestBuilder::set_tool_choice(ToolChoice::Auto)` — let model decide |
| 119 | - `QwenParser` detects `<tool_call>...</tool_call>` tags in output |
| 120 | - Grammar-constrained decoding forces valid JSON inside tool calls |
| 121 | - `<think>...</think>` reasoning is separated from tool calls by the reasoning parser |
| 122 | - Works identically for GGUF and full-precision models |
| 123 | |
| 124 | ### Layer 2: onde (engine-level) |
| 125 | |
| 126 | #### Key types (`onde/src/inference/types.rs`) |
| 127 | |
| 128 | | Type | Purpose | |
| 129 | |------|---------| |
| 130 | | `ToolDefinition` | `{ name, description, parameters_schema: String }` | |
| 131 | | `ToolCallRequest` | `{ id, function_name, arguments: String }` | |
| 132 | | `ToolResult` | `{ tool_call_id, content: String }` | |
| 133 | | `ToolAwareResult` | `{ text, tool_calls: Vec<ToolCallRequest>, duration_secs, ... }` | |
| 134 | |
| 135 | #### Key methods (`onde/src/inference/engine.rs`) |
| 136 | |
| 137 | | Method | Purpose | |
| 138 | |--------|---------| |
| 139 | | `send_message_with_tools(msg, &[ToolDefinition])` | Returns `ToolAwareResult` with possible tool calls | |
| 140 | | `send_tool_results(Vec<ToolResult>, Option<&[ToolDefinition]>)` | Feed results back; `None` forces text response | |
| 141 | |
| 142 | #### Layer 2.5: the `InferenceBackend` abstraction (`src/backend.rs`) |
| 143 | |
| 144 | siGit doesn't call the engine directly from the agent loop — it goes through the |
| 145 | `InferenceBackend` trait so on-device and cloud inference share one code path: |
| 146 | |
| 147 | | Item | Purpose | |
| 148 | |------|---------| |
| 149 | | `trait InferenceBackend` | `send_message_with_tools` / `send_tool_results` / `is_remote` | |
| 150 | | `LocalBackend` | wraps `Arc<ChatEngine>` — on-device inference | |
| 151 | | `OpenAiBackend` | OpenAI-compatible HTTP client — siGit Code Cloud tiers | |
| 152 | | `ToolSpec` | backend-level tool definition (`name`, `description`, `parameters_schema`) | |
| 153 | | `ToolCall` / `ToolResult` / `TurnResult` | backend-level request/result types | |
| 154 | |
| 155 | `handle_prompt` snapshots `self.backend.lock().await.clone()` once per turn so a |
| 156 | mid-turn model/tier switch can't split the conversation across backends. When |
| 157 | `backend.is_remote()` it skips the local model load + readiness wait. Cloud tiers |
| 158 | (`fast`, `balanced`, `large`) come from `src/provider.rs` and are sign-in gated. |
| 159 | |
| 160 | #### Internal details |
| 161 | |
| 162 | - `attach_tools()` converts `ToolDefinition` → mistral.rs `Tool`, sets `ToolChoice::Auto` and `strict: Some(true)` |
| 163 | - `parse_tool_calls()` extracts tool calls from `choice.message.tool_calls`, generates fallback IDs if empty |
| 164 | - `replay_history_with_tools()` uses `.enumerate()` for correct sequential `index` values |
| 165 | - Malformed `parameters_schema` JSON logs a warning instead of silently producing empty params |
| 166 | - Malformed tool call `arguments` JSON logs a warning for debugging |
| 167 | |
| 168 | ### Layer 3: siGit (agent-level) |
| 169 | |
| 170 | #### ACP session handling (`src/main.rs`) |
| 171 | |
| 172 | All session handlers (`load_session`, `fork_session`, `new_session`) do: |
| 173 | |
| 174 | 1. **Store `args.cwd`** in `session_cwd: Mutex<Option<PathBuf>>` |
| 175 | 2. **`std::env::set_current_dir(&args.cwd)`** — so relative paths in tool calls resolve correctly |
| 176 | 3. **`engine.clear_history()`** — siGit doesn't persist sessions |
| 177 | 4. **`engine.push_history(ChatMessage::system(...))`** — injects: *"The user's project working directory is {cwd}. Always use absolute paths..."* |
| 178 | |
| 179 | Without step 4, the model uses the process `cwd` (often `$HOME`) and creates files in the wrong directory. |
| 180 | |
| 181 | #### ACP content block handling (`prompt()`) |
| 182 | |
| 183 | The `prompt()` handler processes all ACP content block types: |
| 184 | |
| 185 | - **`ContentBlock::Text`** — passed through as-is |
| 186 | - **`ContentBlock::Resource` (EmbeddedResource)** — `TextResourceContents` inlined as `--- {uri} ---\n{text}\n--- end ---` |
| 187 | - **`ContentBlock::ResourceLink`** — `file://` URIs are read from disk. **Line range fragments** like `#L207:219` are parsed: the `#` fragment is stripped from the path, and only lines 207–219 are extracted and sent to the model |
| 188 | |
| 189 | Example: Zed sends `@ index.html (207:219)` as: |
| 190 | ``` |
| 191 | ResourceLink(name="index.html (207:219)", uri="file:///path/to/index.html#L207:219") |
| 192 | ``` |
| 193 | siGit parses this into path `/path/to/index.html` + lines 207–219. |
| 194 | |
| 195 | --- |
| 196 | |
| 197 | ## The Agentic Loop |
| 198 | |
| 199 | Both ACP mode (`SiGitAgent::handle_prompt()`) and TUI mode (`run_inference_task()`) |
| 200 | implement the same loop, driven through the active `InferenceBackend`: |
| 201 | |
| 202 | ``` |
| 203 | 1. backend.send_message_with_tools(user_text, &tools) → TurnResult |
| 204 | 2. while result.tool_calls is non-empty AND round < MAX_TOOL_ROUNDS (10): |
| 205 | a. For each tool_call: |
| 206 | - Log: → tool_name(arguments) |
| 207 | - Execute: tools::execute_tool(name, arguments).await |
| 208 | - Log: ← N chars |
| 209 | - Collect ToolResult { tool_call_id, content } |
| 210 | b. Decide next_tools: |
| 211 | - round < MAX_TOOL_ROUNDS → Some(&tools) (allow more calls) |
| 212 | - else → None (force text response) |
| 213 | c. backend.send_tool_results(results, next_tools) → TurnResult |
| 214 | 3. Strip <think> blocks (chat::strip_think_blocks), send final text to user |
| 215 | - Empty reply after tool rounds → log warning (ACP) or show error (TUI) |
| 216 | ``` |
| 217 | |
| 218 | In ACP mode the final text is sent as one `AgentMessageChunk`; the tool-calling |
| 219 | loop is not streamed token-by-token. |
| 220 | |
| 221 | --- |
| 222 | |
| 223 | ## System Prompt |
| 224 | |
| 225 | `main.rs` defines **two** prompts, picked by `system_prompt_for_model(tool_calling)`: |
| 226 | |
| 227 | - **`SYSTEM_PROMPT`** (~120 lines) — the full agentic prompt for tool-calling models: |
| 228 | - **Never tell the user to run commands** — use `run_command` tool instead |
| 229 | - **Can access websites** — use `read_website` tool (overrides RLHF refusal training) |
| 230 | - **Prefer absolute paths** in all tool arguments |
| 231 | - **Git operations** — always use `run_command` with absolute cwd |
| 232 | - **Always re-read a file before `edit_file`** — don't trust stale content |
| 233 | - **smbCloud domain knowledge** — auth boundaries, deploy flows, project structure |
| 234 | - **`SIMPLE_SYSTEM_PROMPT`** — a short prompt for non-tool models; the full one |
| 235 | wastes context and confuses them. |
| 236 | |
| 237 | The session `cwd` is injected as a separate system message at session creation time (not part of the static prompt). |
| 238 | |
| 239 | --- |
| 240 | |
| 241 | ## Model Cache |
| 242 | |
| 243 | Models are stored in the shared Onde App Group container on macOS: |
| 244 | |
| 245 | ``` |
| 246 | ~/Library/Group Containers/group.com.ondeinference.apps/models/hub/ |
| 247 | ``` |
| 248 | |
| 249 | `setup.rs` sets `HF_HOME` and `HF_HUB_CACHE` to point there at startup, so siGit reuses models downloaded by the Onde desktop app (and vice versa). |
| 250 | |
| 251 | --- |
| 252 | |
| 253 | ## Adding a New Tool |
| 254 | |
| 255 | 1. Add an `AgentTool` entry to `all_tools()` in `src/tools.rs` |
| 256 | 2. Add a match arm to `execute_tool()` — use `spawn_blocking` if the implementation blocks |
| 257 | 3. Write `exec_your_tool(arguments: &str) -> String` |
| 258 | 4. Update `test_all_tools_count` test (currently expects 9) |
| 259 | |
| 260 | No changes needed in onde or mistral.rs — tool definitions are passed dynamically. |
| 261 | |
| 262 | --- |
| 263 | |
| 264 | ## Adding a New Model |
| 265 | |
| 266 | 1. **`onde/src/inference/models.rs`** — add `pub const` for repo ID and GGUF filename, add to `SUPPORTED_MODELS` array and `SUPPORTED_MODEL_INFO` |
| 267 | 2. **`onde/src/inference/engine.rs`** — add `pub fn model_name() -> Self` constructor to `impl GgufModelConfig` |
| 268 | 3. **`sigit/src/models.rs`** — add a match arm to `model_id_to_config()` mapping the repo ID to the new constructor; if it supports tool calling, add the repo ID to `is_tool_calling()` (which also drives `max_tokens_for()`). The picker (`build_model_picker_items`) then surfaces it automatically. |
| 269 | 4. **`sigit/src/main.rs`** — only if you're changing the fallback default (`qwen25_3b()`) |
| 270 | |
| 271 | --- |
| 272 | |
| 273 | ## Debugging |
| 274 | |
| 275 | ### Log locations |
| 276 | |
| 277 | - **TUI mode:** `$TMPDIR/sigit.log` (e.g. `/var/folders/.../sigit.log`) |
| 278 | - **ACP mode (Zed):** `~/Library/Logs/Zed/Zed.log` — grep for `agent stderr:.*sigit` |
| 279 | |
| 280 | ### Key log patterns |
| 281 | |
| 282 | ``` |
| 283 | # Model loaded successfully |
| 284 | ChatEngine: model Qwen 3 8B loaded in 6.9s |
| 285 | |
| 286 | # Session cwd captured |
| 287 | load_session: id=..., cwd=/path/to/project, additional_directories=[...] |
| 288 | |
| 289 | # Tool call parsed by mistral.rs |
| 290 | ChatEngine: tool inference END — 12.3s — tool_calls: 1 |
| 291 | |
| 292 | # Tool executed |
| 293 | → read_file({"path":"/absolute/path/to/file.rs"}) |
| 294 | ← 6506 chars |
| 295 | |
| 296 | # Tool result sent back |
| 297 | ChatEngine: tool results inference START — 1 results |
| 298 | |
| 299 | # Model returned empty (exhausted max_tokens on thinking) |
| 300 | model returned empty reply after 7 tool round(s) |
| 301 | |
| 302 | # ResourceLink received from Zed |
| 303 | block[1]: ResourceLink(name=index.html (207:219), uri=file:///path/to/index.html#L207:219) |
| 304 | |
| 305 | # ResourceLink read failed (fragment not stripped — old bug, now fixed) |
| 306 | could not read ResourceLink file:///path/to/index.html#L207:219: No such file or directory |
| 307 | ``` |
| 308 | |
| 309 | ### Common issues |
| 310 | |
| 311 | | Symptom | Cause | Fix | |
| 312 | |---------|-------|-----| |
| 313 | | Model says "I cannot access websites" | RLHF refusal override not in system prompt | System prompt now has CRITICAL block about `read_website` | |
| 314 | | `0 tool call(s)` for every prompt | Wrong model loaded (Qwen 2.5) | Check log for `loading GGUF model` — must be Qwen 3 | |
| 315 | | `edit_file` returns `← 161 chars` repeatedly | `old_text not found` — model can't match exact text | Use Qwen 3 8B (not 4B); consider line-based edit tool | |
| 316 | | Files created in wrong directory | `cwd` not captured from ACP session | Session handlers must call `set_current_dir` + `push_history` with cwd | |
| 317 | | `@ file.html (207:219)` context missing | `#L207:219` fragment not stripped from file path | `prompt()` now parses URI fragments and extracts line ranges | |
| 318 | | `read_website` panics/hangs | `reqwest::blocking` inside tokio runtime | `exec_read_website` wrapped in `spawn_blocking` | |
| 319 | | Empty reply after many tool rounds | Model exhausted `max_tokens` on `<think>` blocks | Set `max_tokens: 8192`; 8B model wastes fewer tokens on thinking | |
| 320 | |
| 321 | --- |
| 322 | |
| 323 | ## Cargo Dependency Note |
| 324 | |
| 325 | `onde` is published on crates.io; `sigit/Cargo.toml` pins it: |
| 326 | |
| 327 | ```toml |
| 328 | onde = "1.1.2" |
| 329 | ``` |
| 330 | |
| 331 | The Qwen 3 / Coder-7B constructors (`qwen3_8b()`, etc.) ship in that release. For |
| 332 | local SDK development against an `onde` checkout, swap to a path dep |
| 333 | (`onde = { path = "../onde" }`) — but the committed form must stay the crates.io |
| 334 | version so CI/release builds resolve. |
| 335 | |
| 336 | --- |
| 337 | |
| 338 | ## File Map |
| 339 | |
| 340 | | File | What it does | |
| 341 | |------|-------------| |
| 342 | | `sigit/src/tools.rs` | 9 tool schemas (`all_tools()`), `execute_tool()` dispatch, all `exec_*` implementations | |
| 343 | | `sigit/src/main.rs` | `SYSTEM_PROMPT`, `SiGitAgent` struct with `session_cwd` + `backend`, ACP handlers (cwd + push_history), `handle_prompt()` content-block parsing + tool loop, `MAX_TOOL_ROUNDS`, ACP builder wiring | |
| 344 | | `sigit/src/backend.rs` | `InferenceBackend` trait, `LocalBackend`, `OpenAiBackend`, `ToolSpec`/`ToolCall`/`ToolResult`/`TurnResult` | |
| 345 | | `sigit/src/models.rs` | `ModelPickerItem`, `model_id_to_config()`, `is_tool_calling()`, `max_tokens_for()`, `build_model_picker_items()` / `local_picker_items()` | |
| 346 | | `sigit/src/provider.rs` | `CLOUD_TIERS`, `cloud_tier_provider()`, cloud endpoint config | |
| 347 | | `sigit/src/chat.rs` | TUI app, model picker UI (uses `build_model_picker_items`), `run_inference_task()` with `tools_enabled` gate, TUI tool loop | |
| 348 | | `sigit/src/setup.rs` | HF cache setup (shared App Group container), `startup_model_selection()` | |
| 349 | | `onde/src/inference/types.rs` | `ToolDefinition`, `ToolCallRequest`, `ToolResult`, `ToolAwareResult` | |
| 350 | | `onde/src/inference/engine.rs` | `send_message_with_tools()`, `send_tool_results()`, `attach_tools()`, `parse_tool_calls()`, `replay_history_with_tools()`, `GgufModelConfig::qwen3_8b()` | |
| 351 | | `onde/src/inference/models.rs` | Model constants and `SUPPORTED_MODELS` array | |