name: tool-calling

#description: Implement or debug tool calling in siGit Code across the app, Onde Inference, and mistral.rs. Use when working on tool schemas, execution loops, model support, session cwd handling, or tool-call troubleshooting.

#Skill: Tool Calling in siGit Code

#Overview

siGit Code supports agentic tool calling — the LLM invokes tools (read/write files, run commands, read websites) to operate on the user's codebase. This works in both interactive TUI mode and ACP server mode (Zed editor).

Tool calling spans these layers:

siGit (agent loop + tool execution)
  → InferenceBackend (src/backend.rs — LocalBackend or OpenAiBackend/cloud)
    → onde ChatEngine (tool-aware API)   ── for LocalBackend
      → mistral.rs (model inference + tool call parsing)
    └ OpenAI-compatible HTTP endpoint    ── for OpenAiBackend (siGit Code Cloud)

The agent loop talks to an InferenceBackend trait object, not the engine directly. LocalBackend wraps the on-device ChatEngine; OpenAiBackend calls a remote OpenAI-compatible endpoint (the siGit Code Cloud tiers). Both implement send_message_with_tools / send_tool_results, so the loop is identical.

#Model Requirement

Tool calling needs a model mistral.rs has a tool-call parser for. The supported set is the Qwen 3 family (<tool_call>...</tool_call> XML) plus Qwen 2.5 Coder 7B. Plain Qwen 2.5 and the smaller Qwen 2.5 Coder variants do NOT support tool calling. The authoritative list is is_tool_calling() in src/models.rs.

#Default model

There is no hardcoded default model. Startup uses the saved selection (setup::startup_model_selection), then the first complete locally-cached model, falling back to GgufModelConfig::platform_default() (Qwen 2.5 3B on macOS) when nothing is cached. The TUI/ACP code in main.rs uses qwen25_3b() as that final fallback. Users pick a tool-calling model via the /models picker.

#max_tokens

max_tokens_for() in src/models.rs gives tool-calling models 4096 tokens and non-tool models 512 (tool models need headroom because <think> blocks eat the budget). The TUI startup load in run_interactive overrides this to 8192. Don't assume a single value.

#Why prefer 8B+ over 4B for editing

4B struggles with edit_file: it reads a file, then fails to reproduce the exact old_text it just saw, spiralling into retry rounds that burn max_tokens on <think> blocks and return nothing. 8B (or larger) lands edits far more reliably.

#bartowski GGUF naming convention

bartowski's repos use the publisher name as a prefix with an underscore:

These constants live in onde/src/inference/models.rs.

#Tools (9 total)

Defined in sigit/src/tools.rs via all_tools():

#Async handling

execute_tool() is async. Most tools run synchronously, except:

• read_website — uses tokio::task::spawn_blocking because reqwest::blocking::Client panics inside a tokio runtime ("Cannot start a runtime from within a runtime")

#Tool gating by model

In TUI mode, run_inference_task() takes a tools_enabled: bool parameter. When the picker item's tool_calling (from models::is_tool_calling) is false, an empty tool list is passed so the model doesn't receive tool schemas it can't use.

In ACP mode, handle_prompt currently always passes the full tool set (agent_tools_as_specs()) regardless of the active model — there is no per-model gate on the ACP path.

#Architecture

#Layer 1: mistral.rs (model-level)

• RequestBuilder::set_tools(Vec<Tool>) — attach tool definitions
• RequestBuilder::set_tool_choice(ToolChoice::Auto) — let model decide
• QwenParser detects <tool_call>...</tool_call> tags in output
• Grammar-constrained decoding forces valid JSON inside tool calls
• <think>...</think> reasoning is separated from tool calls by the reasoning parser
• Works identically for GGUF and full-precision models

#Layer 2: onde (engine-level)

#Key types (onde/src/inference/types.rs)

#Key methods (onde/src/inference/engine.rs)

#Layer 2.5: the InferenceBackend abstraction (src/backend.rs)

siGit doesn't call the engine directly from the agent loop — it goes through the InferenceBackend trait so on-device and cloud inference share one code path:

handle_prompt snapshots self.backend.lock().await.clone() once per turn so a mid-turn model/tier switch can't split the conversation across backends. When backend.is_remote() it skips the local model load + readiness wait. Cloud tiers (fast, balanced, large) come from src/provider.rs and are sign-in gated.

#Internal details

• attach_tools() converts ToolDefinition → mistral.rs Tool, sets ToolChoice::Auto and strict: Some(true)
• parse_tool_calls() extracts tool calls from choice.message.tool_calls, generates fallback IDs if empty
• replay_history_with_tools() uses .enumerate() for correct sequential index values
• Malformed parameters_schema JSON logs a warning instead of silently producing empty params
• Malformed tool call arguments JSON logs a warning for debugging

#Layer 3: siGit (agent-level)

#ACP session handling (src/main.rs)

All session handlers (load_session, fork_session, new_session) do:

1. Store args.cwd in session_cwd: Mutex<Option<PathBuf>>
2. std::env::set_current_dir(&args.cwd) — so relative paths in tool calls resolve correctly
3. engine.clear_history() — siGit doesn't persist sessions
4. engine.push_history(ChatMessage::system(...)) — injects: "The user's project working directory is {cwd}. Always use absolute paths..."

Without step 4, the model uses the process cwd (often $HOME) and creates files in the wrong directory.

#ACP content block handling (prompt())

The prompt() handler processes all ACP content block types:

• ContentBlock::Text — passed through as-is
• ContentBlock::Resource (EmbeddedResource) — TextResourceContents inlined as --- {uri} ---\n{text}\n--- end ---
• ContentBlock::ResourceLink — file:// URIs are read from disk. Line range fragments like #L207:219 are parsed: the # fragment is stripped from the path, and only lines 207–219 are extracted and sent to the model

Example: Zed sends @ index.html (207:219) as: ResourceLink(name="index.html (207:219)", uri="file:///path/to/index.html#L207:219") siGit parses this into path /path/to/index.html + lines 207–219.

#The Agentic Loop

Both ACP mode (SiGitAgent::handle_prompt()) and TUI mode (run_inference_task()) implement the same loop, driven through the active InferenceBackend:

1. backend.send_message_with_tools(user_text, &tools) → TurnResult
2. while result.tool_calls is non-empty AND round < MAX_TOOL_ROUNDS (10):
   a. For each tool_call:
      - Log: → tool_name(arguments)
      - Execute: tools::execute_tool(name, arguments).await
      - Log: ← N chars
      - Collect ToolResult { tool_call_id, content }
   b. Decide next_tools:
      - round < MAX_TOOL_ROUNDS → Some(&tools)  (allow more calls)
      - else → None  (force text response)
   c. backend.send_tool_results(results, next_tools) → TurnResult
3. Strip <think> blocks (chat::strip_think_blocks), send final text to user
   - Empty reply after tool rounds → log warning (ACP) or show error (TUI)

In ACP mode the final text is sent as one AgentMessageChunk; the tool-calling loop is not streamed token-by-token.

#System Prompt

main.rs defines two prompts, picked by system_prompt_for_model(tool_calling):

• SYSTEM_PROMPT (~120 lines) — the full agentic prompt for tool-calling models:
  • Never tell the user to run commands — use run_command tool instead
  • Can access websites — use read_website tool (overrides RLHF refusal training)
  • Prefer absolute paths in all tool arguments
  • Git operations — always use run_command with absolute cwd
  • Always re-read a file before edit_file — don't trust stale content
  • smbCloud domain knowledge — auth boundaries, deploy flows, project structure
• SIMPLE_SYSTEM_PROMPT — a short prompt for non-tool models; the full one wastes context and confuses them.

The session cwd is injected as a separate system message at session creation time (not part of the static prompt).

#Model Cache

Models are stored in the shared Onde App Group container on macOS:

~/Library/Group Containers/group.com.ondeinference.apps/models/hub/

setup.rs sets HF_HOME and HF_HUB_CACHE to point there at startup, so siGit reuses models downloaded by the Onde desktop app (and vice versa).

#Adding a New Tool

1. Add an AgentTool entry to all_tools() in src/tools.rs
2. Add a match arm to execute_tool() — use spawn_blocking if the implementation blocks
3. Write exec_your_tool(arguments: &str) -> String
4. Update test_all_tools_count test (currently expects 9)

No changes needed in onde or mistral.rs — tool definitions are passed dynamically.

#Adding a New Model

1. onde/src/inference/models.rs — add pub const for repo ID and GGUF filename, add to SUPPORTED_MODELS array and SUPPORTED_MODEL_INFO
2. onde/src/inference/engine.rs — add pub fn model_name() -> Self constructor to impl GgufModelConfig
3. sigit/src/models.rs — add a match arm to model_id_to_config() mapping the repo ID to the new constructor; if it supports tool calling, add the repo ID to is_tool_calling() (which also drives max_tokens_for()). The picker (build_model_picker_items) then surfaces it automatically.
4. sigit/src/main.rs — only if you're changing the fallback default (qwen25_3b())

#Debugging

#Log locations

• TUI mode: $TMPDIR/sigit.log (e.g. /var/folders/.../sigit.log)
• ACP mode (Zed): ~/Library/Logs/Zed/Zed.log — grep for agent stderr:.*sigit

#Key log patterns

# Model loaded successfully
ChatEngine: model Qwen 3 8B loaded in 6.9s

# Session cwd captured
load_session: id=..., cwd=/path/to/project, additional_directories=[...]

# Tool call parsed by mistral.rs
ChatEngine: tool inference END — 12.3s — tool_calls: 1

# Tool executed
→ read_file({"path":"/absolute/path/to/file.rs"})
← 6506 chars

# Tool result sent back
ChatEngine: tool results inference START — 1 results

# Model returned empty (exhausted max_tokens on thinking)
model returned empty reply after 7 tool round(s)

# ResourceLink received from Zed
block[1]: ResourceLink(name=index.html (207:219), uri=file:///path/to/index.html#L207:219)

# ResourceLink read failed (fragment not stripped — old bug, now fixed)
could not read ResourceLink file:///path/to/index.html#L207:219: No such file or directory

#Common issues

#Cargo Dependency Note

onde is published on crates.io; sigit/Cargo.toml pins it:

onde = "1.1.2"

The Qwen 3 / Coder-7B constructors (qwen3_8b(), etc.) ship in that release. For local SDK development against an onde checkout, swap to a path dep (onde = { path = "../onde" }) — but the committed form must stay the crates.io version so CI/release builds resolve.

#File Map