@ondeinference / onde / commits / 5cd3dee

Match tool-calling capability to what Onde has actually tested

The capability check matched "qwen3" as a substring of the whole repo id, so acme/my-qwen3-finetune came back Supported even though the doc comment two lines above promised custom repositories stay Unknown. Onde has never tested that model. It now checks an explicit TOOL_CALLING_MODELS list, with a test that fails if a catalogued Qwen 3 repo is ever left out of it. The three answers needed saying out loud too. Unsupported reads like "this cannot work" but means "we have not verified it", and since Ed turns it into a hard error, a model Onde has never seen is currently treated more permissively than one it ships. That is a deliberate call, and now a written one. tool_calling_support returned Unknown with nothing loaded, putting an empty engine in the same bucket as an uncatalogued repo. It returns Unsupported now, as does the fallback on platforms where inference cannot run at all. record_tool_results wrote whatever it was handed. A ToolResult with no AssistantToolCall ahead of it builds a message sequence the model cannot read, and that lands on the next turn rather than the one that caused it. It now walks back to the assistant turn being answered, drops results for calls that were never requested or are already recorded, and returns how many it kept. The Dart and React Native changelogs claimed the feature, but neither package exposes it: the uniffi surface has no tool exports, and the flutter_rust_bridge api only mirrors tool_calls on results. Both now say the version is aligned and the additions are Rust only. Also moved tool_calling_support out of the Construction section it had landed in, down to the tool-aware inference block where it belongs. Co-Authored-By: siGit Code <noreply@sigit.si>

Seto Elkahfi committed Sep 9, 2026 at 23:58 UTC 5cd3deea03050c95258f70cbd5f1975b9b86ffa7

4 files changed +159 -35

sdk/dart/CHANGELOG.md

+3 -1

@@ -1,6 +1,8 @@

1 ## 1.2.5

3 -- Added engine-level tool-calling capability detection and tool-result history support for agent SDKs

3 +- Version aligned with the Rust crate. The 1.2.5 tool-calling additions

4 + (`ChatEngine::tool_calling_support`, `ChatEngine::record_tool_results`) are

5 + Rust-only and are not yet exposed through this package.

7 ## 1.2.4

sdk/react-native/CHANGELOG.md

+3 -1

@@ -1,6 +1,8 @@

1 ## 1.2.5

3 -- Added engine-level tool-calling capability detection and tool-result history support for agent SDKs

3 +- Version aligned with the Rust crate. The 1.2.5 tool-calling additions

4 + (`ChatEngine::tool_calling_support`, `ChatEngine::record_tool_results`) are

5 + Rust-only and are not yet exposed through this package.

7 ## 1.2.4

src/inference/engine.rs

+87 -26

@@ -114,7 +114,7 @@ use mistralrs::{

114 target_os = "linux",

115 target_os = "android"

116 ))]

117 -use std::collections::HashMap;

117 +use std::collections::{HashMap, HashSet};

118 #[cfg(any(

119 target_os = "macos",

120 target_os = "ios",

@@ -278,22 +278,6 @@ pub struct ChatEngine {

278 target_os = "android"

279 ))]

280 impl ChatEngine {

281 - /// Whether the currently loaded model is verified for structured tool calls.

282 - pub async fn tool_calling_support(&self) -> super::types::ToolCallingSupport {

283 - use super::types::ToolCallingSupport;

284 -

285 - let guard = self.inner.lock().await;

286 - let Some(loaded) = guard.as_ref() else {

287 - return ToolCallingSupport::Unknown;

288 - };

289 - let model_id = match &loaded.config {

290 - LoadedModelConfig::Gguf(config) => &config.model_id,

291 - LoadedModelConfig::Uqff(config) => &config.model_id,

292 - #[cfg(target_os = "macos")]

293 - LoadedModelConfig::Isq(config) => &config.model_id,

294 - };

295 - super::models::tool_calling_support(model_id)

296 - }

281 // ── Construction ─────────────────────────────────────────────────────

282

283 /// Create a new engine with no model loaded and no Onde app association.

@@ -1334,6 +1318,28 @@ impl ChatEngine {

1318

1319 // ── Tool-aware inference (Rust-only) ─────────────────────────────────

1320

1321 + /// Whether the currently loaded model is verified for structured tool calls.

1322 + ///

1323 + /// Returns [`Unsupported`](super::types::ToolCallingSupport::Unsupported) when no model

1324 + /// is loaded: an engine with nothing in it cannot produce tool calls, and

1325 + /// reporting `Unknown` there would conflate an empty engine with a model

1326 + /// repository Onde has not catalogued.

1327 + pub async fn tool_calling_support(&self) -> super::types::ToolCallingSupport {

1328 + use super::types::ToolCallingSupport;

1329 +

1330 + let guard = self.inner.lock().await;

1331 + let Some(loaded) = guard.as_ref() else {

1332 + return ToolCallingSupport::Unsupported;

1333 + };

1334 + let model_id = match &loaded.config {

1335 + LoadedModelConfig::Gguf(config) => &config.model_id,

1336 + LoadedModelConfig::Uqff(config) => &config.model_id,

1337 + #[cfg(target_os = "macos")]

1338 + LoadedModelConfig::Isq(config) => &config.model_id,

1339 + };

1340 + super::models::tool_calling_support(model_id)

1341 + }

1342 +

1343 /// Send a user message with tool definitions available. Non-streaming.

1344 ///

1345 /// If the model decides to call tools, [`ToolAwareResult::tool_calls`]

@@ -1503,16 +1509,67 @@ impl ChatEngine {

1509 /// Agent hosts use this when a turn is cancelled after the assistant has

1510 /// requested tools. Recording a result for every request keeps the history

1511 /// structurally valid for the next turn.

1506 - pub async fn record_tool_results(&self, results: Vec<ToolResult>) {

1512 + ///

1513 + /// Only results answering a call the assistant actually requested, and not

1514 + /// already recorded, are appended. Anything else is dropped with a warning:

1515 + /// a `ToolResult` with no matching `AssistantToolCall` ahead of it builds a

1516 + /// message sequence the model cannot interpret, and silently writing one

1517 + /// corrupts every subsequent turn rather than the current one. Returns how

1518 + /// many results were appended.

1519 + pub async fn record_tool_results(&self, results: Vec<ToolResult>) -> usize {

1520 let mut guard = self.inner.lock().await;

1508 - if let Some(loaded) = guard.as_mut() {

1509 - for result in results {

1510 - loaded.history.push(HistoryEntry::ToolResult {

1511 - tool_call_id: result.tool_call_id,

1512 - content: result.content,

1513 - });

1521 + let Some(loaded) = guard.as_mut() else {

1522 + return 0;

1523 + };

1524 +

1525 + // Walk back over results already recorded for this round to find the

1526 + // assistant turn they answer. A `Text` entry means the round is closed.

1527 + let mut requested: Vec<String> = Vec::new();

1528 + let mut seen: HashSet<String> = HashSet::new();

1529 + for entry in loaded.history.iter().rev() {

1530 + match entry {

1531 + HistoryEntry::ToolResult { tool_call_id, .. } => {

1532 + seen.insert(tool_call_id.clone());

1533 + }

1534 + HistoryEntry::AssistantToolCall { tool_calls, .. } => {

1535 + requested = tool_calls.iter().map(|call| call.id.clone()).collect();

1536 + break;

1537 + }

1538 + HistoryEntry::Text(_) => break,

1539 + }

1540 + }

1541 +

1542 + if requested.is_empty() {

1543 + log::warn!(

1544 + "ChatEngine: record_tool_results found no assistant tool call to answer — dropping {} result(s)",

1545 + results.len()

1546 + );

1547 + return 0;

1548 + }

1549 +

1550 + let mut appended = 0;

1551 + for result in results {

1552 + if !requested.contains(&result.tool_call_id) {

1553 + log::warn!(

1554 + "ChatEngine: dropping tool result for unrequested call {:?}",

1555 + result.tool_call_id

1556 + );

1557 + continue;

1558 }

1559 + if !seen.insert(result.tool_call_id.clone()) {

1560 + log::warn!(

1561 + "ChatEngine: dropping duplicate tool result for call {:?}",

1562 + result.tool_call_id

1563 + );

1564 + continue;

1565 + }

1566 + loaded.history.push(HistoryEntry::ToolResult {

1567 + tool_call_id: result.tool_call_id,

1568 + content: result.content,

1569 + });

1570 + appended += 1;

1571 }

1572 + appended

1573 }

1574

1575 /// Stream tool execution results back to the model.

@@ -2070,10 +2127,14 @@ impl ChatEngine {

2127 pub async fn push_history(&self, _message: ChatMessage) {}

2128

2129 pub async fn tool_calling_support(&self) -> super::types::ToolCallingSupport {

2073 - super::types::ToolCallingSupport::Unknown

2130 + // Inference cannot run on this platform at all, so tool calling is not

2131 + // merely unverified — it is unavailable.

2132 + super::types::ToolCallingSupport::Unsupported

2133 }

2134

2076 - pub async fn record_tool_results(&self, _results: Vec<ToolResult>) {}

2135 + pub async fn record_tool_results(&self, _results: Vec<ToolResult>) -> usize {

2136 + 0

2137 + }

2138

2139 pub async fn send_message(

2140 &self,

src/inference/models.rs

+66 -7

@@ -231,18 +231,44 @@ pub const SUPPORTED_MODELS: &[&str] = &[

231 THEBLOKE_DEEPSEEK_CODER_6_7B_INSTRUCT_GGUF,

232 ];

233

234 +/// Catalogued models Onde has verified for structured tool calling.

235 +///

236 +/// Every Qwen 3 repo routes through mistral.rs' Qwen tool parser. Qwen 2.5

237 +/// Coder 7B is the one Qwen 2.5 model in the catalogue verified alongside it.

238 +/// Entries must also appear in [`SUPPORTED_MODELS`].

239 +pub const TOOL_CALLING_MODELS: &[&str] = &[

240 + BARTOWSKI_QWEN3_0_6B_GGUF,

241 + BARTOWSKI_QWEN3_1_7B_GGUF,

242 + BARTOWSKI_QWEN3_4B_GGUF,

243 + BARTOWSKI_QWEN3_8B_GGUF,

244 + BARTOWSKI_QWEN3_14B_GGUF,

245 + BARTOWSKI_QWEN3_32B_GGUF,

246 + BARTOWSKI_QWEN3_4B_INSTRUCT_2507_GGUF,

247 + BARTOWSKI_QWEN3_4B_THINKING_2507_GGUF,

248 + BARTOWSKI_QWEN3_30B_A3B_INSTRUCT_2507_GGUF,

249 + BARTOWSKI_QWEN25_CODER_7B_INSTRUCT_GGUF,

250 +];

251 +

252 /// Return Onde's verified tool-calling capability for a model repository.

253 ///

236 -/// Qwen 3 models use mistral.rs' Qwen tool parser. Qwen 2.5 Coder 7B is the

237 -/// one verified Qwen 2.5 model in the built-in catalogue that also supports

238 -/// structured tool calls. Custom repositories remain `Unknown` rather than

239 -/// being rejected solely because Onde has not catalogued them.

254 +/// The three answers mean different things, and callers should treat them

255 +/// differently:

256 +///

257 +/// - `Supported` — catalogued and verified. Use tools freely.

258 +/// - `Unsupported` — catalogued, and *not* verified for tool calling. This is

259 +/// Onde's own test coverage talking, not an upstream capability claim: some

260 +/// of these models ship tool-call templates and may well work. A caller that

261 +/// wants to try one anyway should, but should not present it as supported.

262 +/// - `Unknown` — not in the catalogue at all. Onde has no opinion. Callers may

263 +/// try these, but should surface that the capability is unverified.

264 +///

265 +/// Matching is exact. A custom repository whose name merely resembles a

266 +/// catalogued one (`acme/my-qwen3-finetune`) is `Unknown`, not `Supported` —

267 +/// Onde has not tested it, and a name is not a verification.

268 pub fn tool_calling_support(model_id: &str) -> super::types::ToolCallingSupport {

269 use super::types::ToolCallingSupport;

270

243 - if model_id.to_ascii_lowercase().contains("qwen3")

244 - || model_id == BARTOWSKI_QWEN25_CODER_7B_INSTRUCT_GGUF

245 - {

271 + if TOOL_CALLING_MODELS.contains(&model_id) {

272 ToolCallingSupport::Supported

273 } else if SUPPORTED_MODELS.contains(&model_id) {

274 ToolCallingSupport::Unsupported

@@ -456,6 +482,39 @@ mod tests {

482 tool_calling_support("acme/custom-agent-model"),

483 ToolCallingSupport::Unknown

484 );

485 + // A custom repo that merely looks like a catalogued one is still

486 + // uncatalogued. Onde has not verified it, so it must not claim to have.

487 + assert_eq!(

488 + tool_calling_support("acme/my-qwen3-finetune"),

489 + ToolCallingSupport::Unknown

490 + );

491 + }

492 +

493 + /// A tool-calling entry that is not a supported model would be dead: the

494 + /// engine only ever asks about repos it has actually loaded.

495 + #[test]

496 + fn every_tool_calling_model_is_a_supported_model() {

497 + for model_id in TOOL_CALLING_MODELS {

498 + assert!(

499 + SUPPORTED_MODELS.contains(model_id),

500 + "{model_id} is listed for tool calling but is not a supported model"

501 + );

502 + }

503 + }

504 +

505 + /// Catches the drift the old substring match papered over: a Qwen 3 repo

506 + /// added to the catalogue without being added to the tool-calling list

507 + /// would silently report `Unsupported`.

508 + #[test]

509 + fn every_catalogued_qwen3_model_supports_tool_calling() {

510 + for model_id in SUPPORTED_MODELS {

511 + if model_id.to_ascii_lowercase().contains("qwen3") {

512 + assert!(

513 + TOOL_CALLING_MODELS.contains(model_id),

514 + "{model_id} is a catalogued Qwen 3 repo but is missing from TOOL_CALLING_MODELS"

515 + );

516 + }

517 + }

518 }

519

520 /// Every supported model must have a corresponding display-metadata entry