@ondeinference / ed / commits / f1a5d11

chore: add agentic resources (.agents/AGENTS.md, symlinks)

Canonical agent instructions live in .agents/AGENTS.md. Root AGENTS.md and CLAUDE.md are symlinks pointing there. Content is tuned to the onde-ed workspace: crate map, architecture rules, LLM provider config, FFI/Tauri boundaries, and commit conventions. Co-Authored-By: siGit Code v1.6.6-acp <noreply@sigit.si>

Seto Elkahfi committed Oct 10, 2026 at 08:17 UTC f1a5d116fad1b65720fd54c9575e04b77100332d

5 files changed +84 -1820

.agents/AGENTS.md

+82 -586

@@ -1,617 +1,113 @@

1 -# Onde Inference — AGENTS.md

1 +# onde-ed – Agent Instructions

3 -> **AI agent reference for the `onde` repository and the `onde-swift` Swift SDK.**

4 -> Keep this file accurate as the codebase evolves.

3 +## What this repo is

4 +`onde-ed` is a **Rust Cargo workspace** providing the embeddable agentic core

5 +used across Onde products (macOS, iOS, Tauri, CLI). The workspace is organised

6 +into focused crates; do not merge them.

8 ---

8 -## What Is Onde?

10 +## Crate map

10 -Onde is an **on-device LLM inference SDK family** built around a shared Rust core. It targets Apple silicon (iOS, tvOS, macOS) plus Android, Windows, and Linux, and ships SDKs for Rust, Swift, Kotlin Multiplatform, Flutter, and React Native. It wraps [mistral.rs](https://github.com/EricLBuehler/mistral.rs) behind platform-friendly APIs with automatic model selection, HuggingFace Hub downloads, cache management, and Metal acceleration where available.

11 -

12 -- **Website:** https://ondeinference.com

13 -- **Rust crate:** https://crates.io/crates/onde

14 -- **Swift package:** https://github.com/ondeinference/onde-swift

15 -- **Maven Central:** https://central.sonatype.com/artifact/com.ondeinference/onde-inference

16 -- **Flutter package:** https://pub.dev/packages/onde_inference

17 -- **React Native package:** https://www.npmjs.com/package/@ondeinference/react-native

18 -- **In production:** [Splitfire AB apps on the Apple AppStore](https://apps.apple.com/se/developer/splitfire-ab/id1831430993)

12 +| Crate | Purpose |

13 +|---|---|

14 +| `ed-agent` | Core library: `Ed` struct, lifecycle, `send`/`stream`/`run`, tool approval, `EventSink`, `SessionStore` seams |

15 +| `ed-acp` | ACP v1 server: `Profile` trait, `serve`/`serve_stdio`, sessions, LLM streaming, tool routing, MCP |

16 +| `ed-mcp` | MCP integration over `rmcp`: tool namespacing (`mcp__<server>__<tool>`, max 64 bytes), `SharedServer`, `McpExecutor`, timeouts, cancellation |

17 +| `ed-acp-tui` | ratatui TUI frontend for ACP agents over stdio |

18 +| `ed-acp-testkit` | Mock Onde Cloud endpoint + conformance test suite |

19 +| `ed-agent-ffi` | UniFFI bridge → Apple XCFramework (`FfiEd`, `FfiToolExecutor`, `FfiApprovalHandler`, `FfiEventListener`) |

20 +| `ed-agent-tauri` | Tauri command bindings that emit events to the webview |

22 ---

22 -## Repository Layout

23 -

24 -```

25 -onde/

26 -├── src/

27 -│ ├── lib.rs # Crate root — uniffi::setup_scaffolding!()

28 -│ ├── hf_cache.rs # HuggingFace Hub cache: list, download, delete, diagnose, repair

29 -│ └── inference/

30 -│ ├── mod.rs # Module exports + top-level re-exports

31 -│ ├── engine.rs # ChatEngine — Rust-native API (generics, mpsc, tool calling)

32 -│ ├── ffi.rs # OndeChatEngine — UniFFI Object (FFI-safe, Arc-wrapped)

33 -│ ├── models.rs # Model ID constants + SupportedModelInfo metadata

34 -│ ├── token.rs # HF token resolution: build-time literal vs cache file

35 -│ └── types.rs # Shared types: ChatMessage, SamplingConfig, InferenceResult, ToolCallInfo, etc.

36 -├── sdk/

37 -│ ├── dart/ # Flutter/Dart package + example app + FRB Rust bridge

38 -│ ├── gem/ # Ruby native extension (Magnus)

39 -│ ├── kotlin/ # Kotlin Multiplatform package (Android + JVM)

40 -│ ├── react-native/ # Expo module wrapping the Rust core for iOS/Android

41 -│ └── python/ # Python bindings (maturin + uniffi)

42 -├── generated/ # UniFFI-generated headers and Swift glue (git-ignored)

43 -├── uniffi-bindgen/ # Standalone bindgen binary crate (pinned uniffi =0.31.0)

44 -├── .github/

45 -│ ├── workflows/

46 -│ │ ├── release-sdk-swift.yml # CI: tag push → build XCFramework → GitHub Release → update onde-swift

47 -│ │ ├── release-sdk-kotlin.yml # CI: tag push → build Android/JVM artifacts → publish to Maven Central

48 -│ │ ├── release-sdk-dart.yml # CI: tag push → publish Flutter package to pub.dev

49 -│ │ ├── release-sdk-npm.yml # CI: tag push → publish React Native package to npm

50 -│ │ └── release-sdk-rust.yml # CI: tag push → publish Rust crate to crates.io

51 -│ └── scripts/

52 -│ └── build-swift-xcframework.sh # Local/CI XCFramework assembly script

53 -├── .cargo/config.toml # Target-specific rustflags (fp16, linker overrides)

54 -├── Cargo.toml # Platform-conditional mistralrs deps

55 -├── build.rs # tvOS ___chkstk_darwin assembly stub

56 -├── scripts/

57 -│ └── tvos_chkstk.s # No-op arm64 stub for missing tvOS symbol

58 -├── uniffi.toml # UniFFI binding config

59 -└── docs/

60 - ├── dev.md # Developer guide (build, architecture, platform table)

61 - ├── swift-package.md # Swift API reference + XCFramework build steps

62 - ├── distribution.md # Release process for all registries

63 - └── ruby-gem.md # Ruby gem API reference

64 -```

65 -

66 -```

67 -onde-swift/ # Swift Package Manager wrapper repo

68 -├── Package.swift # Declares OndeFramework.xcframework binary target

69 -└── Sources/

70 - └── Onde/

71 - └── onde.swift # UniFFI-generated Swift glue (do NOT edit manually)

72 -```

73 -

74 ----

24 +## Non-negotiable architecture rules

76 -## Architecture

26 +1. **One `Ed` instance, one engine, one model.** `Ed` is `Send + Sync`, not

27 + `Clone`. Never try to clone or share it across threads without `Arc`.

78 -### Layer Diagram

29 +2. **`Profile` is the product seam.** A product crate (or binary) implements

30 + `Profile` (`info`, `system_prompt`, `toolsets`, `builtin_mcp_servers`).

31 + `ed-acp` handles the ACP server, session management, and LLM glue. Do not

32 + put product-specific logic inside `ed-acp`.

80 -```

81 -┌──────────────────────────────────────────────────────────┐

82 -│ Swift (iOS / tvOS / macOS) Rust app / CLI / server │

83 -│ import Onde use onde::inference::* │

84 -└──────────┬───────────────────────────────┬───────────────┘

85 - │ UniFFI FFI bindings │ Direct Rust API

86 - ▼ ▼

87 -┌──────────────────────────┐ ┌─────────────────────────────┐

88 -│ OndeChatEngine │ │ ChatEngine │

89 -│ (ffi.rs — uniffi::Object│──│ (engine.rs — Rust-native) │

90 -│ FFI-safe, Arc<Self>) │ │ generics, mpsc channels │

91 -└──────────┬───────────────┘ └─────────────┬───────────────┘

92 - │ │

93 - └────────────────┬───────────────┘

94 - ▼

95 - ┌────────────────────────┐

96 - │ mistralrs::Model │

97 - │ GgufModelBuilder │

98 - │ Metal / CUDA / CPU │

99 - └────────────────────────┘

100 -```

101 -

102 -### Key Design Rules

103 -

104 -1. **`ChatEngine`** (`engine.rs`) owns all Rust-idiomatic logic: `impl Into<String>`, `tokio::sync::mpsc::Receiver`, etc. Never add UniFFI annotations here.

105 -2. **`OndeChatEngine`** (`ffi.rs`) is a thin `Arc`-wrapped UniFFI `Object` with concrete, FFI-safe signatures. It delegates every call to the inner `ChatEngine`.

106 -3. **`StreamChunkListener`** is a `#[uniffi::export(callback_interface)]` trait — it must be passed as a parameter to **free functions** (not Object methods) because UniFFI 0.31 does not support callback_interface on Object methods.

107 -4. **Free functions** in `ffi.rs` (e.g. `stream_chat_message`, `default_model_config`, `user_message`) are exported with `#[uniffi::export]` and generate top-level Swift functions.

108 -5. **`uniffi::setup_scaffolding!()`** lives in `lib.rs`. Never call `uniffi::generate_scaffolding!()` from `build.rs` for the proc-macro path.

109 -

110 ----

34 +3. **Tool risk gates approval.** `ToolRisk::ReadOnly` vs `ToolRisk::Mutating`

35 + determines whether the approval handler is invoked. Per-session grants are

36 + tracked via `ApprovalDecision::AllowForSession`. Do not bypass this gate.

112 -## Platform Support

38 +4. **MCP tool names ≤ 64 bytes.** Names are sanitised to

39 + `mcp__<server>__<tool>`. Anything longer is truncated. Do not assume the

40 + original tool name survived intact.

115 -| -------- | ----------- | ----------------- | ------------- | ------------------ |

117 -| iOS | `ios` | Metal | Qwen 2.5 Coder 1.5B (~941 MB) | `["metal"]` |

42 +5. **FFI types are wrappers, not source of truth.** `FfiEd` delegates to

43 + `Ed`; `FfiToolExecutor` bridges a foreign callback into the

44 + `ToolExecutor` trait. Keep FFI types thin.

123 -- **Current default routing:** `platform_default()` now prefers the Coder variants, 1.5B on mobile and 3B on desktop.

124 -- **iOS / tvOS memory constraint:** iOS gives apps about 2–3 GB. The 3B model (~1.93 GB) can still cause OOM on constrained devices, so mobile defaults stay on the 1.5B model.

125 -- **tvOS tier-3 target:** requires `cargo +nightly -Z build-std`. Stable toolchain cannot build tvOS targets.

126 -- **Android `hf_hub`:** `dirs::home_dir()` panics in the Android sandbox. `hf-hub` is added as an explicit dependency on Android so `HF_HOME` can be seeded programmatically via `hf_hub::api::tokio::ApiBuilder`.

46 +6. **Tauri and FFI are separate bindings.** `ed-agent-tauri` and

47 + `ed-agent-ffi` both wrap `ed-agent` but must not depend on each other.

49 ---

130 -## Supported Models

51 +## LLM providers

132 -All model constants live in `src/inference/models.rs`. When adding a new model:

53 +Configured via `ed-acp/src/llm.rs`. Three variants: `Onde`, `OpenAi`, `Local`.

134 -1. Add `pub const` entries for the HF repo ID, GGUF filename, and, on Android, `TOK_MODEL_ID` when needed.

135 -2. Add the repo ID to `SUPPORTED_MODELS` so `list_local_hf_models` filters it.

136 -3. Add a `SupportedModelInfo` entry to `SUPPORTED_MODEL_INFO` with accurate `expected_size_bytes` from the HF API `siblings[].size`.

137 -4. Add a constructor to `GgufModelConfig` in `engine.rs`.

138 -5. Export a free function in `ffi.rs` for UniFFI consumers when the model should be reachable from Swift and Kotlin.

139 -6. If the GGUF does not ship with a built-in chat template, set `chat_template` explicitly. DeepSeek Coder is the current example.

55 +| Env var | Effect |

56 +|---|---|

57 +| `<PREFIX>_PROVIDER` | `onde` / `openai` / `local` |

58 +| `ONDE_API_KEY` | Cloud key for `Onde` provider |

59 +| `OPENAI_API_KEY` | Key for `OpenAi` provider |

141 -### Current Models

61 +Default cloud model: `onde-kkk`.

62 +Default local model: `Qwen3-4B` (desktop) / `Qwen3-1.7B` (mobile).

144 -|-------|------|------|------|-----------|

64 +`LlmEnv` drives config-dir, env-file loading, and variable namespacing per

65 +product. Do not hard-code env vars; always go through `LlmEnv`.

67 ---

164 -## Key Types (Rust ↔ Swift)

165 -

166 -| Rust Type | Swift Type | Notes |

167 -|-----------|------------|-------|

168 -| `ChatRole` | `enum ChatRole` | `.system`, `.user`, `.assistant` |

169 -| `ChatMessage` | `struct ChatMessage` | `role: ChatRole`, `content: String` |

170 -| `SamplingConfig` | `struct SamplingConfig` | All fields `Optional` |

171 -| `GgufModelConfig` | `struct GgufModelConfig` | `modelId`, `files`, `tokModelId?`, `displayName`, `approxMemory`, `chatTemplate?` |

172 -| `IsqModelConfig` | `struct IsqModelConfig` | `modelId`, `isqBits: UInt8`, `displayName`, `approxMemory` |

173 -| `InferenceResult` | `struct InferenceResult` | `text`, `durationSecs`, `durationDisplay`, `finishReason`, `toolCalls` |

174 -| `ToolCallInfo` | `struct ToolCallInfo` | Structured tool call request emitted by the model |

175 -| `StreamChunk` | `struct StreamChunk` | `delta`, `done`, `finishReason?` |

176 -| `EngineStatus` | `enum EngineStatus` | `.unloaded`, `.loading`, `.ready`, `.generating`, `.error` |

177 -| `EngineInfo` | `struct EngineInfo` | `status`, `modelName?`, `approxMemory?`, `historyLength: UInt64` |

178 -| `InferenceError` | `enum InferenceError: Error` | `noModelLoaded`, `alreadyLoaded`, `modelBuild`, `inference`, `cancelled`, `other` |

179 -| `OndeChatEngine` | `class OndeChatEngine` | Thread-safe; `Arc`-backed; constructed with `OndeChatEngine()` |

180 -| `StreamChunkListener` | `protocol StreamChunkListener` | Implement `onChunk(chunk:) -> Bool` |

181 -

182 ----

183 -

184 -## HuggingFace Token

185 -

186 -`src/inference/token.rs` resolves the HF token in priority order:

187 -

188 -1. **Build-time literal** (`HF_TOKEN` env var baked in via `option_env!`) — required for iOS/tvOS (no filesystem token possible).

189 -2. **On-disk cache** (`~/.cache/huggingface/token`) — works on macOS after `mistralrs login`.

190 -

191 -Set `HF_TOKEN` before building:

192 -

193 -```bash

194 -export HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

195 -cargo build --release

196 -```

197 -

198 -On sandboxed platforms, your app's setup function must also configure `HF_HOME`, `HF_HUB_CACHE`, and `TMPDIR` to point inside the app container before any `OndeChatEngine` method is called. See `docs/swift-package.md` for the full `setupInferenceEnvironment()` Swift snippet.

199 -

200 ----

201 -

202 -## Building (Rust)

203 -

204 -### Prerequisites

205 -

206 -```bash

207 -# Stable toolchain (macOS, iOS, Android, Windows, Linux)

208 -rustup toolchain install stable

209 -

210 -# Nightly toolchain (tvOS tier-3 targets only)

211 -rustup toolchain install nightly

212 -rustup component add rust-src --toolchain nightly

213 -

214 -# Apple targets (stable)

215 -rustup target add aarch64-apple-ios aarch64-apple-ios-sim

216 -rustup target add aarch64-apple-darwin x86_64-apple-darwin

217 -```

218 -

219 -### Commands

220 -

221 -```bash

222 -# Verify compilation (macOS host)

223 -cargo check

224 -

225 -# Run all tests

226 -cargo test

227 -

228 -# Run inference module tests only

229 -cargo test inference::

230 -

231 -# Run FFI wrapper tests only

232 -cargo test inference::ffi::

233 -

234 -# Clippy lint

235 -cargo clippy

236 -

237 -# Format

238 -cargo fmt

239 -```

240 -

241 -### tvOS

242 -

243 -tvOS targets require nightly and `-Z build-std`:

244 -

245 -```bash

246 -cargo +nightly rustc -Z build-std \

247 - --target aarch64-apple-tvos --release --lib --crate-type staticlib

248 -```

249 -

250 ----

251 -

252 -## Building the Swift XCFramework

253 -

254 -The XCFramework bundles staticlibs for iOS device, iOS simulator, tvOS device, tvOS simulator, and macOS into a single distributable that `Package.swift` references.

255 -

256 -### Quick build (local or CI)

257 -

258 -```bash

259 -.github/scripts/build-swift-xcframework.sh

260 -```

261 -

262 -Output lands in `dist/swift/`:

263 -- `OndeFramework.xcframework/` — the framework tree

264 -- `OndeFramework.xcframework.zip` — zipped for SPM remote binary

265 -- `OndeFramework.checksum.txt` — SHA-256 for `Package.swift`

266 -- `version.txt` — semver from `Cargo.toml`

267 -- `Package/Sources/Onde/onde.swift` — generated UniFFI Swift glue

268 -

269 -### Manual steps

270 -

271 -```bash

272 -# 1. Build the uniffi-bindgen binary (pinned to uniffi =0.31.0)

273 -cargo build --manifest-path uniffi-bindgen/Cargo.toml --release

274 -BINDGEN=uniffi-bindgen/target/release/uniffi-bindgen

275 -

276 -# 2. Compile staticlibs per target (use +1.92.0 or current stable)

277 -cargo +stable rustc --target aarch64-apple-ios --release --lib --crate-type staticlib

278 -cargo +stable rustc --target aarch64-apple-ios-sim --release --lib --crate-type staticlib

279 -cargo +stable rustc --target aarch64-apple-darwin --release --lib --crate-type staticlib

280 -cargo +nightly rustc -Z build-std --target aarch64-apple-tvos --release --lib --crate-type staticlib

281 -cargo +nightly rustc -Z build-std --target aarch64-apple-tvos-sim --release --lib --crate-type staticlib

282 -

283 -# 3. Generate Swift bindings from the iOS arm64 slice

284 -$BINDGEN generate target/aarch64-apple-ios/release/libonde.a \

285 - --language swift \

286 - --out-dir sdk/Onde/Sources/Onde \

287 - --config uniffi.toml

288 -

289 -# 4. Assemble the XCFramework (see build-swift-xcframework.sh for full xcodebuild invocation)

290 -```

291 -

292 -### Updating `onde-swift` after a new release

293 -

294 -This is fully automated by CI. Publishing a GitHub Release on `onde` triggers the workflow, which:

295 -

296 -1. Builds the XCFramework and attaches it to the release.

297 -2. Rewrites `onde-swift/Package.swift` with the release `url:` + `checksum:`.

298 -3. Copies the freshly generated `onde.swift` into `onde-swift/Sources/Onde/`.

299 -4. Commits, tags, and pushes `onde-swift` — Swift Package Index picks it up automatically.

300 -

301 -Manual intervention is only needed if `ONDE_SWIFT_PAT` has expired or the `onde-swift` push fails.

302 -

303 ----

304 -

305 -## Swift SDK (`onde-swift`)

306 -

307 -### Package.swift structure

308 -

309 -```

310 -onde-swift/

311 -├── Package.swift # Declares OndeFramework binary target + Onde wrapper target

312 -└── Sources/Onde/

313 - └── onde.swift # UniFFI-generated glue (NEVER edit manually)

314 -```

315 -

316 -`Package.swift` has two targets:

317 -

318 -- **`OndeFramework`** — `.binaryTarget` using `url:` + `checksum:` for distribution. For local development, swap to the `path:` form documented in the `Package.swift` header comment and run `build-swift-xcframework.sh` first.

319 -- **`Onde`** — `.target` depending on `OndeFramework`, used as the public import name in Swift.

320 -

321 -### Swift API Quick Reference

322 -

323 -```swift

324 -import Onde

325 -

326 -// Create engine

327 -let engine = OndeChatEngine()

328 -

329 -// Load the platform default model

330 -let elapsed = try await engine.loadDefaultModel(

331 - systemPrompt: "You are a helpful assistant.",

332 - sampling: nil

333 -)

334 -

335 -// Or load the model assigned to your app in the Onde dashboard

336 -let assignedElapsed = try await engine.loadAssignedModel(

337 - appId: "your-app-id",

338 - appSecret: "your-app-secret",

339 - systemPrompt: "You are a helpful assistant.",

340 - sampling: nil

341 -)

342 -

343 -// Multi-turn chat

344 -let result = try await engine.sendMessage(message: "Hello!")

345 -print(result.text)

346 -print(result.toolCalls)

347 -

348 -// Streaming (free function — callback_interface constraint in UniFFI 0.31)

349 -class Handler: StreamChunkListener {

350 - func onChunk(chunk: StreamChunk) -> Bool {

351 - print(chunk.delta, terminator: "")

352 - return !chunk.done

353 - }

354 -}

355 -try await streamChatMessage(engine: engine, message: "Tell me a story.", listener: Handler())

356 -

357 -// One-shot (does NOT modify conversation history)

358 -let enhanced = try await engine.generate(

359 - messages: [userMessage(content: "Expand: a cat in space")],

360 - sampling: deterministicSamplingConfig()

361 -)

362 -

363 -// Status

364 -let info = await engine.info() // EngineInfo

365 -

366 -// History management

367 -let history = await engine.history()

368 -let removed = await engine.clearHistory()

369 -await engine.pushHistory(message: userMessage(content: "..."))

370 -

371 -// Cleanup

372 -await engine.unloadModel()

373 -```

69 +## Working in this repo

375 -### Free Functions

71 +### Before changing anything

72 +- Read the relevant `lib.rs` and the trait/struct you are modifying first.

73 +- Check whether a public API is re-exported from `ed-agent`'s `lib.rs` before

74 + adding a new one; keep the public surface minimal.

75 +- If touching `ed-agent-ffi`, remember it must compile for Apple targets

76 + (macOS + iOS); avoid anything that does not cross the FFI boundary cleanly.

377 -| Function | Returns | Notes |

378 -|----------|---------|-------|

379 -| `defaultModelConfig()` | `GgufModelConfig` | Platform-aware Coder default (1.5B on iOS/tvOS/Android, 3B on desktop) |

380 -| `qwen251_5bConfig()` | `GgufModelConfig` | Forces Qwen 2.5 1.5B regardless of platform |

381 -| `qwen253bConfig()` | `GgufModelConfig` | Forces Qwen 2.5 3B regardless of platform |

382 -| `qwen306bConfig()` | `GgufModelConfig` | Qwen 3 0.6B (~0.5 GB) |

383 -| `qwen317bConfig()` | `GgufModelConfig` | Qwen 3 1.7B (~1.3 GB) |

384 -| `qwen34bConfig()` | `GgufModelConfig` | Qwen 3 4B (~2.7 GB) |

385 -| `qwen38bConfig()` | `GgufModelConfig` | Qwen 3 8B (~5 GB) |

386 -| `qwen314bConfig()` | `GgufModelConfig` | Qwen 3 14B (~8.4 GB) |

387 -| `qwen332bConfig()` | `GgufModelConfig` | Qwen 3 32B (~19.8 GB) |

388 -| `qwen34bInstruct2507Config()` | `GgufModelConfig` | Qwen 3 4B Instruct 2507 — latest non-thinking 4B (~2.5 GB) |

389 -| `qwen34bThinking2507Config()` | `GgufModelConfig` | Qwen 3 4B Thinking 2507 — latest reasoning 4B (~2.5 GB) |

390 -| `qwen330bA3bInstruct2507Config()` | `GgufModelConfig` | Qwen 3 30B-A3B Instruct 2507 MoE (~18.6 GB) |

391 -| `defaultSamplingConfig()` | `SamplingConfig` | temp=0.7, top_p=0.95, max_tokens=512 |

392 -| `deterministicSamplingConfig()` | `SamplingConfig` | temp=0.0, greedy |

393 -| `mobileSamplingConfig()` | `SamplingConfig` | temp=0.7, max_tokens=128 |

394 -| `systemMessage(content:)` | `ChatMessage` | `.system` role |

395 -| `userMessage(content:)` | `ChatMessage` | `.user` role |

396 -| `assistantMessage(content:)` | `ChatMessage` | `.assistant` role |

397 -| `streamChatMessage(engine:message:listener:)` | `async throws` | Streaming via callback |

398 -

399 ----

400 -

401 -## UniFFI Conventions

402 -

403 -- **Version pin:** `uniffi = "=0.31.0"` everywhere — the `onde` crate, `uniffi-bindgen/`, and `[build-dependencies]` must all use the **same** version. Mixing versions causes bindgen panics.

404 -- **`uniffi::setup_scaffolding!()`** in `lib.rs` — proc-macro approach, no UDL file needed.

405 -- **Object methods** use `#[uniffi::export]` on the `impl OndeChatEngine` block.

406 -- **Callback interfaces** (`StreamChunkListener`) must be parameters of **free functions** only, not Object methods.

407 -- **`Arc<Self>`** is the return type for `#[uniffi::constructor]`. UniFFI automatically handles this.

408 -- **Async:** all async exported methods use `tokio` runtime (enabled via `uniffi = { features = ["tokio"] }`).

409 -- **`uniffi.toml`**: lives at the crate root. Adjust renaming or namespace settings there before regenerating.

410 -

411 ----

412 -

413 -## HuggingFace Cache (`hf_cache.rs`)

414 -

415 -The `hf_cache` module manages the on-device model cache, exposed via UniFFI to Swift / Kotlin as-needed.

416 -

417 -Key public functions:

418 -

419 -| Function | Description |

420 -|----------|-------------|

421 -| `list_local_hf_models()` | List downloaded models that Onde supports |

422 -| `list_supported_hf_models()` | All supported models with download status |

423 -| `download_model(model_id, progress_callback)` | Download a model with progress reporting |

424 -| `delete_local_hf_model(model_id)` | Remove a model from the local cache |

425 -| `diagnose_hf_cache()` | Inspect the cache for corruption |

426 -| `repair_hf_cache_symlinks()` | Fix broken symlinks in the HF cache layout |

427 -| `model_cache_path(model_id)` | Resolve the filesystem path for a model |

428 -| `clean_stale_lock_files()` | Remove leftover `.lock` files from interrupted downloads |

429 -

430 -`ModelDownloadProgress` carries `downloaded_bytes`, `total_bytes`, `progress` (0.0–1.0), and `done`.

431 -

432 ----

433 -

434 -## Sampling Presets

435 -

437 -|--------|--------------|---------|--------------|----------|

438 -| `SamplingConfig::default()` | 0.7 | 0.95 | 512 | General creative chat |

439 -| `SamplingConfig::deterministic()` | 0.0 | — | 512 | Reproducible / coding |

440 -| `SamplingConfig::mobile()` | 0.7 | 0.95 | 128 | Memory/latency constrained |

441 -| `SamplingConfig::coding()` | 0.0 | — | 512 | Code generation |

442 -| `SamplingConfig::coding_mobile()` | 0.0 | — | 128 | Code on mobile |

443 -

444 ----

445 -

446 -## Testing

447 -

448 -```bash

449 -# All tests (requires a macOS host with Metal)

450 -cargo test

451 -

452 -# Unit tests only (no model downloads)

453 -cargo test inference::

454 -cargo test inference::ffi::

455 -cargo test inference::types::

456 -

457 -# hf_cache module

458 -cargo test hf_cache::

459 -

460 -# Clippy (treat warnings as errors)

461 -cargo clippy -- -D warnings

462 -```

463 -

464 -Tests that require model downloads are integration tests and not run by default in CI. Unit tests verify:

465 -- Type constructors and `Display` implementations

466 -- `SamplingConfig` presets

467 -- `GgufModelConfig` constructors and `platform_default()` routing

468 -- `OndeChatEngine` lifecycle: `new()` starts unloaded, `send_message` without model returns `InferenceError::NoModelLoaded`, `clear_history` on empty returns 0, `unload_model` on empty returns `nil`.

469 -

470 ----

471 -

472 -## Known Issues

473 -

474 -### `___chkstk_darwin` linker error on tvOS

475 -

476 -`aws-lc-sys` (transitive via `reqwest → rustls → aws-lc-rs`) references `___chkstk_darwin`, a stack probing symbol that tvOS does not export. `build.rs` compiles `scripts/tvos_chkstk.s` (a no-op `ret` stub) via the `cc` crate to satisfy the linker. **Do not delete `scripts/tvos_chkstk.s`.**

477 -

478 -Affects: tvOS only. macOS, iOS, Android, Windows, Linux are unaffected.

479 -

480 -### Metal Toolchain missing (Xcode 26+)

481 -

482 -On Xcode 26+, the Metal compiler is a separate download. If you see empty `.metallib` files (92 bytes) or `"Error while loading function: fused_glu_float"` at runtime:

483 -

484 -```bash

485 -xcodebuild -downloadComponent MetalToolchain

486 -cargo clean -p mistralrs-quant

487 -cargo check

488 -```

489 -

490 -### Android `home_dir` sandbox panic

491 -

492 -`dirs::home_dir()` panics in the Android sandbox. The crate-level `Cargo.toml` adds `hf-hub` as an explicit Android dependency so `HF_HOME` can be seeded via `ApiBuilder` before any hub request. Never call `home::home_dir()` or `dirs::home_dir()` on Android.

493 -

494 ----

495 -

496 -## Code Conventions

497 -

498 -### Rust

499 -

500 -- **Error handling:** `anyhow` for application-level errors inside `engine.rs`; `thiserror` for the `InferenceError` enum (exported via UniFFI). Never `.unwrap()` or `.expect()` in non-test code.

501 -- **Async:** `tokio` runtime. All async functions in `OndeChatEngine` are `pub async fn`.

502 -- **Logging:** `log` crate macros (`log::debug!`, `log::info!`, `log::warn!`, `log::error!`). No `println!` in library code.

503 -- **Platform gating:** use `#[cfg(target_os = "...")]` blocks. Match `Cargo.toml`'s target-conditional dependency sections.

504 -- **Re-exports:** `mistralrs`, `hf_hub`, and `mistralrs_core` are re-exported from `lib.rs` for downstream Rust consumers. Keep these re-exports in sync with what's actually available per platform.

505 -- **No `mod.rs`:** use named files (`inference/engine.rs`) not `inference/mod.rs` — except that `inference/mod.rs` exists and is the intentional module root for the `inference` module.

506 -

507 -### Swift / SDK

508 -

509 -- **Never manually edit `onde-swift/Sources/Onde/onde.swift`** — it is generated by `uniffi-bindgen`. Regenerate by running the build script.

510 -- **`Package.swift` binary target** uses `url:` + `checksum:` in the committed form. For local development, swap to `path:` (instructions are in the file header). Never commit the `path:` form — CI overwrites it on every release via `release-sdk-swift.yml`.

511 -- **iOS/tvOS sandbox:** always call `setupInferenceEnvironment()` at app launch before any `OndeChatEngine` call.

512 -

513 ----

514 -

515 -## CI / Release Workflow

516 -

517 -### `release-sdk-swift.yml`

518 -

519 -Triggered on **semver tag push** or `workflow_dispatch`. Runs on `macos-15`.

520 -

521 -Steps:

522 -1. Install stable Rust with iOS/macOS targets.

523 -2. Install nightly Rust with `rust-src` for tvOS `-Z build-std`.

524 -3. Run `build-swift-xcframework.sh`.

525 -4. Read `version.txt` and `OndeFramework.checksum.txt` into step outputs.

526 -5. Validate that the tag matches `Cargo.toml`.

527 -6. Upload CI artifacts: zip, checksum, version, generated `onde.swift`.

528 -7. On tag push, create a GitHub Release with the zip and checksum attached.

529 -8. On tag push, check out `ondeinference/onde-swift` using `ONDE_SWIFT_PAT`.

530 -9. On tag push, rewrite the `.binaryTarget` in `onde-swift/Package.swift` with the release URL and checksum.

531 -10. On tag push, copy the generated `onde.swift` into `onde-swift/Sources/Onde/`.

532 -11. On tag push, commit both files, tag the commit with the version, and push to `ondeinference/onde-swift`.

533 -

534 -#### Required secret

535 -

536 -`ONDE_SWIFT_PAT` — a GitHub Personal Access Token with **`contents: write`** scope on the `ondeinference/onde-swift` repository.

537 -

538 -### `release-sdk-kotlin.yml`

539 -

540 -Triggered on **semver tag push** or `workflow_dispatch`. Runs on `macos-15` because it builds both Android artifacts and the macOS Apple Silicon JVM native library.

541 -

542 -Steps:

543 -1. Install stable Rust and the `aarch64-apple-darwin` target.

544 -2. Install Android SDK / NDK and auto-discover the latest installed 29.x NDK for `cargo-ndk`.

545 -3. Build Android JNI libraries for all supported ABIs.

546 -4. Build the macOS Apple Silicon `libonde.dylib` used by the JVM target.

547 -5. Run Gradle publication tasks for the Kotlin Multiplatform package.

548 -6. Validate that the tag matches `sdk/kotlin/gradle.properties` `VERSION_NAME`.

549 -7. On tag push, publish `com.ondeinference:onde-inference` to Maven Central.

550 -

551 -#### Required secrets

552 -

553 -- `ORG_GRADLE_PROJECT_MAVENCENTRALUSERNAME`

554 -- `ORG_GRADLE_PROJECT_MAVENCENTRALPASSWORD`

555 -- `ORG_GRADLE_PROJECT_SIGNINGKEYID`

556 -- `ORG_GRADLE_PROJECT_SIGNINGKEY`

557 -- `ORG_GRADLE_PROJECT_SIGNINGPASSWORD`

558 -

559 -### `release-sdk-dart.yml`

560 -

561 -Triggered on **semver tag push** or `workflow_dispatch`.

562 -

563 -- Publishes `onde_inference` to pub.dev.

564 -- Version must match the Dart package version.

565 -- The example app is part of release validation.

566 -

567 -### `release-sdk-npm.yml`

568 -

569 -Triggered on **semver tag push** or `workflow_dispatch`.

570 -

571 -- Publishes `@ondeinference/react-native` to npm.

572 -- Builds the Expo module package and validates the published version.

573 -

574 -### `release-sdk-rust.yml`

575 -

576 -Triggered on **semver tag push** or `workflow_dispatch`.

577 -

578 -- Publishes `onde` to crates.io.

579 -- Uses registry dependencies, not git refs, because `cargo publish` strips git patches.

580 -

581 -### Release Process (end-to-end)

582 -

583 -1. Bump versions in every affected package manifest:

584 - - `Cargo.toml`

585 - - `sdk/kotlin/gradle.properties`

586 - - `sdk/dart/pubspec.yaml`

587 - - `sdk/react-native/package.json`

588 -2. Merge release branches with `--no-ff`. Do not fast-forward multi-commit feature branches.

589 -3. Commit, tag, and push the release.

590 -4. CI publishes all five SDKs:

591 - - crates.io (`onde`)

592 - - Swift Package Index / GitHub Releases (`onde-swift`)

593 - - Maven Central (`com.ondeinference:onde-inference`)

594 - - pub.dev (`onde_inference`)

595 - - npm (`@ondeinference/react-native`)

596 -5. Verify the registries after publish, especially Maven Central indexing and the generated Swift release assets.

597 -

598 -> **Note:** `onde-swift/Package.swift` is updated automatically by CI. For local development only, switch it to the `path:` form documented in the file header, then switch it back before committing.

599 -

600 ----

78 +### Making changes

79 +- Keep crate boundaries: adding a dependency from a lower-level crate

80 + (`ed-agent`) on a higher-level one (`ed-acp`) is forbidden.

81 +- New tool definitions belong in the crate that owns the domain, exposed via

82 + `ToolExecutor` or MCP server, not inlined into `Ed`.

83 +- Prefer `async_trait` + `Arc<dyn Trait>` for seams; avoid generics on `Ed`

84 + itself.

85 +- When adding an FFI type, add the matching `From<Ffi*>` / `From<*>` impls

86 + immediately so both sides stay in sync.

602 -## Distribution Registry Summary

88 +### After changes

89 +- Run `cargo check --workspace` to catch compile errors across all crates.

90 +- Run `cargo test --workspace` before committing.

91 +- For FFI changes, note that a full XCFramework build requires the Apple SDK

92 + toolchain; CI handles that — local `cargo check` is sufficient to validate

93 + Rust correctness.

604 -| Registry | Name | Import |

605 -|----------|------|--------|

606 -| crates.io | `onde` | `onde = "1.x"` |

607 -| Swift Package Index | `onde-swift` (org: `ondeinference`) | `import Onde` |

608 -| Maven Central | `com.ondeinference:onde-inference` | Gradle `implementation("com.ondeinference:onde-inference:<version>")` |

609 -| pub.dev | `onde_inference` | `import 'package:onde_inference/onde_inference.dart';` |

610 -| npm | `@ondeinference/react-native` | `import { OndeChatEngine } from "@ondeinference/react-native"` |

611 -| PyPI | `onde-inference` | `import onde_inference` |

612 -| RubyGems | `onde-inference` | `require 'onde'` |

95 +### Commit messages

96 +- Scope to the affected crate(s): `ed-agent: …`, `ed-acp: …`, `ed-mcp: …`.

97 +- Keep the subject line under 72 chars.

98 +- Always end the commit with:

99 + ```

100 + Co-Authored-By: siGit Code v1.6.6-acp <noreply@sigit.si>

101 + ```

102

103 ---

104

616 -*This file is for AI agent and human developer reference. Update it when the architecture, API surface, supported models, or toolchain requirements change.*

617 -```

105 +## Things to avoid

106 +- Do not add `#[allow(unused)]` blanket suppressions — fix the unused item.

107 +- Do not store `tokio::runtime::Runtime` inside an FFI object; use `block_on`

108 + only at the outermost FFI boundary.

109 +- Do not panic inside async tasks — convert errors to `AgentError` and surface

110 + them through the event sink or return type.

111 +- Do not hard-code model names anywhere outside `ed-acp/src/llm.rs`.

112 +- Do not bypass `ToolRisk` approval for any mutating tool, even in tests — use

113 + an `AllowAll` test double instead.

AGENTS.md deleted

-617

@@ -1,617 +0,0 @@