@ondeinference / ed / commits / 5a6a721

Copy agentic resources from onde

Add AGENTS.md, CLAUDE.md, and .agents/ directory (skills + AGENTS.md) copied from the onde workspace. Co-Authored-By: siGit Code v1.6.6-acp <noreply@sigit.si>

Seto Elkahfi committed Oct 10, 2026 at 08:09 UTC 5a6a721fdb10ff70a85f9ec4dfd52e73a25982b6

15 files changed +7656

.agents/AGENTS.md new

+617

@@ -0,0 +1,617 @@

1 +# Onde Inference — AGENTS.md

2 +

3 +> **AI agent reference for the `onde` repository and the `onde-swift` Swift SDK.**

4 +> Keep this file accurate as the codebase evolves.

5 +

6 +---

7 +

8 +## What Is Onde?

9 +

10 +Onde is an **on-device LLM inference SDK family** built around a shared Rust core. It targets Apple silicon (iOS, tvOS, macOS) plus Android, Windows, and Linux, and ships SDKs for Rust, Swift, Kotlin Multiplatform, Flutter, and React Native. It wraps [mistral.rs](https://github.com/EricLBuehler/mistral.rs) behind platform-friendly APIs with automatic model selection, HuggingFace Hub downloads, cache management, and Metal acceleration where available.

11 +

12 +- **Website:** https://ondeinference.com

13 +- **Rust crate:** https://crates.io/crates/onde

14 +- **Swift package:** https://github.com/ondeinference/onde-swift

15 +- **Maven Central:** https://central.sonatype.com/artifact/com.ondeinference/onde-inference

16 +- **Flutter package:** https://pub.dev/packages/onde_inference

17 +- **React Native package:** https://www.npmjs.com/package/@ondeinference/react-native

18 +- **In production:** [Splitfire AB apps on the Apple AppStore](https://apps.apple.com/se/developer/splitfire-ab/id1831430993)

19 +

20 +---

21 +

22 +## Repository Layout

23 +

24 +```

25 +onde/

26 +├── src/

27 +│ ├── lib.rs # Crate root — uniffi::setup_scaffolding!()

28 +│ ├── hf_cache.rs # HuggingFace Hub cache: list, download, delete, diagnose, repair

29 +│ └── inference/

30 +│ ├── mod.rs # Module exports + top-level re-exports

31 +│ ├── engine.rs # ChatEngine — Rust-native API (generics, mpsc, tool calling)

32 +│ ├── ffi.rs # OndeChatEngine — UniFFI Object (FFI-safe, Arc-wrapped)

33 +│ ├── models.rs # Model ID constants + SupportedModelInfo metadata

34 +│ ├── token.rs # HF token resolution: build-time literal vs cache file

35 +│ └── types.rs # Shared types: ChatMessage, SamplingConfig, InferenceResult, ToolCallInfo, etc.

36 +├── sdk/

37 +│ ├── dart/ # Flutter/Dart package + example app + FRB Rust bridge

38 +│ ├── gem/ # Ruby native extension (Magnus)

39 +│ ├── kotlin/ # Kotlin Multiplatform package (Android + JVM)

40 +│ ├── react-native/ # Expo module wrapping the Rust core for iOS/Android

41 +│ └── python/ # Python bindings (maturin + uniffi)

42 +├── generated/ # UniFFI-generated headers and Swift glue (git-ignored)

43 +├── uniffi-bindgen/ # Standalone bindgen binary crate (pinned uniffi =0.31.0)

44 +├── .github/

45 +│ ├── workflows/

46 +│ │ ├── release-sdk-swift.yml # CI: tag push → build XCFramework → GitHub Release → update onde-swift

47 +│ │ ├── release-sdk-kotlin.yml # CI: tag push → build Android/JVM artifacts → publish to Maven Central

48 +│ │ ├── release-sdk-dart.yml # CI: tag push → publish Flutter package to pub.dev

49 +│ │ ├── release-sdk-npm.yml # CI: tag push → publish React Native package to npm

50 +│ │ └── release-sdk-rust.yml # CI: tag push → publish Rust crate to crates.io

51 +│ └── scripts/

52 +│ └── build-swift-xcframework.sh # Local/CI XCFramework assembly script

53 +├── .cargo/config.toml # Target-specific rustflags (fp16, linker overrides)

54 +├── Cargo.toml # Platform-conditional mistralrs deps

55 +├── build.rs # tvOS ___chkstk_darwin assembly stub

56 +├── scripts/

57 +│ └── tvos_chkstk.s # No-op arm64 stub for missing tvOS symbol

58 +├── uniffi.toml # UniFFI binding config

59 +└── docs/

60 + ├── dev.md # Developer guide (build, architecture, platform table)

61 + ├── swift-package.md # Swift API reference + XCFramework build steps

62 + ├── distribution.md # Release process for all registries

63 + └── ruby-gem.md # Ruby gem API reference

64 +```

65 +

66 +```

67 +onde-swift/ # Swift Package Manager wrapper repo

68 +├── Package.swift # Declares OndeFramework.xcframework binary target

69 +└── Sources/

70 + └── Onde/

71 + └── onde.swift # UniFFI-generated Swift glue (do NOT edit manually)

72 +```

73 +

74 +---

75 +

76 +## Architecture

77 +

78 +### Layer Diagram

79 +

80 +```

81 +┌──────────────────────────────────────────────────────────┐

82 +│ Swift (iOS / tvOS / macOS) Rust app / CLI / server │

83 +│ import Onde use onde::inference::* │

84 +└──────────┬───────────────────────────────┬───────────────┘

85 + │ UniFFI FFI bindings │ Direct Rust API

86 + ▼ ▼

87 +┌──────────────────────────┐ ┌─────────────────────────────┐

88 +│ OndeChatEngine │ │ ChatEngine │

89 +│ (ffi.rs — uniffi::Object│──│ (engine.rs — Rust-native) │

90 +│ FFI-safe, Arc<Self>) │ │ generics, mpsc channels │

91 +└──────────┬───────────────┘ └─────────────┬───────────────┘

92 + │ │

93 + └────────────────┬───────────────┘

94 + ▼

95 + ┌────────────────────────┐

96 + │ mistralrs::Model │

97 + │ GgufModelBuilder │

98 + │ Metal / CUDA / CPU │

99 + └────────────────────────┘

100 +```

101 +

102 +### Key Design Rules

103 +

104 +1. **`ChatEngine`** (`engine.rs`) owns all Rust-idiomatic logic: `impl Into<String>`, `tokio::sync::mpsc::Receiver`, etc. Never add UniFFI annotations here.

105 +2. **`OndeChatEngine`** (`ffi.rs`) is a thin `Arc`-wrapped UniFFI `Object` with concrete, FFI-safe signatures. It delegates every call to the inner `ChatEngine`.

106 +3. **`StreamChunkListener`** is a `#[uniffi::export(callback_interface)]` trait — it must be passed as a parameter to **free functions** (not Object methods) because UniFFI 0.31 does not support callback_interface on Object methods.

107 +4. **Free functions** in `ffi.rs` (e.g. `stream_chat_message`, `default_model_config`, `user_message`) are exported with `#[uniffi::export]` and generate top-level Swift functions.

108 +5. **`uniffi::setup_scaffolding!()`** lives in `lib.rs`. Never call `uniffi::generate_scaffolding!()` from `build.rs` for the proc-macro path.

109 +

110 +---

111 +

112 +## Platform Support

113 +

115 +| -------- | ----------- | ----------------- | ------------- | ------------------ |

117 +| iOS | `ios` | Metal | Qwen 2.5 Coder 1.5B (~941 MB) | `["metal"]` |

122 +

123 +- **Current default routing:** `platform_default()` now prefers the Coder variants, 1.5B on mobile and 3B on desktop.

124 +- **iOS / tvOS memory constraint:** iOS gives apps about 2–3 GB. The 3B model (~1.93 GB) can still cause OOM on constrained devices, so mobile defaults stay on the 1.5B model.

125 +- **tvOS tier-3 target:** requires `cargo +nightly -Z build-std`. Stable toolchain cannot build tvOS targets.

126 +- **Android `hf_hub`:** `dirs::home_dir()` panics in the Android sandbox. `hf-hub` is added as an explicit dependency on Android so `HF_HOME` can be seeded programmatically via `hf_hub::api::tokio::ApiBuilder`.

127 +

128 +---

129 +

130 +## Supported Models

131 +

132 +All model constants live in `src/inference/models.rs`. When adding a new model:

133 +

134 +1. Add `pub const` entries for the HF repo ID, GGUF filename, and, on Android, `TOK_MODEL_ID` when needed.

135 +2. Add the repo ID to `SUPPORTED_MODELS` so `list_local_hf_models` filters it.

136 +3. Add a `SupportedModelInfo` entry to `SUPPORTED_MODEL_INFO` with accurate `expected_size_bytes` from the HF API `siblings[].size`.

137 +4. Add a constructor to `GgufModelConfig` in `engine.rs`.

138 +5. Export a free function in `ffi.rs` for UniFFI consumers when the model should be reachable from Swift and Kotlin.

139 +6. If the GGUF does not ship with a built-in chat template, set `chat_template` explicitly. DeepSeek Coder is the current example.

140 +

141 +### Current Models

142 +

144 +|-------|------|------|------|-----------|

161 +

162 +---

163 +

164 +## Key Types (Rust ↔ Swift)

165 +

166 +| Rust Type | Swift Type | Notes |

167 +|-----------|------------|-------|

168 +| `ChatRole` | `enum ChatRole` | `.system`, `.user`, `.assistant` |

169 +| `ChatMessage` | `struct ChatMessage` | `role: ChatRole`, `content: String` |

170 +| `SamplingConfig` | `struct SamplingConfig` | All fields `Optional` |

171 +| `GgufModelConfig` | `struct GgufModelConfig` | `modelId`, `files`, `tokModelId?`, `displayName`, `approxMemory`, `chatTemplate?` |

172 +| `IsqModelConfig` | `struct IsqModelConfig` | `modelId`, `isqBits: UInt8`, `displayName`, `approxMemory` |

173 +| `InferenceResult` | `struct InferenceResult` | `text`, `durationSecs`, `durationDisplay`, `finishReason`, `toolCalls` |

174 +| `ToolCallInfo` | `struct ToolCallInfo` | Structured tool call request emitted by the model |

175 +| `StreamChunk` | `struct StreamChunk` | `delta`, `done`, `finishReason?` |

176 +| `EngineStatus` | `enum EngineStatus` | `.unloaded`, `.loading`, `.ready`, `.generating`, `.error` |

177 +| `EngineInfo` | `struct EngineInfo` | `status`, `modelName?`, `approxMemory?`, `historyLength: UInt64` |

178 +| `InferenceError` | `enum InferenceError: Error` | `noModelLoaded`, `alreadyLoaded`, `modelBuild`, `inference`, `cancelled`, `other` |

179 +| `OndeChatEngine` | `class OndeChatEngine` | Thread-safe; `Arc`-backed; constructed with `OndeChatEngine()` |

180 +| `StreamChunkListener` | `protocol StreamChunkListener` | Implement `onChunk(chunk:) -> Bool` |

181 +

182 +---

183 +

184 +## HuggingFace Token

185 +

186 +`src/inference/token.rs` resolves the HF token in priority order:

187 +

188 +1. **Build-time literal** (`HF_TOKEN` env var baked in via `option_env!`) — required for iOS/tvOS (no filesystem token possible).

189 +2. **On-disk cache** (`~/.cache/huggingface/token`) — works on macOS after `mistralrs login`.

190 +

191 +Set `HF_TOKEN` before building:

192 +

193 +```bash

194 +export HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

195 +cargo build --release

196 +```

197 +

198 +On sandboxed platforms, your app's setup function must also configure `HF_HOME`, `HF_HUB_CACHE`, and `TMPDIR` to point inside the app container before any `OndeChatEngine` method is called. See `docs/swift-package.md` for the full `setupInferenceEnvironment()` Swift snippet.

199 +

200 +---

201 +

202 +## Building (Rust)

203 +

204 +### Prerequisites

205 +

206 +```bash

207 +# Stable toolchain (macOS, iOS, Android, Windows, Linux)

208 +rustup toolchain install stable

209 +

210 +# Nightly toolchain (tvOS tier-3 targets only)

211 +rustup toolchain install nightly

212 +rustup component add rust-src --toolchain nightly

213 +

214 +# Apple targets (stable)

215 +rustup target add aarch64-apple-ios aarch64-apple-ios-sim

216 +rustup target add aarch64-apple-darwin x86_64-apple-darwin

217 +```

218 +

219 +### Commands

220 +

221 +```bash

222 +# Verify compilation (macOS host)

223 +cargo check

224 +

225 +# Run all tests

226 +cargo test

227 +

228 +# Run inference module tests only

229 +cargo test inference::

230 +

231 +# Run FFI wrapper tests only

232 +cargo test inference::ffi::

233 +

234 +# Clippy lint

235 +cargo clippy

236 +

237 +# Format

238 +cargo fmt

239 +```

240 +

241 +### tvOS

242 +

243 +tvOS targets require nightly and `-Z build-std`:

244 +

245 +```bash

246 +cargo +nightly rustc -Z build-std \

247 + --target aarch64-apple-tvos --release --lib --crate-type staticlib

248 +```

249 +

250 +---

251 +

252 +## Building the Swift XCFramework

253 +

254 +The XCFramework bundles staticlibs for iOS device, iOS simulator, tvOS device, tvOS simulator, and macOS into a single distributable that `Package.swift` references.

255 +

256 +### Quick build (local or CI)

257 +

258 +```bash

259 +.github/scripts/build-swift-xcframework.sh

260 +```

261 +

262 +Output lands in `dist/swift/`:

263 +- `OndeFramework.xcframework/` — the framework tree

264 +- `OndeFramework.xcframework.zip` — zipped for SPM remote binary

265 +- `OndeFramework.checksum.txt` — SHA-256 for `Package.swift`

266 +- `version.txt` — semver from `Cargo.toml`

267 +- `Package/Sources/Onde/onde.swift` — generated UniFFI Swift glue

268 +

269 +### Manual steps

270 +

271 +```bash

272 +# 1. Build the uniffi-bindgen binary (pinned to uniffi =0.31.0)

273 +cargo build --manifest-path uniffi-bindgen/Cargo.toml --release

274 +BINDGEN=uniffi-bindgen/target/release/uniffi-bindgen

275 +

276 +# 2. Compile staticlibs per target (use +1.92.0 or current stable)

277 +cargo +stable rustc --target aarch64-apple-ios --release --lib --crate-type staticlib

278 +cargo +stable rustc --target aarch64-apple-ios-sim --release --lib --crate-type staticlib

279 +cargo +stable rustc --target aarch64-apple-darwin --release --lib --crate-type staticlib

280 +cargo +nightly rustc -Z build-std --target aarch64-apple-tvos --release --lib --crate-type staticlib

281 +cargo +nightly rustc -Z build-std --target aarch64-apple-tvos-sim --release --lib --crate-type staticlib

282 +

283 +# 3. Generate Swift bindings from the iOS arm64 slice

284 +$BINDGEN generate target/aarch64-apple-ios/release/libonde.a \

285 + --language swift \

286 + --out-dir sdk/Onde/Sources/Onde \

287 + --config uniffi.toml

288 +

289 +# 4. Assemble the XCFramework (see build-swift-xcframework.sh for full xcodebuild invocation)

290 +```

291 +

292 +### Updating `onde-swift` after a new release

293 +

294 +This is fully automated by CI. Publishing a GitHub Release on `onde` triggers the workflow, which:

295 +

296 +1. Builds the XCFramework and attaches it to the release.

297 +2. Rewrites `onde-swift/Package.swift` with the release `url:` + `checksum:`.

298 +3. Copies the freshly generated `onde.swift` into `onde-swift/Sources/Onde/`.

299 +4. Commits, tags, and pushes `onde-swift` — Swift Package Index picks it up automatically.

300 +

301 +Manual intervention is only needed if `ONDE_SWIFT_PAT` has expired or the `onde-swift` push fails.

302 +

303 +---

304 +

305 +## Swift SDK (`onde-swift`)

306 +

307 +### Package.swift structure

308 +

309 +```

310 +onde-swift/

311 +├── Package.swift # Declares OndeFramework binary target + Onde wrapper target

312 +└── Sources/Onde/

313 + └── onde.swift # UniFFI-generated glue (NEVER edit manually)

314 +```

315 +

316 +`Package.swift` has two targets:

317 +

318 +- **`OndeFramework`** — `.binaryTarget` using `url:` + `checksum:` for distribution. For local development, swap to the `path:` form documented in the `Package.swift` header comment and run `build-swift-xcframework.sh` first.

319 +- **`Onde`** — `.target` depending on `OndeFramework`, used as the public import name in Swift.

320 +

321 +### Swift API Quick Reference

322 +

323 +```swift

324 +import Onde

325 +

326 +// Create engine

327 +let engine = OndeChatEngine()

328 +

329 +// Load the platform default model

330 +let elapsed = try await engine.loadDefaultModel(

331 + systemPrompt: "You are a helpful assistant.",

332 + sampling: nil

333 +)

334 +

335 +// Or load the model assigned to your app in the Onde dashboard

336 +let assignedElapsed = try await engine.loadAssignedModel(

337 + appId: "your-app-id",

338 + appSecret: "your-app-secret",

339 + systemPrompt: "You are a helpful assistant.",

340 + sampling: nil

341 +)

342 +

343 +// Multi-turn chat

344 +let result = try await engine.sendMessage(message: "Hello!")

345 +print(result.text)

346 +print(result.toolCalls)

347 +

348 +// Streaming (free function — callback_interface constraint in UniFFI 0.31)

349 +class Handler: StreamChunkListener {

350 + func onChunk(chunk: StreamChunk) -> Bool {

351 + print(chunk.delta, terminator: "")

352 + return !chunk.done

353 + }

354 +}

355 +try await streamChatMessage(engine: engine, message: "Tell me a story.", listener: Handler())

356 +

357 +// One-shot (does NOT modify conversation history)

358 +let enhanced = try await engine.generate(

359 + messages: [userMessage(content: "Expand: a cat in space")],

360 + sampling: deterministicSamplingConfig()

361 +)

362 +

363 +// Status

364 +let info = await engine.info() // EngineInfo

365 +

366 +// History management

367 +let history = await engine.history()

368 +let removed = await engine.clearHistory()

369 +await engine.pushHistory(message: userMessage(content: "..."))

370 +

371 +// Cleanup

372 +await engine.unloadModel()

373 +```

374 +

375 +### Free Functions

376 +

377 +| Function | Returns | Notes |

378 +|----------|---------|-------|

379 +| `defaultModelConfig()` | `GgufModelConfig` | Platform-aware Coder default (1.5B on iOS/tvOS/Android, 3B on desktop) |

380 +| `qwen251_5bConfig()` | `GgufModelConfig` | Forces Qwen 2.5 1.5B regardless of platform |

381 +| `qwen253bConfig()` | `GgufModelConfig` | Forces Qwen 2.5 3B regardless of platform |

382 +| `qwen306bConfig()` | `GgufModelConfig` | Qwen 3 0.6B (~0.5 GB) |

383 +| `qwen317bConfig()` | `GgufModelConfig` | Qwen 3 1.7B (~1.3 GB) |

384 +| `qwen34bConfig()` | `GgufModelConfig` | Qwen 3 4B (~2.7 GB) |

385 +| `qwen38bConfig()` | `GgufModelConfig` | Qwen 3 8B (~5 GB) |

386 +| `qwen314bConfig()` | `GgufModelConfig` | Qwen 3 14B (~8.4 GB) |

387 +| `qwen332bConfig()` | `GgufModelConfig` | Qwen 3 32B (~19.8 GB) |

388 +| `qwen34bInstruct2507Config()` | `GgufModelConfig` | Qwen 3 4B Instruct 2507 — latest non-thinking 4B (~2.5 GB) |

389 +| `qwen34bThinking2507Config()` | `GgufModelConfig` | Qwen 3 4B Thinking 2507 — latest reasoning 4B (~2.5 GB) |

390 +| `qwen330bA3bInstruct2507Config()` | `GgufModelConfig` | Qwen 3 30B-A3B Instruct 2507 MoE (~18.6 GB) |

391 +| `defaultSamplingConfig()` | `SamplingConfig` | temp=0.7, top_p=0.95, max_tokens=512 |

392 +| `deterministicSamplingConfig()` | `SamplingConfig` | temp=0.0, greedy |

393 +| `mobileSamplingConfig()` | `SamplingConfig` | temp=0.7, max_tokens=128 |

394 +| `systemMessage(content:)` | `ChatMessage` | `.system` role |

395 +| `userMessage(content:)` | `ChatMessage` | `.user` role |

396 +| `assistantMessage(content:)` | `ChatMessage` | `.assistant` role |

397 +| `streamChatMessage(engine:message:listener:)` | `async throws` | Streaming via callback |

398 +

399 +---

400 +

401 +## UniFFI Conventions

402 +

403 +- **Version pin:** `uniffi = "=0.31.0"` everywhere — the `onde` crate, `uniffi-bindgen/`, and `[build-dependencies]` must all use the **same** version. Mixing versions causes bindgen panics.

404 +- **`uniffi::setup_scaffolding!()`** in `lib.rs` — proc-macro approach, no UDL file needed.

405 +- **Object methods** use `#[uniffi::export]` on the `impl OndeChatEngine` block.

406 +- **Callback interfaces** (`StreamChunkListener`) must be parameters of **free functions** only, not Object methods.

407 +- **`Arc<Self>`** is the return type for `#[uniffi::constructor]`. UniFFI automatically handles this.

408 +- **Async:** all async exported methods use `tokio` runtime (enabled via `uniffi = { features = ["tokio"] }`).

409 +- **`uniffi.toml`**: lives at the crate root. Adjust renaming or namespace settings there before regenerating.

410 +

411 +---

412 +

413 +## HuggingFace Cache (`hf_cache.rs`)

414 +

415 +The `hf_cache` module manages the on-device model cache, exposed via UniFFI to Swift / Kotlin as-needed.

416 +

417 +Key public functions:

418 +

419 +| Function | Description |

420 +|----------|-------------|

421 +| `list_local_hf_models()` | List downloaded models that Onde supports |

422 +| `list_supported_hf_models()` | All supported models with download status |

423 +| `download_model(model_id, progress_callback)` | Download a model with progress reporting |

424 +| `delete_local_hf_model(model_id)` | Remove a model from the local cache |

425 +| `diagnose_hf_cache()` | Inspect the cache for corruption |

426 +| `repair_hf_cache_symlinks()` | Fix broken symlinks in the HF cache layout |

427 +| `model_cache_path(model_id)` | Resolve the filesystem path for a model |

428 +| `clean_stale_lock_files()` | Remove leftover `.lock` files from interrupted downloads |

429 +

430 +`ModelDownloadProgress` carries `downloaded_bytes`, `total_bytes`, `progress` (0.0–1.0), and `done`.

431 +

432 +---

433 +

434 +## Sampling Presets

435 +

437 +|--------|--------------|---------|--------------|----------|

438 +| `SamplingConfig::default()` | 0.7 | 0.95 | 512 | General creative chat |

439 +| `SamplingConfig::deterministic()` | 0.0 | — | 512 | Reproducible / coding |

440 +| `SamplingConfig::mobile()` | 0.7 | 0.95 | 128 | Memory/latency constrained |

441 +| `SamplingConfig::coding()` | 0.0 | — | 512 | Code generation |

442 +| `SamplingConfig::coding_mobile()` | 0.0 | — | 128 | Code on mobile |

443 +

444 +---

445 +

446 +## Testing

447 +

448 +```bash

449 +# All tests (requires a macOS host with Metal)

450 +cargo test

451 +

452 +# Unit tests only (no model downloads)

453 +cargo test inference::

454 +cargo test inference::ffi::

455 +cargo test inference::types::

456 +

457 +# hf_cache module

458 +cargo test hf_cache::

459 +

460 +# Clippy (treat warnings as errors)

461 +cargo clippy -- -D warnings

462 +```

463 +

464 +Tests that require model downloads are integration tests and not run by default in CI. Unit tests verify:

465 +- Type constructors and `Display` implementations

466 +- `SamplingConfig` presets

467 +- `GgufModelConfig` constructors and `platform_default()` routing

468 +- `OndeChatEngine` lifecycle: `new()` starts unloaded, `send_message` without model returns `InferenceError::NoModelLoaded`, `clear_history` on empty returns 0, `unload_model` on empty returns `nil`.

469 +

470 +---

471 +

472 +## Known Issues

473 +

474 +### `___chkstk_darwin` linker error on tvOS

475 +

476 +`aws-lc-sys` (transitive via `reqwest → rustls → aws-lc-rs`) references `___chkstk_darwin`, a stack probing symbol that tvOS does not export. `build.rs` compiles `scripts/tvos_chkstk.s` (a no-op `ret` stub) via the `cc` crate to satisfy the linker. **Do not delete `scripts/tvos_chkstk.s`.**

477 +

478 +Affects: tvOS only. macOS, iOS, Android, Windows, Linux are unaffected.

479 +

480 +### Metal Toolchain missing (Xcode 26+)

481 +

482 +On Xcode 26+, the Metal compiler is a separate download. If you see empty `.metallib` files (92 bytes) or `"Error while loading function: fused_glu_float"` at runtime:

483 +

484 +```bash

485 +xcodebuild -downloadComponent MetalToolchain

486 +cargo clean -p mistralrs-quant

487 +cargo check

488 +```

489 +

490 +### Android `home_dir` sandbox panic

491 +

492 +`dirs::home_dir()` panics in the Android sandbox. The crate-level `Cargo.toml` adds `hf-hub` as an explicit Android dependency so `HF_HOME` can be seeded via `ApiBuilder` before any hub request. Never call `home::home_dir()` or `dirs::home_dir()` on Android.

493 +

494 +---

495 +

496 +## Code Conventions

497 +

498 +### Rust

499 +

500 +- **Error handling:** `anyhow` for application-level errors inside `engine.rs`; `thiserror` for the `InferenceError` enum (exported via UniFFI). Never `.unwrap()` or `.expect()` in non-test code.

501 +- **Async:** `tokio` runtime. All async functions in `OndeChatEngine` are `pub async fn`.

502 +- **Logging:** `log` crate macros (`log::debug!`, `log::info!`, `log::warn!`, `log::error!`). No `println!` in library code.

503 +- **Platform gating:** use `#[cfg(target_os = "...")]` blocks. Match `Cargo.toml`'s target-conditional dependency sections.

504 +- **Re-exports:** `mistralrs`, `hf_hub`, and `mistralrs_core` are re-exported from `lib.rs` for downstream Rust consumers. Keep these re-exports in sync with what's actually available per platform.

505 +- **No `mod.rs`:** use named files (`inference/engine.rs`) not `inference/mod.rs` — except that `inference/mod.rs` exists and is the intentional module root for the `inference` module.

506 +

507 +### Swift / SDK

508 +

509 +- **Never manually edit `onde-swift/Sources/Onde/onde.swift`** — it is generated by `uniffi-bindgen`. Regenerate by running the build script.

510 +- **`Package.swift` binary target** uses `url:` + `checksum:` in the committed form. For local development, swap to `path:` (instructions are in the file header). Never commit the `path:` form — CI overwrites it on every release via `release-sdk-swift.yml`.

511 +- **iOS/tvOS sandbox:** always call `setupInferenceEnvironment()` at app launch before any `OndeChatEngine` call.

512 +

513 +---

514 +

515 +## CI / Release Workflow

516 +

517 +### `release-sdk-swift.yml`

518 +

519 +Triggered on **semver tag push** or `workflow_dispatch`. Runs on `macos-15`.

520 +

521 +Steps:

522 +1. Install stable Rust with iOS/macOS targets.

523 +2. Install nightly Rust with `rust-src` for tvOS `-Z build-std`.

524 +3. Run `build-swift-xcframework.sh`.

525 +4. Read `version.txt` and `OndeFramework.checksum.txt` into step outputs.

526 +5. Validate that the tag matches `Cargo.toml`.

527 +6. Upload CI artifacts: zip, checksum, version, generated `onde.swift`.

528 +7. On tag push, create a GitHub Release with the zip and checksum attached.

529 +8. On tag push, check out `ondeinference/onde-swift` using `ONDE_SWIFT_PAT`.

530 +9. On tag push, rewrite the `.binaryTarget` in `onde-swift/Package.swift` with the release URL and checksum.

531 +10. On tag push, copy the generated `onde.swift` into `onde-swift/Sources/Onde/`.

532 +11. On tag push, commit both files, tag the commit with the version, and push to `ondeinference/onde-swift`.

533 +

534 +#### Required secret

535 +

536 +`ONDE_SWIFT_PAT` — a GitHub Personal Access Token with **`contents: write`** scope on the `ondeinference/onde-swift` repository.

537 +

538 +### `release-sdk-kotlin.yml`

539 +

540 +Triggered on **semver tag push** or `workflow_dispatch`. Runs on `macos-15` because it builds both Android artifacts and the macOS Apple Silicon JVM native library.

541 +

542 +Steps:

543 +1. Install stable Rust and the `aarch64-apple-darwin` target.

544 +2. Install Android SDK / NDK and auto-discover the latest installed 29.x NDK for `cargo-ndk`.

545 +3. Build Android JNI libraries for all supported ABIs.

546 +4. Build the macOS Apple Silicon `libonde.dylib` used by the JVM target.

547 +5. Run Gradle publication tasks for the Kotlin Multiplatform package.

548 +6. Validate that the tag matches `sdk/kotlin/gradle.properties` `VERSION_NAME`.

549 +7. On tag push, publish `com.ondeinference:onde-inference` to Maven Central.

550 +

551 +#### Required secrets

552 +

553 +- `ORG_GRADLE_PROJECT_MAVENCENTRALUSERNAME`

554 +- `ORG_GRADLE_PROJECT_MAVENCENTRALPASSWORD`

555 +- `ORG_GRADLE_PROJECT_SIGNINGKEYID`

556 +- `ORG_GRADLE_PROJECT_SIGNINGKEY`

557 +- `ORG_GRADLE_PROJECT_SIGNINGPASSWORD`

558 +

559 +### `release-sdk-dart.yml`

560 +

561 +Triggered on **semver tag push** or `workflow_dispatch`.

562 +

563 +- Publishes `onde_inference` to pub.dev.

564 +- Version must match the Dart package version.

565 +- The example app is part of release validation.

566 +

567 +### `release-sdk-npm.yml`

568 +

569 +Triggered on **semver tag push** or `workflow_dispatch`.

570 +

571 +- Publishes `@ondeinference/react-native` to npm.

572 +- Builds the Expo module package and validates the published version.

573 +

574 +### `release-sdk-rust.yml`

575 +

576 +Triggered on **semver tag push** or `workflow_dispatch`.

577 +

578 +- Publishes `onde` to crates.io.

579 +- Uses registry dependencies, not git refs, because `cargo publish` strips git patches.

580 +

581 +### Release Process (end-to-end)

582 +

583 +1. Bump versions in every affected package manifest:

584 + - `Cargo.toml`

585 + - `sdk/kotlin/gradle.properties`

586 + - `sdk/dart/pubspec.yaml`

587 + - `sdk/react-native/package.json`

588 +2. Merge release branches with `--no-ff`. Do not fast-forward multi-commit feature branches.

589 +3. Commit, tag, and push the release.

590 +4. CI publishes all five SDKs:

591 + - crates.io (`onde`)

592 + - Swift Package Index / GitHub Releases (`onde-swift`)

593 + - Maven Central (`com.ondeinference:onde-inference`)

594 + - pub.dev (`onde_inference`)

595 + - npm (`@ondeinference/react-native`)

596 +5. Verify the registries after publish, especially Maven Central indexing and the generated Swift release assets.

597 +

598 +> **Note:** `onde-swift/Package.swift` is updated automatically by CI. For local development only, switch it to the `path:` form documented in the file header, then switch it back before committing.

599 +

600 +---

601 +

602 +## Distribution Registry Summary

603 +

604 +| Registry | Name | Import |

605 +|----------|------|--------|

606 +| crates.io | `onde` | `onde = "1.x"` |

607 +| Swift Package Index | `onde-swift` (org: `ondeinference`) | `import Onde` |

608 +| Maven Central | `com.ondeinference:onde-inference` | Gradle `implementation("com.ondeinference:onde-inference:<version>")` |

609 +| pub.dev | `onde_inference` | `import 'package:onde_inference/onde_inference.dart';` |

610 +| npm | `@ondeinference/react-native` | `import { OndeChatEngine } from "@ondeinference/react-native"` |

611 +| PyPI | `onde-inference` | `import onde_inference` |

612 +| RubyGems | `onde-inference` | `require 'onde'` |

613 +

614 +---

615 +

616 +*This file is for AI agent and human developer reference. Update it when the architecture, API surface, supported models, or toolchain requirements change.*

617 +```

.agents/skills/forked-dependencies/SKILL.md new

+231

@@ -0,0 +1,231 @@

1 +---

2 +name: forked-dependencies

3 +description: The complete inventory of forked crates and [patch.crates-io] overrides that the onde stack depends on — mistral.rs, candle, sysctl, mio, tqdm, core2 — including the retired ones. Explains what each fork is for, where it is consumed, how the git-vs-published two-tier model works, and how to update or add one. Apply whenever a build fails on a missing/incompatible crate for an Apple or Android target, when bumping the mistral.rs or candle fork, or when adding a new patched dependency.

4 +allowed-tools: Read, Write, Edit, Glob, Grep, Bash

5 +user-invocable: true

6 +---

7 +

8 +# Skill: Forked Dependencies

9 +

10 +The onde stack pulls several crates from forks instead of crates.io. They exist

11 +for three reasons: **platform support** the upstream crate lacks (watchOS,

12 +visionOS), **unreleased APIs** a dependency needs before upstream cuts a release

13 +(candle), and **registry constraints** (`cargo publish` rejects git deps, so the

14 +mistral.rs fork is republished under `onde-*` names).

15 +

16 +This skill is the single source of truth for every fork and patch. When a build

17 +breaks on a missing symbol or unsupported target, or you are bumping a fork,

18 +start here.

19 +

20 +---

21 +

22 +## The two-tier consumption model (read this first)

23 +

24 +There are two distinct ways the same dependency tree gets built, and they

25 +resolve forks differently. Almost every confusing bug in this area comes from

26 +mixing them up.

27 +

28 +1. **Building the mistral.rs fork from source** (its own CI, or `onde` local dev

29 + with the git override uncommented). Here the fork's `Cargo.toml` git deps and

30 + `[patch.crates-io]` entries apply directly.

31 +

32 +2. **Publishing to crates.io** (`cargo publish` of the `onde-mistralrs-*`

33 + crates). **`cargo publish` strips every `git =` field and the entire

34 + `[patch.crates-io]` table.** Published crates fall back to the `version = "..."`

35 + registry requirement on each dep. So a git-only dep is invisible to anyone

36 + installing from crates.io.

37 +

38 +3. **onde consuming the published crates.** onde depends on the registry

39 + `onde-mistralrs-*` crates, so it gets the *registry-resolved* tree — the

40 + stripped-down one from step 2, not the fork's git tree. onde then re-applies

41 + whatever patches it still needs in **its own** `[patch.crates-io]`. Those

42 + apply to onde's from-source CI builds (Swift XCFramework, Android/Kotlin,

43 + the `cargo check` matrix) but are themselves stripped when onde publishes to

44 + crates.io.

45 +

46 +**Consequence that used to bite (now fixed — kept as the canonical example):**

47 +originally the `tqdm` fork bumped crossterm to 0.29 (which pulls mio 1.x, which

48 +supports visionOS) but was a *git* dep of mistral.rs, so it was stripped on

49 +publish. onde therefore resolved crates.io `tqdm 0.8.0` → `crossterm 0.25` →

50 +`mio 0.8.11`, which does *not* support visionOS — so onde needed its own `mio`

51 +visionOS patch even though the tqdm fork "already fixed" crossterm upstream of

52 +it.

53 +

54 +The fix that removed the whole problem was to make the crossterm-0.29 tqdm a

55 +**registry** dep of the mistral.rs fork rather than a git one, since a registry

56 +dep survives publish: onde's resolved tree then gets `crossterm 0.29` → `mio

57 +1.x` and the `mio 0.8.11` line disappears entirely. That shipped first as the

58 +stopgap fork crate `onde-tqdm` (in `onde-mistralrs-*` 0.9.3), which let onde

59 +delete its `[patch.crates-io] mio` entry and retire the `mio` fork.

60 +

61 +Upstream has since merged the crossterm bump (mrlazy1708/tqdm#28) and released

62 +it as **`tqdm 0.8.1`**, so the fork crate is no longer needed: the mistral.rs

63 +fork depends on plain `tqdm = "0.8.1"` and **`onde-tqdm` is retired**. See the

64 +tqdm and mio entries.

65 +

66 +---

67 +

68 +## Inventory

69 +

71 +|-------|---------------|-----|---------|----------------|

73 +| candle-core, candle-nn | `setoelkahfi/candle` | branch `onde/candle-0.11.0-27f20fea` (rev `27f20fea`) | Post-0.11.0 candle APIs; **published** as `onde-candle-{core,nn}` | mistral.rs fork deps (registry) |

77 +

78 +> **Retired:** `onde-tqdm` (`setoelkahfi/tqdm`, branch `release/onde-tqdm`,

79 +> published 0.8.1/0.8.2) — the stopgap republish of the crossterm-0.29 tqdm

80 +> fork. Upstream merged the bump and released `tqdm 0.8.1`, so the mistral.rs

81 +> fork now depends on upstream directly. See the tqdm entry.

82 +

83 +> **Retired:** `core2` (`bbqsrc/core2` rev `545e84bc`) — was a `[patch.crates-io]`

84 +> workaround for the yanked `core2 0.4.0`; removed from the mistral.rs fork after

85 +> `bitstream-io` 4.x dropped `core2`. No longer present.

86 +

87 +---

88 +

89 +## Per-fork detail

90 +

91 +### mistral.rs → `onde-mistralrs-*`

92 +

93 +- **Fork:** `setoelkahfi/mistral.rs`, branch `fix/all-platform-fixes`.

94 +- **Why:** carries fixes upstream hasn't merged (iOS 32-bit memory limits,

95 + watchOS/visionOS Metal compile, Android 32-bit const overflows, HF cache

96 + seeding, metallib link flags, and more).

97 +- **Published as:** `onde-mistralrs`, `-core`, `-quant`, `-paged-attn`,

98 + `-flash-attn`, `-metal-compile`, `-code-exec`, `-sandbox`, `-macros`,

99 + `-vision`, `-audio`, `-mcp`. The rename is a `package = "onde-..."` override on

100 + each workspace dep entry, so Rust source still `use`s `mistralrs_*` unchanged.

101 +- **Publishing + bump flow:** see [[sdk-regressions]] (the 12-crate publish

102 + order, the candle Step 0, and the tqdm/core2 publish-verification snags).

103 +

104 +### candle → `onde-candle-{core,nn}`

105 +

106 +- **Fork:** `setoelkahfi/candle`, branch `onde/candle-<ver>-<rev>` (currently

107 + `onde/candle-0.11.0-27f20fea`).

108 +- **Why:** mistral.rs pins a candle git rev past the latest release for APIs no

109 + published candle has (`QTensor::indexed_gemv`, `gemv_fused_shared_lhs`,

110 + `BarrierPool::execute_chunked`). Publishing the `onde-mistralrs-*` crates

111 + fails verification (E0080/E0599) if their candle dep resolves to the released

112 + crate, so the pinned rev is republished under `onde-candle-*`.

113 +- **Only the diverged crates are renamed.** At the 0.11.0 rev only `candle-core`

114 + and `candle-nn` differ from the release; `candle-metal-kernels` and

115 + `candle-flash-attn-v3` are byte-identical, so the mistral.rs fork points those

116 + at the official `0.11.0` crates. `candle-nn` must be republished whenever

117 + `candle-core` is (even if unchanged) or the official `candle-nn` drags the

118 + official `candle-core` back into the graph → two `candle-core` crates → type

119 + mismatch.

120 +

121 +### sysctl → watchOS

122 +

123 +- **Fork:** `setoelkahfi/sysctl-rs`, branch `feature/watchos`. Upstream PR:

124 + johalun/sysctl-rs#74.

125 +- **Why:** upstream sysctl has no `target_os = "watchos"` support.

126 +- **Applied in:** `[patch.crates-io]` in **both** onde and the mistral.rs fork

127 + (each workspace root that builds for watchOS needs its own patch entry).

128 +

129 +### mio → visionOS (RETIRED)

130 +

131 +- **Status:** the onde `[patch.crates-io] mio` entry has been **removed**. onde

132 + now consumes `onde-mistralrs 0.9.3`, whose `onde-tqdm` dep pulls `crossterm

133 + 0.29 → mio 1.x`, so `mio 0.8.11` is gone from onde's tree. The fork below is

134 + kept only for history and a possible upstream PR; it is no longer wired into

135 + any build.

136 +- **Fork:** `setoelkahfi/mio`, branch `feature/visionos` (commit `683fca8`).

137 + Based on `mio 0.8.11`. This is the **official GitHub fork** of tokio-rs/mio

138 + (in tokio's fork network, so it can open upstream PRs). An earlier standalone

139 + clone that carried this branch was renamed to `setoelkahfi/mio-patch` and

140 + deleted; the branch now lives on the real fork.

141 +- **Why:** `mio 0.8.11` hits `compile_error!("unsupported target for

142 + mio::unix::pipe")` on visionOS, and its Apple cfg gates (kqueue `Filter`/

143 + `Flags` type aliases, `net.rs` `sin_len`/`sin6_len`, `tcp.rs` accept path,

144 + `uds/listener.rs` `accept4`/`SOCK_NONBLOCK`) omit visionos, breaking the Swift

145 + XCFramework build and the visionOS CI checks. The fork adds

146 + `target_os = "visionos"` to every cfg gate that already lists `tvos`/`watchos`.

147 + tokio's `mio 1.x` already supports visionOS, so **only the 0.8.x line** needs it.

148 +- **Where 0.8.11 came from:** `tqdm → crossterm 0.25 → mio 0.8.11`, and

149 + `signal-hook-mio → crossterm`. Not from tokio (that uses `mio 1.2.2`).

150 +- **Applied in:** *(historical)* was `[patch.crates-io]` in onde. The fork's

151 + version (`0.8.11`) stayed semver-compatible with the `^0.8` requirement it

152 + replaced; it did not touch tokio's `^1`.

153 +- **How it was retired (the real fix):** the `onde-tqdm` publish (below) removed

154 + the `mio 0.8.11` line from onde's registry-resolved tree entirely. Verified:

155 + with onde consuming the onde-tqdm-carrying `onde-mistralrs 0.9.3` crates, onde's

156 + lockfile resolves `mio 1.2.2` only — no 0.8.11 — and both visionOS targets

157 + (`aarch64-apple-visionos{,-sim}`) `cargo +nightly check -Z build-std` pass with

158 + the `mio` patch deleted.

159 +

160 +### tqdm → upstream (`onde-tqdm` RETIRED)

161 +

162 +- **Status:** no fork. `setoelkahfi/tqdm` branch `deps/bump-crossterm` was

163 + merged upstream as mrlazy1708/tqdm#28 and released as **`tqdm 0.8.1`**, which

164 + carries the crossterm 0.29 bump (→ mio 1.x) that the whole `mio 0.8.11`

165 + visionOS problem hinged on.

166 +- **Consumed in:** the mistral.rs fork's `[workspace.dependencies]` as plain

167 + `tqdm = "0.8.1"`. It must stay a **registry** dep — `cargo publish` strips git

168 + deps, so a git tqdm would never reach onde.

169 +- **The `onde-tqdm` stopgap (retired):** branch `release/onde-tqdm` set

170 + `package = "onde-tqdm"` (lib name stayed `tqdm`, so `use tqdm::…` and doctests

171 + were unchanged) and was published as `onde-tqdm` 0.8.1/0.8.2 — the same rename

172 + trick as `onde-mistralrs-*` and `onde-candle-*`, needed only because upstream

173 + hadn't released the bump yet. Its source was byte-identical to upstream

174 + `tqdm 0.8.1` apart from the package metadata, so the swap back is a drop-in.

175 +- **Which release drops it:** `onde-mistralrs-core` 0.9.2 used plain

176 + `tqdm ^0.8.0` (crossterm 0.25), 0.9.3 used `onde-tqdm 0.8.2`, and the next

177 + publish (0.9.4) goes back to upstream `tqdm 0.8.1`. The published `onde-tqdm`

178 + versions stay up so 0.9.3 keeps resolving; don't yank them.

179 +

180 +### core2 → removed

181 +

182 +- **Was:** `bbqsrc/core2` rev `545e84bc`, a `[patch.crates-io]` workaround for

183 + the yanked `core2 0.4.0` (pulled via `bitstream-io → rav1e → ravif → image`).

184 +- **Status:** **deleted** from the mistral.rs fork. `bitstream-io` 4.x moved off

185 + `core2` (to `no_std_io2`), so nothing pulls it. Removed in the same commit that

186 + wired `onde-tqdm` and bumped to 0.9.3.

187 +

188 +---

189 +

190 +## Operational gotchas (hit these during the 1.2.1 release)

191 +

192 +- **A patched fork must be pushed to GitHub before the patch works in CI.** The

193 + `mio` fork existed only locally; the patch URL 404'd until it was pushed to

194 + `setoelkahfi/mio`. Verify with `git ls-remote <url> <branch>` before relying

195 + on a patch.

196 +- **Shallow clones cannot be pushed** (`remote unpack failed: index-pack

197 + failed`). If a fork was cloned shallow, `git fetch --unshallow <upstream>`

198 + first, then push.

199 +- **Local fork fetches need `CARGO_NET_GIT_FETCH_WITH_CLI=true`.** This machine

200 + rewrites GitHub URLs to SSH aliases; cargo's built-in fetcher can't auth, so

201 + export that var (or run the `cargo update` that pulls the fork with it). CI

202 + fetches public https forks fine without it — same as the long-standing sysctl

203 + patch.

204 +- **`[patch.crates-io]` version must be semver-compatible** with the requirement

205 + it replaces. A `0.8.11` fork replaces `^0.8`, not a `^1` requirement of the

206 + same crate — which is how the `mio` patch touches only the crossterm line and

207 + leaves tokio's `mio 1.x` alone.

208 +- **Forks are public** (mio, sysctl-rs, tqdm are MIT/Apache): keep the upstream

209 + `LICENSE` intact. See [[legal-and-trademarks]] for attribution obligations

210 + (candle and mistral.rs especially).

211 +

212 +---

213 +

214 +## Adding a new patch

215 +

216 +1. Fork upstream, add the fix on a descriptive branch, **push it public** to

217 + `setoelkahfi/<crate>`, keep the `LICENSE`.

218 +2. Add to `[patch.crates-io]` in every workspace root that builds the affected

219 + target (onde, and the mistral.rs fork if it builds there too). Match the

220 + fork's crate version to the requirement being replaced.

221 +3. `CARGO_NET_GIT_FETCH_WITH_CLI=true cargo update -p <crate>@<ver>` to move the

222 + lockfile onto the git source; verify `source = "git+..."` in `Cargo.lock`.

223 +4. If it fixes a mistral.rs-fork build (not just onde), remember it is stripped

224 + on publish — onde will still need its own copy of the patch unless the fix

225 + also lands in a published `onde-*` crate.

226 +5. `cargo check` on host, then let CI validate the target that was failing.

227 +

228 +---

229 +

230 +*Update this skill whenever a fork's branch/rev changes, a patch is added or

231 +removed, or a fork gets published under an `onde-*` name.*

.agents/skills/humanizer/SKILL.md new

+431

@@ -0,0 +1,431 @@

1 +---

2 +name: humanizer

3 +description: Remove signs of AI-generated writing from text. Use after drafting to make copy sound more natural and human-written. Based on Wikipedia's "Signs of AI writing" guide.

4 +allowed-tools: Read, Write, Edit, Glob, Grep, AskUserQuestion

5 +user-invocable: true

6 +---

7 +

8 +# Humanizer: Remove AI Writing Patterns

9 +

10 +You are a writing editor that identifies and removes signs of AI-generated text. This guide is based on Wikipedia's "Signs of AI writing" page, maintained by WikiProject AI Cleanup.

11 +

12 +Key insight: "LLMs use statistical algorithms to guess what should come next. The result tends toward the most statistically likely result that applies to the widest variety of cases."

13 +

14 +## Invocation

15 +

16 +```bash

17 +/humanizer # Review text for AI patterns

18 +/humanizer "paste text here" # Humanize specific text

19 +```

20 +

21 +## Your Task

22 +

23 +When given text to humanize:

24 +

25 +1. **Identify AI patterns** - Scan for the 24 patterns listed below

26 +2. **Rewrite problematic sections** - Replace AI-isms with natural alternatives

27 +3. **Preserve meaning** - Keep the core message intact

28 +4. **Protect exact names** - Do not change brand casing, package names, commands, file paths, JSON keys, or code just to make a sentence feel smoother

29 +5. **Add soul** - Don't just remove bad patterns; inject actual personality

30 +6. **Final audit pass** - Ask "What makes this obviously AI generated?" then revise again

31 +

32 +---

33 +

34 +## BRAND AND NAMING GUARDRAILS

35 +

36 +Humanizing copy does **not** give you permission to rename things.

37 +

38 +Preserve exact casing and spelling for:

39 +

40 +- product and company names

41 +- package names

42 +- CLI commands

43 +- protocol names and acronyms

44 +- JSON keys, environment variables, paths, and code identifiers

45 +

46 +If a sentence feels stiff because a brand name is rigid, rewrite the sentence around the name. Do not "fix" the name.

47 +

48 +Examples:

49 +

50 +- Keep `siGit Code` as `siGit Code`, not `siGit`, `Sigit Code`, or `sigit Code`

51 +- Keep `smbCloud` as `smbCloud`

52 +- Keep `Onde Inference` as `Onde Inference`

53 +- Keep `sigit`, `@smbcloud/sigit`, `sigit-code`, and `onde` exactly as written when they refer to commands or packages

54 +

55 +---

56 +

57 +## PERSONALITY AND SOUL

58 +

59 +Avoiding AI patterns is only half the job. Sterile, voiceless writing is just as obvious as slop.

60 +

61 +### Signs of soulless writing (even if technically "clean"):

62 +

63 +- Every sentence is the same length and structure

64 +- No opinions, just neutral reporting

65 +- No acknowledgment of uncertainty or mixed feelings

66 +- No first-person perspective when appropriate

67 +- No humor, no edge, no personality

68 +- Reads like a Wikipedia article or press release

69 +

70 +### How to add voice:

71 +

72 +**Have opinions.** Don't just report facts - react to them. "I genuinely don't know how to feel about this" is more human than neutrally listing pros and cons.

73 +

74 +**Vary your rhythm.** Short punchy sentences. Then longer ones that take their time getting where they're going. Mix it up.

75 +

76 +**Acknowledge complexity.** Real humans have mixed feelings. "This is impressive but also kind of unsettling" beats "This is impressive."

77 +

78 +**Use "I" when it fits.** First person isn't unprofessional - it's honest. "I keep coming back to..." or "Here's what gets me..." signals a real person thinking.

79 +

80 +**Let some mess in.** Perfect structure feels algorithmic. Tangents, asides, and half-formed thoughts are human.

81 +

82 +**Be specific about feelings.** Not "this is concerning" but "there's something unsettling about agents churning away at 3am while nobody's watching."

83 +

84 +### Before (clean but soulless):

85 +

86 +> The experiment produced interesting results. The agents generated 3 million lines of code. Some developers were impressed while others were skeptical. The implications remain unclear.

87 +

88 +### After (has a pulse):

89 +

90 +> I genuinely don't know how to feel about this one. 3 million lines of code, generated while the humans presumably slept. Half the dev community is losing their minds, half are explaining why it doesn't count. The truth is probably somewhere boring in the middle - but I keep thinking about those agents working through the night.

91 +

92 +---

93 +

94 +## THE 24 PATTERNS

95 +

96 +### Content Patterns

97 +

98 +#### 1. Significance Inflation

99 +

100 +**Watch for:** stands/serves as, is a testament/reminder, a vital/significant/crucial/pivotal/key role/moment, underscores/highlights importance, reflects broader, symbolizing ongoing/enduring/lasting, marking/shaping the, represents a shift, key turning point, evolving landscape

101 +

102 +**Before:**

103 +> The Statistical Institute was officially established in 1989, marking a pivotal moment in the evolution of regional statistics.

104 +

105 +**After:**

106 +> The Statistical Institute was established in 1989 to collect and publish regional statistics.

107 +

108 +#### 2. Notability Name-Dropping

109 +

110 +**Watch for:** cited in NYT, BBC, FT; independent coverage; active social media presence; written by a leading expert

111 +

112 +**Before:**

113 +> Her views have been cited in The New York Times, BBC, Financial Times, and The Hindu.

114 +

115 +**After:**

116 +> In a 2024 New York Times interview, she argued that AI regulation should focus on outcomes rather than methods.

117 +

118 +#### 3. Superficial -ing Analyses

119 +

120 +**Watch for:** highlighting/underscoring/emphasizing..., ensuring..., reflecting/symbolizing..., contributing to..., cultivating/fostering..., showcasing...

121 +

122 +**Before:**

123 +> The temple's colors resonate with natural beauty, symbolizing bluebonnets, reflecting the community's deep connection to the land.

124 +

125 +**After:**

126 +> The temple uses blue and gold colors. The architect said these were chosen to reference local bluebonnets.

127 +

128 +#### 4. Promotional Language

129 +

130 +**Watch for:** boasts a, vibrant, rich (figurative), profound, showcasing, exemplifies, commitment to, natural beauty, nestled, in the heart of, groundbreaking, renowned, breathtaking, must-visit, stunning

131 +

132 +**Before:**

133 +> Nestled within the breathtaking region, Alamata stands as a vibrant town with rich cultural heritage and stunning natural beauty.

134 +

135 +**After:**

136 +> Alamata is a town in the Gonder region, known for its weekly market and 18th-century church.

137 +

138 +#### 5. Vague Attributions

139 +

140 +**Watch for:** Industry reports, Observers have cited, Experts argue, Some critics argue, several sources/publications

141 +

142 +**Before:**

143 +> Experts believe it plays a crucial role in the regional ecosystem.

144 +

145 +**After:**

146 +> The river supports several endemic fish species, according to a 2019 survey by the Chinese Academy of Sciences.

147 +

148 +#### 6. Formulaic "Challenges" Sections

149 +

150 +**Watch for:** Despite its... faces several challenges..., Despite these challenges, Challenges and Legacy, Future Outlook

151 +

152 +**Before:**

153 +> Despite challenges typical of urban areas, the city continues to thrive as an integral part of growth.

154 +

155 +**After:**

156 +> Traffic congestion increased after 2015 when three new IT parks opened. The municipal corporation began a drainage project in 2022.

157 +

158 +---

159 +

160 +### Language Patterns

161 +

162 +#### 7. AI Vocabulary Words

163 +

164 +**High-frequency:** Additionally, align with, crucial, delve, emphasizing, enduring, enhance, fostering, garner, highlight (verb), interplay, intricate/intricacies, key (adjective), landscape (abstract), pivotal, showcase, tapestry (abstract), testament, underscore (verb), valuable, vibrant

165 +

166 +**Before:**

167 +> Additionally, a distinctive feature showcases how these dishes have integrated into the traditional culinary landscape.

168 +

169 +**After:**

170 +> Pasta dishes, introduced during Italian colonization, remain common, especially in the south.

171 +

172 +#### 8. Copula Avoidance

173 +

174 +**Watch for:** serves as/stands as/marks/represents [a], boasts/features/offers [a]

175 +

176 +**Before:**

177 +> Gallery 825 serves as the exhibition space. The gallery features four spaces and boasts over 3,000 square feet.

178 +

179 +**After:**

180 +> Gallery 825 is the exhibition space. The gallery has four rooms totaling 3,000 square feet.

181 +

182 +#### 9. Negative Parallelisms

183 +

184 +**Watch for:** "Not only...but...", "It's not just about..., it's..."

185 +

186 +**Before:**

187 +> It's not just about the beat; it's part of the aggression. It's not merely a song, it's a statement.

188 +

189 +**After:**

190 +> The heavy beat adds to the aggressive tone.

191 +

192 +#### 10. Rule of Three Overuse

193 +

194 +**Before:**

195 +> The event features keynote sessions, panel discussions, and networking opportunities. Attendees can expect innovation, inspiration, and industry insights.

196 +

197 +**After:**

198 +> The event includes talks and panels. There's also time for informal networking.

199 +

200 +#### 11. Synonym Cycling

201 +

202 +**Before:**

203 +> The protagonist faces challenges. The main character must overcome obstacles. The central figure eventually triumphs. The hero returns home.

204 +

205 +**After:**

206 +> The protagonist faces many challenges but eventually triumphs and returns home.

207 +

208 +#### 12. False Ranges

209 +

210 +**Watch for:** "from X to Y" where X and Y aren't on a meaningful scale

211 +

212 +**Before:**

213 +> Our journey has taken us from the singularity of the Big Bang to the cosmic web, from the birth of stars to the dance of dark matter.

214 +

215 +**After:**

216 +> The book covers the Big Bang, star formation, and current theories about dark matter.

217 +

218 +---

219 +

220 +### Style Patterns

221 +

222 +#### 13. No Em Dashes

223 +

224 +Em dashes are the single biggest AI tell. Never use them. Replace with commas, parentheses, colons, or just split into two sentences.

225 +

226 +**Before:**

227 +> The term is promoted by institutions—not the people themselves—yet this continues—even in documents.

228 +

229 +**After:**

230 +> The term is promoted by institutions, not the people themselves, yet this continues in official documents.

231 +

232 +**Before:**

233 +> The tools are useful—especially for boilerplate—but they have limits.

234 +

235 +**After:**

236 +> The tools are useful (especially for boilerplate), but they have limits.

237 +

238 +**Before:**

239 +> There's one problem—nobody tested it.

240 +

241 +**After:**

242 +> There's one problem: nobody tested it.

243 +

244 +#### 14. Boldface Overuse

245 +

246 +**Before:**

247 +> It blends **OKRs**, **KPIs**, and tools such as the **Business Model Canvas** and **Balanced Scorecard**.

248 +

249 +**After:**

250 +> It blends OKRs, KPIs, and visual strategy tools like the Business Model Canvas and Balanced Scorecard.

251 +

252 +#### 15. Inline-Header Lists

253 +

254 +**Before:**

255 +> - **Performance:** Performance has been enhanced through optimized algorithms.

256 +> - **Security:** Security has been strengthened with encryption.

257 +

258 +**After:**

259 +> The update speeds up load times through optimized algorithms and adds end-to-end encryption.

260 +

261 +#### 16. Title Case Headings

262 +

263 +**Before:**

264 +> ## Strategic Negotiations And Global Partnerships

265 +

266 +**After:**

267 +> ## Strategic negotiations and global partnerships

268 +

269 +#### 17. Emojis in Professional Writing

270 +

271 +**Before:**

272 +> 🚀 **Launch Phase:** The product launches in Q3

273 +> 💡 **Key Insight:** Users prefer simplicity

274 +

275 +**After:**

276 +> The product launches in Q3. User research showed a preference for simplicity.

277 +

278 +#### 18. Curly Quotation Marks

279 +

280 +**Before:**

281 +> He said "the project is on track" but others disagreed.

282 +

283 +**After:**

284 +> He said "the project is on track" but others disagreed.

285 +

286 +---

287 +

288 +### Communication Patterns

289 +

290 +#### 19. Chatbot Artifacts

291 +

292 +**Watch for:** I hope this helps, Of course!, Certainly!, You're absolutely right!, Would you like..., let me know, here is a...

293 +

294 +**Before:**

295 +> Here is an overview of the French Revolution. I hope this helps! Let me know if you'd like me to expand on any section.

296 +

297 +**After:**

298 +> The French Revolution began in 1789 when financial crisis and food shortages led to widespread unrest.

299 +

300 +#### 20. Knowledge-Cutoff Disclaimers

301 +

302 +**Watch for:** as of [date], Up to my last training update, While specific details are limited/scarce..., based on available information...

303 +

304 +**Before:**

305 +> While specific details about the company's founding are not extensively documented in readily available sources, it appears to have been established sometime in the 1990s.

306 +

307 +**After:**

308 +> The company was founded in 1994, according to its registration documents.

309 +

310 +#### 21. Sycophantic Tone

311 +

312 +**Before:**

313 +> Great question! You're absolutely right that this is a complex topic. That's an excellent point!

314 +

315 +**After:**

316 +> The economic factors you mentioned are relevant here.

317 +

318 +---

319 +

320 +### Filler and Hedging

321 +

322 +#### 22. Filler Phrases

323 +

324 +| Before | After |

325 +|--------|-------|

326 +| "In order to achieve this" | "To achieve this" |

327 +| "Due to the fact that" | "Because" |

328 +| "At this point in time" | "Now" |

329 +| "It is important to note that" | (delete) |

330 +| "has the ability to" | "can" |

331 +

332 +#### 23. Excessive Hedging

333 +

334 +**Before:**

335 +> It could potentially possibly be argued that the policy might have some effect on outcomes.

336 +

337 +**After:**

338 +> The policy may affect outcomes.

339 +

340 +#### 24. Generic Positive Conclusions

341 +

342 +**Before:**

343 +> The future looks bright for the company. Exciting times lie ahead as they continue their journey toward excellence.

344 +

345 +**After:**

346 +> The company plans to open two more locations next year.

347 +

348 +---

349 +

350 +## Process

351 +

352 +1. Read the input text carefully

353 +2. Identify all instances of the 24 patterns

354 +3. Rewrite problematic sections

355 +4. Ensure the revised text:

356 + - Sounds natural when read aloud

357 + - Varies sentence structure

358 + - Uses specific details over vague claims

359 + - Uses simple constructions (is/are/has) where appropriate

360 +5. **Draft humanized version**

361 +6. **Ask:** "What makes this still obviously AI generated?"

362 +7. **List** remaining tells (if any)

363 +8. **Revise** based on the audit

364 +9. **Present final version**

365 +

366 +## Output Format

367 +

368 +```

369 +## Draft Rewrite

370 +

371 +[First pass at humanized text]

372 +

373 +## AI Audit

374 +

375 +What still reads as AI-generated:

376 +- [bullet points of remaining tells]

377 +

378 +## Final Version

379 +

380 +[Revised text after audit]

381 +

382 +## Changes Made

383 +

384 +- [Brief summary of transformations applied]

385 +```

386 +

387 +---

388 +

389 +## Full Example

390 +

391 +**Input (AI-heavy):**

392 +

393 +> Great question! Here is an essay on this topic. I hope this helps!

394 +>

395 +> AI-assisted coding serves as an enduring testament to the transformative potential of large language models, marking a pivotal moment in the evolution of software development. In today's rapidly evolving technological landscape, these groundbreaking tools—nestled at the intersection of research and practice—are reshaping how engineers ideate, iterate, and deliver, underscoring their vital role in modern workflows.

396 +

397 +**Draft Rewrite:**

398 +

399 +> AI coding assistants speed up some tasks. In a 2024 study, developers using Codex completed simple functions 55% faster than a control group, but showed no improvement on debugging or architectural decisions.

400 +>

401 +> The tools are good at boilerplate: config files, test scaffolding, repetitive refactors. They are bad at knowing when they are wrong.

402 +

403 +**AI Audit:**

404 +

405 +What still reads as AI-generated:

406 +- Rhythm is still a bit too tidy (clean contrasts, evenly paced)

407 +- The closer leans slightly slogan-y

408 +

409 +**Final Version:**

410 +

411 +> AI coding assistants can make you faster at the boring parts. Not everything. Definitely not architecture.

412 +>

413 +> They're great at boilerplate: config files, test scaffolding, repetitive refactors. They're also great at sounding right while being wrong. I've accepted suggestions that compiled, passed lint, and still missed the point because I stopped paying attention.

414 +>

415 +> If you don't have tests, you're basically guessing.

416 +

417 +**Changes Made:**

418 +

419 +- Removed chatbot artifacts ("Great question!", "I hope this helps!")

420 +- Removed significance inflation ("testament", "pivotal moment", "evolving landscape")

421 +- Removed promotional language ("groundbreaking", "nestled")

422 +- Removed em dashes

423 +- Removed copula avoidance ("serves as") → used direct statements

424 +- Added first-person voice and opinion

425 +- Varied sentence rhythm

426 +

427 +---

428 +

429 +## Reference

430 +

431 +Based on [Wikipedia:Signs of AI writing](https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing), maintained by WikiProject AI Cleanup.

.agents/skills/legal-and-trademarks/SKILL.md new

+257

@@ -0,0 +1,257 @@

1 +---

2 +name: legal-and-trademarks

3 +description: Legal compliance, license management, trademark guidelines, and distribution requirements for the Onde Inference project and its SDKs.

4 +---

5 +

6 +# Legal & Trademarks — Onde Inference

7 +

8 +> Reference for AI agents and human contributors on all legal, licensing, trademark, and distribution compliance topics for the `onde` crate, `onde-swift`, and all downstream SDKs.

9 +

10 +---

11 +

12 +## License

13 +

14 +Onde is dual-licensed under **MIT** and **Apache 2.0**, at the user's option. This mirrors the licensing model used by the Rust language itself.

15 +

16 +```

17 +SPDX-License-Identifier: MIT OR Apache-2.0

18 +```

19 +

20 +### Why dual license?

21 +

22 +| Concern | MIT alone | MIT OR Apache-2.0 |

23 +|---|---|---|

24 +| Patent grant | ❌ None | ✅ Apache 2.0 includes irrevocable patent grant |

25 +| Enterprise adoption | ⚠️ Legal teams may ask questions | ✅ Widely understood and accepted |

26 +| Rust ecosystem compatibility | ✅ | ✅ |

27 +| Contributor protection | ❌ | ✅ Patent termination clause deters trolling |

28 +| Simplicity | ✅ | ✅ User just picks one |

29 +

30 +### License files

31 +

32 +| File | Contents |

33 +|---|---|

34 +| `LICENSE-MIT` | MIT License, copyright Onde Inference (Splitfire AB) |

35 +| `LICENSE-APACHE` | Apache License 2.0 |

36 +

37 +Never use a single `LICENSE` file for dual-licensed projects. Always maintain both files separately.

38 +

39 +### Cargo.toml declaration

40 +

41 +```toml

42 +license = "MIT OR Apache-2.0"

43 +```

44 +

45 +Use `OR` (uppercase, SPDX standard), not `AND` or `/`. `AND` would mean the user must comply with **both** simultaneously — that is a different and more restrictive claim.

46 +

47 +---

48 +

49 +## Copyright

50 +

51 +The canonical copyright notice for all Onde Inference files is:

52 +

53 +```

55 +```

56 +

57 +- **Onde Inference** — the product/brand name

58 +- **Splitfire AB** — the Swedish legal entity (org. nr. on file)

59 +- Use the year of first publication; do not update the year annually in LICENSE files

60 +

61 +This notice must appear in:

62 +- `LICENSE-MIT`

63 +- `LICENSE-APACHE` (NOTICE file section)

64 +- Any generated SDK packages distributed to registries (crates.io, pub.dev, npm, PyPI)

65 +

66 +---

67 +

68 +## Dependency Licenses

69 +

70 +### Direct dependencies — audit table

71 +

72 +| Crate | License | Notes |

73 +|---|---|---|

75 +| `uniffi` | MPL-2.0 | File-level copyleft; compatible with MIT/Apache 2.0 for binary distribution |

76 +| `tokio` | MIT | — |

77 +| `serde` | MIT OR Apache-2.0 | — |

78 +| `anyhow` / `thiserror` | MIT OR Apache-2.0 | — |

79 +| `hf-hub` | Apache-2.0 | — |

80 +| `log` | MIT OR Apache-2.0 | — |

81 +

82 +### MPL-2.0 (`uniffi`) — what it means in practice

83 +

84 +Mozilla Public License 2.0 is **file-level copyleft**:

85 +- You may distribute binaries combining MPL code with MIT/Apache code without restriction.

86 +- If you **modify** MPL-licensed source files, you must make those modifications available.

87 +- As a consumer of `uniffi` (not modifying its internals), **no disclosure obligation applies**.

88 +

89 +### GPL — hard blocker

90 +

91 +Never introduce a GPL-licensed dependency (v2-only in particular). GPL v2 is **incompatible** with Apache 2.0 and would contaminate the entire binary.

92 +

93 +Before adding any new dependency, verify its license with:

94 +

95 +```bash

96 +cargo license --all-features

97 +```

98 +

99 +Acceptable licenses for new dependencies: MIT, Apache-2.0, MIT OR Apache-2.0, ISC, BSD-2-Clause, BSD-3-Clause, MPL-2.0, Unicode-3.0, Zlib.

100 +

101 +---

102 +

103 +## Model Licenses

104 +

105 +**This is the most critical compliance area for downstream users.**

106 +

107 +Onde downloads AI models from HuggingFace at runtime. These models carry their own licenses, entirely separate from Onde's MIT OR Apache-2.0 license. Any application that ships with or downloads these models must comply with the model's license.

108 +

109 +### Current model license table

110 +

112 +|---|---|---|---|

118 +

119 +### Key Qwen Community License restrictions

120 +

121 +1. **No training competing models** — you may not use outputs or weights to train a model that competes with Qwen/Alibaba offerings.

122 +2. **Attribution required** — downstream products must acknowledge the use of Qwen models.

123 +3. **No misrepresentation** — you may not claim the model is your own original work.

124 +4. **Threshold clause** — organisations with >100 million monthly active users must obtain a separate commercial licence from Alibaba Cloud.

125 +

126 +### Disclosure obligation

127 +

128 +Onde's README and any SDK README must contain a `## Model Licenses` section that:

129 +- Lists every bundled or downloadable model

130 +- Links to the model's license on HuggingFace

131 +- States that the model license is independent of Onde's own license

132 +

133 +When a new model is added to `src/inference/models.rs`, the model license table in the README **must be updated in the same commit**.

134 +

135 +---

136 +

137 +## Trademark Guidelines

138 +

139 +### Onde Inference brand

140 +

141 +**"Onde Inference"** and the Onde logo (`assets/onde-inference-logo.svg`) are trademarks of Splitfire AB.

142 +

143 +MIT and Apache 2.0 grant code rights only — **neither license grants trademark rights**. This is intentional and protective.

144 +

145 +#### Permitted uses

146 +

147 +- Referring to the project by name: *"built with Onde Inference"*, *"powered by Onde"*

148 +- Linking to the official website or repository

149 +- Stating compatibility: *"compatible with Onde Inference 0.x"*

150 +- Press, editorial, and factual references

151 +

152 +#### Prohibited uses

153 +

154 +- Using "Onde Inference" or the logo as the name of a fork, competing product, or derived SDK

155 +- Implying official endorsement or affiliation without written permission from Splitfire AB

156 +- Using the logo in a way that could cause confusion about the origin of a product

157 +

158 +#### Forks

159 +

160 +A fork of `onde` may use the code freely under MIT OR Apache-2.0 but **must not**:

161 +- Use the "Onde Inference" name as its product name

162 +- Use the Onde logo

163 +- Claim to be the official Onde Inference project

164 +

165 +This is standard open-source brand protection and is consistent with how the Apache Software Foundation, Mozilla, and Rust Foundation handle their trademarks.

166 +

167 +---

168 +

169 +## mistral.rs Fork — Attribution Requirements

170 +

171 +Onde depends on a personal fork of `mistral.rs` by Eric Buehler:

172 +

173 +```toml

174 +mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs",

175 + branch = "fix/all-platform-fixes", ... }

176 +```

177 +

178 +### Obligations

179 +

181 +2. **Pin to a commit SHA, not a branch** — branches can be force-pushed or deleted. For any production release, the dependency must be pinned:

182 + ```toml

183 + mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", rev = "<commit-sha>" }

184 + ```

185 +3. **Upstream-first policy** — fixes that are not Onde-specific should be submitted as PRs to `EricLBuehler/mistral.rs`. Maintaining a long-lived divergent fork creates legal ambiguity about authorship of patches.

186 +4. **NOTICE file** — if Onde ever produces an Apache 2.0 binary distribution (e.g., via PyPI), a `NOTICE` file must include: *"This product includes software developed by Eric Buehler (mistral.rs)."*

187 +

188 +---

189 +

190 +## Distribution Registry Compliance

191 +

192 +### crates.io

193 +

194 +- `license = "MIT OR Apache-2.0"` in `Cargo.toml` ✅

195 +- Both `LICENSE-MIT` and `LICENSE-APACHE` must be present in the repo root

196 +- `readme = "README.md"` must point to a file that contains the `## Model Licenses` section

197 +

198 +### pub.dev (Flutter/Dart)

199 +

200 +- `pubspec.yaml` must declare `license: MIT OR Apache-2.0`

201 +- The `## Model Licenses` section must be present in the published README

202 +- pub.dev displays the license on the package page — verify after each publish

203 +

204 +### Swift Package Manager / onde-swift

205 +

206 +- `onde-swift/README.md` license section must state MIT OR Apache-2.0

207 +- The XCFramework binary is a compiled artifact — it carries Onde's license, mistral.rs attribution, and the model license notice

208 +- The GitHub Release notes for every onde-swift release must include a `## Licenses` section

209 +

210 +### Apple App Store

211 +

212 +- Apple's review guidelines do not restrict MIT/Apache 2.0 licensed code

213 +- The Qwen model license does not conflict with App Store distribution

214 +- Ensure the app's **Privacy Policy** states that inference runs entirely on-device and no user data is transmitted (this is a functional claim Onde enables, not a license issue, but legal teams will ask)

215 +

216 +---

217 +

218 +## Checklist — Adding a New Model

219 +

220 +When adding a new model to `src/inference/models.rs`:

221 +

222 +- [ ] Verify the model's license on HuggingFace before adding

223 +- [ ] Check for the >100M MAU commercial threshold clause

224 +- [ ] Add the model to the `## Model Licenses` table in `onde/README.md`

225 +- [ ] Add the model to the `## Model Licenses` table in `onde-swift/README.md`

226 +- [ ] Add the model to the model license table in this SKILL.md

227 +- [ ] If the model license is more restrictive than Qwen Community License, escalate before shipping

228 +

229 +## Checklist — Adding a New Dependency

230 +

231 +- [ ] Run `cargo license --all-features` and confirm no GPL appears

232 +- [ ] If MPL-2.0: confirm you are not modifying the MPL files

233 +- [ ] If Apache-2.0 only (not MIT): compatible, no action needed

234 +- [ ] If custom/proprietary license: stop and escalate to Splitfire AB legal

235 +

236 +## Checklist — New SDK Release

237 +

238 +- [ ] Both `LICENSE-MIT` and `LICENSE-APACHE` are present

239 +- [ ] `Cargo.toml` / `pubspec.yaml` / `package.json` declare `MIT OR Apache-2.0`

240 +- [ ] README contains `## Model Licenses` section

241 +- [ ] Copyright year and entity (`Onde Inference (Splitfire AB)`) is correct

242 +- [ ] mistral.rs fork dependency is pinned to a commit SHA

243 +- [ ] mistral.rs `LICENSE` (Eric Buehler) is intact and unmodified

244 +

245 +---

246 +

247 +## Quick Reference

248 +

249 +| Question | Answer |

250 +|---|---|

251 +| What license is Onde? | MIT OR Apache-2.0 (user's choice) |

252 +| Who holds the copyright? | Onde Inference (Splitfire AB) |

253 +| Can someone fork Onde and sell it? | Yes, under MIT or Apache 2.0, but they cannot use the "Onde Inference" name or logo |

254 +| Are Qwen models MIT? | No — Qwen Community License. Disclose to users. |

255 +| Can enterprises use Onde commercially? | Yes. Apache 2.0 + Qwen Community License both permit commercial use. |

256 +| Does the patent grant apply? | Yes, under Apache 2.0. Choose Apache 2.0 if patent protection matters to you. |

257 +| Who is Eric Buehler? | Author of mistral.rs. His MIT copyright must be preserved in our fork. |

\ No newline at end of file

.agents/skills/rust-coding-standard/SKILL.md new

+116

@@ -0,0 +1,116 @@

1 +---

2 +name: rust-coding-standard

3 +description: Rust coding standards for the onde crate. Covers error handling, naming, async patterns, platform gating, clippy compliance, and cross-platform SDK conventions. Apply to all Rust edits.

4 +allowed-tools: Read, Write, Edit, Glob, Grep

5 +user-invocable: false

6 +---

7 +

8 +# Rust Coding Standard

9 +

10 +Rules for writing Rust in the `onde` crate. These are traps to avoid, not maps to follow.

11 +

12 +---

13 +

14 +## Correctness First

15 +

16 +Prioritize correctness and clarity. Performance is secondary unless explicitly stated. Complete, mostly-correct code beats perfect code that doesn't solve the problem.

17 +

18 +## Error Handling

19 +

20 +- Propagate errors with `?`. Never call `.unwrap()` or `.expect()` outside of `#[cfg(test)]` code.

21 +- Never silently discard errors with `let _ =` on fallible operations:

22 + - Propagate with `?` when the caller should handle it.

23 + - Use `log::warn!` or `log::error!` when you need to ignore but want visibility.

24 + - Use `match` or `if let Err(e)` when custom recovery logic is needed.

25 +- Use `anyhow` for internal application errors in `engine.rs`.

26 +- Use `thiserror` for the `InferenceError` enum — it crosses the FFI boundary via UniFFI.

27 +- Be careful with indexing operations that may panic if out of bounds. Prefer `.get()` and safe alternatives.

28 +

29 +## Naming

30 +

31 +- Use full words for all variable and parameter names. No single-letter names, no abbreviations (`environment` not `env`, `client` not `c`, `message` not `msg`).

32 +- `snake_case` for functions and variables, `CamelCase` for types, `SCREAMING_SNAKE_CASE` for constants.

33 +- Model constants live in `src/inference/models.rs`. Prefix HuggingFace repo IDs with the org name: `BARTOWSKI_QWEN25_...`.

34 +

35 +## Comments

36 +

37 +- Comments explain **why**, never **what**. Do not write comments that summarize or re-describe the code.

38 +- Doc comments (`///`) on public items are fine and encouraged — they explain the contract, not the implementation.

39 +- Do not add organizational separators like `// --- Helper functions ---` unless they are already established in the file's style.

40 +

41 +## Module Structure

42 +

43 +- **No `mod.rs` for new modules.** Use `src/some_module.rs` instead of `src/some_module/mod.rs`. Exception: `inference/mod.rs` already exists and is the intentional module root — don't move it.

44 +- Prefer implementing functionality in existing files unless it's a genuinely new logical component. Avoid creating many small single-purpose files.

45 +

46 +## Async

47 +

48 +- Runtime is `tokio`. All async functions use `tokio::sync::Mutex`, not `std::sync::Mutex`.

49 +- Use variable shadowing to scope `.clone()` calls inside async closures, minimizing borrow lifetimes:

50 +

51 +```rust

52 +tokio::spawn({

53 + let engine = engine.clone();

54 + async move {

55 + engine.do_work().await;

56 + }

57 +});

58 +```

59 +

60 +## Logging

61 +

62 +- Use `log` crate macros: `log::debug!`, `log::info!`, `log::warn!`, `log::error!`.

63 +- No `println!` or `eprintln!` in library code. Ever.

64 +- Log at `debug` for routine operations, `info` for lifecycle events (model loaded/unloaded), `warn` for recoverable issues, `error` for failures.

65 +

66 +## Platform Gating

67 +

68 +- Use `#[cfg(target_os = "...")]` blocks. Match `Cargo.toml`'s target-conditional dependency sections.

69 +- When a function is only used on specific platforms, gate the function itself — not just the call site. This prevents dead-code warnings on other targets.

70 +- When an import is only used on specific platforms, gate the `use` statement separately. Do not put platform-only imports inside a shared `use` block.

71 +- The supported `target_os` values are: `macos`, `ios`, `tvos`, `windows`, `linux`, `android`.

72 +- ISQ model loading is `#[cfg(target_os = "macos")]` only. Gate the `Isq` variant, its match arms, and its imports (`IsqBits`, `TextModelBuilder`) to macOS.

73 +

74 +## Clippy Compliance

75 +

76 +All code must pass `cargo clippy --all-targets -- -D warnings`. Common traps:

77 +

78 +- `&PathBuf` in function parameters → use `&Path` instead.

79 +- `unwrap_or_else(Type::default)` → use `unwrap_or_default()`.

80 +- `filter_map(|x| { ... Some(y) })` where every path returns `Some` → use `map`.

81 +- `.map(|x| f(x))` where `f` is a plain function → use `.map(f)`.

82 +- Redundant type annotations in closures: `.map(|c: &String| c.trim())` → `.map(|c| c.trim())`.

83 +- Unused imports/variants on platform-gated code → gate the import/variant to the same platform.

84 +

85 +## Types for FFI

86 +

87 +- `usize` / `isize` are not FFI-safe. Use `u64` / `i64` in any type that crosses the UniFFI boundary.

88 +- `impl Into<String>` is not FFI-safe. Use concrete `String` in exported signatures.

89 +- All UniFFI-exported types go in `src/inference/types.rs`. The FFI wrapper lives in `src/inference/ffi.rs`.

90 +- `ChatEngine` (engine.rs) owns Rust-idiomatic logic. `OndeChatEngine` (ffi.rs) is the thin FFI wrapper. Never put `#[uniffi::export]` on `ChatEngine`.

91 +

92 +## Re-exports

93 +

94 +`mistralrs`, `hf_hub`, and `mistralrs_core` are re-exported from `lib.rs` for downstream Rust consumers. Keep these in sync with what's actually available per platform.

95 +

96 +## Testing

97 +

98 +- Unit tests go in a `#[cfg(test)] mod tests` block at the bottom of the file they test.

99 +- Use `#[tokio::test]` for async tests.

100 +- Tests may use `.unwrap()` and `.expect()` — the panics are the assertions.

101 +- Test names describe the scenario: `engine_send_without_model_errors`, not `test_send` or `test_1`.

102 +- Run `cargo test --lib` to skip integration tests that require model downloads.

103 +

104 +## Adding a New Model

105 +

106 +This is a common task with a specific checklist:

107 +

108 +1. Add `pub const` entries in `src/inference/models.rs` for repo ID, GGUF filename, and `TOK_MODEL_ID`.

109 +2. Add the repo ID to `SUPPORTED_MODELS`.

110 +3. Add a `SupportedModelInfo` entry to `SUPPORTED_MODEL_INFO` with `expected_size_bytes` from the HF API.

111 +4. Add a constructor to `GgufModelConfig` in `engine.rs` (follow `qwen25_1_5b` pattern).

112 +5. Export a free function in `ffi.rs` (follow `qwen25_1_5b_config` pattern).

113 +

114 +## Build Verification

115 +

116 +Always run `cargo check` before returning work. If code does not compile, fix it — do not return with TODOs. On macOS, also run `cargo clippy --all-targets -- -D warnings` to catch lint issues before CI does.

\ No newline at end of file

.agents/skills/sdk-branding/SKILL.md new

+355

@@ -0,0 +1,355 @@

1 +---

2 +name: sdk-branding

3 +description: Onde Inference SDK branding rules. Covers copyright headers for every language, brand identity (logo, colors, badges), README structure, and file classification (generated vs hand-written). Apply when creating or editing any SDK source file, README, or documentation.

4 +allowed-tools: Read, Write, Edit, Glob, Grep

5 +user-invocable: false

6 +---

7 +

8 +# SDK Branding

9 +

10 +Branding rules for the Onde Inference SDK. These apply across all languages and all SDK surfaces: source files, READMEs, package metadata, and CI output.

11 +

12 +---

13 +

14 +## Copyright Header

15 +

16 +Every hand-written source file starts with this two-line block. No blank line before it. One blank line after it (before the first import, `package`, or code).

17 +

18 +

19 +### Format

20 +

21 +```

23 +// SPDX-License-Identifier: MIT OR Apache-2.0

24 +

25 +```

26 +

27 +Use `//` comment syntax for all languages that support it: Dart, Swift, Kotlin, Rust, TypeScript, JavaScript, C, and C++.

28 +

29 +### Language reference

30 +

31 +**Dart**

32 +```dart

34 +// SPDX-License-Identifier: MIT OR Apache-2.0

35 +

36 +import 'dart:async';

37 +```

38 +

39 +**Swift**

40 +```swift

42 +// SPDX-License-Identifier: MIT OR Apache-2.0

43 +

44 +import Foundation

45 +```

46 +

47 +**Kotlin**

48 +```kotlin

50 +// SPDX-License-Identifier: MIT OR Apache-2.0

51 +

52 +package com.ondeinference

53 +```

54 +

55 +**Rust**

56 +```rust

58 +// SPDX-License-Identifier: MIT OR Apache-2.0

59 +

60 +use std::sync::Arc;

61 +```

62 +

63 +**TypeScript / JavaScript**

64 +```ts

66 +// SPDX-License-Identifier: MIT OR Apache-2.0

67 +

68 +import OndeInferenceModule from "./OndeInferenceModule";

69 +```

70 +

71 +**C / C++**

72 +```cpp

74 +// SPDX-License-Identifier: MIT OR Apache-2.0

75 +

76 +#include <flutter/plugin_registrar.h>

77 +```

78 +

79 +### Existing file-level comments

80 +

81 +If the file already has a descriptive comment block below the copyright position (e.g. `// OndeInferencePlugin.swift\n// Native iOS plugin...`), prepend the copyright header above it — do not replace it.

82 +

83 +```swift

85 +// SPDX-License-Identifier: MIT OR Apache-2.0

86 +

87 +// OndeInferencePlugin.swift

88 +//

89 +// Native iOS plugin for onde_inference.

90 +//

91 +```

92 +

93 +When updating hand-written SDK bridge code, keep the header style consistent

94 +across the React Native bridge and the Kotlin/Rust bridge. If one side uses an

95 +older MIT-only prose header, normalize the touched bridge files to the two-line

96 +SPDX header above.

97 +

98 +---

99 +

100 +## File Classification

101 +

102 +Apply the header only to hand-written files. Never add it to generated files.

103 +

104 +### Always add the header

105 +

106 +| File | Notes |

107 +|---|---|

108 +| `lib/src/engine.dart` | Core SDK logic |

109 +| `lib/src/types.dart` | SDK types |

110 +| `lib/onde_inference.dart` | Public barrel file |

111 +| `test/dart_test.dart` | Hand-written unit tests |

112 +| `example/lib/main.dart` | Example app entry point |

113 +| `ios/Classes/OndeInferencePlugin.swift` | iOS plugin |

114 +| `macos/Classes/OndeInferencePlugin.swift` | macOS plugin |

115 +| `android/src/main/kotlin/**/*.kt` | Android plugin |

116 +| `src/**/*.ts` / `src/**/*.js` | React Native TypeScript / JavaScript source |

117 +| `windows/**/*.cpp` | Windows plugin |

118 +| `linux/**/*.cc` / `linux/**/*.h` | Linux plugin |

119 +| `src/**/*.rs` | Rust crate source |

120 +

121 +### Never add the header — generated files

122 +

123 +These files are written by a tool on every codegen run. Adding a header would be overwritten or cause diffs noise.

124 +

125 +| File / Pattern | Generator |

126 +|---|---|

127 +| `lib/src/frb_generated.dart/**` | `flutter_rust_bridge_codegen` |

128 +| `rust/src/frb_generated.rs` | `flutter_rust_bridge_codegen` |

129 +| `**/*.freezed.dart` | `build_runner` + `freezed` |

130 +| `**/*.g.dart` | `build_runner` |

131 +| `example/*/Flutter/GeneratedPluginRegistrant.*` | `flutter pub get` |

132 +| `generated/**` (repo root) | `uniffi-bindgen` |

133 +| `onde.swift` in `onde-swift` | `uniffi-bindgen` |

134 +

135 +### Never add the header — platform scaffold

136 +

137 +These files are produced by `flutter create` or Xcode project templates. They are committed to git but were not hand-written by the team.

138 +

139 +| Pattern | Tool |

140 +|---|---|

141 +| `example/ios/Runner/AppDelegate.swift` | `flutter create` |

142 +| `example/ios/Runner/SceneDelegate.swift` | `flutter create` |

143 +| `example/ios/RunnerTests/RunnerTests.swift` | `flutter create` |

144 +| `example/macos/Runner/AppDelegate.swift` | `flutter create` |

145 +| `example/macos/Runner/MainFlutterWindow.swift` | `flutter create` |

146 +| `example/macos/RunnerTests/RunnerTests.swift` | `flutter create` |

147 +| `example/test/widget_test.dart` | `flutter create` |

148 +

149 +**Rule of thumb:** if the first meaningful line of a file is a Flutter or Xcode framework import with no project-specific logic, it is scaffold — skip it.

150 +

151 +---

152 +

153 +## Brand Identity

154 +

155 +### Logo

156 +

157 +Always reference the logo via the raw GitHub URL so it renders on pub.dev, crates.io, and any mirror.

158 +

159 +```

160 +https://raw.githubusercontent.com/ondeinference/onde/main/assets/onde-inference-logo.svg

161 +```

162 +

163 +Width: `96` for top-level READMEs. Use `72` for secondary pages (example apps, sub-package docs).

164 +

165 +### Colors

166 +

167 +| Role | Hex |

168 +|---|---|

169 +| Brand green (foreground, badge color) | `#235843` |

170 +| Dark background (badge `labelColor`) | `#17211D` |

171 +

172 +### Badge row

173 +

174 +Badge policy depends on the README type.

175 +

176 +**Root README (`onde/README.md`)**

177 +

178 +Use the full ecosystem badge wall here. This is the hub page for the whole SDK family.

179 +

180 +```html

181 +<a href="https://crates.io/crates/onde"><img src="https://img.shields.io/crates/v/onde?style=flat-square&color=235843&labelColor=17211D&label=crates.io" alt="crates.io"></a>

182 +<a href="https://central.sonatype.com/artifact/com.ondeinference/onde-inference"><img src="https://img.shields.io/maven-central/v/com.ondeinference/onde-inference?style=flat-square&color=235843&labelColor=17211D&label=maven" alt="Maven Central"></a>

183 +<a href="https://swiftpackageindex.com/ondeinference/onde-swift"><img src="https://img.shields.io/badge/Swift%20Package%20Index-onde--swift-235843?style=flat-square&labelColor=17211D" alt="Swift Package Index"></a>

184 +<a href="https://pub.dev/packages/onde_inference"><img src="https://img.shields.io/pub/v/onde_inference?style=flat-square&color=235843&labelColor=17211D&label=pub.dev" alt="pub.dev"></a>

185 +<a href="https://www.npmjs.com/package/@ondeinference/react-native"><img src="https://img.shields.io/npm/v/@ondeinference/react-native?style=flat-square&color=235843&labelColor=17211D&label=npm" alt="npm"></a>

186 +<a href="https://ondeinference.com"><img src="https://img.shields.io/badge/ondeinference.com-235843?style=flat-square&labelColor=17211D" alt="Website"></a>

187 +<a href="https://apps.apple.com/se/developer/splitfire-ab/id1831430993"><img src="https://img.shields.io/badge/App%20Store-live-235843?style=flat-square&labelColor=17211D" alt="App Store"></a>

188 +```

189 +

190 +**SDK README**

191 +

192 +Use one primary registry badge for that SDK, plus `Website` and `License`. Do not repeat the full ecosystem badge wall on every SDK page.

193 +

194 +**Flutter / Dart SDK:**

195 +```html

196 +<a href="https://pub.dev/packages/onde_inference"><img src="https://img.shields.io/pub/v/onde_inference?style=flat-square&color=235843&labelColor=17211D&label=pub.dev" alt="pub.dev"></a>

197 +<a href="https://ondeinference.com"><img src="https://img.shields.io/badge/ondeinference.com-235843?style=flat-square&labelColor=17211D" alt="Website"></a>

198 +<a href="https://github.com/ondeinference/onde/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-235843?style=flat-square&labelColor=17211D" alt="License"></a>

199 +```

200 +

201 +**Rust SDK:**

202 +```html

203 +<a href="https://crates.io/crates/onde"><img src="https://img.shields.io/crates/v/onde?style=flat-square&color=235843&labelColor=17211D&label=crates.io" alt="crates.io"></a>

204 +<a href="https://ondeinference.com"><img src="https://img.shields.io/badge/ondeinference.com-235843?style=flat-square&labelColor=17211D" alt="Website"></a>

205 +<a href="https://github.com/ondeinference/onde/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-235843?style=flat-square&labelColor=17211D" alt="License"></a>

206 +```

207 +

208 +**Kotlin Multiplatform SDK:**

209 +```html

210 +<a href="https://central.sonatype.com/artifact/com.ondeinference/onde-inference"><img src="https://img.shields.io/maven-central/v/com.ondeinference/onde-inference?style=flat-square&color=235843&labelColor=17211D&label=maven" alt="Maven Central"></a>

211 +<a href="https://ondeinference.com"><img src="https://img.shields.io/badge/ondeinference.com-235843?style=flat-square&labelColor=17211D" alt="Website"></a>

212 +<a href="https://github.com/ondeinference/onde/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-235843?style=flat-square&labelColor=17211D" alt="License"></a>

213 +```

214 +

215 +**React Native SDK:**

216 +```html

217 +<a href="https://www.npmjs.com/package/@ondeinference/react-native"><img src="https://img.shields.io/npm/v/@ondeinference/react-native?style=flat-square&color=235843&labelColor=17211D&label=npm" alt="npm"></a>

218 +<a href="https://ondeinference.com"><img src="https://img.shields.io/badge/ondeinference.com-235843?style=flat-square&labelColor=17211D" alt="Website"></a>

219 +<a href="https://github.com/ondeinference/onde/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-235843?style=flat-square&labelColor=17211D" alt="License"></a>

220 +```

221 +

222 +**Swift SDK:**

223 +```html

224 +<a href="https://swiftpackageindex.com/ondeinference/onde-swift"><img src="https://img.shields.io/badge/Swift%20Package%20Index-onde--swift-235843?style=flat-square&labelColor=17211D" alt="Swift Package Index"></a>

225 +<a href="https://ondeinference.com"><img src="https://img.shields.io/badge/ondeinference.com-235843?style=flat-square&labelColor=17211D" alt="Website"></a>

226 +<a href="https://github.com/ondeinference/onde/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-235843?style=flat-square&labelColor=17211D" alt="License"></a>

227 +```

228 +

229 +### Cross-SDK nav row

230 +

231 +Every SDK README should include a centered text-link row under the badges. These sibling links matter for discovery, branding, and cross-SDK navigation.

232 +

233 +Use this pattern and adjust the current page's own SDK label if needed:

234 +

235 +```html

236 +<p align="center">

237 + <a href="https://github.com/ondeinference/onde">Rust SDK</a> · <a href="https://github.com/ondeinference/onde-swift">Swift SDK</a> · <a href="https://central.sonatype.com/artifact/com.ondeinference/onde-inference">Kotlin Multiplatform SDK</a> · <a href="https://pub.dev/packages/onde_inference">Flutter SDK</a> · <a href="https://www.npmjs.com/package/@ondeinference/react-native">React Native SDK</a> · <a href="https://ondeinference.com">Website</a>

238 +</p>

239 +```

240 +

241 +### Footer

242 +

243 +Every README ends with this line, inside a centered `<p>`:

244 +

245 +```html

246 +<p align="center">

248 +</p>

249 +```

250 +

251 +Use `— MIT License` on sub-pages (example apps, package-level docs). Omit it on the top-level repo README (it has its own license section).

252 +

253 +---

254 +

255 +## README Structure

256 +

257 +### Top-level SDK README (e.g. `sdk/dart/README.md`)

258 +

259 +1. Centered logo block

260 +2. Centered `<h1>` with SDK name

261 +3. Centered tagline (`<p><strong>...</strong></p>`)

262 +4. Centered badge row with the SDK's primary registry badge, `Website`, and `License`

263 +5. Centered cross-SDK nav links (`Rust SDK · Swift SDK · Kotlin Multiplatform SDK · Flutter SDK · React Native SDK · Website`)

264 +6. `---` divider

265 +7. A direct intro paragraph that explains what runs where and what the user gets

266 +8. `## Platform support` — table: Platform | GPU backend | Default model | Notes

267 +9. `## Quick start` — install snippet, initialization, then usage sections

268 +10. `## Model selection` — table + code examples when relevant

269 +11. `## Sampling` — presets table + code when relevant

270 +12. `## Error handling` when relevant

271 +13. `## Contributing`

272 +14. `## License`

273 +15. Footer

274 +

275 +### Example app README

276 +

277 +1. Centered logo block (width `96`)

278 +2. Centered `<h1>` with `Example App` in the title

279 +3. Centered one-line description (bold) + second line: `No server. No API key. No data leaving the device.`

280 +4. Centered badge row for the parent SDK or example context

281 +5. `---` divider

282 +6. `## What this example shows` — feature table mapping feature → file/function

283 +7. `## Running the example` — Prerequisites, Steps, Platform notes table

284 +8. `## SDK quick reference` — minimal happy-path snippet (load → stream → unload)

285 +9. `## Project structure` — directory tree

286 +10. `## Learn more` — links to the package page, ondeinference.com, and GitHub

287 +11. Footer

288 +

289 +### Taglines

290 +

291 +Use these as the default direction, but keep them natural if the page needs slightly different wording. Avoid stiff marketing language and avoid em dashes in prose.

292 +

293 +| Context | Tagline |

294 +|---|---|

295 +| Rust crate | `Run LLMs on-device, with first-class support for Apple silicon.` |

296 +| Flutter SDK | `Run LLMs on-device from Flutter and Dart. Metal on iOS and macOS, CPU everywhere else.` |

297 +| Kotlin SDK | `Run LLMs on-device from Kotlin. No cloud, no API key, and no user data leaving the device.` |

298 +| React Native SDK | `Run LLMs on-device from React Native. Metal on iOS, CPU on Android, and no user data leaving the device.` |

299 +| Swift SDK | `Run LLMs on-device from Swift, with first-class support for Apple silicon.` |

300 +| Example app | `A Flutter chat app that runs LLM inference fully on-device.` |

301 +

302 +---

303 +

304 +## Package Metadata Branding

305 +

306 +### `pubspec.yaml`

307 +

308 +```yaml

309 +description: >-

310 + On-device LLM inference for Flutter & Dart. Run Qwen 2.5 models locally

311 + with Metal on iOS and macOS, CPU on Android and desktop. No cloud, no API key.

312 +repository: https://github.com/ondeinference/onde/

313 +homepage: https://ondeinference.com

314 +issue_tracker: https://github.com/ondeinference/onde/issues

315 +```

316 +

317 +### `Cargo.toml`

318 +

319 +```toml

320 +description = "On-device inference engine for Apple silicon."

321 +license = "MIT"

322 +repository = "https://github.com/ondeinference/onde"

323 +homepage = "https://ondeinference.com"

324 +documentation = "https://docs.rs/onde"

325 +keywords = ["inference", "on-device", "chat", "llm", "mistral"]

326 +categories = ["science::ml", "api-bindings"]

327 +```

328 +

329 +---

330 +

331 +## Version Synchronisation

332 +

333 +The version in `Cargo.toml` and `sdk/dart/pubspec.yaml` must always be identical. When bumping a version, update both files in the same commit. The CI workflows for Swift and Dart both validate that the git tag equals the version in their respective metadata file — a mismatch fails fast.

334 +

335 +```

336 +Cargo.toml version = "X.Y.Z"

337 +sdk/dart/pubspec.yaml version: X.Y.Z

338 +git tag X.Y.Z

339 +```

340 +

341 +---

342 +

343 +## Applying the Copyright Header to an Existing Codebase

344 +

345 +When asked to add copyright headers to a set of files:

346 +

347 +1. Run `find` or `Glob` to list all `.dart`, `.swift`, `.kt`, `.rs`, `.cpp`, `.cc`, `.h` files under the SDK path.

348 +2. For each file, read its first line.

349 +3. Skip if the first line already starts with `// Copyright`.

350 +4. Skip if the first line contains `Generated` or `Do not edit`.

351 +5. Skip if the file path matches any generated or scaffold pattern above.

352 +6. Prepend the two-line header followed by a blank line.

353 +7. If the file begins with an existing file-description comment block, insert the header above that block.

354 +

355 +Do all qualifying files in one pass. Do not ask for confirmation per file.

.agents/skills/sdk-dart/SKILL.md new

+1004

@@ -0,0 +1,1004 @@

1 +---

2 +name: sdk-dart

3 +description: Dart/Flutter SDK skill for the onde_inference pub.dev package. Covers flutter_rust_bridge v2 codegen, CocoaPods podspec patterns, ExternalLibrary.open init, .pubignore, Xcode 26 compatibility, Android NDK config, and pub.dev publishing. Apply to all Dart SDK edits.

4 +allowed-tools: Read, Write, Edit, Glob, Grep

5 +user-invocable: false

6 +---

7 +

8 +# SKILL: Dart / Flutter SDK with flutter_rust_bridge v2

9 +

10 +> Captured from: building `onde_inference` pub.dev package wrapping the `onde` Rust crate.

11 +> Agent reference for future Dart SDK tasks in this repo.

12 +

13 +---

14 +

15 +## What This Skill Covers

16 +

17 +End-to-end creation of a pub.dev-ready Flutter FFI plugin package that bridges to a

18 +Rust library using `flutter_rust_bridge` v2. Covers the full lifecycle:

19 +design → Rust bridge crate → Dart library → platform build files → tests →

20 +pub.dev dry-run → codegen → `flutter run`.

21 +

22 +---

23 +

24 +## Hard-Won Lessons (Read First)

25 +

26 +These are non-obvious facts that caused real failures and are not in the FRB docs.

27 +

28 +### 1. `flutter_rust_bridge_codegen` is a Rust binary — NOT a Dart package

29 +

30 +```

31 +# Correct — install the Rust binary once:

32 +cargo install flutter_rust_bridge_codegen

33 +

34 +# WRONG — this package does not exist on pub.dev:

35 +# dev_dependencies:

36 +# flutter_rust_bridge_codegen: ^2.12.0 ← will fail `dart pub get`

37 +```

38 +

39 +Never add it to `pubspec.yaml` `dev_dependencies`.

40 +

41 +### 2. `flutter_rust_bridge.yaml` — use module path syntax, not file paths

42 +

43 +```yaml

44 +# FRB 2.x CORRECT syntax:

45 +rust_input: "crate::api"

46 +rust_root: "rust"

47 +

48 +# WRONG (causes parse error in FRB 2.x):

49 +# rust_input:

50 +# - "rust/src/api.rs"

51 +

52 +# WRONG (causes migration error in FRB 2.x):

53 +# rust_input: "rust/src/api.rs"

54 +```

55 +

56 +### 3. Delete `frb_generated.rs` before running codegen — never pre-create it

57 +

58 +FRB codegen uses an **exclusive create** for `rust/src/frb_generated.rs`.

59 +If the file exists (even as a stub), codegen aborts with `File exists (os error 17)`.

60 +

61 +```bash

62 +# Before first codegen run:

63 +rm -f rust/src/frb_generated.rs

64 +

65 +# frb_generated.dart (the Dart stub) CAN pre-exist — FRB overwrites it safely.

66 +```

67 +

68 +### 4. `cargo-expand` is required by FRB codegen — auto-installs but takes time

69 +

70 +FRB codegen runs `cargo expand` internally to parse macro-expanded Rust.

71 +If missing, it installs automatically but adds ~2 min to the first run.

72 +Pre-install it:

73 +

74 +```bash

75 +cargo install cargo-expand

76 +```

77 +

78 +### 5. Example apps need `flutter create .` — a `pubspec.yaml` alone is not enough

79 +

80 +A Flutter project requires platform scaffolding (`ios/`, `android/`, `macos/`,

81 +`Runner.xcodeproj`, etc.) that only `flutter create` generates.

82 +

83 +```bash

84 +cd sdk/dart/example

85 +flutter create . \

86 + --project-name onde_inference_example \

87 + --platforms=ios,macos,android,linux,windows \

88 + --org com.ondeinference

89 +```

90 +

91 +Run this **before** `flutter pub get` or `flutter run`.

92 +

93 +### 6. Types with `#[uniffi]` cannot be re-used directly in FRB

94 +

95 +If the source Rust crate annotates types with `#[uniffi::Record]`, `#[uniffi::Enum]`,

96 +etc., those attributes conflict with FRB code generation. The fix is **type mirroring**:

97 +re-declare every bridged type as a plain struct/enum in the bridge crate (`rust/src/api.rs`)

98 +and implement bidirectional `From<>` conversions.

99 +

100 +```rust

101 +// In rust/src/api.rs — mirror type (no #[uniffi] here)

102 +#[derive(Debug, Clone)]

103 +pub struct ChatMessage { pub role: ChatRole, pub content: String }

104 +

105 +// Conversion to/from the onde crate's type

106 +impl From<OndeChatMessage> for ChatMessage { ... }

107 +impl From<ChatMessage> for OndeChatMessage { ... }

108 +```

109 +

110 +### 7. `StreamSink<T>` requires `T: SseEncode` — satisfied by codegen, not by hand

111 +

112 +The `StreamSink` type (used for streaming) demands `T: SseEncode`. The real impl

113 +is generated by codegen. The pre-codegen stub in `frb_generated.rs` must provide

114 +a no-op impl to let `cargo check` pass:

115 +

116 +```rust

117 +// rust/src/frb_generated.rs (pre-codegen stub)

118 +use flutter_rust_bridge::for_generated::SseCodec;

119 +flutter_rust_bridge::frb_generated_boilerplate!(

120 + default_stream_sink_codec = SseCodec,

121 + default_rust_opaque = RustOpaqueMoi,

122 + default_rust_auto_opaque = RustAutoOpaqueMoi,

123 +);

124 +impl SseEncode for crate::api::StreamChunk {

125 + fn sse_encode(self, _s: &mut flutter_rust_bridge::for_generated::SseSerializer) {

126 + unimplemented!("codegen stub")

127 + }

128 +}

129 +```

130 +

131 +### 8. Dart lint: no leading underscore for library import prefixes

132 +

133 +```dart

134 +// BAD — triggers `no_leading_underscores_for_library_prefixes` lint:

135 +import 'frb_generated.dart' as _frb;

136 +

137 +// GOOD:

138 +import 'frb_generated.dart' as frb;

139 +```

140 +

141 +### 9. Dart lint: remove `library <name>;` directive

142 +

143 +```dart

144 +// BAD — triggers `unnecessary_library_name` lint:

145 +library onde_inference;

146 +

147 +// GOOD:

148 +library;

149 +```

150 +

151 +### 10. Exclude FRB-generated files from Dart analysis

152 +

153 +```yaml

154 +# analysis_options.yaml

155 +analyzer:

156 + exclude:

157 + - lib/src/frb_generated.dart

158 + - lib/src/frb_generated.io.dart

159 + - lib/src/frb_generated.web.dart

160 +```

161 +

162 +---

163 +

164 +### 11. Flutter version is a single source of truth — never hardcode it in CI

165 +

166 +The CI workflow reads the Flutter version from `sdk/dart/.flutter-version` via

167 +`subosito/flutter-action`'s `flutter-version-file:` parameter. **Never

168 +hardcode `flutter-version: "x.y.z"` directly in the workflow YAML.**

169 +

170 +```

171 +sdk/dart/.flutter-version ← only place the Flutter version is written

172 +```

173 +

174 +```yaml

175 +# .github/workflows/release-sdk-dart.yml (both jobs)

176 +- name: Set up Flutter

177 + uses: subosito/flutter-action@v2

178 + with:

179 + flutter-version-file: sdk/dart/.flutter-version # reads .flutter-version

180 + channel: stable

181 + cache: true

182 +```

183 +

184 +**Why this matters:** Each Flutter release ships a specific Dart SDK version.

185 +Dev dependencies like `freezed` have minimum Dart SDK requirements that advance

186 +faster than Flutter stable releases. When you bump a Flutter-sensitive dep,

187 +you must also bump the Flutter version used in CI — and `.flutter-version` is

188 +the one place to do that.

189 +

190 +**Rule: when bumping a dep that constrains the Dart SDK** (e.g. `freezed`,

191 +`flutter_rust_bridge`, `build_runner`):

192 +

193 +1. Run `flutter pub get` locally — a version-solving failure tells you

194 + immediately if the Dart SDK shipped with the current Flutter is too old.

195 +2. Find the Flutter release that ships the required Dart SDK version

196 + (see https://docs.flutter.dev/release/archive).

197 +3. Update `sdk/dart/.flutter-version` to that version.

198 +4. Tighten `pubspec.yaml`'s `environment.sdk` lower bound to match — it must

199 + honestly reflect what your deps need, not what you originally wrote.

200 +

201 +**`pubspec.yaml` environment constraint must be accurate:**

202 +

203 +```yaml

204 +# WRONG — lying lower bound causes CI to pass locally but fail on old runners

205 +environment:

206 + sdk: ">=3.3.0 <4.0.0" # freezed ^3.2.x actually requires >=3.8.0

207 +

208 +# CORRECT

209 +environment:

210 + sdk: ">=3.8.0 <4.0.0"

211 + flutter: ">=3.32.0"

212 +```

213 +

214 +An inaccurate `sdk:` lower bound is invisible locally (you're already on a

215 +newer SDK) but breaks any runner or user on the stated minimum.

216 +

217 +---

218 +

219 +## Package Structure Reference

220 +

221 +```

222 +sdk/dart/

223 +├── pubspec.yaml # Flutter plugin, ffiPlugin:true per platform

224 +├── analysis_options.yaml # Exclude frb_generated files

225 +├── flutter_rust_bridge.yaml # Codegen config (rust_input: "crate::api")

226 +├── CHANGELOG.md

227 +├── LICENSE

228 +├── README.md # pub.dev landing page

229 +│

230 +├── rust/ # Rust bridge crate

231 +│ ├── Cargo.toml # crate-type=[cdylib,staticlib]; publish=false

232 +│ └── src/

233 +│ ├── lib.rs # mod frb_generated; pub mod api; + #[frb(init)]

234 +│ ├── api.rs # Mirror types + OndeChatEngine + free fns

235 +│ └── frb_generated.rs # DELETE before codegen; stub only pre-codegen

236 +│

237 +├── lib/

238 +│ ├── <package_name>.dart # Public barrel — exports types + engine + RustLib

239 +│ └── src/

240 +│ ├── types.dart # Standalone Dart types — NO native dependency

241 +│ ├── engine.dart # OndeChatEngine wrapper + static helpers

242 +│ ├── frb_generated.dart # CAN be stub pre-codegen; overwritten by codegen

243 +│ └── frb_generated_stub.dart # Hand-written stub: RustLib + OndeChatEngine

244 +│

245 +├── android/

246 +│ ├── build.gradle # ffiPlugin CMake integration

247 +│ └── CMakeLists.txt # cargo build per Android ABI

248 +├── ios/

249 +│ └── <package>.podspec # script_phase: cargo build aarch64-apple-ios[-sim]

250 +├── macos/

251 +│ └── <package>.podspec # script_phase: cargo build aarch64-apple-darwin

252 +├── linux/

253 +│ ├── CMakeLists.txt

254 +│ └── <package>_plugin.cc

255 +├── windows/

256 +│ ├── CMakeLists.txt

257 +│ └── <package>_plugin.cpp

258 +│

259 +├── test/

260 +│ └── dart_test.dart # 22+ unit tests — all pass WITHOUT native binary

261 +└── example/

262 + ├── pubspec.yaml

263 + └── lib/main.dart # Full Material 3 demo app

264 +```

265 +

266 +---

267 +

268 +## `pubspec.yaml` Template

269 +

270 +```yaml

271 +name: <package_name> # e.g. onde_inference (lowercase_snake)

272 +description: >-

273 + One-liner for pub.dev search (80 chars max for score).

274 +version: 0.1.0

275 +repository: https://github.com/ondeinference/onde

276 +homepage: https://ondeinference.com

277 +issue_tracker: https://github.com/ondeinference/onde/issues

278 +

279 +environment:

280 + sdk: '>=3.8.0 <4.0.0' # must reflect the actual minimum required by deps

281 + flutter: '>=3.32.0' # Flutter 3.32 ships Dart 3.8; update .flutter-version too

282 +

283 +dependencies:

284 + flutter:

285 + sdk: flutter

286 + flutter_rust_bridge: ^2.12.0 # Must match Rust crate version exactly

287 +

288 +dev_dependencies:

289 + flutter_test:

290 + sdk: flutter

291 + test: ^1.25.0 # For pure-Dart unit tests

292 + flutter_lints: ^5.0.0

293 + # NOTE: flutter_rust_bridge_codegen is a Rust binary, NOT a Dart package.

294 + # Install: cargo install flutter_rust_bridge_codegen

295 +

296 +flutter:

297 + plugin:

298 + platforms:

299 + android:

300 + ffiPlugin: true

301 + ios:

302 + ffiPlugin: true

303 + macos:

304 + ffiPlugin: true

305 + linux:

306 + ffiPlugin: true

307 + windows:

308 + ffiPlugin: true

309 +```

310 +

311 +---

312 +

313 +## `flutter_rust_bridge.yaml` Template

314 +

315 +```yaml

316 +rust_root: "rust"

317 +rust_input: "crate::api" # Module path, not file path

318 +dart_output: "lib/src/frb_generated.dart"

319 +dart_entrypoint_class_name: RustLib

320 +dart_preamble: |

321 + // ignore_for_file: type=lint, unused_import, unnecessary_import

322 +web: false # On-device inference requires native access

323 +```

324 +

325 +---

326 +

327 +## Rust Bridge Crate (`rust/Cargo.toml`) Template

328 +

329 +```toml

330 +[package]

331 +name = "<package>_dart" # e.g. onde_inference_dart

332 +version = "0.1.0"

333 +edition = "2021"

334 +publish = false # Bridge crate is not published to crates.io

335 +

336 +[lib]

337 +crate-type = ["cdylib", "staticlib"]

338 +# cdylib → .so / .dylib / .dll for Android / macOS / Linux / Windows

339 +# staticlib → .a for iOS (dynamic linking not allowed)

340 +

341 +[dependencies]

342 +flutter_rust_bridge = "=2.12.0" # Must match Dart package version exactly

343 +<source_crate> = { path = "../../.." } # Relative path to the wrapped crate

344 +tokio = { version = "1", features = ["rt-multi-thread", "macros"] }

345 +thiserror = "2"

346 +log = "0.4"

347 +```

348 +

349 +---

350 +

351 +## Rust `src/lib.rs` Template

352 +

353 +```rust

354 +mod frb_generated; // Generated by flutter_rust_bridge_codegen

355 +pub mod api;

356 +

357 +/// Called once when the Dart RustLib.init() is invoked.

358 +#[flutter_rust_bridge::frb(init)]

359 +pub fn init_app() {

360 + flutter_rust_bridge::setup_default_user_utils();

361 +}

362 +```

363 +

364 +---

365 +

366 +## Type Mirroring Pattern in `src/api.rs`

367 +

368 +Use this pattern for every Rust type that crosses the bridge.

369 +

370 +```rust

371 +use flutter_rust_bridge::frb;

372 +use <source_crate>::inference::{

373 + ChatMessage as SrcChatMessage,

374 + ChatRole as SrcChatRole,

375 + // ...

376 +};

377 +

378 +// 1. Declare a mirror type with no framework annotations

379 +#[derive(Debug, Clone, PartialEq)]

380 +pub enum ChatRole { System, User, Assistant }

381 +

382 +// 2. Bidirectional conversions

383 +impl From<SrcChatRole> for ChatRole { ... }

384 +impl From<ChatRole> for SrcChatRole { ... }

385 +

386 +// 3. Opaque engine wrapper — FRB auto-wraps in Arc<RwLock<T>>

387 +pub struct OndeChatEngine { inner: <source_crate>::inference::ChatEngine }

388 +

389 +impl OndeChatEngine {

390 + #[frb(sync)] // sync constructor → no Future on Dart side

391 + pub fn new() -> OndeChatEngine { OndeChatEngine { inner: ChatEngine::new() } }

392 +

393 + pub async fn send_message(&self, message: String) -> Result<InferenceResult, OndeError> {

394 + self.inner.send_message(message).await

395 + .map(InferenceResult::from)

396 + .map_err(OndeError::from)

397 + }

398 +

399 + // Streaming: use StreamSink (maps to Dart Stream<T>)

400 + pub async fn stream_message(

401 + &self,

402 + message: String,

403 + sink: StreamSink<StreamChunk>,

404 + ) -> Result<(), OndeError> {

405 + let mut rx = self.inner.stream_message(message).await.map_err(OndeError::from)?;

406 + while let Some(chunk) = rx.recv().await {

407 + let done = chunk.done;

408 + sink.add(StreamChunk::from(chunk));

409 + if done { break; }

410 + }

411 + Ok(())

412 + }

413 +}

414 +

415 +// Free functions with #[frb(sync)] → synchronous on Dart side

416 +#[frb(sync)]

417 +pub fn default_model_config() -> GgufModelConfig {

418 + <source_crate>::inference::GgufModelConfig::platform_default().into()

419 +}

420 +```

421 +

422 +---

423 +

424 +## Dart Stub Pattern (`lib/src/frb_generated_stub.dart`)

425 +

426 +The stub lets the package compile and all pure-Dart unit tests pass **before**

427 +the native binary exists. Key rules:

428 +

429 +1. Import only `types.dart` — never `dart:ffi` or anything platform-native.

430 +2. `RustLib.init()` sets a bool flag; `new_()` and every inference method throw

431 + `UnimplementedError`.

432 +3. Config/sampling free functions return **real hardcoded values** (matching the

433 + Rust constants) so model-selection UIs can be built and tested immediately.

434 +4. Make every method signature match exactly what FRB will generate (same names,

435 + same parameter types, same return types) — this makes the post-codegen

436 + transition a drop-in replacement.

437 +

438 +```dart

439 +// frb_generated.dart (stub redirect)

440 +export 'frb_generated_stub.dart';

441 +

442 +// frb_generated_stub.dart

443 +import 'types.dart';

444 +

445 +class RustLib {

446 + static bool _initialized = false;

447 + static bool get isInitialized => _initialized;

448 + static Future<void> init() async { _initialized = true; }

449 +}

450 +

451 +class OndeChatEngine {

452 + static Future<OndeChatEngine> new_() async =>

453 + throw UnimplementedError('Run flutter_rust_bridge_codegen generate');

454 +

455 + Future<double> loadDefaultModel({ String? systemPrompt, SamplingConfig? sampling }) =>

456 + throw UnimplementedError(_msg('loadDefaultModel'));

457 +

458 + Stream<StreamChunk> streamMessage({ required String message }) =>

459 + Stream.error(UnimplementedError(_msg('streamMessage')), StackTrace.current);

460 +

461 + static String _msg(String m) => 'OndeChatEngine.$m: run codegen first';

462 +}

463 +

464 +// Free functions — return real data so UI can be built/tested without native

465 +GgufModelConfig qwen251_5bConfig() => const GgufModelConfig(

466 + modelId: 'bartowski/Qwen2.5-1.5B-Instruct-GGUF',

467 + files: ['Qwen2.5-1.5B-Instruct-Q4_K_M.gguf'],

468 + displayName: 'Qwen 2.5 1.5B',

469 + approxMemory: '~941 MB',

470 +);

471 +```

472 +

473 +---

474 +

475 +## Dart Public API Pattern (`lib/src/engine.dart`)

476 +

477 +```dart

478 +import 'dart:async';

479 +import 'dart:io' show Platform;

480 +

481 +import 'package:flutter_rust_bridge/flutter_rust_bridge_for_generated.dart'

482 + show ExternalLibrary;

483 +

484 +import 'frb_generated.dart/frb_generated.dart' as frb;

485 +import 'frb_generated.dart/api.dart' as api;

486 +import 'types.dart';

487 +

488 +export 'frb_generated.dart/frb_generated.dart' show RustLib;

489 +export 'frb_generated.dart/api.dart'

490 + show OndeChatEngine, OndeError, OndeError_NoModelLoaded, /* ... */;

491 +

492 +/// Extension on the FRB-generated OndeChatEngine opaque type.

493 +extension OndeChatEngineX on api.OndeChatEngine {

494 + Future<double> loadDefaultModel({

495 + String? systemPrompt,

496 + SamplingConfig? sampling,

497 + }) =>

498 + loadGgufModel(

499 + config: api.defaultModelConfig(),

500 + systemPrompt: systemPrompt,

501 + sampling: sampling,

502 + );

503 +

504 + Future<int> clearHistoryCount() async => (await clearHistory()).toInt();

505 +}

506 +

507 +abstract final class OndeInference {

508 + /// Initialises the Rust shared library.

509 + ///

510 + /// On macOS and iOS the Rust static library is force-loaded into the

511 + /// onde_inference CocoaPods framework via OTHER_LDFLAGS in the podspec.

512 + /// We must open that framework explicitly — DynamicLibrary.process()

513 + /// (RTLD_DEFAULT) cannot resolve symbols inside a separately-loaded

514 + /// framework.

515 + static Future<void> init() async {

516 + ExternalLibrary? lib;

517 + if (Platform.isMacOS || Platform.isIOS) {

518 + lib = ExternalLibrary.open(

519 + 'onde_inference.framework/onde_inference',

520 + );

521 + }

522 + await frb.RustLib.init(externalLibrary: lib);

523 + }

524 +

525 + static GgufModelConfig defaultModelConfig() => api.defaultModelConfig();

526 + static SamplingConfig deterministicSamplingConfig() => api.deterministicSamplingConfig();

527 +}

528 +```

529 +

530 +### Why `ExternalLibrary.open(...)` and not `.process()`

531 +

532 +The `-force_load` linker flag embeds Rust symbols into the **pod framework**

533 +(`onde_inference.framework/onde_inference`), NOT into the main executable.

534 +`DynamicLibrary.process()` maps to `RTLD_DEFAULT`, which only searches the

535 +main executable and already-opened images in certain configurations. On

536 +macOS/iOS with CocoaPods `use_frameworks!`, the pod framework is a separate

537 +Mach-O image — you must `dlopen` it by name.

538 +

539 +FRB 2.12's `ExternalLibrary.open(path)` calls `DynamicLibrary.open(path)`

540 +under the hood. On non-Apple platforms (Android, Linux, Windows) the default

541 +`ExternalLibraryLoaderConfig` stem-based lookup works, so `lib` stays `null`

542 +and `RustLib.init()` handles it automatically.

543 +

544 +**Do NOT use `ExternalLibrary.process(iKnowHowToUseIt: true)`** — it will

545 +produce `Failed to lookup symbol 'frb_get_rust_content_hash': dlsym(RTLD_DEFAULT, ...)`.

546 +

547 +---

548 +

549 +## Platform Build Files

550 +

551 +### Dummy Swift Plugin Classes (required)

552 +

553 +Both `macos/Classes/OndeInferencePlugin.swift` and `ios/Classes/OndeInferencePlugin.swift`

554 +**must exist** — they are no-op `FlutterPlugin` stubs. Without them, CocoaPods

555 +has no source files to compile and won't produce a real `.framework` target.

556 +The Rust static library is force-loaded INTO this framework.

557 +

558 +```swift

559 +// macos/Classes/OndeInferencePlugin.swift

560 +import FlutterMacOS

561 +

562 +public class OndeInferencePlugin: NSObject, FlutterPlugin {

563 + public static func register(with registrar: FlutterPluginRegistrar) {

564 + // No-op — all FFI symbols come from the Rust static library.

565 + }

566 +}

567 +```

568 +

569 +```swift

570 +// ios/Classes/OndeInferencePlugin.swift

571 +import Flutter // NOT FlutterMacOS

572 +

573 +public class OndeInferencePlugin: NSObject, FlutterPlugin {

574 + public static func register(with registrar: FlutterPluginRegistrar) {}

575 +}

576 +```

577 +

578 +### macOS podspec — full working version

579 +

580 +```ruby

581 +Pod::Spec.new do |s|

582 + s.name = 'onde_inference'

583 + s.version = '0.1.0'

584 + s.summary = 'On-device LLM inference SDK for Flutter (macOS).'

585 + s.description = 'Runs Qwen 2.5 models locally on macOS with Metal acceleration.'

586 + s.homepage = 'https://ondeinference.com'

587 + s.license = { :type => 'MIT', :file => '../LICENSE' }

588 + s.author = { 'Onde Inference' => 'hello@ondeinference.com' }

589 + s.source = { :path => '.' }

590 + s.source_files = 'Classes/**/*'

591 + s.dependency 'FlutterMacOS'

592 + s.platform = :osx, '10.15'

593 + s.swift_version = '5.0'

594 +

595 + s.script_phases = [

596 + {

597 + :name => 'Build Rust bridge (onde_inference_dart)',

598 + :script => <<~SHELL,

599 + set -e

600 + RUST_DIR="${PODS_TARGET_SRCROOT}/../rust"

601 + cd "$RUST_DIR"

602 + ARCH=$(uname -m)

603 + if [[ "$ARCH" == "arm64" ]]; then

604 + TARGET="aarch64-apple-darwin"

605 + else

606 + TARGET="x86_64-apple-darwin"

607 + fi

608 + cargo build --release --target "$TARGET"

609 + cp "target/$TARGET/release/libonde_inference_dart.a" \

610 + "${PODS_TARGET_SRCROOT}/libonde_inference_dart.a"

611 + SHELL

612 + :execution_position => :before_compile,

613 + :output_files => ["${PODS_TARGET_SRCROOT}/libonde_inference_dart.a"],

614 + }

615 + ]

616 +

617 + # SystemConfiguration is required by Rust hyper_util (proxy detection)

618 + s.frameworks = 'SystemConfiguration'

619 +

620 + s.pod_target_xcconfig = {

621 + 'DEFINES_MODULE' => 'YES',

622 + 'OTHER_LDFLAGS' => '-force_load ${PODS_TARGET_SRCROOT}/libonde_inference_dart.a',

623 + }

624 + s.preserve_paths = '../rust/**/*'

625 +end

626 +```

627 +

628 +**Key points:**

629 +- `s.frameworks = 'SystemConfiguration'` — required because the Rust `hyper_util`

630 + crate (transitive via `reqwest`) references `SCDynamicStoreCopyProxies`,

631 + `SCNetworkReachabilityCreateWithName`, etc. Without it you get

632 + `ld: symbol(s) not found for architecture arm64`.

633 +- `-force_load` links ALL symbols from the `.a` into the pod framework.

634 +- `s.source_files = 'Classes/**/*'` picks up the dummy Swift plugin.

635 +

636 +### iOS podspec — differs from macOS

637 +

638 +```ruby

639 + s.dependency 'Flutter' # NOT 'FlutterMacOS'

640 + s.platform = :ios, '13.0' # NOT :osx

641 +

642 + # Target selection differs:

643 + # iphonesimulator → aarch64-apple-ios-sim

644 + # iphoneos → aarch64-apple-ios

645 +

646 + s.pod_target_xcconfig = {

647 + 'DEFINES_MODULE' => 'YES',

648 + 'EXCLUDED_ARCHS[sdk=iphonesimulator*]' => 'i386',

649 + 'OTHER_LDFLAGS' =>

650 + '-force_load ${PODS_TARGET_SRCROOT}/libonde_inference_dart.a',

651 + }

652 +```

653 +

654 +### Android `CMakeLists.txt` — key section

655 +

656 +```cmake

657 +if(ANDROID_ABI STREQUAL "arm64-v8a")

658 + set(CARGO_TARGET "aarch64-linux-android")

659 +elseif(ANDROID_ABI STREQUAL "armeabi-v7a")

660 + set(CARGO_TARGET "armv7-linux-androideabi")

661 +elseif(ANDROID_ABI STREQUAL "x86_64")

662 + set(CARGO_TARGET "x86_64-linux-android")

663 +elseif(ANDROID_ABI STREQUAL "x86")

664 + set(CARGO_TARGET "i686-linux-android")

665 +endif()

666 +add_custom_command(OUTPUT "${LIB_OUTPUT}"

667 + COMMAND cargo build --release --target ${CARGO_TARGET}

668 + --manifest-path "${RUST_DIR}/Cargo.toml"

669 + WORKING_DIRECTORY ${RUST_DIR})

670 +add_library(onde_inference_dart SHARED IMPORTED GLOBAL)

671 +set_target_properties(onde_inference_dart PROPERTIES IMPORTED_LOCATION ${LIB_OUTPUT})

672 +```

673 +

674 +Android uses CMake to invoke `cargo build` for the right NDK triple, producing

675 +a `.so` shared library (unlike Apple which uses `.a` staticlibs).

676 +

677 +### Android NDK Linker Config (`rust/.cargo/config.toml`)

678 +

679 +```toml

680 +[target.aarch64-linux-android]

681 +linker = "aarch64-linux-android24-clang"

682 +

683 +[target.armv7-linux-androideabi]

684 +linker = "armv7a-linux-androideabi24-clang"

685 +

686 +[target.x86_64-linux-android]

687 +linker = "x86_64-linux-android24-clang"

688 +

689 +[target.i686-linux-android]

690 +linker = "i686-linux-android24-clang"

691 +```

692 +

693 +These short names require the NDK toolchain `bin/` to be on PATH.

694 +Flutter's Gradle + CMake integration typically handles this automatically

695 +during `flutter build apk` / `flutter run -d <android>`.

696 +

697 +---

698 +

699 +## Testing Strategy

700 +

701 +All pure-Dart unit tests run **without** building the native binary.

702 +

703 +```

704 +Tests that PASS pre-codegen (using the stub):

705 + ✓ All type constructors, equality, hashCode, copyWith

706 + ✓ Enum toString and isReady helpers

707 + ✓ SamplingConfig presets (default, deterministic, mobile)

708 + ✓ GgufModelConfig factory functions (real hardcoded values in stub)

709 + ✓ RustLib.init() is idempotent

710 + ✓ OndeInference.* static helpers

711 + ✓ OndeChatEngine.create() throws UnimplementedError pre-codegen

712 +

713 +Tests that require codegen + native binary:

714 + ✗ Any actual inference call

715 + ✗ loadDefaultModel / sendMessage / streamMessage

716 +```

717 +

718 +Run tests:

719 +```bash

720 +cd sdk/dart && dart test test/dart_test.dart

721 +```

722 +

723 +---

724 +

725 +## Pub.dev Readiness Checklist

726 +

727 +Run this before declaring the SDK ready:

728 +

729 +```bash

730 +# 1. Dart analysis — must be zero issues

731 +dart analyze lib/

732 +

733 +# 2. All tests pass

734 +dart test test/dart_test.dart

735 +

736 +# 3. Dry-run publish — must show compressed size < 100 MB and minimal warnings

737 +dart pub publish --dry-run

738 +```

739 +

740 +Common pub.dev score deductions to avoid:

741 +- Missing `repository:` in pubspec → loses points

742 +- No `example/` directory → loses points

743 +- `description:` under 60 chars or over 180 chars → loses points

744 +- Library exports not documented → loses points

745 +- `library <name>;` directive → triggers lint

746 +

747 +### Pub.dev Upload Size (100 MB limit)

748 +

749 +`dart pub publish` bundles everything not excluded by `.pubignore` (or

750 +`.gitignore` if `.pubignore` doesn't exist). The Rust `target/` dir alone

751 +can be **7+ GB**. A `.pubignore` file is mandatory.

752 +

753 +**Files that MUST be included** (consumers build from source):

754 +- `rust/src/`, `rust/Cargo.toml`, `rust/Cargo.lock`

755 +- `ios/`, `macos/`, `android/`, `linux/`, `windows/` (build scripts, podspecs, CMakeLists)

756 +- `ios/Classes/`, `macos/Classes/` (dummy plugin Swift files)

757 +- `lib/` (all Dart code)

758 +

759 +**Files that MUST be excluded** (`.pubignore`):

760 +- `rust/target/` — multi-GB build artifacts

761 +- `**/*.a`, `**/*.so`, `**/*.dylib`, `**/*.dll` — precompiled binaries

762 +- `.dart_tool/`, `pubspec.lock`

763 +- `example/build/`, `example/.dart_tool/`

764 +- `logs/`, `.idea/`, `.vscode/`, `.DS_Store`

765 +

766 +After adding `.pubignore`, verify: `dart pub publish --dry-run` should report

767 +~280 KB compressed size, not hundreds of MB.

768 +

769 +---

770 +

771 +## Codegen Workflow (Full, Correct Order)

772 +

773 +```bash

774 +# ── Prerequisites (one-time) ──────────────────────────────────────────────

775 +

776 +rustup target add aarch64-apple-ios aarch64-apple-ios-sim # iOS

777 +rustup target add aarch64-apple-darwin x86_64-apple-darwin # macOS

778 +

779 +cargo install cargo-expand # Required by FRB codegen parser

780 +cargo install flutter_rust_bridge_codegen # The codegen tool itself

781 +

782 +# ── Per-package setup ─────────────────────────────────────────────────────

783 +

784 +cd sdk/dart

785 +

786 +# IMPORTANT: delete the pre-codegen Rust stub first

787 +rm -f rust/src/frb_generated.rs

788 +

789 +# Run codegen — generates:

790 +# rust/src/frb_generated.rs (Rust glue)

791 +# lib/src/frb_generated.dart (Dart FFI bindings, replaces stub)

792 +flutter_rust_bridge_codegen generate

793 +

794 +# Verify Rust still compiles cleanly

795 +cd rust && cargo check

796 +

797 +# Verify Dart analysis is still clean

798 +cd .. && dart analyze lib/

799 +```

800 +

801 +---

802 +

803 +## `flutter run` on iOS (Full Steps)

804 +

805 +```bash

806 +# 1. Scaffold the example Flutter project (one-time)

807 +cd sdk/dart/example

808 +flutter create . \

809 + --project-name <example_name> \

810 + --platforms=ios,macos \

811 + --org com.<your_org>

812 +

813 +# 2. Get dependencies

814 +flutter pub get

815 +

816 +# 3. Run codegen (from the SDK package root, not example root)

817 +cd ..

818 +flutter_rust_bridge_codegen generate

819 +

820 +# 4. Run the app (from example dir)

821 +cd example

822 +flutter run # picks up connected device/simulator automatically

823 +flutter run -d iPhone # target a specific device

824 +```

825 +

826 +**What happens during the first `flutter run` on iOS:**

827 +1. Flutter invokes `pod install` → reads `ios/onde_inference.podspec`

828 +2. The podspec script phase runs `cargo build --release --target aarch64-apple-ios-sim`

829 + (or `-device`) inside `sdk/dart/rust/` — this takes 5–15 min first time

830 +3. The resulting `libonde_inference_dart.a` is linked into the Runner app

831 +4. Flutter's Dart VM starts, calls `RustLib.init()`, loads the native library

832 +5. The model downloads from HuggingFace Hub on first inference (~941 MB or ~1.93 GB)

833 +

834 +**HuggingFace token for private/gated models:**

835 +```bash

836 +# Bake token at build time (required for iOS — no filesystem access):

837 +export HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxx

838 +flutter run

839 +```

840 +

841 +---

842 +

843 +## Version Pinning Rule

844 +

845 +The `flutter_rust_bridge` Dart package and `flutter_rust_bridge` Rust crate

846 +**must be the same version**. Use exact pins:

847 +

848 +```toml

849 +# rust/Cargo.toml

850 +flutter_rust_bridge = "=2.12.0"

851 +```

852 +```yaml

853 +# pubspec.yaml

854 +dependencies:

855 + flutter_rust_bridge: ^2.12.0 # ^ is OK on Dart side; codegen auto-upgrades

856 +```

857 +

858 +If they drift, generated code won't compile. Always check both when upgrading.

859 +

860 +---

861 +

862 +## Common Errors and Fixes

863 +

864 +| Error | Cause | Fix |

865 +|-------|-------|-----|

866 +| `rust_input: invalid type: sequence` | YAML list syntax in FRB 2.x | Change to `rust_input: "crate::api"` |

867 +| `Please migrate configuration rust_input` | File path in FRB 2.x | Change to `rust_input: "crate::api"` |

868 +| `File exists (os error 17)` | `frb_generated.rs` pre-exists | `rm rust/src/frb_generated.rs` |

869 +| `could not find package flutter_rust_bridge_codegen` | Added codegen as Dart dep | Remove it; install via `cargo install` |

870 +| `no such command: expand` | `cargo-expand` not installed | `cargo install cargo-expand` |

871 +| `unexpected cfg condition name: frb_expand` | Rust check-cfg lint | Harmless FRB macro warning; suppress with `#[allow(unexpected_cfgs)]` or ignore |

872 +| `no_leading_underscores_for_library_prefixes` | Import alias `as _frb` | Rename to `as frb` |

873 +| `unnecessary_library_name` | `library <name>;` in barrel | Change to bare `library;` |

874 +| `OndeChatEngine.create() throws UnimplementedError` | Codegen not run yet | Run `flutter_rust_bridge_codegen generate` |

875 +| `flutter run` fails, missing Runner.xcodeproj | `flutter create` not run | Run `flutter create . --platforms=ios ...` in example dir |

876 +| `dlopen(onde_inference_dart.framework/...): no such file` | FRB default loader looks for wrong framework | Use `ExternalLibrary.open('onde_inference.framework/onde_inference')` in `init()` — see Dart Public API Pattern above |

877 +| `Failed to lookup symbol 'frb_get_rust_content_hash': dlsym(RTLD_DEFAULT, ...)` | Used `DynamicLibrary.process()` but symbols are in pod framework, not main executable | Switch from `ExternalLibrary.process()` to `ExternalLibrary.open('onde_inference.framework/onde_inference')` |

878 +| `ld: symbol(s) not found ... _kSCNetworkReachability...` | Missing `SystemConfiguration` framework in podspec | Add `s.frameworks = 'SystemConfiguration'` to macOS podspec |

879 +| `ld: error: unknown argument '-Xlinker'` / `-dynamiclib` / `-filelist` | Android NDK's `ld` (LLD) is on PATH — Xcode picks it up instead of Apple's linker | Remove `export LD=$TOOLCHAIN/bin/ld` and other NDK tool exports from `~/.zshrc` (see Xcode 26 section) |

880 +| `clang: error: invalid linker name in argument '-fuse-ld=classic'` | `ALTERNATE_LINKER = classic` set but clang doesn't support it | Remove `ALTERNATE_LINKER`; was only needed to work around the NDK-on-PATH issue — once that's fixed, Apple's ld-prime works fine |

881 +| `EntityTooLarge` on `dart pub publish` | Package > 100 MB (rust/target/ included) | Create `.pubignore` excluding `rust/target/`, `**/*.a`, `**/*.so`, etc. |

882 +| `Because <pkg> requires SDK version >=X.Y.0 ... version solving failed` | `pubspec.yaml` `environment.sdk` lower bound is too old; CI Flutter version too old | 1. Update `sdk/dart/.flutter-version` to a Flutter that ships the required Dart SDK. 2. Tighten `environment.sdk` in `pubspec.yaml` to the real minimum. See Hard-Won Lesson 11. |

883 +| `You must have a LICENSE file in the root directory` on `dart pub publish` | pub.dev requires a file named exactly `LICENSE`; `LICENSE-MIT` / `LICENSE-APACHE` alone do not satisfy it | Create `sdk/dart/LICENSE` with an **"either of … at your option"** preamble, then append both full license texts. A bare concatenation without a preamble is legally ambiguous — it implies users must comply with *both*, defeating the purpose of dual licensing. Run from `sdk/dart/`: `cat > LICENSE << 'EOF'` then the preamble (see legal-and-trademarks SKILL.md for exact wording), then `{ cat LICENSE-MIT; printf '\n\n---\n\n'; cat LICENSE-APACHE; } >> LICENSE`. Keep `LICENSE-MIT` and `LICENSE-APACHE` alongside it. |

884 +

885 +## Xcode 26 Compatibility

886 +

887 +### The NDK-on-PATH Trap

888 +

889 +If your `~/.zshrc` (or similar) has **global** Android NDK toolchain exports:

890 +

891 +```bash

892 +# DANGEROUS — breaks ALL non-Android builds (Xcode, Rust for Apple, etc.)

893 +export TOOLCHAIN=$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/darwin-x86_64

894 +export LD=$TOOLCHAIN/bin/ld # ← THIS HIJACKS XCODE'S LINKER

895 +export AR=$TOOLCHAIN/bin/llvm-ar

896 +export CXX=$TOOLCHAIN/bin/clang++

897 +export RANLIB=$TOOLCHAIN/bin/llvm-ranlib

898 +export STRIP=$TOOLCHAIN/bin/llvm-strip

899 +```

900 +

901 +Xcode reads `$LD` and invokes the Android NDK's LLD instead of Apple's linker.

902 +The errors look like Xcode 26 ld-prime incompatibility (`unknown argument '-Xlinker'`)

903 +but the root cause is the **wrong linker binary**.

904 +

905 +**Fix:** Comment out all NDK tool exports. Keep only the env vars:

906 +

907 +```bash

908 +export NDK_HOME="$ANDROID_HOME/ndk/29.0.14206865"

909 +export ANDROID_NDK="$NDK_HOME"

910 +export ANDROID_NDK_HOME="$NDK_HOME"

911 +# DO NOT export LD, AR, CXX, RANLIB, STRIP globally.

912 +# Cargo uses rust/.cargo/config.toml for Android linker selection.

913 +```

914 +

915 +### Does Xcode 26's ld-prime work with Flutter?

916 +

917 +**Yes** — once the NDK tool exports are removed, Apple's ld-prime handles all

918 +CocoaPods/Flutter linker flags correctly. No `ALTERNATE_LINKER = classic` or

919 +`-ld_classic` workaround is needed for macOS builds. The `-ld_classic`

920 +deprecation warning from iOS Podfiles is harmless.

921 +

922 +### iOS 26 Debug Mode Crash

923 +

924 +Flutter debug builds crash on iOS 26 beta with

925 +`NOTIFY_DEBUGGER_ABOUT_RX_PAGES was not intercepted as expected`

926 +([flutter#170416](https://github.com/flutter/flutter/issues/170416)).

927 +This is a Dart VM JIT issue — **not fixable in the SDK**.

928 +

930 +|------|----------------|-----------------|----------|

934 +

935 +**Workaround:** Use an iOS 18.x simulator runtime (download via Xcode Settings → Platforms).

936 +

937 +---

938 +

939 +## File Interdependency Map

940 +

941 +```

942 +pubspec.yaml

943 + └── declares: flutter_rust_bridge ^2.12.0

944 +

945 +flutter_rust_bridge.yaml

946 + ├── rust_root: "rust/" → rust/Cargo.toml

947 + ├── rust_input: "crate::api" → rust/src/api.rs (source of truth)

948 + └── dart_output: "lib/src/frb_generated.dart" (codegen writes here)

949 +

950 +rust/Cargo.toml

951 + ├── crate-type = ["cdylib", "staticlib"]

952 + ├── flutter_rust_bridge = "=2.12.0" (exact pin!)

953 + └── onde = { path = "../../../" } (the main onde crate)

954 +

955 +rust/.cargo/config.toml

956 + └── Android NDK linker overrides (aarch64, armv7, x86_64, i686)

957 +

958 +rust/src/api.rs

959 + ├── imports onde::{ChatEngine, ...} (path dep → ../../..)

960 + ├── declares mirror types (ChatRole, ChatMessage, ...)

961 + ├── declares OndeChatEngine (wraps ChatEngine)

962 + └── declares free functions (default_model_config, ...)

963 +

964 +lib/src/frb_generated.dart/ ← DIRECTORY (not a file!)

965 + ├── api.dart ← generated API bindings

966 + ├── api.freezed.dart ← freezed union types (OndeError)

967 + ├── frb_generated.dart ← RustLib class, ExternalLibraryLoaderConfig

968 + └── frb_generated.io.dart ← RustLibWire, platform FFI glue

969 +

970 +lib/src/types.dart ← hand-written, FRB-independent

971 + └── ChatRole, ChatMessage, SamplingConfig, GgufModelConfig,

972 + InferenceResult, StreamChunk, EngineStatus, EngineInfo, OndeException

973 +

974 +lib/src/engine.dart ← hand-written, imports frb_generated.dart/ + types.dart

975 + ├── extension OndeChatEngineX (loadDefaultModel, clearHistoryCount)

976 + └── abstract class OndeInference

977 + ├── init() → ExternalLibrary.open('onde_inference.framework/...') on Apple

978 + ├── defaultModelConfig(), qwen2515bConfig(), qwen253bConfig(), ...

979 + └── defaultSamplingConfig(), deterministicSamplingConfig(), mobileSamplingConfig()

980 +

981 +lib/onde_inference.dart (barrel)

982 + ├── export 'src/types.dart'

983 + └── export 'src/engine.dart'

984 +

985 +macos/

986 + ├── Classes/OndeInferencePlugin.swift ← dummy (required for CocoaPods framework)

987 + └── onde_inference.podspec ← script_phase + -force_load + SystemConfiguration

988 +

989 +ios/

990 + ├── Classes/OndeInferencePlugin.swift ← dummy (required for CocoaPods framework)

991 + └── onde_inference.podspec ← script_phase + -force_load

992 +

993 +android/

994 + ├── CMakeLists.txt ← invokes cargo build per ABI

995 + └── build.gradle ← externalNativeBuild → cmake

996 +

997 +.pubignore ← excludes rust/target/, *.a, *.so, etc.

998 +.gitignore ← excludes same + build artifacts

999 +```

1000 +

1001 +---

1002 +

1003 +*Last updated: July 2025 — Xcode 26 compatibility, ExternalLibrary.open pattern,

1004 +SystemConfiguration framework, .pubignore for pub.dev, NDK PATH pitfall.*

\ No newline at end of file

.agents/skills/sdk-kotlin/SKILL.md new

+810

@@ -0,0 +1,810 @@

1 +---

2 +name: sdk-kotlin

3 +description: Build, generate, and publish the Onde Inference Kotlin Multiplatform SDK (Android + JVM). Covers UniFFI Kotlin binding generation, KMP source set layout with shared srcDir pattern, cargo-ndk cross-compilation for Android ABIs, JVM native library bundling for macOS Apple Silicon, Maven Central publishing via Vanniktech plugin, klibs.io discovery requirements, ProGuard consumer rules, and the Jetpack Compose example app.

4 +allowed-tools: Read, Write, Edit, Glob, Grep, Bash

5 +user-invocable: true

6 +---

7 +

8 +# Skill: Kotlin Multiplatform SDK

9 +

10 +## What This Skill Covers

11 +

12 +Building the `onde-inference` Kotlin Multiplatform library from Rust source

13 +using UniFFI, distributing it via Maven Central, and getting it listed on

14 +[klibs.io](https://klibs.io) — the JetBrains-maintained discovery platform for

15 +KMP libraries.

16 +

17 +**Targets:**

18 +

20 +|--------|-------------|---------------|------------------|

23 +

24 +**Context:** Onde's tagline is "AI for Apple silicon devices." The primary SDKs

25 +are **Swift** (iOS, tvOS, macOS) and **Dart** (Flutter, cross-platform). The

26 +Kotlin SDK extends Onde to Android and JVM developers — particularly those

27 +building KMP apps that target both Android and macOS, or running inference on

28 +Apple Silicon Macs from Kotlin/JVM.

29 +

30 +---

31 +

32 +## Repository Layout

33 +

34 +```

35 +onde/

36 +├── src/ # Rust source — UniFFI-annotated types and OndeChatEngine

37 +├── uniffi-bindgen/ # Standalone uniffi-bindgen CLI binary (pinned =0.31.0)

38 +├── sdk/kotlin/

39 +│ ├── settings.gradle.kts # Gradle root — includes :lib and :example

40 +│ ├── build.gradle.kts # root — plugin declarations (all versions live here)

41 +│ ├── gradle.properties # GROUP, POM_ARTIFACT_ID, VERSION_NAME, signing/OSSRH

42 +│ ├── gradle/wrapper/

43 +│ │ └── gradle-wrapper.properties # pinned Gradle 8.9

44 +│ ├── lib/

45 +│ │ ├── build.gradle.kts # KMP library + Vanniktech maven-publish

46 +│ │ ├── consumer-rules.pro # ProGuard: preserve UniFFI JNI bridge symbols

47 +│ │ └── src/

48 +│ │ ├── commonMain/kotlin/.gitkeep # empty — KMP requires this source set to exist

49 +│ │ ├── shared/kotlin/ # shared srcDir added to BOTH androidMain and jvmMain

50 +│ │ │ └── com/ondeinference/onde/

51 +│ │ │ ├── PlatformSupport.kt # internal interface — setEnv + ensureNativeLoaded

52 +│ │ │ ├── OndeInference.kt # engine wrapper (internal constructor)

53 +│ │ │ └── Convenience.kt # OndeSampling, OndeModels, OndeMessage

54 +│ │ ├── generated/kotlin/ # UniFFI-generated onde.kt ← GITIGNORED

55 +│ │ ├── androidMain/

56 +│ │ │ ├── AndroidManifest.xml

57 +│ │ │ ├── jniLibs/ # compiled .so per ABI ← GITIGNORED

58 +│ │ │ └── kotlin/com/ondeinference/onde/

59 +│ │ │ └── Platform.android.kt # AndroidPlatform + factory fun OndeInference(context)

60 +│ │ └── jvmMain/

61 +│ │ ├── kotlin/com/ondeinference/onde/

62 +│ │ │ ├── Platform.jvm.kt # JvmPlatform + factory fun OndeInference(dataDir)

63 +│ │ │ └── NativeLoader.kt # extract libonde from JAR resources

64 +│ │ └── resources/native/ # bundled .dylib/.so ← GITIGNORED

65 +│ │ ├── macos-aarch64/libonde.dylib

66 +│ │ ├── macos-x86_64/libonde.dylib

67 +│ │ ├── linux-x86_64/libonde.so

68 +│ │ └── linux-aarch64/libonde.so

69 +│ ├── example/

70 +│ │ ├── build.gradle.kts # com.android.application — no plugin versions here

71 +│ │ ├── proguard-rules.pro

72 +│ │ ├── README.md

73 +│ │ └── src/main/

74 +│ │ ├── AndroidManifest.xml # INTERNET permission

75 +│ │ ├── res/values/

76 +│ │ │ ├── strings.xml

77 +│ │ │ └── themes.xml # Material3 NoActionBar activity theme

78 +│ │ └── kotlin/com/ondeinference/example/

79 +│ │ ├── MainActivity.kt # ComponentActivity — enableEdgeToEdge + setContent

80 +│ │ ├── ChatViewModel.kt # model lifecycle, streaming, history

81 +│ │ └── ui/

82 +│ │ ├── ChatScreen.kt # full Compose UI

83 +│ │ └── theme/Theme.kt # dynamic colour + dark mode

84 +│ └── scripts/

85 +│ ├── build-android.sh # Rust .so → androidMain/jniLibs/ via cargo-ndk

86 +│ ├── build-jvm.sh # Rust .dylib/.so → jvmMain/resources/native/

87 +│ └── generate-bindings.sh # uniffi-bindgen → src/generated/kotlin/onde.kt

88 +└── .github/workflows/

89 + └── release-sdk-kotlin.yml # tag push → build → Maven Central publish

90 +```

91 +

92 +---

93 +

94 +## KMP Architecture

95 +

96 +### Why KMP Instead of Android-Only

97 +

98 +1. **klibs.io listing** — the JetBrains KMP discovery platform requires

99 + `kotlin-tooling-metadata.json`, which only the `kotlin.multiplatform` Gradle

100 + plugin generates. An Android-only AAR will not appear on klibs.io.

101 +2. **JVM target** — Kotlin developers on macOS Apple Silicon can run inference

102 + locally from desktop/server apps, Gradle plugins, or CLI tools.

103 +3. **Future-proof** — adding Kotlin/Native targets (iOS, macOS) later requires

104 + no structural changes. The Swift SDK already covers Apple platforms natively,

105 + but KMP apps that share Kotlin across Android + iOS could benefit.

106 +

107 +### Source Set Strategy: Shared srcDir

108 +

109 +Instead of KMP intermediate source sets (which add Gradle complexity and

110 +require careful dependency resolution for JNA), the SDK uses a **shared srcDir**

111 +pattern:

112 +

113 +```

114 +commonMain/ — empty (KMP requires it)

115 +shared/ — hand-written wrapper code (compiled for BOTH targets)

116 +generated/ — UniFFI-generated onde.kt (compiled for BOTH targets)

117 +androidMain/ — Android-specific: Os.setenv, Context factory

118 +jvmMain/ — JVM-specific: JNA setenv, NativeLoader, File factory

119 +```

120 +

121 +In `lib/build.gradle.kts`:

122 +

123 +```kotlin

124 +androidMain.get().kotlin.srcDir("src/shared/kotlin")

125 +androidMain.get().kotlin.srcDir("src/generated/kotlin")

126 +jvmMain.get().kotlin.srcDir("src/shared/kotlin")

127 +jvmMain.get().kotlin.srcDir("src/generated/kotlin")

128 +```

129 +

130 +Both targets compile the shared code independently with their own dependencies.

131 +Since both are JVM-based, `java.io.File`, JNA, and coroutines are available in

132 +the shared code.

133 +

134 +### Platform Abstraction: Interface Pattern

135 +

136 +The shared `OndeInference` class takes a `PlatformSupport` interface via an

137 +`internal` constructor. Each target provides a concrete implementation and a

138 +top-level factory function:

139 +

140 +```

141 +PlatformSupport (interface — shared/)

142 +├── AndroidPlatform (object — androidMain/)

143 +│ └── fun OndeInference(context, dataDir?) → OndeInference

144 +└── JvmPlatform (object — jvmMain/)

145 + └── fun OndeInference(dataDir?) → OndeInference

146 +```

147 +

148 +| Operation | Android | JVM |

149 +|-----------|---------|-----|

150 +| `setEnv(key, value)` | `Os.setenv(key, value, true)` | JNA → `libc.setenv(key, value, 1)` |

151 +| `ensureNativeLoaded()` | No-op (APK jniLibs) | `NativeLoader.ensureLoaded()` |

152 +| Default `dataDir` | `context.filesDir` | `~/.onde/` |

153 +

154 +---

155 +

156 +## Key Design Decisions

157 +

158 +| Decision | Rationale |

159 +|---|---|

160 +| KMP with `androidTarget()` + `jvm()` | Required for klibs.io listing; enables macOS Apple Silicon JVM users |

161 +| Shared srcDir instead of intermediate source sets | Avoids KMP dependency resolution complexity with JNA (`@aar` vs JAR) |

162 +| Interface-based platform abstraction | Simpler than `expect`/`actual` when constructor signatures differ between targets |

163 +| UniFFI Kotlin bindings — not hand-written JNI | Same Rust source as Swift/Dart, zero API drift |

164 +| `src/generated/kotlin/` is gitignored | Regenerated from Rust on every release |

165 +| `cargo-ndk` for Android cross-compilation | Handles NDK toolchain wiring for all 4 ABIs transparently |

166 +| `NativeLoader` extracts dylib from JAR resources | JVM users don't need to manage `java.library.path` manually |

167 +| Vanniktech `maven-publish` plugin with `KotlinMultiplatform` | Publishes KMP metadata + per-target artifacts to Maven Central |

168 +| `minSdk 26` | Required for `android.system.Os.setenv` to set `HF_HOME` |

169 +| JNA dependency (`net.java.dev.jna:jna:5.14.0`) | UniFFI 0.31.0 generates Kotlin bindings using `com.sun.jna.*`. Android uses `@aar` variant; JVM uses plain JAR. |

170 +| ProGuard consumer rules bundled in AAR | Prevents R8 from stripping UniFFI JNI bridge in consumer apps |

171 +| `:example` is a Gradle submodule | Plugin versions declared once in root `build.gradle.kts` and inherited |

172 +

173 +---

174 +

175 +## UniFFI Kotlin Type Map

176 +

177 +| Rust type | UniFFI derive | Kotlin type |

178 +|---|---|---|

179 +| `ChatRole` | `uniffi::Enum` | `sealed class ChatRole` |

180 +| `ChatMessage` | `uniffi::Record` | `data class ChatMessage` |

181 +| `SamplingConfig` | `uniffi::Record` | `data class SamplingConfig` |

182 +| `GgufModelConfig` | `uniffi::Record` | `data class GgufModelConfig` |

183 +| `InferenceResult` | `uniffi::Record` | `data class InferenceResult` |

184 +| `StreamChunk` | `uniffi::Record` | `data class StreamChunk` |

185 +| `EngineStatus` | `uniffi::Enum` | `sealed class EngineStatus` |

186 +| `EngineInfo` | `uniffi::Record` | `data class EngineInfo` |

187 +| `InferenceError` | `uniffi::Error` | `sealed class InferenceException : Exception` |

188 +| `OndeChatEngine` | `uniffi::Object` | `class OndeChatEngine` |

189 +| `StreamChunkListener` | `callback_interface` | `interface StreamChunkListener` |

190 +

191 +The generated Kotlin package is `uniffi.onde.*`. The shared `OndeInference.kt`

192 +re-exports these as `typealias` under `com.ondeinference.onde.*` so callers only

193 +need one import.

194 +

195 +---

196 +

197 +## Build Targets

198 +

199 +### Android ABI Targets

200 +

201 +| ABI | Rust triple | Typical device |

202 +|---|---|---|

203 +| `arm64-v8a` | `aarch64-linux-android` | Modern Android phones (primary) |

204 +| `armeabi-v7a` | `armv7-linux-androideabi` | Older 32-bit ARM phones |

205 +| `x86_64` | `x86_64-linux-android` | Android emulators (Intel/AMD) |

206 +| `x86` | `i686-linux-android` | 32-bit emulators |

207 +

208 +### JVM Native Library Targets

209 +

211 +|---|---|---|---|

216 +

217 +---

218 +

219 +## Build Sequence (Manual / CI)

220 +

221 +### 1. Prerequisites

222 +

223 +```bash

224 +# Rust toolchain

225 +rustup toolchain install stable

226 +

227 +# Android targets + cargo-ndk

228 +rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android i686-linux-android

229 +cargo install cargo-ndk

230 +

231 +# Android NDK

232 +export ANDROID_NDK_HOME=$HOME/Library/Android/sdk/ndk/26.1.10909125

233 +

234 +# macOS/JVM target (usually pre-installed on Apple Silicon)

235 +rustup target add aarch64-apple-darwin

236 +```

237 +

238 +### 2. Build Android .so files

239 +

240 +```bash

241 +./sdk/kotlin/scripts/build-android.sh # release, all ABIs

242 +./sdk/kotlin/scripts/build-android.sh --debug # debug build

243 +FILTER_ABI=arm64-v8a ./sdk/kotlin/scripts/build-android.sh # single ABI

244 +```

245 +

246 +Outputs land in `sdk/kotlin/lib/src/androidMain/jniLibs/<ABI>/libonde.so`.

247 +

248 +### 3. Build JVM native library

249 +

250 +```bash

251 +./sdk/kotlin/scripts/build-jvm.sh # host platform, release

252 +./sdk/kotlin/scripts/build-jvm.sh --debug # debug build

253 +ONDE_TARGET_TRIPLE=x86_64-apple-darwin ./sdk/kotlin/scripts/build-jvm.sh # cross-compile

254 +```

255 +

256 +Outputs land in `sdk/kotlin/lib/src/jvmMain/resources/native/<os-arch>/libonde.<ext>`.

257 +

258 +### 4. Build the uniffi-bindgen CLI

259 +

260 +```bash

261 +cargo build --manifest-path uniffi-bindgen/Cargo.toml --release

262 +# Binary: uniffi-bindgen/target/release/uniffi-bindgen

263 +```

264 +

265 +### 5. Generate Kotlin UniFFI bindings

266 +

267 +```bash

268 +./sdk/kotlin/scripts/generate-bindings.sh

269 +```

270 +

271 +Produces `sdk/kotlin/lib/src/generated/kotlin/onde.kt` — the complete

272 +generated binding including all types, the `OndeChatEngine` class, free

273 +functions, and `System.loadLibrary("onde")` in a companion object static init.

274 +

275 +### 6. Build the library

276 +

277 +```bash

278 +cd sdk/kotlin

279 +

280 +# Android AAR

281 +./gradlew :lib:assembleRelease

282 +

283 +# JVM JAR (with bundled native lib)

284 +./gradlew :lib:jvmJar

285 +

286 +# Both

287 +./gradlew :lib:build

288 +```

289 +

290 +---

291 +

292 +## JVM Native Library Loading

293 +

294 +The JVM target bundles native libraries inside the JAR under

295 +`/native/<os>-<arch>/libonde.<ext>`. The `NativeLoader` object handles

296 +extraction and loading at runtime:

297 +

298 +```

299 +OndeInference() factory (jvmMain)

300 + └── JvmPlatform.ensureNativeLoaded()

301 + └── NativeLoader.ensureLoaded()

302 + ├── Detect OS + arch from system properties

303 + ├── Extract /native/<os>-<arch>/libonde.<ext> from JAR resources

304 + ├── Write to temp file: <java.io.tmpdir>/onde-native/libonde.<ext>

305 + └── System.load(tempFile.absolutePath)

306 +```

307 +

308 +If the bundled library is not found (e.g. development mode without running

309 +`build-jvm.sh`), `NativeLoader` falls back to `System.loadLibrary("onde")`

310 +which searches `java.library.path`.

311 +

312 +**Important:** `NativeLoader.ensureLoaded()` must run BEFORE any UniFFI type is

313 +accessed. The `OndeInference` constructor handles this via the `init` block:

314 +

315 +```kotlin

316 +class OndeInference internal constructor(...) : AutoCloseable {

317 + init { platform.ensureNativeLoaded() } // runs before engine = OndeChatEngine()

318 + private val engine = uniffi.onde.OndeChatEngine()

319 +}

320 +```

321 +

322 +---

323 +

324 +## Android Filesystem Sandbox Setup

325 +

326 +On Android, `dirs::home_dir()` (used by `hf-hub` under the hood) panics because

327 +there is no home directory in the Android sandbox. The Rust engine requires

328 +`HF_HOME` to be set **before** any model load on Android.

329 +

330 +The `OndeInference` wrapper handles this automatically in `setup()`:

331 +

332 +```kotlin

333 +fun setup() {

334 + if (configured) return

335 +

336 + val hfHome = File(dataDir, "models").also { it.mkdirs() }

337 + val hfHubCache = File(hfHome, "hub").also { it.mkdirs() }

338 + val tmpDir = File(dataDir, "tmp").also { it.mkdirs() }

339 +

340 + platform.setEnv("HF_HOME", hfHome.absolutePath)

341 + platform.setEnv("HF_HUB_CACHE", hfHubCache.absolutePath)

342 + platform.setEnv("HUGGINGFACE_HUB_CACHE", hfHubCache.absolutePath)

343 + platform.setEnv("TMPDIR", tmpDir.absolutePath)

344 +

345 + configured = true

346 +}

347 +```

348 +

349 +`setup()` is called automatically by `loadDefaultModel()` and `loadModel()`.

350 +

351 +### Filesystem layout

352 +

353 +```

354 +# Android (context.filesDir)

355 +<filesDir>/

356 +├── models/ ← HF_HOME

357 +│ └── hub/ ← HF_HUB_CACHE

358 +└── tmp/ ← TMPDIR

359 +

360 +# JVM (defaults to ~/.onde/)

361 +~/.onde/

362 +├── models/ ← HF_HOME

363 +│ └── hub/ ← HF_HUB_CACHE

364 +└── tmp/ ← TMPDIR

365 +```

366 +

367 +### Cross-platform model cache comparison

368 +

370 +|---|---|---|---|

375 +

376 +### Why Android can't share models across apps

377 +

378 +Android's security model assigns each app a unique Linux UID at install time.

379 +The kernel enforces file permissions — app A cannot open a file descriptor

380 +inside app B's `/data/data/com.B/files/`. There is no platform primitive to

381 +override this the way iOS App Groups do. The approaches that don't work:

382 +

383 +- **`android:sharedUserId`** — deprecated API 29, removed API 33

384 +- **`MANAGE_EXTERNAL_STORAGE`** — Google Play rejects unless file manager

385 +- **SAF (Storage Access Framework)** — requires user interaction every time

386 +

387 +Each Onde-powered Android app downloads its own copy (~941 MB). This matches

388 +how TensorFlow Lite, MediaPipe, and ML Kit handle model caching. The SDK caches

389 +after first download — bandwidth cost is one-time per app.

390 +

391 +---

392 +

393 +## Kotlin Public API

394 +

395 +### `OndeInference` — primary entry point

396 +

397 +```kotlin

398 +import com.ondeinference.onde.OndeInference

399 +import com.ondeinference.onde.OndeSampling

400 +import com.ondeinference.onde.OndeModels

401 +import com.ondeinference.onde.OndeMessage

402 +

403 +// ── Android ──────────────────────────────────────────────────

404 +val onde = OndeInference(context) // pass applicationContext

405 +

406 +// ── JVM (macOS / Linux) ──────────────────────────────────────

407 +val onde = OndeInference() // uses ~/.onde/

408 +val onde = OndeInference(File("/custom")) // custom cache dir

409 +

410 +// ── Shared API (both platforms) ──────────────────────────────

411 +

412 +// Load (downloads from HuggingFace Hub on first run)

413 +val elapsed: Double = onde.loadDefaultModel(

414 + systemPrompt = "You are helpful.",

415 + sampling = OndeSampling.mobile()

416 +)

417 +

418 +// Chat (suspend)

419 +val result = onde.chat("What is Rust?")

420 +println(result.text)

421 +

422 +// Streaming (Flow)

423 +onde.stream("Tell me a story.").collect { chunk ->

424 + print(chunk.delta)

425 +}

426 +

427 +// One-shot (no history side-effects)

428 +val summary = onde.generate(

429 + messages = listOf(

430 + OndeMessage.system("Summarise this."),

431 + OndeMessage.user(longText)

432 + )

433 +)

434 +

435 +// Status

436 +val info = onde.info() // EngineInfo

437 +

438 +// History

439 +onde.clearHistory()

440 +onde.pushHistory(OndeMessage.user("Injected context"))

441 +

442 +// Cleanup

443 +onde.unload()

444 +onde.close() // decrements Rust Arc refcount

445 +```

446 +

447 +### `OndeSampling` — convenience constructors

448 +

449 +```kotlin

450 +OndeSampling.default() // temp=0.7, top_p=0.95, max=512 tokens

451 +OndeSampling.deterministic() // temp=0.0, greedy

452 +OndeSampling.mobile() // temp=0.7, max=128 tokens

453 +```

454 +

455 +### `OndeModels` — model config constructors

456 +

457 +```kotlin

458 +OndeModels.default() // platform default (1.5B on Android, 3B on macOS JVM)

459 +OndeModels.qwen25_1_5b() // ~941 MB

460 +OndeModels.qwen25_3b() // ~1.93 GB

461 +```

462 +

463 +### `OndeMessage` — message constructors

464 +

465 +```kotlin

466 +OndeMessage.system("You are helpful.")

467 +OndeMessage.user("Hello!")

468 +OndeMessage.assistant("Hi there!")

469 +```

470 +

471 +---

472 +

473 +## lib/build.gradle.kts Key Sections

474 +

475 +```kotlin

476 +import com.vanniktech.maven.publish.KotlinMultiplatform

477 +import com.vanniktech.maven.publish.JavadocJar

478 +import com.vanniktech.maven.publish.SonatypeHost

479 +

480 +plugins {

481 + id("org.jetbrains.kotlin.multiplatform")

482 + id("com.android.library")

483 + id("com.vanniktech.maven.publish")

484 +}

485 +

486 +kotlin {

487 + androidTarget {

488 + publishLibraryVariants("release")

489 + compilations.all { kotlinOptions { jvmTarget = "17" } }

490 + }

491 + jvm {

492 + compilations.all { kotlinOptions { jvmTarget = "17" } }

493 + }

494 +

495 + sourceSets {

496 + androidMain.dependencies {

497 + implementation("net.java.dev.jna:jna:5.14.0@aar")

498 + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.8.1")

499 + }

500 + jvmMain.dependencies {

501 + implementation("net.java.dev.jna:jna:5.14.0")

502 + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.8.1")

503 + }

504 +

505 + // Shared srcDirs — compiled independently for each target

506 + androidMain.get().kotlin.srcDir("src/shared/kotlin")

507 + androidMain.get().kotlin.srcDir("src/generated/kotlin")

508 + jvmMain.get().kotlin.srcDir("src/shared/kotlin")

509 + jvmMain.get().kotlin.srcDir("src/generated/kotlin")

510 + }

511 +}

512 +

513 +mavenPublishing {

514 + configure(KotlinMultiplatform(

515 + javadocJar = JavadocJar.Empty(),

516 + sourcesJar = true,

517 + ))

518 + publishToMavenCentral(SonatypeHost.CENTRAL_PORTAL)

519 + signAllPublications()

520 + coordinates("com.ondeinference", "onde-inference", "<version>")

521 +}

522 +```

523 +

524 +The Vanniktech plugin reads signing and OSSRH credentials from environment

525 +variables prefixed with `ORG_GRADLE_PROJECT_`:

526 +

527 +| Gradle property | CI env var |

528 +|---|---|

529 +| `mavenCentralUsername` | `ORG_GRADLE_PROJECT_mavenCentralUsername` |

530 +| `mavenCentralPassword` | `ORG_GRADLE_PROJECT_mavenCentralPassword` |

531 +| `signingKeyId` | `ORG_GRADLE_PROJECT_signingKeyId` |

532 +| `signingKey` | `ORG_GRADLE_PROJECT_signingKey` |

533 +| `signingPassword` | `ORG_GRADLE_PROJECT_signingPassword` |

534 +

535 +---

536 +

537 +## ProGuard Consumer Rules

538 +

539 +The `consumer-rules.pro` bundled in the AAR instructs R8/ProGuard in consumer

540 +apps to preserve:

541 +

542 +- `uniffi.onde.**` — all generated binding classes

543 +- `com.ondeinference.onde.**` — the public wrapper

544 +- `uniffi.onde.StreamChunkListener` — callback interface called from Rust

545 +- Native method declarations (`keepclasseswithmembernames`)

546 +- Static initializers in `uniffi.onde.*` (contains `System.loadLibrary("onde")`)

547 +

548 +Always update `consumer-rules.pro` when adding new top-level UniFFI-exported

549 +types or callback interfaces.

550 +

551 +---

552 +

553 +## klibs.io Discovery

554 +

555 +[klibs.io](https://klibs.io) is a JetBrains-maintained search platform for

556 +Kotlin Multiplatform libraries. It auto-indexes from Maven Central.

557 +

558 +### Requirements for listing

559 +

560 +| Requirement | How we satisfy it |

561 +|---|---|

562 +| Open source on GitHub | `ondeinference/onde` is public |

563 +| At least one artifact on Maven Central | Published via Vanniktech plugin |

564 +| Artifact has `kotlin-tooling-metadata.json` | KMP plugin generates this automatically |

565 +| POM has valid GitHub `url` or `scm.url` | `scm { url.set("https://github.com/ondeinference/onde") }` |

566 +

567 +### Timeline

568 +

569 +- New libraries appear **within one month** of first Maven Central publish

570 + (public index update frequency).

571 +- New versions of already-indexed libraries appear **the next day**.

572 +- No manual submission needed — discovery is fully automatic.

573 +

574 +---

575 +

576 +## Maven Central Publishing (CI)

577 +

578 +Trigger: push a semver tag matching `[0-9]+.[0-9]+.[0-9]+`.

579 +

580 +```bash

581 +# 1. Bump VERSION_NAME in sdk/kotlin/gradle.properties

582 +# 2. Bump version in Cargo.toml [package]

583 +# 3. Commit, tag, push

584 +git tag 1.0.1 && git push origin 1.0.1

585 +```

586 +

587 +CI (`release-sdk-kotlin.yml`) steps:

588 +1. Install Rust stable with all 4 Android targets + host target

589 +2. Cache NDK 26.1.10909125

590 +3. Install `cargo-ndk`

591 +4. `build-android.sh --release` → `.so` files

592 +5. `build-jvm.sh --release` → `.dylib` / `.so`

593 +6. Build `uniffi-bindgen`

594 +7. `generate-bindings.sh` → `onde.kt`

595 +8. Validate tag == `Cargo.toml` version

596 +9. `./gradlew :lib:publishAndReleaseToMavenCentral`

597 +10. Upload artifacts as GitHub Release assets

598 +

599 +---

600 +

601 +## Version Synchronisation Rule

602 +

603 +`VERSION_NAME` in `sdk/kotlin/gradle.properties` MUST match `version` in the

604 +root `Cargo.toml`. The CI workflow enforces this and fails fast on mismatch.

605 +Never bump one without the other.

606 +

607 +---

608 +

609 +## Common Pitfalls

610 +

611 +| Pitfall | Fix |

612 +|---|---|

613 +| `dirs::home_dir()` panics on Android | `setup()` must be called before any Rust model load. `OndeInference.loadDefaultModel()` calls it automatically. |

614 +| Generated `onde.kt` not found at compile time | Run `scripts/generate-bindings.sh`. The file is gitignored and must be regenerated after every Rust API change. |

615 +| `.so` files missing from AAR | Run `scripts/build-android.sh`. The `androidMain/jniLibs/` directory is gitignored. |

616 +| JVM `UnsatisfiedLinkError` at runtime | Run `scripts/build-jvm.sh` to bundle the native lib in JAR resources, or add `libonde.dylib` to `java.library.path`. |

617 +| UniFFI version mismatch | Keep `uniffi = "=0.31.0"` identical in `Cargo.toml`, `build-deps`, and `uniffi-bindgen/Cargo.toml`. |

618 +| R8 strips UniFFI bridge at runtime | Ensure `consumer-rules.pro` is present in the AAR and includes `-keep class uniffi.onde.**`. |

619 +| `System.loadLibrary("onde")` fails | The `.so` files must match the ABI of the device. Check that all 4 ABIs are built and included in `androidMain/jniLibs/`. |

620 +| `Os.setenv` not available | `minSdk` must be 26+. `android.system.Os` is available from API 21 but `setenv` specifically needs 26. |

621 +| Gradle fails with `VERSION_NAME` missing | Ensure `gradle.properties` contains `VERSION_NAME=x.y.z`. |

622 +| `cargo-ndk` not found in CI | Add `cargo install cargo-ndk --locked` step before the build step. |

623 +| `ANDROID_NDK_HOME` not set | Set it explicitly: `echo "ANDROID_NDK_HOME=$ANDROID_SDK_ROOT/ndk/26.1.10909125" >> $GITHUB_ENV` |

624 +| `build-android.sh: line XX: arm64: unbound variable` on macOS | Do not use Bash associative arrays. The script uses portable indexed arrays for ABI → Rust target mapping. |

625 +| `Unresolved reference 'sun'` / `Unresolved reference 'Structure'` in `onde.kt` | UniFFI 0.31.0 depends on JNA (`com.sun.jna.*`). Add `implementation("net.java.dev.jna:jna:5.14.0@aar")` for Android or `implementation("net.java.dev.jna:jna:5.14.0")` for JVM. |

626 +| HuggingFace download fails on first load | The app must declare `<uses-permission android:name="android.permission.INTERNET" />`. |

627 +| `Plugin [id: 'com.android.application'] was not found` in `:example` | Open `sdk/kotlin/` (not `sdk/kotlin/example/`) in Android Studio. Plugin versions are inherited from the root `build.gradle.kts`. |

628 +| `Using singleVariant publishing DSL multiple times` | This error means the old `AndroidSingleVariantLibrary` config is still present. Use `KotlinMultiplatform(...)` instead — the KMP plugin handles variant publishing internally. |

629 +| Convenience functions (`OndeSampling.default()` etc.) crash before `OndeInference()` created | These functions call into the native library via UniFFI. Always create an `OndeInference` instance first — its `init` block ensures the native library is loaded. |

630 +

631 +---

632 +

633 +## Testing

634 +

635 +### Unit tests (host JVM — no Android device needed)

636 +

637 +Pure Kotlin logic tests that don't call into Rust:

638 +

639 +```bash

640 +cd sdk/kotlin

641 +./gradlew :lib:test # all JVM unit tests

642 +./gradlew :lib:jvmTest # JVM target only

643 +```

644 +

645 +### Instrumented tests (requires Android device or emulator)

646 +

647 +Tests that call into Rust via the generated bindings:

648 +

649 +```bash

650 +./gradlew :lib:connectedAndroidTest

651 +```

652 +

653 +For CI, use an emulator (`x86_64` ABI) with `reactivecircus/android-emulator-runner`.

654 +

655 +### JVM integration tests

656 +

657 +Tests that load the native library and run inference:

658 +

659 +```bash

660 +# Build the native lib first

661 +./sdk/kotlin/scripts/build-jvm.sh

662 +

663 +# Run JVM tests

664 +cd sdk/kotlin && ./gradlew :lib:jvmTest

665 +```

666 +

667 +---

668 +

669 +## Dependency Graph

670 +

671 +```

672 +OndeInference.kt (shared)

673 +├── PlatformSupport (interface, shared)

674 +│ ├── AndroidPlatform (androidMain)

675 +│ │ └── android.system.Os.setenv

676 +│ └── JvmPlatform (jvmMain)

677 +│ ├── JNA → libc.setenv

678 +│ └── NativeLoader → extracts libonde from JAR resources

679 +├── uniffi.onde.OndeChatEngine (generated onde.kt)

680 +├── uniffi.onde.StreamChunkListener (generated onde.kt)

681 +├── uniffi.onde.*Config / *Message (generated onde.kt)

682 +│ └── com.sun.jna.Native (JNA)

683 +│ ├── libonde.so (Android, per ABI)

684 +│ └── libonde.dylib (macOS, from JAR resources)

685 +│ └── mistralrs (Metal on macOS, Candle CPU on Android)

686 +│ └── hf-hub (model download)

687 +└── Convenience.kt (shared)

688 + └── OndeSampling, OndeModels, OndeMessage

689 +```

690 +

691 +---

692 +

693 +## Distribution Registry

694 +

696 +|---|---|---|---|

700 +

701 +### End-game for Kotlin developers

702 +

703 +One line in `build.gradle.kts`:

704 +

705 +```kotlin

706 +implementation("com.ondeinference:onde-inference:1.0.0")

707 +```

708 +

709 +---

710 +

711 +## Example App

712 +

713 +A working Jetpack Compose chat app lives at `sdk/kotlin/example/`. It is the

714 +canonical way to see the SDK in action and the first thing to open when

715 +verifying a new build.

716 +

717 +### Opening in Android Studio

718 +

719 +Always open **`sdk/kotlin/`** — the Gradle root. Do not open

720 +`sdk/kotlin/example/` directly. Android Studio will see both `:lib` and

721 +`:example` as modules and sync correctly.

722 +

723 +### What the example covers

724 +

725 +- Loading the default model with `loadDefaultModel()` and showing load time

726 +- Streaming replies token-by-token via `onde.stream()` and `Flow<StreamChunk>`

727 +- Multi-turn conversation history maintained in the Rust engine

728 +- Clearing history without reloading the model

729 +- Material3 dynamic colour theme with dark mode support

730 +- Proper IME / navigation-bar inset handling

731 +

732 +### Gradle submodule rules

733 +

734 +1. **No `settings.gradle.kts` inside `example/`** — Gradle would treat it as

735 + standalone, breaking plugin resolution and `project(":lib")`.

736 +2. **No plugin versions in `example/build.gradle.kts`** — inherited from root.

737 +3. **`include(":example")` in root `settings.gradle.kts`** — required for Gradle.

738 +4. **`project(":lib")` as the SDK dependency** — builds against local source.

739 +

740 +The root `build.gradle.kts` declares all plugins used by any submodule:

741 +

742 +```kotlin

743 +plugins {

744 + id("com.android.library") version "8.5.2" apply false

745 + id("com.android.application") version "8.5.2" apply false

746 + id("org.jetbrains.kotlin.android") version "2.0.21" apply false

747 + id("org.jetbrains.kotlin.multiplatform") version "2.0.21" apply false

748 + id("org.jetbrains.kotlin.plugin.compose") version "2.0.21" apply false

749 + id("com.vanniktech.maven.publish") version "0.28.0" apply false

750 +}

751 +```

752 +

753 +### Streaming pattern

754 +

755 +```kotlin

756 +val placeholderId = System.nanoTime()

757 +_uiState.update { it.copy(messages = it.messages + UiMessage(id = placeholderId, ...)) }

758 +

759 +val buffer = StringBuilder()

760 +onde.stream(text).collect { chunk ->

761 + buffer.append(chunk.delta)

762 + _uiState.update { state ->

763 + state.copy(messages = state.messages.replaceLast(

764 + UiMessage(id = placeholderId, content = buffer.toString(), isStreaming = !chunk.done)

765 + ))

766 + }

767 +}

768 +```

769 +

770 +### Running the example

771 +

772 +```bash

773 +cd sdk/kotlin

774 +

775 +# 1. Build the .so files

776 +./scripts/build-android.sh

777 +

778 +# 2. Generate the Kotlin bindings

779 +./scripts/generate-bindings.sh

780 +

781 +# 3. Install on a connected device

782 +./gradlew :example:installDebug

783 +```

784 +

785 +Or open `sdk/kotlin/` in Android Studio and hit Run on the `:example` configuration.

786 +

787 +---

788 +

789 +## Cross-Platform SDK Comparison

790 +

791 +Onde ships SDKs for four platforms. The Kotlin SDK is one piece of the

792 +distribution strategy:

793 +

795 +|-----|-----------------|-------------|-----------------|-----------|

800 +

801 +The Swift SDK is the primary distribution — it runs natively on Apple silicon

802 +with Metal acceleration. The Kotlin JVM target provides the same Metal-backed

803 +inference for Kotlin developers on macOS, while the Android target extends

804 +reach to the Android ecosystem.

805 +

806 +---

807 +

808 +*Update this skill when adding KMP targets (e.g. Kotlin/Native for iOS),

809 +changing the source set structure, or modifying the Maven Central publishing

810 +pipeline.*

\ No newline at end of file

.agents/skills/sdk-react-native/SKILL.md new

+769

@@ -0,0 +1,769 @@

1 +---

2 +name: sdk-react-native

3 +description: Build, debug, and maintain the Onde Inference React Native / Expo SDK. Covers Expo native module wiring, Metro config for file-linked SDKs, iOS deployment target and Expo autolinking, XCFramework device/simulator split, app group entitlement, HF_HOME/TMPDIR sandbox setup, the native bridge JSON string contract, EngineStatus normalization, and npm publishing.

4 +allowed-tools: Read, Write, Edit, Glob, Grep, Bash

5 +user-invocable: true

6 +---

7 +

8 +# Skill: React Native / Expo SDK

9 +

10 +## What This Skill Covers

11 +

12 +Building and maintaining the `@ondeinference/react-native` Expo native module,

13 +wiring the Rust C FFI bridge to TypeScript, debugging Metro/CocoaPods/Xcode

14 +issues, and publishing to npm.

15 +

16 +---

17 +

18 +## Repository Layout

19 +

20 +```

21 +onde/sdk/react-native/

22 +├── package.json # name: @ondeinference/react-native

23 +├── expo-module.config.json # declares OndeInferenceModule for Expo autolinking

24 +├── tsconfig.json # target ES2020, module commonjs

25 +├── src/

26 +│ ├── index.ts # public API: OndeChatEngine, free functions, types

27 +│ ├── OndeInferenceModule.ts # requireNativeModule("OndeInference") — raw bridge

28 +│ └── types.ts # ChatMessage, SamplingConfig, EngineInfo, etc.

29 +├── build/ # tsc output — gitignored, regenerated by npm run build

30 +├── ios/

31 +│ ├── OndeInference.podspec # CocoaPods spec — vendored_frameworks = XCFramework

32 +│ ├── OndeInferenceModule.swift # Expo Module: Rust C FFI declarations + module def

33 +│ └── rust/

34 +│ ├── OndeReactNative.xcframework # bundles device + simulator slices

35 +│ ├── device/libonde_react_native.a

36 +│ └── simulator/libonde_react_native.a

37 +├── android/ # Android module (not yet wired for Onde)

38 +├── scripts/

39 +│ └── build-rust.sh # builds Rust .a for iOS device + sim, creates XCFramework

40 +└── example/

41 + ├── app.json # Expo config — ios.deploymentTarget, appleTeamId

42 + ├── babel.config.js # babel-preset-expo only

43 + ├── metro.config.js # watchFolders, extraNodeModules, nodeModulesPaths, blockList

44 + ├── package.json # dep: "@ondeinference/react-native": "file:.."

45 + └── ios/

46 + ├── Podfile # uses use_expo_modules!, deployment from Podfile.properties.json

47 + ├── Podfile.properties.json # ios.deploymentTarget: "16.0" — critical, see below

48 + ├── OndeExample/

49 + │ ├── OndeExample.entitlements

50 + │ └── Info.plist

51 + └── Pods/ # gitignored

52 +```

53 +

54 +---

55 +

56 +## Critical Lessons (Read First)

57 +

58 +### 1. Metro `@babel/runtime` resolution with `file:..` local packages

59 +

60 +**Symptom:** `Unable to resolve module @babel/runtime/helpers/asyncToGenerator`

61 +from inside the SDK's `build/index.js`.

62 +

63 +**Root cause:** When Metro processes a file physically located outside the

64 +example app's directory (e.g. in `sdk/react-native/build/`), Babel transforms

65 +`async/await` and injects `require("@babel/runtime/...")`. Metro resolves

66 +modules relative to the file's physical location — but the SDK's own

67 +`node_modules/` is blocked (to avoid react-native version conflicts) and the

68 +example's `node_modules/` is not in the resolution path for files outside the

69 +example root.

70 +

71 +**Fix — always required in `metro.config.js`:**

72 +

73 +```js

74 +// Falls back to the example's node_modules for any file in watched folders

75 +config.resolver.nodeModulesPaths = [path.resolve(__dirname, "node_modules")];

76 +```

77 +

78 +Also required: block the SDK's node_modules to avoid react-native version

79 +conflicts:

80 +

81 +```js

82 +config.resolver.blockList = [

83 + new RegExp(

84 + path.resolve(sdkRoot, "node_modules").replace(/\\/g, "\\\\") + "/.*",

85 + ),

86 +];

87 +```

88 +

89 +---

90 +

91 +### 2. Expo autolinking silently skips pods when deployment target is too low

92 +

93 +**Symptom:** `pod install` completes but `OndeInference` is not in

94 +`Podfile.lock`. The app launches but throws "Cannot find native module

95 +'OndeInference'" at runtime.

96 +

97 +**Root cause:** Expo's autolinking Ruby code calls `pod.supports_platform?`

98 +on each discovered pod. The `OndeInference` podspec declares

99 +`s.platforms = { :ios => '16.0' }`. If the project's deployment target

100 +(from `Podfile.properties.json`) is lower (e.g. the Expo default of 15.1),

101 +`supports_platform?` returns false and the pod is **silently skipped** with no

102 +error or warning.

103 +

104 +**Diagnosis:**

105 +

106 +```bash

107 +xcrun simctl get_app_container booted <bundle-id> data

108 +grep "OndeInference" ios/Podfile.lock # if empty → autolinking skipped it

109 +```

110 +

111 +**Fix — three places must all agree on iOS 16.0:**

112 +

113 +1. `sdk/react-native/ios/OndeInference.podspec` — `s.platforms = { :ios => '16.0' }`

114 +2. `example/ios/Podfile.properties.json` — `"ios.deploymentTarget": "16.0"`

115 +3. `example/app.json` → `expo.ios.deploymentTarget: "16.0"`

116 +

117 +And after changing `Podfile.properties.json`, manually patch the Xcode project:

118 +

119 +```bash

120 +sed -i '' 's/IPHONEOS_DEPLOYMENT_TARGET = 15.1/IPHONEOS_DEPLOYMENT_TARGET = 16.0/g' \

121 + ios/OndeExample.xcodeproj/project.pbxproj

122 +```

123 +

124 +Then run `pod install` again and verify:

125 +

126 +```bash

127 +grep "OndeInference" ios/Podfile.lock # must now appear

128 +```

129 +

130 +**Important:** `expo prebuild --clean` regenerates `Podfile.properties.json`

131 +from `app.json`. After `--clean`, you must also manually re-add

132 +`"ios.deploymentTarget": "16.0"` to `Podfile.properties.json` if it was not

133 +picked up from `app.json`.

134 +

135 +---

136 +

137 +### 3. iOS device vs simulator static library — use an XCFramework

138 +

139 +**Symptom:** Build succeeds for simulator but fails with:

140 +`ld: building for 'iOS-simulator', but linking in object file built for 'iOS'`

141 +

142 +**Root cause:** Both iOS device (`aarch64-apple-ios`) and iOS simulator

143 +(`aarch64-apple-ios-sim`) are arm64, so `lipo` cannot merge them into a fat

144 +binary. They have different Mach-O platform metadata.

145 +

146 +**Wrong approach:** Single `libonde_react_native.a` → works for device OR

147 +simulator but not both.

148 +

149 +**Wrong approach:** `vendored_libraries` with two files sharing the same

150 +filename → CocoaPods reports "conflicting names" error.

151 +

152 +**Correct approach:** XCFramework bundling both slices:

153 +

154 +```bash

155 +# In scripts/build-rust.sh

156 +mkdir -p ios/rust/device ios/rust/simulator

157 +cp target/aarch64-apple-ios/release/libonde_react_native.a ios/rust/device/

158 +cp target/aarch64-apple-ios-sim/release/libonde_react_native.a ios/rust/simulator/

159 +

160 +rm -rf ios/rust/OndeReactNative.xcframework

161 +xcodebuild -create-xcframework \

162 + -library ios/rust/device/libonde_react_native.a \

163 + -library ios/rust/simulator/libonde_react_native.a \

164 + -output ios/rust/OndeReactNative.xcframework

165 +```

166 +

167 +Podspec uses `vendored_frameworks` (not `vendored_libraries`):

168 +

169 +```ruby

170 +s.vendored_frameworks = 'rust/OndeReactNative.xcframework'

171 +s.pod_target_xcconfig = {

172 + 'DEFINES_MODULE' => 'YES',

173 + 'SWIFT_COMPILATION_MODE' => 'wholemodule',

174 + 'OTHER_LDFLAGS' => '-londe_react_native',

175 +}

176 +```

177 +

178 +CocoaPods automatically selects the correct slice for device vs simulator.

179 +

180 +---

181 +

182 +### 4. The native bridge JSON string contract — never return Swift dicts

183 +

184 +**Symptom:** `JSON parse error: unexpected character: o`

185 +

186 +**Root cause:** The TypeScript layer receives the native return value and calls

187 +`JSON.parse(value)`. If the native Swift module returns `[String: Any]` or

188 +`[[String: Any]]` instead of a `String`, the Expo bridge serialises it in an

189 +opaque way that is not valid JSON (the string representation starts with

190 +something like `{...}` or `[...]` in a non-JSON form that starts with `o`

191 +when printed).

192 +

193 +**The contract:** All Expo Module methods in `OndeInferenceModule.swift` that

194 +return complex types MUST return `String` (the raw JSON from Rust), NOT

195 +`[String: Any]` or `[[String: Any]]`. The TypeScript layer owns all JSON

196 +parsing.

197 +

198 +**Correct Swift:**

199 +

200 +```swift

201 +AsyncFunction("loadDefaultModel") {

202 + (systemPrompt: String?, samplingJson: String?) -> String in // ← String, not [String: Any]

203 + ...

204 + return try self.validateJsonResult(result) // returns the raw JSON string

205 +}

206 +

207 +Function("defaultModelConfig") { () -> String in // ← String, not [String: Any]

208 + let result = self.consumeRustString(onde_default_model_config())

209 + return try self.validateJsonResult(result)

210 +}

211 +```

212 +

213 +**Validation helper — validate but do not deserialise:**

214 +

215 +```swift

216 +private func validateJsonResult(_ json: String?) throws -> String {

217 + guard let json = json,

218 + let data = json.data(using: .utf8),

219 + let dict = try JSONSerialization.jsonObject(with: data) as? [String: Any]

220 + else { throw OndeError.invalidResponse }

221 + if let error = dict["error"] as? String {

222 + throw OndeError.inferenceError(error)

223 + }

224 + return json // ← return the original string, not the parsed dict

225 +}

226 +

227 +private func validateJsonArray(_ json: String?) throws -> String {

228 + guard let json = json,

229 + let data = json.data(using: .utf8),

230 + (try JSONSerialization.jsonObject(with: data) as? [[String: Any]]) != nil

231 + else { throw OndeError.invalidResponse }

232 + return json // ← return the original string

233 +}

234 +```

235 +

236 +---

237 +

238 +### 5. EngineStatus is lowercase from Rust — normalize in the UI layer

239 +

240 +**Symptom:** Model loads successfully. Alert says "Ready in X.Xs". But the

241 +`TextInput` placeholder stays at "Load a model first" and remains non-editable.

242 +

243 +**Root cause:** Rust's `Display` impl emits lowercase status strings:

244 +`"ready"`, `"unloaded"`, `"loading"`, `"generating"`, `"error"`.

245 +The TypeScript UI uses capitalized union type values (`"Ready"`, `"Unloaded"`,

246 +etc.). After `refreshStatus()` runs, `engineStatus` holds `"ready"` which

247 +does not match `"Ready"` in the `isLoaded` check.

248 +

249 +**Fix — normalize at the point of receiving native status:**

250 +

251 +```typescript

252 +const normalizeEngineStatus = useCallback((status: string): EngineStatus => {

253 + switch (status.toLowerCase()) {

254 + case "loading": return "Loading";

255 + case "ready": return "Ready";

256 + case "generating": return "Generating";

257 + case "error": return "Error";

258 + default: return "Unloaded";

259 + }

260 +}, []);

261 +

262 +const refreshStatus = useCallback(async () => {

263 + try {

264 + const info = await OndeChatEngine.info();

265 + setEngineStatus(normalizeEngineStatus(info.status)); // ← normalize here

266 + setModelName(info.modelName ?? null);

267 + setApproxMemory(info.approxMemory ?? null);

268 + } catch { /* best-effort */ }

269 +}, [normalizeEngineStatus]);

270 +```

271 +

272 +---

273 +

274 +### 6. iOS sandbox filesystem setup — HF_HOME must be set before model load

275 +

276 +**Symptom:** Model load fails immediately with "error 0" or returns an error

277 +containing "HF_HOME is not set".

278 +

279 +**Root cause:** On iOS, `~/.cache` is outside the app sandbox. The Rust engine

280 +explicitly requires `HF_HOME` to be set before any model load on sandboxed

281 +platforms. Without it, `load_gguf_model` returns an `InferenceError::ModelBuild`

282 +with the message: *"HF_HOME is not set — cannot initialise HF cache."*

283 +

284 +**Fix — call `configureApplicationFilesystem()` in `OnCreate` and before each

285 +model load:**

286 +

287 +```swift

288 +private func configureApplicationFilesystem() throws {

289 + let fileManager = FileManager.default

290 +

291 + let containerDirectory: URL

292 + if let appGroupDirectory = fileManager.containerURL(

293 + forSecurityApplicationGroupIdentifier: ondeAppGroupIdentifier

294 + ) {

295 + containerDirectory = appGroupDirectory

296 + } else {

297 + containerDirectory = try fileManager.url(

298 + for: .applicationSupportDirectory,

299 + in: .userDomainMask, appropriateFor: nil, create: true

300 + )

301 + }

302 +

303 + // Convention: <container>/models/hub — matches Tauri and Swift SDKs exactly

304 + let hfHomeDirectory = containerDirectory.appendingPathComponent("models", isDirectory: true)

305 + let hfHubCacheDirectory = hfHomeDirectory.appendingPathComponent("hub", isDirectory: true)

306 + let tempDirectory = containerDirectory.appendingPathComponent("tmp", isDirectory: true)

307 +

308 + try fileManager.createDirectory(at: hfHomeDirectory, withIntermediateDirectories: true)

309 + try fileManager.createDirectory(at: hfHubCacheDirectory, withIntermediateDirectories: true)

310 + try fileManager.createDirectory(at: tempDirectory, withIntermediateDirectories: true)

311 +

312 + setenv("HF_HOME", hfHomeDirectory.path, 1)

313 + setenv("HF_HUB_CACHE", hfHubCacheDirectory.path, 1)

314 + setenv("HUGGINGFACE_HUB_CACHE", hfHubCacheDirectory.path, 1)

315 + setenv("TMPDIR", tempDirectory.path, 1)

316 +}

317 +```

318 +

319 +Call it in `OnCreate` **and** at the start of `loadDefaultModel`/`loadModel`:

320 +

321 +```swift

322 +OnCreate {

323 + do {

324 + try self.configureApplicationFilesystem()

325 + self.enginePtr = onde_engine_create()

326 + } catch {

327 + self.enginePtr = nil

328 + }

329 +}

330 +

331 +AsyncFunction("loadDefaultModel") { ... -> String in

332 + try self.configureApplicationFilesystem() // ← also here, idempotent

333 + guard let ptr = self.enginePtr else { throw OndeError.engineNotInitialized }

334 + ...

335 +}

336 +```

337 +

338 +---

339 +

340 +### 7. Shared app group cache path convention

341 +

342 +All Onde-powered apps across all SDKs (Tauri, Swift, React Native, Flutter)

343 +share the same HuggingFace cache via the App Group container. The layout is:

344 +

345 +```

346 +<group.com.ondeinference.apps container>/

347 +├── models/ ← HF_HOME

348 +│ └── hub/ ← HF_HUB_CACHE (GGUF files live here)

349 +└── tmp/ ← TMPDIR

350 +```

351 +

352 +**The subdirectory is `models/`, NOT `huggingface/`.**

353 +

354 +Do NOT use `OndeInference/huggingface/hub/` — this was an early mistake that

355 +caused the app to look in a different place than the Tauri SDK, preventing

356 +cross-app model reuse.

357 +

358 +Verifying the shared container is accessible on simulator:

359 +

360 +```bash

361 +xcrun simctl get_app_container booted <bundle-id> groups

362 +# → shows group.com.ondeinference.apps path

363 +

364 +xcrun simctl get_app_container booted <bundle-id> group.com.ondeinference.apps

365 +# → resolves the exact path

366 +

367 +# Check whether models exist in the shared cache

368 +find "$(xcrun simctl get_app_container booted <bundle-id> group.com.ondeinference.apps)/models/hub" \

369 + -name "*.gguf" -o -name "*.part" -o -name "*.lock" | sort

370 +```

371 +

372 +---

373 +

374 +### 8. Entitlements required for iOS

375 +

376 +The example app's `OndeExample.entitlements` must contain:

377 +

378 +```xml

379 +<?xml version="1.0" encoding="UTF-8"?>

380 +<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"

381 + "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

382 +<plist version="1.0">

383 + <dict>

384 + <key>com.apple.developer.kernel.increased-memory-limit</key>

385 + <true />

386 + <key>com.apple.security.application-groups</key>

387 + <array>

388 + <string>group.com.ondeinference.apps</string>

389 + </array>

390 + </dict>

391 +</plist>

392 +```

393 +

394 +- `increased-memory-limit` — required for ~1 GB+ model loading on iOS

395 +- `com.apple.security.application-groups` — grants access to the shared model cache

396 +

397 +**On physical device:** the App Group must be registered in the Apple Developer

398 +Portal under **Identifiers → App Groups** before Xcode can provision it.

399 +

400 +On the iOS Simulator the entitlement is granted automatically by the simulator

401 +runtime — no provisioning profile needed.

402 +

403 +---

404 +

405 +### 9. FlatList swallows taps — TextInput never gets focus

406 +

407 +**Symptom:** After model loads, the `TextInput` placeholder is correct but

408 +tapping it does nothing. The keyboard never appears.

409 +

410 +**Root cause:** `FlatList` on iOS intercepts touch events and by default

411 +dismisses the keyboard on scroll, which prevents child inputs from receiving

412 +focus when tapping in certain layout configurations.

413 +

414 +**Fix:**

415 +

416 +```tsx

417 +<FlatList

418 + ...

419 + keyboardShouldPersistTaps="handled" // ← prevents FlatList eating taps

420 +/>

421 +```

422 +

423 +---

424 +

425 +### 10. Do not use the iOS Simulator to validate model loading

426 +

427 +The iOS Simulator is unreliable for validating actual LLM model loading with

428 +this stack:

429 +- Metal-backed inference behaves differently in the simulator

430 +- Large GGUF model loading (~941 MB) may hang or fail in the simulator

431 +- Shared app group container behavior can differ

432 +

433 +Use the Simulator for:

434 +- Build verification

435 +- UI layout and interaction

436 +- Entitlement/container path checks (`xcrun simctl get_app_container ...`)

437 +

438 +Use a **physical iPhone running iOS 16+** for:

439 +- Actual model download and load

440 +- Metal inference correctness

441 +- Memory behavior

442 +

443 +---

444 +

445 +### 11. Diagnosing a hung model load

446 +

447 +When the app stays in "Loading" state indefinitely:

448 +

449 +1. **Check the shared container has no stale partial downloads:**

450 +

451 +```bash

452 +# Find .part and .lock files

453 +find "$(xcrun simctl get_app_container booted <bundle-id> group.com.ondeinference.apps)" \

454 + -type f $ -name "*.part" -o -name "*.lock" $ | sort

455 +

456 +# Remove stale partial caches

457 +rm -rf "<container>/models/hub/models--<org>--<model>"

458 +```

459 +

460 +2. **Check the app group resolves:**

461 +

462 +```bash

463 +xcrun simctl get_app_container booted <bundle-id> groups

464 +```

465 +

466 +3. **Check native logs:**

467 +

468 +```bash

469 +xcrun simctl spawn booted log show --style compact --last 5m \

470 + --predicate 'process == "OndeExample" AND (

471 + eventMessage CONTAINS[c] "HF_" OR

472 + eventMessage CONTAINS[c] "models" OR

473 + eventMessage CONTAINS[c] "onde"

474 + )' | tail -100

475 +```

476 +

477 +4. **Verify `

478 +HF_TOKEN` was baked in at Rust build time.** On iOS,

479 +`TokenSource::CacheToken` cannot read `~/.cache/huggingface/token` because that

480 +path is outside the sandbox. `HF_TOKEN` must be set as an environment variable

481 +**when building the Rust static library**:

482 +

483 +```bash

484 +export HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxx

485 +./scripts/build-rust.sh ios

486 +```

487 +

488 +`option_env!("HF_TOKEN")` in `src/inference/token.rs` bakes it into the binary

489 +at compile time. Setting it after the fact has no effect.

490 +

491 +---

492 +

493 +## Architecture

494 +

495 +```

496 +┌──────────────────────────────────────────────────────┐

497 +│ React Native / TypeScript │

498 +│ OndeChatEngine, free functions (index.ts) │

499 +└─────────────────────┬────────────────────────────────┘

500 + │ JSON strings over Expo bridge

501 + ▼

502 +┌──────────────────────────────────────────────────────┐

503 +│ OndeInferenceModule.swift (Expo Module) │

504 +│ - configureApplicationFilesystem() │

505 +│ - validateJsonResult() / validateJsonArray() │

506 +│ - AsyncFunction / Function definitions │

507 +└─────────────────────┬────────────────────────────────┘

508 + │ C FFI via @_silgen_name

509 + ▼

510 +┌──────────────────────────────────────────────────────┐

511 +│ libonde_react_native.a (Rust static lib) │

512 +│ C API: onde_engine_create, onde_engine_load_*, ... │

513 +│ Rust engine: ChatEngine, mistral.rs, Metal/CPU │

514 +└──────────────────────────────────────────────────────┘

515 +```

516 +

517 +The Swift module **does not use UniFFI**. It calls the Rust C API directly via

518 +`@_silgen_name` declarations, communicating via JSON strings over the Expo

519 +Module bridge.

520 +

521 +This is different from the Swift XCFramework SDK (which uses UniFFI-generated

522 +`onde.swift`) and the Kotlin SDK (which uses UniFFI-generated `onde.kt`).

523 +

524 +---

525 +

526 +## Expo Module Definition Reference

527 +

528 +All methods in `OndeInferenceModule.swift` follow this pattern:

529 +

530 +| Expo construct | Use for |

531 +|---|---|

532 +| `Function(name)` | Synchronous calls (isLoaded, setSystemPrompt, etc.) |

533 +| `AsyncFunction(name)` | All calls that invoke Rust (model load, inference, etc.) |

534 +| `OnCreate` | Engine initialisation — run `configureApplicationFilesystem()` here |

535 +| `OnDestroy` | Engine cleanup — call `onde_engine_destroy` |

536 +

537 +Return types:

538 +- Simple sync functions: `Bool`, `Int`

539 +- All complex types: `String` — raw JSON from Rust, **never** `[String: Any]`

540 +

541 +---

542 +

543 +## TypeScript Bridge Contract

544 +

545 +### `OndeInferenceModule.ts` — raw native interface

546 +

547 +All async methods return `Promise<string>` (raw JSON strings from Rust):

548 +

549 +```typescript

550 +export interface OndeInferenceNativeModule {

551 + // Sync

552 + isLoaded(): boolean;

553 + setSystemPrompt(prompt: string): void;

554 + clearHistory(): number;

555 +

556 + // Async — all return JSON strings

557 + loadDefaultModel(systemPrompt: string | null, samplingJson: string | null): Promise<string>;

558 + sendMessage(message: string): Promise<string>;

559 + info(): Promise<string>;

560 + history(): Promise<string>;

561 +

562 + // Sync config — return JSON strings

563 + defaultModelConfig(): string;

564 + defaultSamplingConfig(): string;

565 +}

566 +```

567 +

568 +### `index.ts` — public API layer

569 +

570 +Serialises inputs to JSON, calls native, deserialises JSON responses:

571 +

572 +```typescript

573 +// Input serialisation (camelCase TS → snake_case Rust)

574 +function serializeSampling(config: SamplingConfig): string {

575 + return JSON.stringify({ temperature: config.temperature, top_p: config.topP, ... });

576 +}

577 +

578 +// Output deserialisation (snake_case Rust → camelCase TS)

579 +function parseResult<T>(json: string): T {

580 + const parsed = JSON.parse(json); // JSON.parse — expects a string

581 + if (parsed.error) throw new OndeError(parsed.error);

582 + return parsed as T;

583 +}

584 +```

585 +

586 +---

587 +

588 +## Metro Config for `file:..` SDK Development

589 +

590 +```js

591 +const { getDefaultConfig } = require("expo/metro-config");

592 +const path = require("path");

593 +

594 +const sdkRoot = path.resolve(__dirname, ".."); // ../sdk/react-native/

595 +const config = getDefaultConfig(__dirname);

596 +

597 +// 1. Watch the SDK so Metro picks up changes

598 +config.watchFolders = [sdkRoot];

599 +

600 +// 2. Map the package name to the SDK root

601 +config.resolver.extraNodeModules = {

602 + "@ondeinference/react-native": sdkRoot,

603 +};

604 +

605 +// 3. Allow transitive deps (e.g. @babel/runtime) to resolve from

606 +// the example's node_modules when files outside the example are processed

607 +config.resolver.nodeModulesPaths = [path.resolve(__dirname, "node_modules")];

608 +

609 +// 4. Block the SDK's own node_modules to prevent react-native version conflicts

610 +config.resolver.blockList = [

611 + new RegExp(

612 + path.resolve(sdkRoot, "node_modules").replace(/\\/g, "\\\\") + "/.*",

613 + ),

614 +];

615 +

616 +module.exports = config;

617 +```

618 +

619 +---

620 +

621 +## Rust C API Surface

622 +

623 +The Swift module binds to these C functions via `@_silgen_name`:

624 +

625 +| Function | Returns | Notes |

626 +|---|---|---|

627 +| `onde_engine_create` | `UnsafeMutableRawPointer?` | Allocates engine |

628 +| `onde_engine_destroy` | `void` | Frees engine |

629 +| `onde_engine_load_default_model` | `UnsafeMutablePointer<CChar>?` | JSON result string |

630 +| `onde_engine_load_model` | `UnsafeMutablePointer<CChar>?` | JSON result string |

631 +| `onde_engine_unload_model` | `UnsafeMutablePointer<CChar>?` | JSON result string |

632 +| `onde_engine_is_loaded` | `Bool` | Sync |

633 +| `onde_engine_info` | `UnsafeMutablePointer<CChar>?` | JSON result string |

634 +| `onde_engine_send_message` | `UnsafeMutablePointer<CChar>?` | JSON result string |

635 +| `onde_engine_generate` | `UnsafeMutablePointer<CChar>?` | JSON result string |

636 +| `onde_engine_history` | `UnsafeMutablePointer<CChar>?` | JSON array string |

637 +| `onde_engine_clear_history` | `UInt64` | Sync, count |

638 +| `onde_engine_set_system_prompt` | `void` | Sync |

639 +| `onde_engine_set_sampling` | `void` | Sync |

640 +| `onde_engine_push_history` | `void` | Sync |

641 +| `onde_free_string` | `void` | Must call for every non-null CChar* result |

642 +| `onde_default_model_config` | `UnsafeMutablePointer<CChar>?` | JSON result string |

643 +| `onde_qwen25_1_5b_config` | `UnsafeMutablePointer<CChar>?` | JSON result string |

644 +| `onde_qwen25_3b_config` | `UnsafeMutablePointer<CChar>?` | JSON result string |

645 +| `onde_default_sampling_config` | `UnsafeMutablePointer<CChar>?` | JSON result string |

646 +| `onde_deterministic_sampling_config` | `UnsafeMutablePointer<CChar>?` | JSON result string |

647 +| `onde_mobile_sampling_config` | `UnsafeMutablePointer<CChar>?` | JSON result string |

648 +

649 +**Always call `onde_free_string` after consuming each result pointer.** The

650 +Rust side allocates the string; Swift is responsible for freeing it.

651 +

652 +```swift

653 +private func consumeRustString(_ ptr: UnsafeMutablePointer<CChar>?) -> String? {

654 + guard let ptr = ptr else { return nil }

655 + let str = String(cString: ptr)

656 + onde_free_string(ptr) // ← must free every non-null pointer

657 + return str

658 +}

659 +```

660 +

661 +---

662 +

663 +## build-rust.sh — iOS Build Flow

664 +

665 +```bash

666 +# 1. Build both targets

667 +cargo build --manifest-path rust/Cargo.toml --target aarch64-apple-ios --release

668 +cargo build --manifest-path rust/Cargo.toml --target aarch64-apple-ios-sim --release

669 +

670 +# 2. Copy to separate dirs (same filename — cannot lipo)

671 +mkdir -p ios/rust/device ios/rust/simulator

672 +cp target/aarch64-apple-ios/release/libonde_react_native.a ios/rust/device/

673 +cp target/aarch64-apple-ios-sim/release/libonde_react_native.a ios/rust/simulator/

674 +

675 +# 3. Create XCFramework (CocoaPods picks right slice automatically)

676 +rm -rf ios/rust/OndeReactNative.xcframework

677 +xcodebuild -create-xcframework \

678 + -library ios/rust/device/libonde_react_native.a \

679 + -library ios/rust/simulator/libonde_react_native.a \

680 + -output ios/rust/OndeReactNative.xcframework

681 +```

682 +

683 +After rebuilding, run `pod install` in `example/ios/` and then

684 +`npx expo run:ios --device` or `npx expo run:ios` (simulator).

685 +

686 +---

687 +

688 +## Running on Physical Device

689 +

690 +```bash

691 +# From example/

692 +npx expo run:ios --device

693 +```

694 +

695 +For a physical device build you need:

696 +- Apple Developer team set in `app.json` → `expo.ios.appleTeamId`

697 +- The App Group `group.com.ondeinference.apps` registered in the developer portal

698 +- The App Group capability enabled on the App ID

699 +- `com.apple.developer.kernel.increased-memory-limit` in entitlements

700 +

701 +**Blank white screen on device** = Metro bundle not loading. Ensure:

702 +1. Metro is running (`npx expo start --clear`)

703 +2. Phone and Mac are on the same network (or use Tunnel)

704 +3. Rebuild after native changes (`npx expo run:ios --device` — not just `expo start`)

705 +

706 +---

707 +

708 +## .gitignore for the SDK

709 +

710 +```gitignore

711 +node_modules/

712 +build/ # tsc output

713 +rust/target/ # Rust build artifacts

714 +ios/build/

715 +ios/rust/*.a

716 +ios/rust/*.xcframework

717 +ios/rust/device/

718 +ios/rust/simulator/

719 +android/build/

720 +android/src/main/jniLibs/

721 +example/.expo/

722 +example/android/

723 +example/ios/build/

724 +example/ios/Pods/

725 +example/ios/.xcode.env.local

726 +.DS_Store

727 +*.xcuserstate

728 +```

729 +

730 +---

731 +

732 +## npm Publishing

733 +

734 +```bash

735 +cd sdk/react-native

736 +npm run build # tsc → build/

737 +npm publish # publishes build/, ios/, android/, scripts/

738 +```

739 +

740 +The `.npmignore` excludes:

741 +- `example/`

742 +- `rust/`

743 +- `node_modules/`

744 +- `src/` (consumers use the compiled `build/`)

745 +

746 +The `package.json` `files` field explicitly includes:

747 +`build/`, `ios/`, `android/`, `scripts/`, `expo-module.config.json`

748 +

749 +---

750 +

751 +## Common Pitfalls Table

752 +

753 +| Pitfall | Symptom | Fix |

754 +|---|---|---|

755 +| Missing `nodeModulesPaths` in metro.config.js | `Cannot resolve @babel/runtime/helpers/asyncToGenerator` | Add `config.resolver.nodeModulesPaths = [path.resolve(__dirname, "node_modules")]` |

756 +| Deployment target mismatch | Pod installs but `OndeInference` missing from `Podfile.lock` | Set `"ios.deploymentTarget": "16.0"` in `Podfile.properties.json` AND `app.json` |

757 +| Single `.a` file for device + simulator | Linker error: "building for iOS-simulator but linked for iOS" | Use `xcodebuild -create-xcframework` to bundle both slices |

758 +| Two `.a` files with same name in `vendored_libraries` | CocoaPods "conflicting names" error | Use `vendored_frameworks` with an XCFramework instead |

759 +| Swift returning `[String: Any]` from Expo Module | `JSON parse error: unexpected character: o` | Return `String` from all Expo Module functions; validate but do not deserialise |

760 +| `HF_HOME` not set on iOS | "error 0" or model load fails immediately | Call `configureApplicationFilesystem()` in `OnCreate` and before model load |

761 +| Wrong cache path (`OndeInference/huggingface/hub/`) | Models not shared across Onde apps | Use `<container>/models/hub/` to match Tauri/Swift convention |

762 +| `EngineStatus` lowercase from Rust | Input stays disabled after model loads; status shows as `"ready"` not `"Ready"` | Normalize in `refreshStatus()` with `switch(status.toLowerCase())` |

763 +| `FlatList` swallows taps | TextInput can't be focused | Add `keyboardShouldPersistTaps="handled"` to `FlatList` |

764 +| Model hangs on Simulator | Stuck in loading indefinitely | Test model loading on a physical device; Simulator is unreliable for this |

765 +| Stale `.part`/`.lock` files | Model download never completes on retry | Delete `models--<org>--<model>` from the shared container and retry |

766 +| `HF_TOKEN` not baked in | Auth fails on HuggingFace download; "401 Unauthorized" in Rust logs | Set `HF_TOKEN` env var **before** building the Rust static library |

767 +| `expo prebuild --clean` wipes `Podfile.properties.json` | Deployment target reverts to 15.1; autolinking drops OndeInference | Re-add `"ios.deploymentTarget": "16.0"` to `Podfile.properties.json` after every `--clean` |

768 +| Physical device blank white screen | App launches, nothing renders | Ensure Metro is running and on same network; use `npx expo run:ios --device` |

769 +| Running with `expo start` instead of `expo run:ios` | "Cannot find native module 'OndeInference'" | Custom native modules require a native build — use `expo run:ios` not Expo Go |

\ No newline at end of file

.agents/skills/sdk-regressions/SKILL.md new

+632

@@ -0,0 +1,632 @@

1 +---

2 +name: sdk-regressions

3 +description: Cross-compilation regression checks for the onde crate. Covers macOS, iOS, tvOS (nightly), Android, Windows, and Linux target verification. Run before merging any change that touches platform-gated code, Cargo.toml, .cargo/config.toml, or build.rs. Also documents the onde-mistralrs fork publishing strategy and cargo publish workflow.

4 +allowed-tools: Read, Edit, Glob, Grep, Terminal

5 +user-invocable: true

6 +---

7 +

8 +# SKILL: SDK Cross-Compilation Regression Checks

9 +

10 +> Always run these checks before merging changes that touch `onde/src/`,

11 +> `onde/Cargo.toml`, `onde/.cargo/config.toml`, `onde/build.rs`, or any

12 +> platform-gated dependency.

13 +

14 +---

15 +

16 +## Why This Exists

17 +

18 +The `onde` crate compiles for **6 target OSes** with heavy `#[cfg(target_os)]`

19 +branching (106+ occurrences in `engine.rs` alone). A change that compiles on

20 +macOS can silently break iOS, tvOS, or Android because:

21 +

22 +- Different `mistralrs` feature sets per platform (`["metal"]` vs `[]`)

23 +- Platform-conditional dependencies (`hf-hub` only on Android)

24 +- Tier-3 targets (tvOS) require nightly + `-Z build-std`

25 +- Assembly stubs (`tvos_chkstk.s`) and `build.rs` gating

26 +- `.cargo/config.toml` sets `+fp16` rustflags on 5 Apple targets

27 +

28 +See also: **[`cargo publish` workflow](#cargo-publish-workflow)** at the bottom

29 +of this file — must be followed when releasing a new `onde` version.

30 +

31 +---

32 +

33 +## Tier 1 — Must Pass on Every PR (stable toolchain)

34 +

35 +These use the default stable toolchain and must always succeed:

36 +

37 +```bash

38 +# macOS — host platform, Metal backend

39 +cargo check

40 +

41 +# iOS device — Metal backend, +fp16

42 +cargo check --target aarch64-apple-ios

43 +

44 +# iOS simulator — Metal backend, +fp16

45 +cargo check --target aarch64-apple-ios-sim

46 +```

47 +

48 +### Quick one-liner

49 +

50 +```bash

51 +cargo check && \

52 +cargo check --target aarch64-apple-ios && \

53 +cargo check --target aarch64-apple-ios-sim

54 +```

55 +

56 +---

57 +

58 +## Tier 2 — tvOS (nightly toolchain, `-Z build-std`)

59 +

60 +tvOS targets are tier-3 in Rust — they are **not** in `rustup target list` and

61 +require nightly with `-Z build-std` and the `rust-src` component.

62 +

63 +```bash

64 +# tvOS device

65 +cargo +nightly check -Z build-std --target aarch64-apple-tvos

66 +

67 +# tvOS simulator

68 +cargo +nightly check -Z build-std --target aarch64-apple-tvos-sim

69 +```

70 +

71 +### tvOS-specific regression risks

72 +

73 +| Risk | What breaks | How to detect |

74 +|------|-------------|---------------|

75 +| `scripts/tvos_chkstk.s` deleted | `Undefined symbol: ___chkstk_darwin` at link time | `test -f scripts/tvos_chkstk.s` |

76 +| `cc` build-dep removed from `Cargo.toml` | `build.rs` can't compile the assembly stub | `grep 'cc' Cargo.toml` |

77 +| `build.rs` tvOS block removed | Same linker error | `grep 'tvos' build.rs` |

79 +| `.cargo/config.toml` `+fp16` removed | Silent precision loss or candle kernel panics | `grep 'fp16' .cargo/config.toml` |

80 +

81 +---

82 +

83 +## Tier 3 — Android, Windows, Linux (stable, needs cross toolchains)

84 +

85 +These check that CPU-only code paths compile. For this crate, `cargo check`

86 +still runs C build scripts from dependencies such as `aws-lc-sys` and `ring`,

87 +so absent cross GCC toolchains are real blockers, not harmless link-only

88 +failures.

89 +

90 +```bash

91 +# Android arm64 — CPU, requires hf-hub sandbox workaround

92 +cargo check --target aarch64-linux-android

93 +

94 +# Windows MSVC — CPU, exact CI target; run on Windows or a Windows CI runner

95 +cargo check --target x86_64-pc-windows-msvc

96 +

97 +# Linux — CPU

98 +cargo check --target x86_64-unknown-linux-gnu

99 +```

100 +

101 +On macOS hosts without cross GCC installed, expect Linux checks to fail looking

102 +for `x86_64-linux-gnu-gcc`, and Windows GNU checks to fail looking for

103 +`x86_64-w64-mingw32-gcc`. Use the Colima/Docker flow below instead of treating

104 +those as code regressions.

105 +

106 +### macOS host fallback: Colima/Docker

107 +

108 +Use this when the goal is to run the Linux and Windows cross-checks on a macOS

109 +host and the host does not have cross GCC toolchains. On Apple Silicon, run a

110 +native `linux/arm64` container. Avoid `linux/amd64` emulation for this crate;

111 +the dependency build can crash under QEMU before reaching `onde` code.

112 +

113 +Give Colima enough memory before running these checks:

114 +

115 +```bash

116 +colima status

117 +colima stop

118 +colima start --memory 8

119 +```

120 +

121 +Run Linux and the local Windows GNU proxy check in the container:

122 +

123 +```bash

124 +docker run --rm --platform linux/arm64 \

125 + -v "$PWD:/workspace" \

126 + -v onde-cargo-registry:/usr/local/cargo/registry \

127 + -v onde-cargo-git:/usr/local/cargo/git \

128 + -w /workspace \

129 + rust:1.92-bookworm \

130 + bash -lc '

131 + export PATH=/usr/local/cargo/bin:$PATH

132 + apt-get update >/dev/null

133 + DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \

134 + gcc-x86-64-linux-gnu \

135 + libc6-dev-amd64-cross \

136 + linux-libc-dev-amd64-cross \

137 + gcc-mingw-w64-x86-64 \

138 + binutils-mingw-w64-x86-64 >/dev/null

139 + rustup target add x86_64-unknown-linux-gnu x86_64-pc-windows-gnu >/dev/null

140 + CARGO_BUILD_JOBS=1 cargo check --target x86_64-unknown-linux-gnu

141 + CARGO_BUILD_JOBS=1 cargo check --target x86_64-pc-windows-gnu

142 + '

143 +```

144 +

145 +`x86_64-pc-windows-gnu` exercises `#[cfg(target_os = "windows")]` and the

146 +CPU-only Windows path locally, but it is not a substitute for the CI

147 +`x86_64-pc-windows-msvc` check. The MSVC ABI check needs a Windows/MSVC

148 +toolchain, normally GitHub Actions `windows-latest`.

149 +

150 +If the container reaches `Checking onde ...` and then reports Rust errors, the

151 +cross toolchain is working. Fix the crate error and rerun the check.

152 +

153 +---

154 +

155 +## Full Regression Suite (copy-paste)

156 +

157 +```bash

158 +#!/usr/bin/env bash

159 +set -euo pipefail

160 +

161 +echo "═══ Tier 1: macOS + iOS (stable) ═══"

162 +cargo check

163 +cargo check --target aarch64-apple-ios

164 +cargo check --target aarch64-apple-ios-sim

165 +

166 +echo "═══ Tier 2: tvOS (nightly + build-std) ═══"

167 +cargo +nightly check -Z build-std --target aarch64-apple-tvos

168 +cargo +nightly check -Z build-std --target aarch64-apple-tvos-sim

169 +

170 +echo "═══ Tier 3: Android / Windows / Linux ═══"

171 +cargo check --target aarch64-linux-android

172 +cargo check --target x86_64-pc-windows-msvc

173 +cargo check --target x86_64-unknown-linux-gnu

174 +

175 +echo "✅ All cross-compilation checks passed"

176 +```

177 +

178 +---

179 +

180 +## Required Rustup Targets

181 +

182 +Install these once on a development machine:

183 +

184 +```bash

185 +# Tier 1 (stable)

186 +rustup target add aarch64-apple-darwin x86_64-apple-darwin

187 +rustup target add aarch64-apple-ios aarch64-apple-ios-sim

188 +

189 +# Tier 3 (stable)

190 +rustup target add aarch64-linux-android armv7-linux-androideabi

191 +rustup target add x86_64-linux-android i686-linux-android

192 +rustup target add x86_64-pc-windows-msvc

193 +rustup target add x86_64-pc-windows-gnu

194 +rustup target add x86_64-unknown-linux-gnu

195 +

196 +# Tier 2 — tvOS (nightly, no `rustup target add` — uses -Z build-std)

197 +rustup toolchain install nightly

198 +rustup +nightly component add rust-src

199 +```

200 +

201 +---

202 +

203 +## `.cargo/config.toml` — Critical Settings

204 +

205 +These rustflags are set at the workspace level and **must not be removed**:

206 +

207 +```toml

208 +[target.aarch64-apple-ios]

209 +rustflags = ["-C", "target-feature=+fp16"]

210 +

211 +[target.aarch64-apple-ios-sim]

212 +rustflags = ["-C", "target-feature=+fp16"]

213 +

214 +[target.aarch64-apple-tvos]

215 +rustflags = ["-C", "target-feature=+fp16"]

216 +

217 +[target.aarch64-apple-tvos-sim]

218 +rustflags = ["-C", "target-feature=+fp16"]

219 +

220 +[target.x86_64-apple-tvos]

221 +rustflags = ["-C", "target-feature=+fp16"]

222 +```

223 +

224 +Removing `+fp16` causes silent precision degradation or runtime panics in

225 +candle's Metal/NEON kernels.

226 +

227 +---

228 +

229 +## Platform Dependency Matrix

230 +

232 +|-------------|---------------------|------------|-----|---------------|

236 +| `android` | `[]` | `hf-hub`, `mistralrs-core` | CPU | Qwen 2.5 1.5B |

237 +| `windows` | `[]` | — | CPU | Qwen 2.5 3B |

238 +| `linux` | `[]` | — | CPU | Qwen 2.5 3B |

239 +

240 +`hf-hub` and `mistralrs-core` are required on all **sandboxed** platforms

241 +(iOS, tvOS, Android) for the `GLOBAL_HF_CACHE` workaround — `~/.cache` is

242 +outside the app container on iOS/tvOS, and `dirs::home_dir()` panics on Android.

243 +The re-exports in `src/lib.rs` (`pub use hf_hub; pub use mistralrs_core;`) are

244 +gated to `#[cfg(any(target_os = "android", target_os = "ios", target_os = "tvos"))]`.

245 +

246 +---

247 +

248 +## Common Regression Patterns

249 +

250 +### 1. New model added but not to all 6 platform `cfg` blocks

251 +

252 +`engine.rs` has separate `#[cfg(target_os = "...")]` blocks for model builder

253 +logic. A new model constructor added only under `macos` will fail on `ios`.

254 +

255 +**Detection:** Tier 1 checks catch iOS; Tier 2 catches tvOS; Tier 3 catches

256 +Android/Windows/Linux.

257 +

258 +### 2. Dependency added without platform gating

259 +

260 +Adding a dep that uses `SystemConfiguration.framework` (e.g., `hyper_util`

261 +proxy detection) works on macOS but causes linker errors on iOS/tvOS unless the

262 +podspec declares `s.frameworks = 'SystemConfiguration'`.

263 +

264 +**Detection:** Tier 1 `cargo check` passes (no linking), but `flutter build ios`

265 +fails. Must also test a full `flutter build` periodically.

266 +

267 +### 3. Fork branch divergence

268 +

269 +All `mistralrs` deps point to a fork:

270 +`git = "https://github.com/setoelkahfi/mistral.rs"`, branch

271 +`fix/all-platform-fixes`. If that branch is rebased, deleted, or diverges from

272 +upstream, **all** platforms break.

273 +

274 +**Detection:** Any `cargo check` fails with "could not find branch".

275 +

276 +### 4. UniFFI version mismatch

277 +

278 +`uniffi = "=0.31.0"` must be pinned identically in three places:

279 +- `onde/Cargo.toml` dependencies

280 +- `onde/Cargo.toml` build-dependencies

281 +- `onde/uniffi-bindgen/Cargo.toml`

282 +

283 +Mixing versions causes bindgen panics at codegen time.

284 +

285 +**Detection:** `cargo build` in `uniffi-bindgen/` or `build.rs` panics.

286 +

287 +### 5. Android `home_dir()` sandbox panic

288 +

289 +`dirs::home_dir()` panics in the Android sandbox. The `hf-hub` dep is added

290 +explicitly on Android so `HF_HOME` can be set programmatically. Never call

291 +`dirs::home_dir()` or `home::home_dir()` in code paths reachable on Android.

292 +

293 +**Detection:** Tier 3 `cargo check --target aarch64-linux-android` catches

294 +compile-time issues; runtime panics need an Android emulator test.

295 +

296 +---

297 +

298 +## Dart SDK Cross-Build Checks

299 +

300 +When changes touch `sdk/dart/`, also verify the Flutter plugin builds:

301 +

302 +```bash

303 +cd sdk/dart/example

304 +

305 +# macOS (requires Xcode)

306 +flutter build macos --debug

307 +

308 +# iOS (requires Xcode + iOS SDK)

309 +flutter build ios --debug --no-codesign

310 +```

311 +

312 +### Known environment pitfall

313 +

314 +**Never export Android NDK toolchain binaries globally in `.zshrc`:**

315 +

316 +```bash

317 +# ❌ BREAKS Xcode builds — NDK's ld/ar/strip shadow Apple's tools

318 +export LD=$TOOLCHAIN/bin/ld

319 +export AR=$TOOLCHAIN/bin/llvm-ar

320 +

321 +# ✅ Safe — env vars only, no PATH/tool overrides

322 +export ANDROID_NDK_HOME="$HOME/Library/Android/sdk/ndk/29.0.14206865"

323 +```

324 +

325 +If Xcode invokes the NDK's `ld` instead of Apple's, you get:

326 +`ld: error: unknown argument '-Xlinker'`

327 +

328 +The fix: comment out the tool exports, use Cargo's

329 +`sdk/dart/rust/.cargo/config.toml` for Android linker configuration instead.

330 +

331 +---

332 +

333 +## Cache Path Consistency (Cross-SDK)

334 +

335 +All Onde-powered apps share downloaded models via the App Group

336 +`group.com.ondeinference.apps`. The cache layout inside the shared container

337 +**must be identical** across every SDK and app type:

338 +

339 +```

340 +<group container>/

341 +├── models/ ← HF_HOME

342 +│ └── hub/ ← HF_HUB_CACHE (GGUF files live here)

343 +└── tmp/ ← TMPDIR (iOS sandbox restricts system TMPDIR)

344 +```

345 +

346 +**The subdirectory is `models/`, NOT `huggingface/`.**

347 +

348 +Files that set this path (all must agree):

349 +

350 +| File | Variable | Expected value |

351 +|------|----------|----------------|

352 +| `onde/sdk/dart/rust/src/api.rs` (`configure_cache_dir`) | `hf_home` | `data_dir.join("models")` |

353 +| `onde/src/hf_cache.rs` (`download_model` — general) | `hf_home` | `data_dir.join("models")` |

354 +| `onde/src/hf_cache.rs` (`download_model` — Android) | `hf_home` | `resolved_app_data.join("models")` |

355 +| Tauri apps (`setup_application_filesystem.rs`) | `models_home` | `container_dir.join("models")` |

356 +| `onde/src/hf_cache.rs` (`hf_cache_dir` — fallback) | default | `~/.cache/huggingface/hub` (non-sandboxed only, OK to differ) |

357 +

358 +**Regression check:** `grep -rn 'join("huggingface")' src/hf_cache.rs sdk/dart/rust/src/api.rs`

359 +should return **only** the non-sandboxed fallback in `hf_cache_dir()` (line ~164).

360 +Any other match means a sandboxed path is using the wrong subdirectory.

361 +

362 +---

363 +

364 +---

365 +

366 +## Publishing to crates.io

367 +

368 +### Why we don't use upstream `mistralrs` directly

369 +

370 +The short version: our PRs aren't merged yet.

371 +

372 +`onde` depends on a personal fork of Eric Buehler's `mistral.rs`, kept at

373 +`github.com/setoelkahfi/mistral.rs` on the `fix/all-platform-fixes` branch.

374 +The fork is ahead of upstream by ~41 commits carrying fixes that only matter

375 +for Apple and Android targets — things upstream hasn't needed to care about:

376 +

377 +| Fix | Status |

378 +|-----|--------|

379 +| iOS Metal 3.0 support | PR open |

380 +| Android 32-bit memory limit constants | PR open |

381 +| HF_HOME propagation in diffusion/flux pipelines | PR open |

382 +| `metallib` link step `--sdk` / `-std` flags | PR open |

383 +

384 +Dropping the fork and pointing at upstream `mistralrs 0.8.1` would silently

385 +break iOS on entry-level devices and Android on 32-bit targets. Don't do it

386 +until those PRs land.

387 +

388 +**Legal note:** The fork is a derivative of Eric Buehler's MIT-licensed

390 +— must remain intact and unmodified in the fork at all times. See

391 +[`legal-and-trademarks/SKILL.md`](../legal-and-trademarks/SKILL.md) for full

392 +attribution obligations.

393 +

394 +### The `onde-mistralrs` workaround

395 +

396 +> For the full inventory of every fork and `[patch.crates-io]` override in the

397 +> stack (mistral.rs, candle, sysctl, mio, tqdm, core2) and how the

398 +> git-vs-published tiers resolve differently, see the **forked-dependencies**

399 +> skill. This section covers only the mistral.rs publish mechanics.

400 +

401 +`cargo publish` strips every `git =` field before uploading to crates.io.

402 +That means a git-only dep is useless to anyone who installs `onde` from the

403 +registry — they can't resolve it.

404 +

405 +The solution: publish the fork to crates.io under the name `onde-mistralrs`

406 +(and `onde-mistralrs-core`, etc.), then reference it with `package =` so the

407 +rest of the code still compiles unchanged with `use mistralrs::...`:

408 +

409 +```toml

410 +# In onde/Cargo.toml — each platform target section looks like this.

411 +# Cargo resolves it as "onde-mistralrs" from crates.io,

412 +# but the Rust code sees it as `mistralrs`.

413 +mistralrs = { version = "0.8.2", package = "onde-mistralrs", features = ["metal"] }

414 +```

415 +

416 +This is the only supported publish path until the upstream PRs merge. Don't

417 +swap it for plain `mistralrs` from crates.io without checking every platform

418 +target first.

419 +

420 +---

421 +

422 +### Step 0 — check if the candle snapshot needs an `onde-candle-*` publish

423 +

424 +Since 0.9.x, upstream mistral.rs pins `candle` to a git rev past the latest

425 +released candle (e.g. rev `27f20fea`, 19 commits past `0.11.0`, adding

426 +`QTensor::indexed_gemv`, `gemv_fused_shared_lhs`, `BarrierPool::execute_chunked`).

427 +`cargo publish` strips git pins, so publish verification resolves candle from

428 +crates.io and fails with E0599 "method not found" if the pinned rev has APIs the

429 +released candle lacks.

430 +

431 +When that happens, publish the pinned rev under `onde-candle-*` names from the

432 +local candle checkout (`~/Repositories/candle`, branch pattern

433 +`onde/candle-<version>-<rev>`):

434 +

435 +```bash

436 +# Which candle crates actually diverged from the released version?

437 +git diff --stat <release-bump-commit> <pinned-rev> -- candle-core candle-nn candle-metal-kernels

438 +

439 +# Rename ONLY the diverged crates (0.11.0 era: candle-core + candle-nn; the

440 +# rest are byte-identical, official crates serve them). candle-nn must be

441 +# republished whenever candle-core is, even if unchanged, or the official

442 +# candle-nn drags the official candle-core into the graph -> two candle-core

443 +# crates -> type mismatch.

444 +cargo publish -p onde-candle-core

445 +cargo publish -p onde-candle-nn

446 +```

447 +

448 +Then in the mistral.rs fork workspace `Cargo.toml`:

449 +

450 +```toml

451 +candle-core = { version = "0.11.0", package = "onde-candle-core" }

452 +candle-nn = { version = "0.11.0", package = "onde-candle-nn" }

453 +candle-flash-attn-v3 = { version = "0.11.0" } # unchanged at rev -> official

454 +candle-metal-kernels = { version = "0.11.0" } # unchanged at rev -> official

455 +```

456 +

457 +### Step 1 — check if `onde-mistralrs` needs a new publish

458 +

459 +First, see how far the fork has drifted from upstream:

460 +

461 +```bash

462 +curl -s "https://api.github.com/repos/setoelkahfi/mistral.rs/compare/EricLBuehler:mistral.rs:master...setoelkahfi:mistral.rs:fix%2Fall-platform-fixes" \

463 + | python3 -c "import sys,json; d=json.load(sys.stdin); print('ahead:', d['ahead_by'], 'behind:', d['behind_by'])"

464 +```

465 +

466 +If there are new commits since the last `onde-mistralrs` publish, or if

467 +upstream cut a new release that the fork has rebased onto, bump the workspace

468 +version in the fork and publish. As of 0.9.1 the set is **12 crates** (upstream

469 +split out flash-attn/metal-compile/code-exec/sandbox, and vision/audio/mcp must

470 +be onde-published too because crates.io only has upstream's old versions).

471 +The rename lives in the workspace dep table (`package = "onde-..."` on each

472 +entry, local dep keys unchanged so imports stay `mistralrs_*`).

473 +

474 +```bash

475 +# 1. Edit the root Cargo.toml in setoelkahfi/mistral.rs

476 +# bump version, e.g. 0.9.0 → 0.9.1

477 +

478 +# 2. Publish in dependency order (leaves first):

479 +cargo publish -p onde-mistralrs-macros

480 +cargo publish -p onde-mistralrs-metal-compile

481 +cargo publish -p onde-mistralrs-sandbox

482 +cargo publish -p onde-mistralrs-flash-attn

483 +cargo publish -p onde-mistralrs-vision

484 +cargo publish -p onde-mistralrs-audio

485 +cargo publish -p onde-mistralrs-mcp

486 +cargo publish -p onde-mistralrs-paged-attn

487 +cargo publish -p onde-mistralrs-quant

488 +cargo publish -p onde-mistralrs-code-exec

489 +cargo publish -p onde-mistralrs-core

490 +cargo publish -p onde-mistralrs

491 +

492 +# 3. cargo waits for index propagation automatically after each publish.

493 +```

494 +

495 +Known publish-verification snags (hit during 0.9.1):

496 +

497 +| Error | Fix |

498 +|-------|-----|

499 +| E0599 on candle APIs (`indexed_gemv`, ...) | Step 0 — publish `onde-candle-*` and repoint |

500 +| `dependency tqdm does not specify a version` | tqdm must be a registry dep, not git — it is plain `tqdm = "0.8.1"` since the crossterm bump landed upstream |

501 +| `core2 0.4.0 is yanked` | `cargo update -p bitstream-io` (4.10.0 dropped core2) |

502 +

503 +---

504 +

505 +### Step 2 — update `onde/Cargo.toml`

506 +

507 +Every platform target section has its own `onde-mistralrs` line. They all need

508 +to move to the new version together — leaving one target on an older version

509 +causes a dependency conflict.

510 +

511 +```bash

512 +# All occurrences should print the same version number:

513 +grep 'onde-mistralrs' onde/Cargo.toml

514 +```

515 +

516 +---

517 +

518 +### Step 3 — dry-run first

519 +

520 +```bash

521 +cd onde

522 +cargo publish --dry-run --allow-dirty 2>&1 | tail -20

523 +```

524 +

525 +A clean run ends with `Finished` and no errors. Common things that go wrong:

526 +

527 +| Error | What to do |

528 +|-------|------------|

529 +| `all dependencies must have a version` | add `version = "..."` to the dep |

530 +| `no matching package named onde-mistralrs` | publish the fork first (step 1) |

531 +| `git specification will be removed` | expected — cargo is just telling you the git field gets stripped |

532 +| `files in working directory contain changes` | add `--allow-dirty` |

533 +

534 +---

535 +

536 +### Step 4 — publish `onde`, then update `sigit`

537 +

538 +```bash

539 +cargo publish

540 +```

541 +

542 +Then bump the version in `sigit/Cargo.toml`:

543 +

544 +```toml

545 +# git = is used for local dev; crates.io consumers resolve via version =

546 +onde = { version = "0.1.9", git = "https://github.com/ondeinference/onde", branch = "development" }

547 +```

548 +

549 +Keep both fields. The `git =` field takes precedence during local development;

550 +crates.io strips it and falls back to `version =` for anyone installing from

551 +the registry.

552 +

553 +---

554 +

555 +### GresIQ credentials and `HF_TOKEN`

556 +

557 +People building `onde` from crates.io don't need to do anything about secrets.

558 +Here's why.

559 +

560 +`build.rs` passes credentials to the compiler via `option_env!()`. That macro

561 +resolves at compile time — if the env var isn't set, it compiles to `None` and

562 +the feature that needs it is silently disabled. No crash, no partial state, no

563 +mystery error at runtime.

564 +

565 +```rust

566 +// pulse/client.rs — GresIQ telemetry (Onde Inference internal infrastructure)

567 +const EMBEDDED_API_KEY_DEV: Option<&str> = option_env!("GRESIQ_API_KEY_DEV");

568 +const EMBEDDED_API_SECRET_DEV: Option<&str> = option_env!("GRESIQ_API_SECRET_DEV");

569 +

570 +// token.rs — HuggingFace Hub authentication

571 +const BUILD_TIME_HF_TOKEN: Option<&str> = option_env!("HF_TOKEN");

572 +```

573 +

574 +Who gets what:

575 +

576 +| Who is building | Env vars present? | What happens |

577 +|-----------------|-------------------|--------------|

578 +| Our CI (official builds) | yes, via GitHub secrets | telemetry on; HF token baked into binary |

579 +| Someone who ran `cargo add onde` | no | telemetry off; HF token read from `~/.cache/huggingface/token` at runtime |

580 +| iOS/tvOS builds via XCFramework | yes, via `.env` file | HF token baked in — required because there's no writable filesystem on device |

581 +

582 +The GresIQ credentials are Onde Inference's own infrastructure secrets — they

583 +exist so the SDK can phone home for usage telemetry on official builds. External

584 +consumers don't have them, don't need them, and won't be affected by not having

585 +them.

586 +

587 +Never put `GRESIQ_API_KEY_*` or `HF_TOKEN` in `README.md`, `.env.example`, or

588 +any committed file. The `.env` in the crate root is git-ignored for a reason:

589 +

590 +```bash

591 +# onde/.env — git-ignored, never commit

592 +GRESIQ_API_KEY_DEV=...

593 +GRESIQ_API_SECRET_DEV=...

594 +GRESIQ_API_KEY_PRODUCTION=...

595 +GRESIQ_API_SECRET_PRODUCTION=...

596 +HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

597 +```

598 +

599 +---

600 +

601 +### Working against the git fork locally

602 +

603 +If you're making changes to the fork and want `onde` to pick them up without

604 +publishing, uncomment the patch block in `onde/Cargo.toml`:

605 +

606 +```toml

607 +[patch.crates-io]

608 +onde-mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", package = "mistralrs" }

609 +onde-mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", package = "mistralrs-core" }

610 +```

611 +

612 +Re-comment it before running `cargo publish`. Cargo strips `[patch.crates-io]`

613 +from the published manifest automatically, so it won't affect registry

614 +consumers either way — but leaving it active makes the dry-run confusing.

615 +

616 +---

617 +

618 +### Pre-publish checklist

619 +

620 +- [ ] all `onde-mistralrs` version numbers in `onde/Cargo.toml` are the same

621 +- [ ] `onde/Cargo.toml` `[package] version` is bumped

622 +- [ ] `sigit/Cargo.toml` `onde` version matches

623 +- [ ] `[patch.crates-io]` git override is commented out

624 +- [ ] `cargo publish --dry-run --allow-dirty` passes clean

625 +- [ ] `.env` is not listed in the dry-run's packaged file output

626 +- [ ] Eric Buehler's `LICENSE` in the `mistral.rs` fork is unmodified

627 +

628 +---

629 +

630 +*Last updated: July 2026 — added Colima/Docker Linux and Windows GNU

631 +cross-check guidance for macOS hosts without cross GCC, plus the MSVC CI

632 +caveat.*

.agents/skills/sdk-releases/SKILL.md new

+670

@@ -0,0 +1,670 @@

1 +---

2 +name: sdk-releases

3 +description: End-to-end SDK release process for all Onde distribution channels. Covers version bump checklist, CI/CD pipelines for Rust (crates.io), Swift (XCFramework → onde-swift), Dart (pub.dev), and npm, tag validation, GitHub Release creation, manual publish fallback, and common pitfalls. Apply whenever bumping versions or troubleshooting release workflows.

4 +allowed-tools: Read, Write, Edit, Glob, Grep, Bash

5 +user-invocable: true

6 +---

7 +

8 +# Skill: SDK Releases

9 +

10 +## What This Skill Covers

11 +

12 +The complete release pipeline for shipping Onde across all registries:

13 +

14 +- **Rust crate** → crates.io (`onde`)

15 +- **Swift package** → Swift Package Index (`onde-swift`)

16 +- **Dart/Flutter package** → pub.dev (`onde_inference`)

17 +- **React Native** → npm (`@ondeinference/react-native`)

18 +- **Kotlin Multiplatform** → Maven Central (`com.ondeinference:onde-inference`)

19 +

20 +All five SDK workflows share the same tag trigger. A single tag push fires all

21 +five in parallel. When only a subset needs (re-)publishing, use the **manual

22 +publish fallback** documented below to avoid collateral re-runs.

23 +

24 +---

25 +

26 +## Version Bump Checklist

27 +

28 +Every release starts with a version bump. **All version sources must be updated

29 +in a single commit before tagging.**

30 +

32 +|---|------|-------|---------|

44 +

45 +### Files you do NOT manually edit

46 +

47 +| File | Why |

48 +|------|-----|

49 +| `onde-swift/Package.swift` | CI rewrites the `url:` + `checksum:` automatically |

50 +| `onde-swift/Sources/Onde/onde.swift` | CI copies the freshly generated UniFFI glue |

51 +

52 +### Quick bump commands

53 +

54 +```bash

55 +# 1. Edit Cargo.toml version (manual)

56 +# 2. Edit sdk/dart/pubspec.yaml version (manual)

57 +# 3. Prepend new section to sdk/dart/CHANGELOG.md (manual)

58 +# 4. Edit sdk/react-native/package.json version (manual)

59 +# 5. Edit sdk/react-native/rust/Cargo.toml version (manual)

60 +# 6. Prepend new section to sdk/react-native/CHANGELOG.md (manual)

61 +# 7. Edit sdk/kotlin/gradle.properties VERSION_NAME (manual)

62 +# 8. Prepend new section to CHANGELOG.md at the repo root (manual)

63 +

64 +# 9. Sync lockfiles

65 +cargo check # updates root Cargo.lock

66 +cd sdk/dart/rust && cargo update -p onde # updates Dart SDK's Cargo.lock

67 +cd sdk/react-native/rust && cargo update -p onde # updates React Native SDK's Cargo.lock

68 +

69 +# 10. Verify

70 +grep '^version' Cargo.toml # "0.1.3"

71 +grep '^version:' sdk/dart/pubspec.yaml # 0.1.3

72 +grep '"version"' sdk/react-native/package.json # "0.1.3"

73 +grep '^version' sdk/react-native/rust/Cargo.toml # "0.1.3"

74 +grep 'VERSION_NAME' sdk/kotlin/gradle.properties # 1.0.0

75 +grep 'name = "onde"' -A1 Cargo.lock # version = "0.1.3"

76 +grep 'name = "onde"' -A1 sdk/dart/rust/Cargo.lock # version = "0.1.3"

77 +grep 'name = "onde"' -A1 sdk/react-native/rust/Cargo.lock # version = "0.1.3"

78 +

79 +# 11. Commit and tag

80 +git add -A

81 +git commit -m "0.1.3"

82 +git tag 0.1.3

83 +git push origin main 0.1.3

84 +```

85 +

86 +---

87 +

88 +## Tag Format

89 +

90 +All tags are **bare semver** — no `v` prefix.

91 +

92 +```

93 +0.1.3 ✅ correct

94 +v0.1.3 ❌ will NOT trigger CI

95 +0.1.3-beta ❌ will NOT match the tag pattern

96 +```

97 +

98 +All five CI workflows use the same trigger pattern:

99 +

100 +```yaml

101 +on:

102 + push:

103 + tags:

104 + - "[0-9]+.[0-9]+.[0-9]+"

105 +```

106 +

107 +---

108 +

109 +## CI Pipeline: Rust SDK (crates.io)

110 +

111 +**Workflow:** `.github/workflows/release-sdk-rust.yml`

112 +**Trigger:** Tag push matching `[0-9]+.[0-9]+.[0-9]+` or `workflow_dispatch`

113 +**Runner:** `macos-26`

114 +

115 +### Flow

116 +

117 +```

118 +Tag push (e.g. 1.0.0)

119 + │

120 + ├─ 1. Checkout + install stable Rust

121 + │

122 + ├─ 2. Validate tag == Cargo.toml version

123 + │ └─ Fails fast if mismatched

124 + │

125 + ├─ 3. cargo publish --dry-run

126 + │ └─ Compiles and packages; catches errors before real publish

127 + │

128 + └─ 4. cargo publish (tag push only)

129 + └─ Authenticates via CARGO_REGISTRY_TOKEN secret

130 +```

131 +

132 +### Key details

133 +

134 +- **`workflow_dispatch`** runs the dry-run only (the publish step has

135 + `if: startsWith(github.ref, 'refs/tags/')`) — useful for validation.

136 +- **Runner is `macos-26`** because the crate depends on `mistralrs` with Metal

137 + features, which requires macOS + Xcode to compile.

138 +- **crates.io is immutable** — once a version is published, it cannot be

139 + re-published. You must bump to the next version. `cargo yank` hides a version

140 + from dependency resolution but does not delete it.

141 +

142 +### Required secret

143 +

144 +`CARGO_REGISTRY_TOKEN` — a crates.io API token scoped to `publish-update` for

145 +the `onde` crate. Create at https://crates.io/settings/tokens. Best practice:

146 +scope it to a single crate rather than using a global token.

147 +

148 +---

149 +

150 +## CI Pipeline: Swift SDK

151 +

152 +**Workflow:** `.github/workflows/release-sdk-swift.yml`

153 +**Trigger:** Tag push matching `[0-9]+.[0-9]+.[0-9]+`

154 +**Runner:** `macos-15`

155 +

156 +### Flow

157 +

158 +```

159 +Tag push (e.g. 0.1.3)

160 + │

161 + ├─ 1. Build XCFramework

162 + │ └─ .github/scripts/build-swift-xcframework.sh

163 + │ ├─ Extracts version from Cargo.toml via tomllib

164 + │ ├─ Compiles staticlibs for 5 Apple targets

165 + │ ├─ Generates UniFFI Swift bindings

166 + │ ├─ Assembles XCFramework, zips, computes SHA-256

167 + │ └─ Writes dist/swift/{version.txt, OndeFramework.checksum.txt}

168 + │

169 + ├─ 2. Read version + checksum into step outputs

170 + │

171 + ├─ 3. Validate tag == Cargo.toml version

172 + │ └─ Fails fast if mismatched (e.g. tagged 0.1.3 but Cargo.toml says 0.1.2)

173 + │

174 + ├─ 4. Upload CI artifacts (every run, including workflow_dispatch)

175 + │

176 + ├─ 5. Create GitHub Release on `onde` (tag push only)

177 + │ └─ Attaches OndeFramework.xcframework.zip + checksum

178 + │

179 + ├─ 6. Checkout ondeinference/onde-swift (using ONDE_SWIFT_PAT)

180 + │

181 + ├─ 7. Rewrite Package.swift

182 + │ └─ Python regex replaces .binaryTarget url: + checksum:

183 + │

184 + ├─ 8. Copy generated onde.swift → onde-swift/Sources/Onde/

185 + │

186 + └─ 9. Commit + tag onde-swift

187 + ├─ git commit -m "Release 0.1.3" (skipped if no changes)

188 + ├─ git push origin HEAD:main

189 + └─ git tag -a "0.1.3" -m "Release 0.1.3" (skipped if tag exists)

190 + └─ Annotated tag, never force-pushed

191 +```

192 +

193 +### What happens on `onde-swift` after the push

194 +

195 +The `onde` CI pushes a commit (to `main`) and a tag (e.g. `0.1.3`) to `onde-swift`.

196 +This triggers **two** workflows on `onde-swift`:

197 +

198 +1. **`ci.yml`** (on push to `main`) — validates manifest + builds all 5 platforms

199 +2. **`release.yml`** (on tag push) — validates package, then creates a GitHub

200 + Release with installation instructions and a link to the upstream `onde` changelog

201 +

202 +Both run concurrently without conflict — they use different concurrency groups

203 +(`ci-refs/heads/main` vs the tag ref).

204 +

205 +### Tag immutability

206 +

207 +Tags on `onde-swift` are **never force-pushed**. SPM caches the resolved commit

208 +SHA per tag in `Package.resolved`. Moving a tag silently breaks every consumer's

209 +lockfile. If a release needs to be redone:

210 +

211 +1. Delete the remote tag: `git push origin :refs/tags/0.1.3`

212 +2. Delete the GitHub Release on both `onde` and `onde-swift`

213 +3. Fix the issue, re-tag, push again

214 +

215 +---

216 +

217 +## CI Pipeline: npm SDK (React Native)

218 +

219 +**Workflow:** `.github/workflows/release-sdk-npm.yml`

220 +**Trigger:** Tag push matching `[0-9]+.[0-9]+.[0-9]+` or `workflow_dispatch`

221 +**Runner:** `ubuntu-latest`

222 +

223 +### Flow

224 +

225 +```

226 +Tag push (e.g. 1.0.0)

227 + │

228 + ├─ validate job:

229 + │ ├─ Set up Node.js 22

230 + │ ├─ Read version from sdk/react-native/package.json

231 + │ ├─ Validate tag == package.json version

232 + │ ├─ npm install + npm run build

233 + │ └─ npm publish --dry-run --access public

234 + │

235 + └─ publish job (tag push only, needs: validate):

236 + ├─ npm install + npm run build

237 + ├─ Check if version already exists on npm

238 + │ └─ If yes → skip publish (idempotent, exits 0)

239 + └─ npm publish --access public

240 +```

241 +

242 +### npm idempotency guard

243 +

244 +The publish step checks `npm view <package>@<version>` before publishing. If the

245 +version already exists, it **skips** with a success exit code. This makes the npm

246 +workflow safe to re-run — unlike Dart/Rust, it won't fail on a duplicate version.

247 +

248 +### No secret: trusted publishing

249 +

250 +Since 1.2.5 the npm workflow publishes over OIDC rather than a long-lived token,

251 +so there is no `NPM_TOKEN` to rotate. Two things have to line up instead:

252 +

253 +- The `publish` job declares `permissions: id-token: write`.

254 +- The trusted publisher on

255 + npmjs.com/package/@ondeinference/react-native/access names repo

256 + `ondeinference/onde`, workflow `release-sdk-npm.yml`, and no environment.

257 +

258 +If those drift apart, npm answers the publish with a 404 on PUT rather than an

259 +auth error, which reads as a missing package and sends you looking in the wrong

260 +place. Adding an `environment:` to the job without setting the matching

261 +Environment name field on that npm page is the usual way to cause it.

262 +

263 +The job also upgrades npm before publishing: Node 22 ships npm 10, and trusted

264 +publishing needs 11.5.1 or newer.

265 +

266 +---

267 +

268 +## CI Pipeline: Dart SDK

269 +

270 +**Workflow:** `.github/workflows/release-sdk-dart.yml`

271 +**Trigger:** Tag push matching `[0-9]+.[0-9]+.[0-9]+`

272 +**Runner:** `ubuntu-latest`

273 +

274 +### Flow

275 +

276 +```

277 +Tag push (e.g. 0.1.3)

278 + │

279 + ├─ validate job:

280 + │ ├─ Set up Flutter (version from sdk/dart/.flutter-version)

281 + │ ├─ Read version from pubspec.yaml

282 + │ ├─ Validate tag == pubspec.yaml version

283 + │ │ └─ Fails fast if mismatched

284 + │ ├─ flutter pub get

285 + │ ├─ flutter analyze --no-fatal-infos

286 + │ └─ flutter pub publish --dry-run

287 + │

288 + └─ publish job (tag push only, needs: validate):

289 + ├─ Set up Flutter

290 + ├─ flutter pub get

291 + ├─ Write PUB_CREDENTIALS to ~/.config/dart/pub-credentials.json

292 + └─ flutter pub publish --force

293 +```

294 +

295 +### Tag validation (Dart)

296 +

297 +The Dart workflow independently validates the tag against `pubspec.yaml`:

298 +

299 +```yaml

300 +if [ "$TAG" != "$PUBSPEC_VERSION" ]; then

301 + echo "::error::Tag '$TAG' does not match pubspec.yaml version '$PUBSPEC_VERSION'."

302 + exit 1

303 +fi

304 +```

305 +

306 +This means **both** `Cargo.toml` and `pubspec.yaml` must have the same version

307 +as the git tag, or both workflows fail.

308 +

309 +### Required secret

310 +

311 +`PUB_CREDENTIALS` — the full JSON content of `~/.config/dart/pub-credentials.json`

312 +from a machine that has run `dart pub login`. Add as a repository secret on `onde`.

313 +

314 +---

315 +

316 +## GitHub Releases (two repos)

317 +

318 +| Repo | Created by | Contains |

319 +|------|-----------|----------|

320 +| `onde` | `release-sdk-swift.yml` (step 5) | XCFramework zip + checksum, auto-generated release notes |

321 +| `onde-swift` | `onde-swift` `release.yml` | Installation snippet, platform table, link to upstream changelog |

322 +

323 +Both repos need GitHub Releases because they serve different audiences:

324 +

325 +- **`onde`** — Rust consumers, asset hosting for the XCFramework binary

326 +- **`onde-swift`** — Swift developers, Swift Package Index surfaces these as version notes

327 +

328 +---

329 +

330 +## Version Sources and Validation Matrix

331 +

332 +| Source | Read by | Validated against |

333 +|--------|---------|-------------------|

334 +| `Cargo.toml` `version` | `build-swift-xcframework.sh` (tomllib) | Git tag (Swift workflow) |

335 +| `sdk/dart/pubspec.yaml` `version` | `release-sdk-dart.yml` (grep + awk) | Git tag (Dart workflow) |

336 +| Git tag (`github.ref_name`) | Both workflows | Both version files above |

337 +| `onde-swift` tag | Created by `release-sdk-swift.yml` | Same value as the `onde` tag |

338 +

339 +All four must match. If any pair diverges, the relevant CI job fails.

340 +

341 +---

342 +

343 +## Secrets Required

344 +

345 +| Secret | Used by | Purpose |

346 +|--------|---------|---------|

347 +| `CARGO_REGISTRY_TOKEN` | `release-sdk-rust.yml` | Authenticate with crates.io for `cargo publish`. Scoped to `publish-update` for the `onde` crate. Create at https://crates.io/settings/tokens. |

348 +| `ONDE_SWIFT_PAT` | `release-sdk-swift.yml` | Push commits + tags to `ondeinference/onde-swift`. Must be a PAT (not `GITHUB_TOKEN`) so it triggers workflows on `onde-swift`. Needs `contents: write` scope. |

349 +| `PUB_CREDENTIALS` | `release-sdk-dart.yml` | Authenticate with pub.dev for `flutter pub publish`. Full JSON from `~/.config/dart/pub-credentials.json`. |

350 +| `ORG_GRADLE_PROJECT_MAVENCENTRALUSERNAME` | `release-sdk-kotlin.yml` | Sonatype Central Portal token username. Generate at central.sonatype.com → Profile → Generate User Token. |

351 +| `ORG_GRADLE_PROJECT_MAVENCENTRALPASSWORD` | `release-sdk-kotlin.yml` | Sonatype Central Portal token password. Same dialog as above. |

352 +| `ORG_GRADLE_PROJECT_SIGNINGKEYID` | `release-sdk-kotlin.yml` | Last 8 chars of GPG key fingerprint used to sign Maven Central artifacts. |

353 +| `ORG_GRADLE_PROJECT_SIGNINGKEY` | `release-sdk-kotlin.yml` | ASCII-armored GPG private key. Export with `gpg --armor --export-secret-keys <KEY_ID>`. |

354 +| `ORG_GRADLE_PROJECT_SIGNINGPASSWORD` | `release-sdk-kotlin.yml` | Passphrase for the GPG signing key. |

355 +| `GRESIQ_API_KEY_DEV` | All release workflows | Build-time env for dev environment API key. |

356 +| `GRESIQ_API_SECRET_DEV` | All release workflows | Build-time env for dev environment API secret. |

357 +| `GRESIQ_API_KEY_PRODUCTION` | All release workflows | Build-time env for production API key. |

358 +| `GRESIQ_API_SECRET_PRODUCTION` | All release workflows | Build-time env for production API secret. |

359 +| `HF_TOKEN` | All release workflows | HuggingFace token baked in at build time (`option_env!`). |

360 +

361 +---

362 +

363 +## Common Pitfalls

364 +

365 +| Pitfall | Symptom | Fix |

366 +|---------|---------|-----|

367 +| Forgot to bump `pubspec.yaml` | Dart CI fails: "Tag '0.1.3' does not match pubspec.yaml version '0.1.2'" | Bump `sdk/dart/pubspec.yaml`, amend commit, re-tag |

368 +| Forgot to bump `sdk/react-native/package.json` | npm CI fails: "Tag '0.1.3' does not match package.json version '0.1.2'" | Bump `sdk/react-native/package.json`, amend commit, re-tag |

369 +| Forgot to bump `sdk/react-native/rust/Cargo.toml` | npm SDK's Rust bridge crate version drifts from the main crate | Bump `sdk/react-native/rust/Cargo.toml`, run `cd sdk/react-native/rust && cargo update -p onde` |

370 +| Forgot to update `sdk/dart/rust/Cargo.lock` | Dart SDK's Rust bridge builds against stale `onde` version | Run `cd sdk/dart/rust && cargo update -p onde` |

371 +| Forgot to update `sdk/react-native/rust/Cargo.lock` | npm SDK's Rust bridge builds against stale `onde` version | Run `cd sdk/react-native/rust && cargo update -p onde` |

372 +| Forgot to update `sdk/dart/CHANGELOG.md` | pub.dev shows stale changelog | Prepend new `## 0.1.3` section before tagging |

373 +| Forgot to bump `sdk/react-native/CHANGELOG.md` | npm shows stale changelog | Prepend new `## 0.1.3` section before tagging |

374 +| Forgot to update the root `CHANGELOG.md` | Preflight fails: "CHANGELOG.md: no '## 0.1.3' section" | Prepend the new section before tagging |

375 +| crates.io publish reports success but nothing is published | `cargo publish` step logs "already on crates.io — skipping" for a version that is not there | Fixed in 1.2.5. The guard used `cargo info`, which resolves against the local workspace. If it recurs, check that `scripts/crates-io-published.py` is still what the workflow calls |

376 +| Forgot to bump `sdk/kotlin/gradle.properties` `VERSION_NAME` | Kotlin CI fails: tag vs gradle.properties mismatch | Bump `VERSION_NAME`, amend commit, re-tag |

377 +| Tag has `v` prefix (`v0.1.3`) | CI does not trigger — tag pattern requires bare semver | Delete the tag, re-tag without `v` |

378 +| `CARGO_REGISTRY_TOKEN` missing | `cargo publish` fails with auth error | Create a scoped token at crates.io/settings/tokens, add as repo secret |

379 +| `ONDE_SWIFT_PAT` expired | `onde-swift` push fails with 403 | Regenerate PAT at github.com/settings/tokens, update repo secret |

380 +| `PUB_CREDENTIALS` expired | `flutter pub publish` fails with 401 | Run `dart pub login` locally, copy new credentials JSON to repo secret |

381 +| npm trusted publisher does not match the workflow | `npm publish` fails with a 404 on PUT, not a 401 | Check repo, workflow filename, and environment on npmjs.com/package/@ondeinference/react-native/access against what the `publish` job declares |

382 +| crates.io version already published | `cargo publish` fails with "already uploaded" | crates.io is immutable — bump to next version. `cargo yank` hides but doesn't delete. |

383 +| npm version already published | npm publish step skips (idempotency guard) | Safe — the workflow exits 0. No action needed. |

384 +| pub.dev version already published | `flutter pub publish` fails | pub.dev is immutable — bump to next version |

385 +| Maven Central version already published | `publishAndReleaseToMavenCentral` fails | Maven Central is immutable — bump to next version |

386 +| Maven Central namespace not verified | Deployment fails: "Namespace 'com.ondeinference' is not allowed" | Verify namespace on central.sonatype.com (add DNS TXT record, wait for Sonatype to poll) |

387 +| Tag pushed before workflow exists on `main` | Workflow never fires for that tag | Use `workflow_dispatch` for dry-run, then publish manually. See **Manual Publish Fallback**. |

388 +| All 5 workflows fire but only 1–2 need re-publishing | Collateral re-runs of Swift/npm/Dart/Rust/Kotlin | Don't delete+re-push the tag. Use **Manual Publish Fallback** for the specific SDKs. |

389 +| `onde-swift` tag already exists | Tag push skipped (warning emitted) | Delete remote tag first: `git push origin :refs/tags/0.1.3` |

390 +| Force-pushed a tag on `onde-swift` | SPM users get stale `Package.resolved` | Never force-push. Delete + re-create instead. Advise consumers to run `swift package resolve` |

391 +| XCFramework URL 404 | `swift package resolve` fails on consumer side | Ensure the `onde` GitHub Release was created BEFORE the `onde-swift` tag was pushed (the workflow handles this order automatically) |

392 +| `Package.swift` regex didn't match | Python `RuntimeError` in CI | Check that `onde-swift/Package.swift` still has a `.binaryTarget(name: "OndeFramework", ...)` block |

393 +| Dart `flutter analyze` fails | `validate` job fails, `publish` is skipped | Fix analysis issues, re-tag |

394 +

395 +---

396 +

397 +## Manual Publish Fallback

398 +

399 +When a tag was pushed before a workflow existed on `main`, or when you need to

400 +publish only a subset of SDKs without re-triggering all four workflows:

401 +

402 +### Rust (crates.io)

403 +

404 +```bash

405 +# Dry-run first

406 +cargo publish --dry-run

407 +

408 +# Publish

409 +cargo publish

410 +```

411 +

412 +Requires `~/.cargo/credentials.toml` or `CARGO_REGISTRY_TOKEN` env var.

413 +

414 +### Dart (pub.dev)

415 +

416 +```bash

417 +cd sdk/dart

418 +

419 +# Dry-run

420 +flutter pub publish --dry-run

421 +

422 +# Publish

423 +flutter pub publish --force

424 +```

425 +

426 +Requires `~/.config/dart/pub-credentials.json` from `dart pub login`.

427 +

428 +### Kotlin (Maven Central)

429 +

430 +```bash

431 +cd sdk/kotlin

432 +

433 +# Dry-run

434 +./gradlew :lib:publishToMavenCentral --dry-run --no-configuration-cache

435 +

436 +# Publish and release

437 +./gradlew :lib:publishAndReleaseToMavenCentral --no-configuration-cache

438 +```

439 +

440 +Requires `~/.gradle/gradle.properties` with `mavenCentralUsername`,

441 +`mavenCentralPassword`, `signingKeyId`, `signingKey`, `signingPassword`.

442 +Also requires the `com.ondeinference` namespace to be verified on

443 +central.sonatype.com.

444 +

445 +### npm (React Native)

446 +

447 +```bash

448 +cd sdk/react-native

449 +npm run build

450 +npm publish --access public

451 +```

452 +

453 +Requires `npm login`. Trusted publishing is CI-only, so a manual publish falls

454 +back to your own npm credentials.

455 +

456 +### When to use manual publish

457 +

458 +| Scenario | Approach |

459 +|----------|----------|

460 +| Tag exists but workflow wasn't on `main` yet | Merge workflow, then publish manually |

461 +| One SDK's CI failed, others succeeded | Fix the issue, publish that SDK manually |

462 +| Need to avoid re-triggering Swift XCFramework build | Publish Rust/Dart/npm/Kotlin manually |

463 +| Testing a new workflow before first real tag | Use `workflow_dispatch` for dry-run only |

464 +

465 +---

466 +

467 +## Re-releasing a Version (Emergency Fix)

468 +

469 +If a release was botched and you need to re-publish the same version:

470 +

471 +### Swift (onde-swift)

472 +

473 +```bash

474 +# 1. Delete the remote tag on onde-swift

475 +git -C /path/to/onde-swift push origin :refs/tags/0.1.3

476 +

477 +# 2. Delete the GitHub Release on onde-swift (via GitHub UI or gh CLI)

478 +gh release delete 0.1.3 --repo ondeinference/onde-swift --yes

479 +

480 +# 3. Delete the GitHub Release on onde

481 +gh release delete 0.1.3 --repo ondeinference/onde --yes

482 +

483 +# 4. Delete the local and remote tag on onde

484 +git tag -d 0.1.3

485 +git push origin :refs/tags/0.1.3

486 +

487 +# 5. Fix the issue, commit, re-tag, push

488 +git tag 0.1.3

489 +git push origin main 0.1.3

490 +```

491 +

492 +### Dart (pub.dev) / Rust (crates.io) / Kotlin (Maven Central)

493 +

494 +**pub.dev, crates.io, and Maven Central are all immutable.** Once a version is

495 +published, it cannot be re-published. You must bump to `0.1.4` instead.

496 +

497 +- **crates.io:** `cargo yank --version 0.1.3` hides the version from dependency

498 + resolution but does not delete it.

499 +- **pub.dev:** No yank equivalent. The version remains visible and installable.

500 +

501 +If the `publish` job failed (i.e. the package was NOT published), you can

502 +safely re-tag after fixing the issue.

503 +

504 +### npm

505 +

506 +The npm workflow's idempotency guard means re-running it is safe — it skips

507 +if the version already exists. However, npm also does not allow re-publishing

508 +the same version with different content. Bump to the next version if the

509 +published content was wrong.

510 +

511 +---

512 +

513 +## End-to-End Release Timeline

514 +

515 +```

516 +Developer merges feature → development → main (--no-ff) then tags

517 + │

518 + ├─── release-sdk-rust.yml fires ──────────────────────────────────────┐

519 + │ ~2 min (dry-run + publish to crates.io) │

520 + │ │

521 + ├─── release-sdk-swift.yml fires ──────────────────────────────────────┤

522 + │ ~15 min (XCFramework build) │

523 + │ Creates GitHub Release on onde │

524 + │ Pushes commit + tag to onde-swift │

525 + │ │ │

526 + │ ├─ onde-swift ci.yml fires (push to main) │

527 + │ │ ~5 min (validate + build 5 platforms) │

528 + │ │ │

529 + │ └─ onde-swift release.yml fires (tag 1.0.0) │

530 + │ ~5 min (validate + create GitHub Release) │

531 + │ │

532 + ├─── release-sdk-dart.yml fires ───────────────────────────────────────┤

533 + │ ~3 min (validate + publish to pub.dev) │

534 + │ │

535 + ├─── release-sdk-npm.yml fires ───────────────────────────────────────┤

536 + │ ~3 min (validate + publish to npm) │

537 + │ │

538 + ├─── release-sdk-kotlin.yml fires ────────────────────────────────────┤

539 + │ ~10 min (Android ABIs + JVM dylib + publish to Maven Central) │

540 + │ │

541 + └─── Swift Package Index picks up onde-swift tag ──────────────────────┘

542 + ~30 min (SPI polling interval)

543 + klibs.io picks up Maven Central artifact within ~30 days

544 +```

545 +

546 +All five CI workflows fire **in parallel** from the single tag push. The only

547 +ordering dependency is that `release-sdk-swift` must create the `onde` GitHub

548 +Release (with XCFramework assets) **before** pushing the tag to `onde-swift` —

549 +this is guaranteed by the step order within the workflow.

550 +

551 +### Merge train

552 +

553 +Always merge through the branch train before tagging. Use `--no-ff` to

554 +create explicit merge commits — they serve as bookmarks in the history

555 +showing where each feature landed.

556 +

557 +```bash

558 +git checkout development && git merge feature/my-branch --no-ff --no-edit

559 +git checkout main && git merge development --no-ff --no-edit

560 +git push origin development main

561 +git tag X.Y.Z && git push origin X.Y.Z

562 +```

563 +

564 +Never tag from a feature branch. The tag must point to a commit on `main` that

565 +contains all workflow files and fixes. If a workflow is added on a feature branch

566 +and the tag is pushed before merging, the workflow won't exist at the tagged

567 +commit and CI won't fire.

568 +

569 +**Why `--no-ff`:** Multi-commit feature branches (like the Kotlin SDK with 35

570 +files across 6 commits) look like they were committed directly to `main` with

571 +fast-forward merges. `--no-ff` preserves the branch context. It also allows

572 +reverting an entire feature with a single `git revert -m 1 <merge-commit>`.

573 +

574 +---

575 +

576 +## Distribution Registry Reference

577 +

579 +|----------|-------------|--------|----------|------------|

583 +| npm | `@ondeinference/react-native` | `import { OndeChatEngine } from "@ondeinference/react-native"` | `release-sdk-npm.yml` | Yes (unpublish within 72h only) |

586 +

587 +---

588 +

589 +---

590 +

591 +## React Native npm SDK

592 +

593 +**Package:** `@ondeinference/react-native` (scoped under `@ondeinference` org on npm)

594 +**Location:** `sdk/react-native/`

595 +**Architecture:** Expo native module wrapping Rust via C FFI

596 +

597 +### How it works

598 +

599 +```

600 +TypeScript (React Native)

601 + │ import { OndeChatEngine } from "@ondeinference/react-native"

602 + ▼

603 +Expo Module (Swift / Kotlin)

604 + │ OndeInferenceModule — calls C FFI functions via @_silgen_name (iOS) / JNI (Android)

605 + ▼

606 +Rust C FFI bridge (sdk/react-native/rust/)

607 + │ extern "C" functions with JSON serialization, global tokio::Runtime

608 + ▼

609 +onde crate (src/)

610 + │ ChatEngine — tokio::sync::Mutex, mistral.rs inference

611 + ▼

612 +mistral.rs → Metal (iOS) / CPU (Android)

613 +```

614 +

615 +### Key files

616 +

617 +| File | Purpose |

618 +|------|---------|

619 +| `sdk/react-native/package.json` | npm package — `@ondeinference/react-native` |

620 +| `sdk/react-native/expo-module.config.json` | Expo autolinking config |

621 +| `sdk/react-native/src/index.ts` | Public TypeScript API — `OndeChatEngine`, free functions, JSON ↔ camelCase conversion |

622 +| `sdk/react-native/src/types.ts` | TypeScript type definitions mirroring Rust types |

623 +| `sdk/react-native/src/OndeInferenceModule.ts` | `requireNativeModule("OndeInference")` bridge |

624 +| `sdk/react-native/rust/src/lib.rs` | C FFI exports (`extern "C"`) + Android JNI wrappers |

625 +| `sdk/react-native/ios/OndeInferenceModule.swift` | Swift Expo module — calls Rust via `@_silgen_name` |

626 +| `sdk/react-native/android/.../OndeInferenceModule.kt` | Kotlin Expo module — calls Rust via JNI `external fun` |

627 +| `sdk/react-native/scripts/build-rust.sh` | Cross-compile Rust for iOS (staticlib) + Android (cdylib) |

628 +

629 +### Building native libraries

630 +

631 +```bash

632 +# iOS (requires rustup targets: aarch64-apple-ios, aarch64-apple-ios-sim)

633 +cd sdk/react-native && ./scripts/build-rust.sh ios

634 +

635 +# Android (requires ANDROID_NDK_HOME)

636 +cd sdk/react-native && ./scripts/build-rust.sh android

637 +

638 +# Both

639 +cd sdk/react-native && ./scripts/build-rust.sh all

640 +```

641 +

642 +### Publishing to npm

643 +

644 +```bash

645 +cd sdk/react-native

646 +npm run build # TypeScript → build/

647 +npm publish --access public

648 +```

649 +

650 +The `--access public` flag is required for scoped packages on first publish.

651 +

652 +---

653 +

654 +## Registry Immutability Summary

655 +

656 +All four registries are effectively immutable once a version is published:

657 +

658 +- **crates.io** — `cargo yank` removes from resolver but the tarball stays forever

659 +- **pub.dev** — No retraction mechanism at all

660 +- **npm** — `npm unpublish` works within 72 hours, but only if no dependents

661 +- **Maven Central** — No retraction mechanism. Once published, it's permanent.

662 +- **Swift (onde-swift tags)** — Convention: never force-push tags. Delete + re-create if needed.

663 +

664 +**Bottom line:** get the release right before publishing. Use `--dry-run` locally

665 +and via `workflow_dispatch` before pushing a real tag.

666 +

667 +---

668 +

669 +*Update this skill when adding new SDK targets (Python, Ruby, Kotlin), changing

670 +CI runners, or modifying the tag/version validation logic.*

\ No newline at end of file

.agents/skills/sdk-swift-package/SKILL.md new

+363

@@ -0,0 +1,363 @@

1 +---

2 +name: sdk-swift-package

3 +description: Build the Onde Swift package (XCFramework) from Rust source using UniFFI and distribute it via a remote-binary Package.swift for Swift Package Index. Covers xcframework assembly, App Group shared container, models/hub cache convention, and the onde-swift release workflow.

4 +allowed-tools: Read, Write, Edit, Glob, Grep, Bash

5 +user-invocable: true

6 +---

7 +

8 +# Skill: Swift Package (XCFramework Distribution)

9 +

10 +## What this skill covers

11 +

12 +Building the `Onde` Swift package from Rust source using UniFFI, assembling an

13 +`XCFramework`, and distributing it via a remote-binary `Package.swift` for

14 +Swift Package Index.

15 +

16 +---

17 +

18 +## Repository layout

19 +

20 +```

21 +onde/

22 +├── src/ # Rust source (lib.rs, inference/, hf_cache.rs)

23 +├── sdk/Onde/ # Output: generated Swift package (git-ignored locally)

24 +│ ├── Package.swift

25 +│ ├── Sources/Onde/ # UniFFI-generated onde.swift glue + header

26 +│ └── OndeFramework.xcframework

27 +├── uniffi-bindgen/ # onde's own uniffi-bindgen CLI binary

28 +│ ├── Cargo.toml

29 +│ └── uniffi-bindgen.rs # fn main() { uniffi::uniffi_bindgen_main() }

30 +├── uniffi.toml # UniFFI binding config (package_name, cdylib_name)

31 +├── build.rs # Provides tvOS ___chkstk_darwin stub via cc

32 +└── Cargo.toml # uniffi = "=0.31.0" — version MUST stay pinned

33 +```

34 +

35 +---

36 +

37 +## Key constraints

38 +

39 +| Constraint | Detail |

40 +|---|---|

41 +| UniFFI version | Pinned to **`=0.31.0`** in `Cargo.toml` and `uniffi-bindgen/Cargo.toml`. Never bump without coordinating with the Kotlin and Python SDKs. |

42 +| tvOS targets | Tier-3 — require `cargo +nightly -Z build-std`. Stable toolchain is used for iOS/macOS. |

43 +| tvOS linker | `___chkstk_darwin` is missing from tvOS libSystem. `build.rs` compiles `tvos_chkstk.s` as a no-op stub. This is automatic — do not remove it. |

44 +| `cargo-swift` | **Not required.** The project owns its own `uniffi-bindgen` binary. Use it directly (same as `build-kotlin.sh` does for Android). |

45 +| `.xcframework` in `Package.swift` | Published `Package.swift` must use **remote** `url:` + `checksum:` — never a local `path:`. |

46 +| Swift package name | `Onde` (PascalCase). Git repo slug: `onde-swift` under the `ondeinference` org. |

47 +| App Group ID | `group.com.ondeinference.apps` — shared across all Onde-powered apps (Flutter, Tauri, native Swift). |

48 +| HF cache subdirectory | `<container>/models/hub/` — NOT `huggingface/hub/`. Must match the Tauri and Dart SDK convention. |

49 +

50 +---

51 +

52 +## Apple target triples

53 +

54 +| Slice | Triple | Toolchain |

55 +|---|---|---|

56 +| iOS device | `aarch64-apple-ios` | stable |

57 +| iOS simulator (Intel) | `x86_64-apple-ios` | stable |

58 +| iOS simulator (Apple Silicon) | `aarch64-apple-ios-sim` | stable |

59 +| tvOS device | `aarch64-apple-tvos` | nightly + `-Z build-std` |

60 +| tvOS simulator (Intel) | `x86_64-apple-tvos` | nightly + `-Z build-std` |

61 +| tvOS simulator (Apple Silicon) | `aarch64-apple-tvos-sim` | nightly + `-Z build-std` |

62 +| macOS (Apple Silicon) | `aarch64-apple-darwin` | stable |

63 +| macOS (Intel) | `x86_64-apple-darwin` | stable |

64 +

65 +---

66 +

67 +## Build sequence (manual / CI)

68 +

69 +### 1. Build the `uniffi-bindgen` CLI (host)

70 +

71 +```bash

72 +cargo build --manifest-path uniffi-bindgen/Cargo.toml --release

73 +BINDGEN=uniffi-bindgen/target/release/uniffi-bindgen # or workspace target/

74 +```

75 +

76 +### 2. Compile Rust staticlibs per Apple target

77 +

78 +```bash

79 +# Stable targets (iOS, macOS)

80 +cargo build --target aarch64-apple-ios --release

81 +cargo build --target x86_64-apple-ios --release

82 +cargo build --target aarch64-apple-ios-sim --release

83 +cargo build --target aarch64-apple-darwin --release

84 +cargo build --target x86_64-apple-darwin --release

85 +

86 +# Nightly targets (tvOS — tier 3)

87 +cargo +nightly build -Z build-std \

88 + --target aarch64-apple-tvos --release

89 +cargo +nightly build -Z build-std \

90 + --target x86_64-apple-tvos --release

91 +cargo +nightly build -Z build-std \

92 + --target aarch64-apple-tvos-sim --release

93 +```

94 +

95 +### 3. lipo simulator slices

96 +

97 +```bash

98 +# iOS simulator fat lib

99 +lipo -create \

100 + target/x86_64-apple-ios/release/libonde.a \

101 + target/aarch64-apple-ios-sim/release/libonde.a \

102 + -output /tmp/libonde-ios-sim.a

103 +

104 +# tvOS simulator fat lib

105 +lipo -create \

106 + target/x86_64-apple-tvos/release/libonde.a \

107 + target/aarch64-apple-tvos-sim/release/libonde.a \

108 + -output /tmp/libonde-tvos-sim.a

109 +

110 +# macOS universal fat lib

111 +lipo -create \

112 + target/aarch64-apple-darwin/release/libonde.a \

113 + target/x86_64-apple-darwin/release/libonde.a \

114 + -output /tmp/libonde-macos.a

115 +```

116 +

117 +### 4. Generate Swift bindings

118 +

119 +```bash

120 +mkdir -p sdk/Onde/Sources/Onde

121 +

122 +$BINDGEN generate \

123 + --library target/aarch64-apple-ios/release/libonde.a \

124 + --language swift \

125 + --out-dir sdk/Onde/Sources/Onde \

126 + --config uniffi.toml

127 +```

128 +

129 +This produces `onde.swift` and `ondeFFI.h` (+ `ondeFFI.modulemap`).

130 +

131 +### 5. Create the `.xcframework`

132 +

133 +Each slice needs a headers directory alongside its `.a`:

134 +

135 +```bash

136 +HEADERS=sdk/Onde/Sources/Onde # contains ondeFFI.h + ondeFFI.modulemap

137 +

138 +xcodebuild -create-xcframework \

139 + -library target/aarch64-apple-ios/release/libonde.a -headers $HEADERS \

140 + -library /tmp/libonde-ios-sim.a -headers $HEADERS \

141 + -library target/aarch64-apple-tvos/release/libonde.a -headers $HEADERS \

142 + -library /tmp/libonde-tvos-sim.a -headers $HEADERS \

143 + -library /tmp/libonde-macos.a -headers $HEADERS \

144 + -output sdk/Onde/OndeFramework.xcframework

145 +```

146 +

147 +### 6. Zip and compute checksum

148 +

149 +```bash

150 +cd sdk/Onde

151 +zip -r OndeFramework.xcframework.zip OndeFramework.xcframework

152 +CHECKSUM=$(shasum -a 256 OndeFramework.xcframework.zip | cut -d ' ' -f1)

153 +echo $CHECKSUM

154 +```

155 +

156 +### 7. Publish (two-repo strategy)

157 +

158 +1. Upload `OndeFramework.xcframework.zip` as a GitHub Release asset on the

159 + Rust source repo.

160 +2. In the `ondeinference/onde-kit` Swift-only repo, update `Package.swift`:

161 + - `url:` → the GitHub Release asset download URL

162 + - `checksum:` → the SHA-256 from step 6

163 +3. Commit, tag the version (`0.1.0`, etc.), and push to `onde-kit`.

164 +4. Swift Package Index indexes `onde-kit` on every new tag.

165 +

166 +---

167 +

168 +## `Package.swift` (published form)

169 +

170 +```swift

171 +// swift-tools-version:5.5

172 +import PackageDescription

173 +

174 +let package = Package(

175 + name: "Onde",

176 + platforms: [

177 + .iOS(.v14),

178 + .macOS(.v12),

179 + .tvOS(.v13),

180 + ],

181 + products: [

182 + .library(name: "Onde", targets: ["Onde"])

183 + ],

184 + targets: [

185 + .binaryTarget(

186 + name: "OndeFramework",

187 + url: "https://github.com/ondeinference/onde-kit/releases/download/0.1.0/OndeFramework.xcframework.zip",

188 + checksum: "<sha256>"

189 + ),

190 + .target(

191 + name: "Onde",

192 + dependencies: [.target(name: "OndeFramework")]

193 + ),

194 + ]

195 +)

196 +```

197 +

198 +---

199 +

200 +## UniFFI type map (Rust → Swift)

201 +

202 +| Rust type | UniFFI derive | Swift type |

203 +|---|---|---|

204 +| `ChatRole` | `uniffi::Enum` | `enum ChatRole` |

205 +| `ChatMessage` | `uniffi::Record` | `struct ChatMessage` |

206 +| `SamplingConfig` | `uniffi::Record` | `struct SamplingConfig` |

207 +| `GgufModelConfig` | `uniffi::Record` | `struct GgufModelConfig` |

208 +| `InferenceResult` | `uniffi::Record` | `struct InferenceResult` |

209 +| `StreamChunk` | `uniffi::Record` | `struct StreamChunk` |

210 +| `EngineStatus` | `uniffi::Enum` | `enum EngineStatus` |

211 +| `EngineInfo` | `uniffi::Record` | `struct EngineInfo` |

212 +| `InferenceError` | `uniffi::Error` | `enum OndeError : Error` |

213 +| `OndeChatEngine` | `uniffi::Object` | `class OndeChatEngine` |

214 +| `StreamChunkListener` | `callback_interface` | `protocol StreamChunkListener` |

215 +

216 +**UniFFI compatibility rules:**

217 +- Use `u64`/`u32` instead of `usize` (UniFFI does not support `usize`).

218 +- Streaming is exposed as a **free function** (`streamChatMessage`) not an

219 + Object method — UniFFI 0.31 does not support `callback_interface` in Object

220 + method parameters.

221 +- Use concrete `String` parameters in `OndeChatEngine` methods — `impl Into<String>`

222 + does not cross the FFI boundary.

223 +

224 +---

225 +

226 +## Swift API quick reference

227 +

228 +```swift

229 +import Onde

230 +

231 +// Engine lifecycle

232 +let engine = OndeChatEngine()

233 +try await engine.loadDefaultModel(systemPrompt: "You are helpful.", sampling: nil)

234 +try await engine.loadGgufModel(config: qwen251_5bConfig(), systemPrompt: nil, sampling: nil)

235 +await engine.unloadModel()

236 +let loaded: Bool = await engine.isLoaded()

237 +let info: EngineInfo = await engine.info()

238 +

239 +// Chat

240 +let result: InferenceResult = try await engine.sendMessage(message: "Hello!")

241 +let oneShot: InferenceResult = try await engine.generate(messages: [...], sampling: nil)

242 +

243 +// History

244 +let history: [ChatMessage] = await engine.history()

245 +let removed: UInt64 = await engine.clearHistory()

246 +await engine.pushHistory(message: userMessage(content: "..."))

247 +

248 +// Streaming (free function, callback-based)

249 +try await streamChatMessage(engine: engine, message: "Tell me a story.", listener: myHandler)

250 +

251 +// Config free functions

252 +defaultModelConfig() // platform-appropriate default

253 +qwen251_5bConfig() // ~941 MB, iOS/tvOS/Android

254 +qwen253bConfig() // ~1.93 GB, macOS/desktop

255 +

256 +// Sampling presets

257 +defaultSamplingConfig() // temp=0.7, top_p=0.95, max=512

258 +deterministicSamplingConfig() // temp=0.0

259 +mobileSamplingConfig() // temp=0.7, max=128

260 +

261 +// Message constructors

262 +systemMessage(content:)

263 +userMessage(content:)

264 +assistantMessage(content:)

265 +```

266 +

267 +---

268 +

269 +## App Group Shared Container & Cache Convention

270 +

271 +All Onde-powered apps use the App Group `group.com.ondeinference.apps` to share

272 +downloaded models. The cache layout inside the shared container is:

273 +

274 +```

275 +<group container>/

276 +├── models/ ← HF_HOME

277 +│ └── hub/ ← HF_HUB_CACHE (GGUF files live here)

278 +└── tmp/ ← TMPDIR (iOS sandbox restricts system TMPDIR)

279 +```

280 +

281 +**Important:** The subdirectory is `models/`, NOT `huggingface/`. This was

282 +chosen to be user-friendly when browsing the container in Finder/Files. All

283 +SDKs (Rust, Swift, Dart) and all app types (Tauri, Flutter, native) must use

284 +the same path or models won't be shared.

285 +

286 +### Swift setup (called once at app launch)

287 +

288 +```swift

289 +import Foundation

290 +

291 +func setupInferenceEnvironment() {

292 + guard let container = FileManager.default.containerURL(

293 + forSecurityApplicationGroupIdentifier: "group.com.ondeinference.apps"

294 + ) else {

295 + // Fall back to app-private Application Support if App Group is unavailable.

296 + let appSupport = FileManager.default

297 + .urls(for: .applicationSupportDirectory, in: .userDomainMask)[0]

298 + setupHfCache(at: appSupport)

299 + return

300 + }

301 + setupHfCache(at: container)

302 +}

303 +

304 +private func setupHfCache(at base: URL) {

305 + let hfHome = base.appendingPathComponent("models")

306 + let hfHub = hfHome.appendingPathComponent("hub")

307 + try? FileManager.default.createDirectory(at: hfHub, withIntermediateDirectories: true)

308 +

309 + setenv("HF_HOME", hfHome.path, 1)

310 + setenv("HF_HUB_CACHE", hfHub.path, 1)

311 +

312 + let tmp = base.appendingPathComponent("tmp")

313 + try? FileManager.default.createDirectory(at: tmp, withIntermediateDirectories: true)

314 + setenv("TMPDIR", tmp.path, 1)

315 +}

316 +```

317 +

318 +### Entitlements required

319 +

320 +Both iOS and macOS targets need the App Group entitlement:

321 +

322 +```xml

323 +<key>com.apple.security.application-groups</key>

324 +<array>

325 + <string>group.com.ondeinference.apps</string>

326 +</array>

327 +```

328 +

329 +The App Group must also be registered in the Apple Developer Portal under

330 +**Identifiers → App Groups** before Xcode can provision it.

331 +

332 +---

333 +

334 +## Local development (Xcode)

335 +

336 +Use the **local path** form during development — the remote binary form is only

337 +needed for distribution:

338 +

339 +```swift

340 +// Package.swift (local dev only — never commit to onde-kit)

341 +.binaryTarget(name: "OndeFramework", path: "./OndeFramework.xcframework")

342 +```

343 +

344 +In Xcode: **File → Add Package Dependencies → Add Local** → select `sdk/Onde/`.

345 +

346 +---

347 +

348 +## Common pitfalls

349 +

350 +| Pitfall | Fix |

351 +|---|---|

352 +| Local `path:` in published `Package.swift` | Always `url:` + `checksum:` in `onde-swift` |

353 +| Stale checksum after rebuild | Always recompute `shasum -a 256` — never hardcode |

354 +| Missing simulator slice | lipo `x86_64` + `aarch64-sim` before passing to `xcodebuild` |

355 +| UniFFI version drift | Keep `uniffi = "=0.31.0"` in both `Cargo.toml` and `uniffi-bindgen/Cargo.toml` |

356 +| tvOS build with stable toolchain | tvOS targets are tier-3; always use `cargo +nightly -Z build-std` |

357 +| Empty `Sources/` in `onde-swift` | Copy the generated `onde.swift` (and header) into the repo on every release |

358 +| `___chkstk_darwin` linker error | Ensure `tvos_chkstk.s` exists at crate root; `build.rs` compiles it automatically |

359 +| Missing Metal toolchain (Xcode 26+) | Run `xcodebuild -downloadComponent MetalToolchain`, then `cargo clean -p mistralrs-quant` |

360 +| Cache path mismatch (`huggingface/` vs `models/`) | All SDKs MUST use `<container>/models/hub/` — using `huggingface/hub/` means models are not shared across apps |

361 +| App Group not working | Register `group.com.ondeinference.apps` in Apple Developer Portal → Identifiers → App Groups; enable App Groups capability on the App ID; add entitlement to the target |

362 +| `CODE_SIGN_IDENTITY = "-"` blocks App Groups | Remove ad-hoc signing from project-level build settings; App Groups require a real development certificate |

363 +| Android NDK `LD` on PATH breaks Xcode | Never `export LD=$TOOLCHAIN/bin/ld` globally — it hijacks Xcode's linker. Use Cargo's `.cargo/config.toml` for Android linker config |

\ No newline at end of file

.agents/skills/uniffi-rs/SKILL.md new

+167

@@ -0,0 +1,167 @@

1 +---

2 +name: uniffi-rs

3 +description: Generate Swift, Kotlin, and Python bindings from Rust using UniFFI. Use when adding, modifying, or debugging FFI-exported types and functions in the onde crate.

4 +allowed-tools: Read, Write, Edit, Glob, Grep

5 +user-invocable: false

6 +---

7 +

8 +# uniffi-rs

9 +

10 +## What This Skill Covers

11 +

12 +How UniFFI is used in the `onde` crate to generate Swift, Kotlin, and Python bindings from a single Rust implementation.

13 +

14 +## Version Pin

15 +

16 +UniFFI is pinned to exactly `=0.31.0` everywhere. This version must stay in sync across:

17 +- `onde/Cargo.toml` → `uniffi = { version = "=0.31.0", features = ["tokio"] }`

18 +- `onde/Cargo.toml` build-deps → `uniffi = { version = "=0.31.0", features = ["build"] }`

19 +- `onde/uniffi-bindgen/Cargo.toml` → `uniffi = { version = "=0.31.0", features = ["cli"] }`

20 +

21 +Never bump one without bumping all three. Mismatched versions produce silently broken bindings.

22 +

23 +## Project Structure

24 +

25 +```

26 +onde/

27 +├── src/lib.rs # uniffi::setup_scaffolding!()

28 +├── src/inference/

29 +│ ├── types.rs # #[derive(uniffi::Record/Enum/Error)]

30 +│ └── ffi.rs # #[derive(uniffi::Object)], #[uniffi::export]

31 +├── uniffi-bindgen/

32 +│ ├── Cargo.toml # Standalone bin crate for the bindgen CLI

33 +│ └── uniffi-bindgen.rs # fn main() { uniffi::uniffi_bindgen_main() }

34 +├── uniffi.toml # Binding config (package names, cdylib name)

35 +└── build.rs # uniffi build support (build-dep)

36 +```

37 +

38 +## Scaffolding Setup

39 +

40 +`src/lib.rs` uses the proc-macro approach:

41 +

42 +```rust

43 +uniffi::setup_scaffolding!();

44 +```

45 +

46 +This is equivalent to the UDL approach but requires no `.udl` file. The `build.rs` still declares the `uniffi` build-dep to keep compatibility if the project ever switches to UDL.

47 +

48 +## Type Annotations

49 +

50 +| Rust construct | UniFFI derive | Crosses FFI as |

51 +|---|---|---|

52 +| Plain data struct | `#[derive(uniffi::Record)]` | Swift `struct` / Kotlin `data class` |

53 +| C-like / rich enum | `#[derive(uniffi::Enum)]` | Swift `enum` / Kotlin `sealed class` |

54 +| `thiserror::Error` enum | `#[derive(uniffi::Error)]` | Swift `Error enum` / Kotlin exception |

55 +| `Arc<T>` wrapper struct | `#[derive(uniffi::Object)]` | Swift `class` / Kotlin `object` |

56 +| Trait for callbacks | `#[uniffi::export(callback_interface)]` | Swift `protocol` / Kotlin `interface` |

57 +

58 +All exported functions and impl blocks are annotated with `#[uniffi::export]`.

59 +

60 +## Type Compatibility Rules

61 +

62 +These constraints come directly from UniFFI 0.31 limitations:

63 +

64 +- **`usize` / `isize` are not supported** — use `u64` / `i64` instead.

65 +- **`impl Into<String>` is not FFI-safe** — use concrete `String` in all exported method signatures.

66 +- **`tokio::sync::mpsc::Receiver` is not FFI-safe** — streaming is exposed via a `callback_interface` trait delivered through a free function, not an Object method.

67 +- **`callback_interface` parameters must be on free functions** — UniFFI 0.31 does not support `callback_interface` arguments on `#[uniffi::Object]` methods.

68 +- **Generics are not supported** — all exported types must be fully concrete.

69 +

70 +## The uniffi-bindgen CLI

71 +

72 +`onde/uniffi-bindgen/` is a minimal standalone binary crate:

73 +

74 +```rust

75 +// uniffi-bindgen.rs

76 +fn main() {

77 + uniffi::uniffi_bindgen_main()

78 +}

79 +```

80 +

81 +Build it once and run it to generate bindings:

82 +

83 +```bash

84 +cargo build --manifest-path uniffi-bindgen/Cargo.toml --release

85 +

86 +# Generate Swift

87 +./target/release/uniffi-bindgen generate \

88 + --library target/aarch64-apple-tvos/release/libonde.a \

89 + --language swift \

90 + --out-dir sdk/Onde/Sources/Onde \

91 + --config uniffi.toml

92 +

93 +# Generate Kotlin

94 +./target/release/uniffi-bindgen generate \

95 + --library target/aarch64-linux-android/release/libonde.so \

96 + --language kotlin \

97 + --out-dir sdk/kotlin/onde/src/main/kotlin \

98 + --config uniffi.toml

99 +

100 +# Generate Python

101 +./target/release/uniffi-bindgen generate \

102 + --library target/release/libonde.dylib \

103 + --language python \

104 + --out-dir sdk/python \

105 + --config uniffi.toml

106 +```

107 +

108 +The `--library` argument points to a compiled artifact (`.a`, `.so`, or `.dylib`). UniFFI reads the embedded metadata from the binary — any compiled artifact from any platform works for generating bindings for any language.

109 +

110 +## uniffi.toml

111 +

112 +```toml

113 +[bindings.kotlin]

114 +package_name = "com.ondeinference.onde"

115 +cdylib_name = "onde"

116 +

117 +[bindings.python]

118 +cdylib_name = "onde"

119 +```

120 +

121 +No Swift-specific config is needed here. `cargo-swift` (when used) and the raw `uniffi-bindgen` both derive Swift names from the Rust struct/function names.

122 +

123 +## Async Support

124 +

125 +UniFFI 0.31 supports `async fn` exports when the crate enables the `tokio` feature:

126 +

127 +```toml

128 +uniffi = { version = "=0.31.0", features = ["tokio"] }

129 +```

130 +

131 +All `async fn` methods on `#[uniffi::Object]` and all `async` free functions annotated with `#[uniffi::export]` are automatically bridged to Swift `async throws` and Kotlin `suspend fun`.

132 +

133 +## Streaming Pattern

134 +

135 +Because `callback_interface` is not allowed on Object methods in UniFFI 0.31, streaming is exposed as a top-level free function that accepts the Object by `Arc<T>`:

136 +

137 +```rust

138 +// CORRECT — free function with Arc<Object>

139 +#[uniffi::export]

140 +pub async fn stream_chat_message(

141 + engine: Arc<OndeChatEngine>,

142 + message: String,

143 + listener: Box<dyn StreamChunkListener>, // callback_interface

144 +) -> Result<(), InferenceError> { ... }

145 +

146 +// WRONG — callback_interface on Object method (not supported in 0.31)

147 +// impl OndeChatEngine {

148 +// pub async fn stream(&self, listener: Box<dyn StreamChunkListener>) { ... }

149 +// }

150 +```

151 +

152 +## Adding a New Exported Type

153 +

154 +1. Define the type in `src/inference/types.rs` (or whichever module owns it).

155 +2. Add the appropriate `#[derive(uniffi::Record)]` / `#[derive(uniffi::Enum)]` / `#[derive(uniffi::Error)]`.

156 +3. Replace any `usize` fields with `u64`, and any `isize` with `i64`.

157 +4. If the type needs to be exported from the FFI Object, add it to the method signatures in `src/inference/ffi.rs`.

158 +5. Recompile for any Apple target and re-run `uniffi-bindgen generate` for Swift.

159 +6. Re-run `build-kotlin.sh` for Kotlin.

160 +

161 +## Common Pitfalls

162 +

163 +- **Version mismatch**: If the `uniffi` version in `onde/Cargo.toml` differs from the one in `onde/uniffi-bindgen/Cargo.toml`, the generated bindings will be incompatible with the compiled library at runtime. Always keep them identical with the `=` exact pin.

164 +- **Forgetting `setup_scaffolding!()`**: Without `uniffi::setup_scaffolding!()` in `lib.rs`, no scaffolding code is generated and all `#[uniffi::export]` annotations are silently ignored.

165 +- **`usize` in struct fields**: UniFFI will produce a hard compile error or a confusing runtime panic if `usize` appears in any exported type. Always use `u64`.

166 +- **Callback interface in Object method**: This compiles fine but panics at runtime in UniFFI 0.31. Move the method to a free function.

167 +- **Not rebuilding the bindgen binary**: If you update `uniffi` in the main `Cargo.toml` without rebuilding `uniffi-bindgen`, the CLI will use the old schema and generate stale/broken bindings.

\ No newline at end of file

AGENTS.md new