@ondeinference / onde / commits / fcb073c

Fix running iOS example

keypair34 committed Apr 5, 2026 at 15:40 UTC fcb073ce05e05d6e50fef6c1a21a4616d788fd68
27 files changed +591 -174
.agents/skills/sdk-regressions/SKILL.md
+42 -4
@@ -173,12 +173,18 @@ candle's Metal/NEON kernels.
173 | `target_os` | `mistralrs` features | Extra deps | GPU | Default model |
174 |-------------|---------------------|------------|-----|---------------|
175 | `macos` | `["metal"]` | — | Metal | Qwen 2.5 3B |
176 -| `ios` | `["metal"]` | — | Metal | Qwen 2.5 1.5B |
177 -| `tvos` | `["metal"]` | — | Metal | Qwen 2.5 1.5B |
176 +| `ios` | `["metal"]` | `hf-hub`, `mistralrs-core` | Metal | Qwen 2.5 1.5B |
177 +| `tvos` | `["metal"]` | `hf-hub`, `mistralrs-core` | Metal | Qwen 2.5 1.5B |
178 | `android` | `[]` | `hf-hub`, `mistralrs-core` | CPU | Qwen 2.5 1.5B |
179 | `windows` | `[]` | — | CPU | Qwen 2.5 3B |
180 | `linux` | `[]` | — | CPU | Qwen 2.5 3B |
181
182 +`hf-hub` and `mistralrs-core` are required on all **sandboxed** platforms
183 +(iOS, tvOS, Android) for the `GLOBAL_HF_CACHE` workaround — `~/.cache` is
184 +outside the app container on iOS/tvOS, and `dirs::home_dir()` panics on Android.
185 +The re-exports in `src/lib.rs` (`pub use hf_hub; pub use mistralrs_core;`) are
186 +gated to `#[cfg(any(target_os = "android", target_os = "ios", target_os = "tvos"))]`.
187 +
188 ---
189
190 ## Common Regression Patterns
@@ -266,5 +272,37 @@ The fix: comment out the tool exports, use Cargo's
272
273 ---
274
269 -*Last updated: July 2025 — based on onde 0.2.x with Xcode 26, Rust 1.92–1.94,
270 -nightly for tvOS.*
\ No newline at end of file
275 +## Cache Path Consistency (Cross-SDK)
276 +
277 +All Onde-powered apps share downloaded models via the App Group
278 +`group.com.ondeinference.apps`. The cache layout inside the shared container
279 +**must be identical** across every SDK and app type:
280 +
281 +```
282 +<group container>/
283 +├── models/ ← HF_HOME
284 +│ └── hub/ ← HF_HUB_CACHE (GGUF files live here)
285 +└── tmp/ ← TMPDIR (iOS sandbox restricts system TMPDIR)
286 +```
287 +
288 +**The subdirectory is `models/`, NOT `huggingface/`.**
289 +
290 +Files that set this path (all must agree):
291 +
292 +| File | Variable | Expected value |
293 +|------|----------|----------------|
294 +| `onde/sdk/dart/rust/src/api.rs` (`configure_cache_dir`) | `hf_home` | `data_dir.join("models")` |
295 +| `onde/src/hf_cache.rs` (`download_model` — general) | `hf_home` | `data_dir.join("models")` |
296 +| `onde/src/hf_cache.rs` (`download_model` — Android) | `hf_home` | `resolved_app_data.join("models")` |
297 +| Tauri apps (`setup_application_filesystem.rs`) | `models_home` | `container_dir.join("models")` |
298 +| `onde/src/hf_cache.rs` (`hf_cache_dir` — fallback) | default | `~/.cache/huggingface/hub` (non-sandboxed only, OK to differ) |
299 +
300 +**Regression check:** `grep -rn 'join("huggingface")' src/hf_cache.rs sdk/dart/rust/src/api.rs`
301 +should return **only** the non-sandboxed fallback in `hf_cache_dir()` (line ~164).
302 +Any other match means a sandboxed path is using the wrong subdirectory.
303 +
304 +---
305 +
306 +*Last updated: July 2025 — added hf-hub/mistralrs-core iOS/tvOS deps,
307 +cache path consistency checks, App Group convention.*
308 +
.agents/skills/sdk-swift-package/SKILL.md
+76 -5
@@ -1,6 +1,6 @@
1 ---
2 name: sdk-swift-package
3 -description: Build the Onde Swift package (XCFramework) from Rust source using UniFFI and distribute it via a remote-binary Package.swift for Swift Package Index. Use when working on Swift SDK distribution, xcframework assembly, or the onde-kit release workflow.
3 +description: Build the Onde Swift package (XCFramework) from Rust source using UniFFI and distribute it via a remote-binary Package.swift for Swift Package Index. Covers xcframework assembly, App Group shared container, models/hub cache convention, and the onde-swift release workflow.
4 allowed-tools: Read, Write, Edit, Glob, Grep, Bash
5 user-invocable: true
6 ---
@@ -43,7 +43,9 @@ onde/
43 | tvOS linker | `___chkstk_darwin` is missing from tvOS libSystem. `build.rs` compiles `tvos_chkstk.s` as a no-op stub. This is automatic — do not remove it. |
44 | `cargo-swift` | **Not required.** The project owns its own `uniffi-bindgen` binary. Use it directly (same as `build-kotlin.sh` does for Android). |
45 | `.xcframework` in `Package.swift` | Published `Package.swift` must use **remote** `url:` + `checksum:` — never a local `path:`. |
46 -| Swift package name | `Onde` (PascalCase). Git repo slug: `onde-kit` under the `ondeinference` org. |
46 +| Swift package name | `Onde` (PascalCase). Git repo slug: `onde-swift` under the `ondeinference` org. |
47 +| App Group ID | `group.com.ondeinference.apps` — shared across all Onde-powered apps (Flutter, Tauri, native Swift). |
48 +| HF cache subdirectory | `<container>/models/hub/` — NOT `huggingface/hub/`. Must match the Tauri and Dart SDK convention. |
49
50 ---
51
@@ -264,6 +266,71 @@ assistantMessage(content:)
266
267 ---
268
269 +## App Group Shared Container & Cache Convention
270 +
271 +All Onde-powered apps use the App Group `group.com.ondeinference.apps` to share
272 +downloaded models. The cache layout inside the shared container is:
273 +
274 +```
275 +<group container>/
276 +├── models/ ← HF_HOME
277 +│ └── hub/ ← HF_HUB_CACHE (GGUF files live here)
278 +└── tmp/ ← TMPDIR (iOS sandbox restricts system TMPDIR)
279 +```
280 +
281 +**Important:** The subdirectory is `models/`, NOT `huggingface/`. This was
282 +chosen to be user-friendly when browsing the container in Finder/Files. All
283 +SDKs (Rust, Swift, Dart) and all app types (Tauri, Flutter, native) must use
284 +the same path or models won't be shared.
285 +
286 +### Swift setup (called once at app launch)
287 +
288 +```swift
289 +import Foundation
290 +
291 +func setupInferenceEnvironment() {
292 + guard let container = FileManager.default.containerURL(
293 + forSecurityApplicationGroupIdentifier: "group.com.ondeinference.apps"
294 + ) else {
295 + // Fall back to app-private Application Support if App Group is unavailable.
296 + let appSupport = FileManager.default
297 + .urls(for: .applicationSupportDirectory, in: .userDomainMask)[0]
298 + setupHfCache(at: appSupport)
299 + return
300 + }
301 + setupHfCache(at: container)
302 +}
303 +
304 +private func setupHfCache(at base: URL) {
305 + let hfHome = base.appendingPathComponent("models")
306 + let hfHub = hfHome.appendingPathComponent("hub")
307 + try? FileManager.default.createDirectory(at: hfHub, withIntermediateDirectories: true)
308 +
309 + setenv("HF_HOME", hfHome.path, 1)
310 + setenv("HF_HUB_CACHE", hfHub.path, 1)
311 +
312 + let tmp = base.appendingPathComponent("tmp")
313 + try? FileManager.default.createDirectory(at: tmp, withIntermediateDirectories: true)
314 + setenv("TMPDIR", tmp.path, 1)
315 +}
316 +```
317 +
318 +### Entitlements required
319 +
320 +Both iOS and macOS targets need the App Group entitlement:
321 +
322 +```xml
323 +<key>com.apple.security.application-groups</key>
324 +<array>
325 + <string>group.com.ondeinference.apps</string>
326 +</array>
327 +```
328 +
329 +The App Group must also be registered in the Apple Developer Portal under
330 +**Identifiers → App Groups** before Xcode can provision it.
331 +
332 +---
333 +
334 ## Local development (Xcode)
335
336 Use the **local path** form during development — the remote binary form is only
@@ -282,11 +349,15 @@ In Xcode: **File → Add Package Dependencies → Add Local** → select `sdk/On
349
350 | Pitfall | Fix |
351 |---|---|
285 -| Local `path:` in published `Package.swift` | Always `url:` + `checksum:` in `onde-kit` |
352 +| Local `path:` in published `Package.swift` | Always `url:` + `checksum:` in `onde-swift` |
353 | Stale checksum after rebuild | Always recompute `shasum -a 256` — never hardcode |
354 | Missing simulator slice | lipo `x86_64` + `aarch64-sim` before passing to `xcodebuild` |
355 | UniFFI version drift | Keep `uniffi = "=0.31.0"` in both `Cargo.toml` and `uniffi-bindgen/Cargo.toml` |
356 | tvOS build with stable toolchain | tvOS targets are tier-3; always use `cargo +nightly -Z build-std` |
290 -| Empty `Sources/` in `onde-kit` | Copy the generated `onde.swift` (and header) into the repo on every release |
357 +| Empty `Sources/` in `onde-swift` | Copy the generated `onde.swift` (and header) into the repo on every release |
358 | `___chkstk_darwin` linker error | Ensure `tvos_chkstk.s` exists at crate root; `build.rs` compiles it automatically |
292 -| Missing Metal toolchain (Xcode 26+) | Run `xcodebuild -downloadComponent MetalToolchain`, then `cargo clean -p mistralrs-quant` |
\ No newline at end of file
359 +| Missing Metal toolchain (Xcode 26+) | Run `xcodebuild -downloadComponent MetalToolchain`, then `cargo clean -p mistralrs-quant` |
360 +| Cache path mismatch (`huggingface/` vs `models/`) | All SDKs MUST use `<container>/models/hub/` — using `huggingface/hub/` means models are not shared across apps |
361 +| App Group not working | Register `group.com.ondeinference.apps` in Apple Developer Portal → Identifiers → App Groups; enable App Groups capability on the App ID; add entitlement to the target |
362 +| `CODE_SIGN_IDENTITY = "-"` blocks App Groups | Remove ad-hoc signing from project-level build settings; App Groups require a real development certificate |
363 +| Android NDK `LD` on PATH breaks Xcode | Never `export LD=$TOOLCHAIN/bin/ld` globally — it hijacks Xcode's linker. Use Cargo's `.cargo/config.toml` for Android linker config |
\ No newline at end of file
Cargo.toml
+11 -4
@@ -40,20 +40,27 @@ uniffi = { version = "=0.31.0", features = ["build"] }
40 # Used by build.rs to compile the tvOS ___chkstk_darwin assembly stub.
41 cc = "1"
42
43 -# Android: CPU-only GGUF inference via candle; also needs hf-hub for explicit
44 -# cache path seeding (dirs::home_dir() panics in the Android sandbox).
43 +# Android: CPU-only GGUF inference via candle; also needs hf-hub and
44 +# mistralrs-core for explicit cache path seeding (dirs::home_dir() panics
45 +# in the Android sandbox).
46 [target.'cfg(target_os = "android")'.dependencies]
47 hf-hub = { version = "0.4", features = ["tokio", "rustls-tls"], default-features = false }
48 mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = [] }
49 mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = [] }
50
50 -# iOS: Metal-accelerated inference.
51 +# iOS: Metal-accelerated inference; hf-hub + mistralrs-core needed for
52 +# GLOBAL_HF_CACHE sandbox workaround (~/.cache is outside the container).
53 [target.'cfg(target_os = "ios")'.dependencies]
54 +hf-hub = { version = "0.4", features = ["tokio", "rustls-tls"], default-features = false }
55 mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
56 +mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
57
54 -# tvOS: Metal-accelerated inference (Apple TV has Apple GPU).
58 +# tvOS: Metal-accelerated inference (Apple TV has Apple GPU); hf-hub +
59 +# mistralrs-core needed for GLOBAL_HF_CACHE sandbox workaround.
60 [target.'cfg(target_os = "tvos")'.dependencies]
61 +hf-hub = { version = "0.4", features = ["tokio", "rustls-tls"], default-features = false }
62 mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
63 +mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
64
65 # macOS: Metal-accelerated inference.
66 [target.'cfg(target_os = "macos")'.dependencies]
README.md
+1 -1
@@ -15,7 +15,7 @@
15 </p>
16
17 <p align="center">
18 - <a href="https://github.com/ondeinference/onde-swift">Swift SDK</a> · <a href="https://ondeinference.com">Website</a>
18 + <a href="https://github.com/ondeinference/onde-swift">Swift SDK</a> · <a href="https://pub.dev/packages/onde_inference">Flutter SDK</a> · <a href="https://ondeinference.com">Website</a>
19 </p>
20
21 ---
sdk/dart/README.md
+49 -76
@@ -1,12 +1,22 @@
1 -# onde_inference
1 +<p align="center">
2 + <img src="https://raw.githubusercontent.com/ondeinference/onde/main/assets/onde-inference-logo.svg" alt="Onde Inference" width="96">
3 +</p>
4
3 -[![pub.dev](https://img.shields.io/pub/v/onde_inference.svg)](https://pub.dev/packages/onde_inference)
4 -[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/ondeinference/onde/blob/main/sdk/dart/LICENSE)
5 -[![Platform](https://img.shields.io/badge/platform-iOS%20%7C%20macOS%20%7C%20Android%20%7C%20Linux%20%7C%20Windows-lightgrey)](https://pub.dev/packages/onde_inference)
5 +<h1 align="center">Onde Inference</h1>
6
7 -**On-device LLM inference for Flutter & Dart.**
7 +<p align="center">
8 + <strong>On-device LLM inference for Flutter & Dart — optimized for <a href="https://en.wikipedia.org/wiki/Apple_silicon">Apple silicon</a>.</strong>
9 +</p>
10
9 -Run [Qwen 2.5](https://huggingface.co/bartowski/Qwen2.5-3B-Instruct-GGUF) language models locally — no cloud, no API keys, no data leaving the device. Powered by the [Onde](https://ondeinference.com) Rust engine and [mistral.rs](https://github.com/EricLBuehler/mistral.rs), bridged to Flutter via [flutter_rust_bridge v2](https://pub.dev/packages/flutter_rust_bridge).
11 +<p align="center">
12 + <a href="https://pub.dev/packages/onde_inference"><img src="https://img.shields.io/pub/v/onde_inference?style=flat-square&color=235843&labelColor=17211D&label=pub.dev" alt="pub.dev"></a>
13 + <a href="https://ondeinference.com"><img src="https://img.shields.io/badge/ondeinference.com-235843?style=flat-square&labelColor=17211D" alt="Website"></a>
14 + <a href="https://apps.apple.com/se/developer/splitfire-ab/id1831430993"><img src="https://img.shields.io/badge/App%20Store-live-235843?style=flat-square&labelColor=17211D" alt="App Store"></a>
15 +</p>
16 +
17 +<p align="center">
18 + <a href="https://crates.io/crates/onde">Rust SDK</a> · <a href="https://github.com/ondeinference/onde-swift">Swift SDK</a> · <a href="https://ondeinference.com">Website</a>
19 +</p>
20
21 ---
22
@@ -37,67 +47,20 @@ Run [Qwen 2.5](https://huggingface.co/bartowski/Qwen2.5-3B-Instruct-GGUF) langua
47
48 ---
49
40 -## Getting started
50 +## Quick start
51
42 -### 1. Add the dependency
52 +### Add the dependency
53
54 ```yaml
55 dependencies:
56 onde_inference: ^0.1.0
57 ```
58
49 -### 2. Install Rust (required to build the native bridge)
50 -
51 -```sh
52 -curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
53 -```
54 -
55 -Add the targets for your platform(s):
56 -
57 -```sh
58 -# iOS
59 -rustup target add aarch64-apple-ios aarch64-apple-ios-sim
60 -
61 -# macOS
62 -rustup target add aarch64-apple-darwin x86_64-apple-darwin
63 -
64 -# Android (requires NDK r25+)
65 -rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android
66 -
67 -# Linux / Windows — already covered by the host toolchain
68 -```
69 -
70 -### 3. Run the code generator
71 -
72 -From your **Flutter project root** (not the package root):
73 -
74 -```sh
75 -dart pub get
76 -dart run flutter_rust_bridge_codegen generate
77 -```
78 -
79 -This reads `onde_inference`'s `rust/src/api.rs` and writes the FFI glue into
80 -`lib/src/rust/frb_generated.dart` inside the package. You only need to re-run
81 -this when the package is updated.
82 -
83 -### 4. Build the native library
59 +> **Note:** The native inference engine is written in Rust and compiled automatically during the Flutter build. A working [Rust toolchain](https://rustup.rs) is required. The first build compiles the full dependency tree (~5–10 minutes cold, <1 minute incremental).
60
85 -The native Rust library is compiled automatically as part of the normal Flutter
86 -build. On iOS and macOS it is driven by the CocoaPods script phase in the
87 -podspec; on Android by the CMake step in `android/build.gradle`; on Linux and
88 -Windows by the `add_custom_command` in the platform `CMakeLists.txt`.
61 +### Initialize
62
90 -For the very first build, allow extra time for Cargo to compile the dependency
91 -tree (~5–10 minutes cold, <1 minute incremental).
92 -
93 ----
94 -
95 -## Usage
96 -
97 -### Initialize the library
98 -
99 -Call `OndeInference.init()` **once** at application startup, before creating
100 -any `OndeChatEngine`:
63 +Call `OndeInference.init()` **once** at application startup, before creating any `OndeChatEngine`:
64
65 ```dart
66 import 'package:onde_inference/onde_inference.dart';
@@ -109,7 +72,7 @@ void main() async {
72 }
73 ```
74
112 -### Create an engine and load the default model
75 +### Load a model
76
77 ```dart
78 // Create the engine (synchronous — no model is loaded yet).
@@ -124,7 +87,7 @@ final elapsed = await engine.loadDefaultModel(
87 print('Model loaded in ${elapsed.toStringAsFixed(1)} s');
88 ```
89
127 -### Send a message (non-streaming)
90 +### Chat
91
92 ```dart
93 final result = await engine.sendMessage('What is Rust's ownership model?');
@@ -132,7 +95,7 @@ print(result.text);
95 print('Generated in ${result.durationDisplay}');
96 ```
97
135 -### Stream a response
98 +### Stream
99
100 ```dart
101 final buffer = StringBuffer();
@@ -229,7 +192,7 @@ await engine.loadGgufModel(
192
193 ---
194
232 -## Sampling configuration
195 +## Sampling
196
197 ```dart
198 // All fields are optional — null means "use the engine default".
@@ -269,23 +232,29 @@ Common causes:
232
233 ---
234
272 -## Running codegen
235 +## Sandboxed app setup (iOS / macOS)
236
274 -The Dart bindings are generated from the Rust source using
275 -`flutter_rust_bridge_codegen`. Run this command from the package root
276 -whenever `rust/src/api.rs` changes:
237 +```dart
238 +import 'package:onde_inference/onde_inference.dart';
239 +import 'package:path_provider/path_provider.dart';
240
278 -```sh
279 -# From onde/sdk/dart/
280 -dart pub get
281 -dart run flutter_rust_bridge_codegen generate
241 +void main() async {
242 + WidgetsFlutterBinding.ensureInitialized();
243 + await OndeInference.init();
244 +
245 + // Resolve shared App Group container (iOS/macOS) or private sandbox (Android).
246 + String? fallback;
247 + if (Platform.isIOS || Platform.isAndroid) {
248 + final dir = await getApplicationSupportDirectory();
249 + fallback = dir.path;
250 + }
251 + await OndeInference.setupCacheDir(fallbackDir: fallback);
252 +
253 + runApp(const MyApp());
254 +}
255 ```
256
284 -The generated output is committed to `lib/src/frb_generated.dart` (and
285 -platform-specific siblings). A hand-written stub at
286 -`lib/src/frb_generated_stub.dart` stands in for the generated code before
287 -the first codegen run, allowing the package to be compiled and the type
288 -system to be checked without a built Rust binary.
257 +> On iOS and macOS, `setupCacheDir()` first tries the App Group shared container (`group.com.ondeinference.apps`) so all Onde-powered apps share downloaded models. If unavailable, it falls back to the app's private directory.
258
259 ---
260
@@ -305,4 +274,8 @@ Please open an issue before submitting a pull request for significant changes.
274
275 ## License
276
308 -MIT © [Splitfire AB](https://splitfire.se) — see [LICENSE](LICENSE).
\ No newline at end of file
277 +MIT © [Splitfire AB](https://splitfire.se) — see [LICENSE](LICENSE).
278 +
279 +<p align="center">
280 + <sub>© 2026 <a href="https://ondeinference.com">Onde Inference</a></sub>
281 +</p>
\ No newline at end of file
sdk/dart/example/android/app/src/main/java/io/flutter/plugins/GeneratedPluginRegistrant.java
+5
@@ -15,5 +15,10 @@ import io.flutter.embedding.engine.FlutterEngine;
15 public final class GeneratedPluginRegistrant {
16 private static final String TAG = "GeneratedPluginRegistrant";
17 public static void registerWith(@NonNull FlutterEngine flutterEngine) {
18 + try {
19 + flutterEngine.getPlugins().add(new io.flutter.plugins.pathprovider.PathProviderPlugin());
20 + } catch (Exception e) {
21 + Log.e(TAG, "Error registering plugin path_provider_android, io.flutter.plugins.pathprovider.PathProviderPlugin", e);
22 + }
23 }
24 }
sdk/dart/example/ios/Podfile
-23
@@ -39,28 +39,5 @@ end
39 post_install do |installer|
40 installer.pods_project.targets.each do |target|
41 flutter_additional_ios_build_settings(target)
42 - # Xcode 26 (Xcode 17) ships a new linker (ld-prime) that dropped legacy
43 - # flags used by Flutter's build scripts (-Xlinker, -target, -dynamiclib,
44 - # -isysroot, -reproducible). Force every pod target to use the classic
45 - # ld64 linker until Flutter officially supports the new linker.
46 - target.build_configurations.each do |config|
47 - config.build_settings['OTHER_LDFLAGS'] ||= ['$(inherited)']
48 - unless config.build_settings['OTHER_LDFLAGS'].include?('-ld_classic')
49 - config.build_settings['OTHER_LDFLAGS'] << '-ld_classic'
50 - end
51 - end
52 - end
53 -
54 - # Also patch the main Runner project targets (not just pods).
55 - installer.aggregate_targets.each do |aggregate_target|
56 - aggregate_target.user_project.targets.each do |target|
57 - target.build_configurations.each do |config|
58 - config.build_settings['OTHER_LDFLAGS'] ||= ['$(inherited)']
59 - unless config.build_settings['OTHER_LDFLAGS'].include?('-ld_classic')
60 - config.build_settings['OTHER_LDFLAGS'] << '-ld_classic'
61 - end
62 - end
63 - end
64 - aggregate_target.user_project.save
42 end
43 end
sdk/dart/example/ios/Podfile.lock
+1 -1
@@ -17,6 +17,6 @@ SPEC CHECKSUMS:
17 Flutter: cabc95a1d2626b1b06e7179b784ebcf0c0cde467
18 onde_inference: ae09386bb943bed458d530d3af9cf1442634e5d1
19
20 -PODFILE CHECKSUM: b02416e1885eb65c170e8950e827f65afad13d48
20 +PODFILE CHECKSUM: 3c63482e143d1b91d2d2560aee9fb04ecc74ac7e
21
22 COCOAPODS: 1.16.2
sdk/dart/example/ios/Runner.xcodeproj/project.pbxproj
+21
@@ -201,6 +201,7 @@
201 97C146EC1CF9000F007C117D /* Resources */,
202 9705A1C41CF9048500538489 /* Embed Frameworks */,
203 3B06AD1E1E4923F5004D2608 /* Thin Binary */,
204 + 9DB519066FB980475B418C5A /* [CP] Embed Pods Frameworks */,
205 );
206 buildRules = (
207 );
@@ -325,6 +326,23 @@
326 shellPath = /bin/sh;
327 shellScript = "/bin/sh \"$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh\" build";
328 };
329 + 9DB519066FB980475B418C5A /* [CP] Embed Pods Frameworks */ = {
330 + isa = PBXShellScriptBuildPhase;
331 + buildActionMask = 2147483647;
332 + files = (
333 + );
334 + inputFileListPaths = (
335 + "${PODS_ROOT}/Target Support Files/Pods-Runner/Pods-Runner-frameworks-${CONFIGURATION}-input-files.xcfilelist",
336 + );
337 + name = "[CP] Embed Pods Frameworks";
338 + outputFileListPaths = (
339 + "${PODS_ROOT}/Target Support Files/Pods-Runner/Pods-Runner-frameworks-${CONFIGURATION}-output-files.xcfilelist",
340 + );
341 + runOnlyForDeploymentPostprocessing = 0;
342 + shellPath = /bin/sh;
343 + shellScript = "\"${PODS_ROOT}/Target Support Files/Pods-Runner/Pods-Runner-frameworks.sh\"\n";
344 + showEnvVarsInLog = 0;
345 + };
346 E946D061797B66127AE20F9A /* [CP] Check Pods Manifest.lock */ = {
347 isa = PBXShellScriptBuildPhase;
348 buildActionMask = 2147483647;
@@ -456,6 +474,7 @@
474 buildSettings = {
475 ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
476 CLANG_ENABLE_MODULES = YES;
477 + CODE_SIGN_ENTITLEMENTS = Runner/Runner.entitlements;
478 CURRENT_PROJECT_VERSION = "$(FLUTTER_BUILD_NUMBER)";
479 DEVELOPMENT_TEAM = 2TQF86ZACD;
480 ENABLE_BITCODE = NO;
@@ -655,6 +674,7 @@
674 buildSettings = {
675 ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
676 CLANG_ENABLE_MODULES = YES;
677 + CODE_SIGN_ENTITLEMENTS = Runner/Runner.entitlements;
678 CURRENT_PROJECT_VERSION = "$(FLUTTER_BUILD_NUMBER)";
679 DEVELOPMENT_TEAM = 2TQF86ZACD;
680 ENABLE_BITCODE = NO;
@@ -682,6 +702,7 @@
702 buildSettings = {
703 ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
704 CLANG_ENABLE_MODULES = YES;
705 + CODE_SIGN_ENTITLEMENTS = Runner/Runner.entitlements;
706 CURRENT_PROJECT_VERSION = "$(FLUTTER_BUILD_NUMBER)";
707 DEVELOPMENT_TEAM = 2TQF86ZACD;
708 ENABLE_BITCODE = NO;
sdk/dart/example/ios/Runner/Runner.entitlements new
+10
@@ -0,0 +1,10 @@
1 +<?xml version="1.0" encoding="UTF-8"?>
2 +<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
3 +<plist version="1.0">
4 +<dict>
5 + <key>com.apple.security.application-groups</key>
6 + <array>
7 + <string>group.com.ondeinference.apps</string>
8 + </array>
9 +</dict>
10 +</plist>
sdk/dart/example/lib/main.dart
+21
@@ -14,8 +14,11 @@
14
15 import 'dart:async';
16
17 +import 'dart:io' show Platform;
18 +
19 import 'package:flutter/material.dart';
20 import 'package:onde_inference/onde_inference.dart';
21 +import 'package:path_provider/path_provider.dart' show getApplicationSupportDirectory;
22
23 // ---------------------------------------------------------------------------
24 // Entry point
@@ -24,6 +27,24 @@ import 'package:onde_inference/onde_inference.dart';
27 void main() async {
28 WidgetsFlutterBinding.ensureInitialized();
29 await OndeInference.init();
30 +
31 + // Resolve the model cache directory for sandboxed platforms.
32 + //
33 + // On iOS/macOS this tries the App Group shared container first
34 + // (group.com.ondeinference.apps) so all Onde-powered apps share
35 + // downloaded models. If the App Group is unavailable it falls back
36 + // to the app's private Application Support directory.
37 + //
38 + // On Android there is no App Group — the fallback is always used.
39 + //
40 + // On desktop Linux/Windows this is a no-op (default ~/.cache works).
41 + String? fallback;
42 + if (Platform.isIOS || Platform.isAndroid) {
43 + final dir = await getApplicationSupportDirectory();
44 + fallback = dir.path;
45 + }
46 + await OndeInference.setupCacheDir(fallbackDir: fallback);
47 +
48 runApp(const OndeInferenceApp());
49 }
50
sdk/dart/example/macos/Flutter/GeneratedPluginRegistrant.swift
+2
@@ -5,6 +5,8 @@
5 import FlutterMacOS
6 import Foundation
7
8 +import onde_inference
9
10 func RegisterGeneratedPlugins(registry: FlutterPluginRegistry) {
11 + OndeInferencePlugin.register(with: registry.registrar(forPlugin: "OndeInferencePlugin"))
12 }
sdk/dart/example/macos/Runner.xcodeproj/project.pbxproj
+3 -3
@@ -543,7 +543,6 @@
543 CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
544 CLANG_WARN_RANGE_LOOP_ANALYSIS = YES;
545 CLANG_WARN_SUSPICIOUS_MOVE = YES;
546 - CODE_SIGN_IDENTITY = "-";
546 COPY_PHASE_STRIP = NO;
547 DEAD_CODE_STRIPPING = YES;
548 DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym";
@@ -575,6 +574,7 @@
574 CODE_SIGN_ENTITLEMENTS = Runner/DebugProfile.entitlements;
575 CODE_SIGN_STYLE = Automatic;
576 COMBINE_HIDPI_IMAGES = YES;
577 + DEVELOPMENT_TEAM = 2TQF86ZACD;
578 INFOPLIST_FILE = Runner/Info.plist;
579 LD_RUNPATH_SEARCH_PATHS = (
580 "$(inherited)",
@@ -620,7 +620,6 @@
620 CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
621 CLANG_WARN_RANGE_LOOP_ANALYSIS = YES;
622 CLANG_WARN_SUSPICIOUS_MOVE = YES;
623 - CODE_SIGN_IDENTITY = "-";
623 COPY_PHASE_STRIP = NO;
624 DEAD_CODE_STRIPPING = YES;
625 DEBUG_INFORMATION_FORMAT = dwarf;
@@ -677,7 +676,6 @@
676 CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
677 CLANG_WARN_RANGE_LOOP_ANALYSIS = YES;
678 CLANG_WARN_SUSPICIOUS_MOVE = YES;
680 - CODE_SIGN_IDENTITY = "-";
679 COPY_PHASE_STRIP = NO;
680 DEAD_CODE_STRIPPING = YES;
681 DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym";
@@ -709,6 +707,7 @@
707 CODE_SIGN_ENTITLEMENTS = Runner/DebugProfile.entitlements;
708 CODE_SIGN_STYLE = Automatic;
709 COMBINE_HIDPI_IMAGES = YES;
710 + DEVELOPMENT_TEAM = 2TQF86ZACD;
711 INFOPLIST_FILE = Runner/Info.plist;
712 LD_RUNPATH_SEARCH_PATHS = (
713 "$(inherited)",
@@ -729,6 +728,7 @@
728 CODE_SIGN_ENTITLEMENTS = Runner/Release.entitlements;
729 CODE_SIGN_STYLE = Automatic;
730 COMBINE_HIDPI_IMAGES = YES;
731 + DEVELOPMENT_TEAM = 2TQF86ZACD;
732 INFOPLIST_FILE = Runner/Info.plist;
733 LD_RUNPATH_SEARCH_PATHS = (
734 "$(inherited)",
sdk/dart/example/macos/Runner/DebugProfile.entitlements
+4
@@ -10,5 +10,9 @@
10 <true/>
11 <key>com.apple.security.network.server</key>
12 <true/>
13 + <key>com.apple.security.application-groups</key>
14 + <array>
15 + <string>group.com.ondeinference.apps</string>
16 + </array>
17 </dict>
18 </plist>
sdk/dart/example/macos/Runner/Release.entitlements
+4
@@ -4,5 +4,9 @@
4 <dict>
5 <key>com.apple.security.app-sandbox</key>
6 <true/>
7 + <key>com.apple.security.application-groups</key>
8 + <array>
9 + <string>group.com.ondeinference.apps</string>
10 + </array>
11 </dict>
12 </plist>
sdk/dart/example/pubspec.yaml
+1
@@ -12,6 +12,7 @@ dependencies:
12 sdk: flutter
13 onde_inference:
14 path: ../
15 + path_provider: ^2.1.0
16
17 dev_dependencies:
18 flutter_test:
sdk/dart/ios/Classes/OndeInferencePlugin.swift
+42 -7
@@ -1,14 +1,49 @@
1 -// This file exists so CocoaPods creates a real framework target for the
2 -// onde_inference pod. The actual native code is the Rust static library
3 -// (libonde_inference_dart.a) which is force-loaded into this framework
4 -// via OTHER_LDFLAGS in the podspec.
1 +// OndeInferencePlugin.swift
2 //
6 -// Do not add any logic here.
3 +// Native iOS plugin for onde_inference.
4 +//
5 +// Responsibilities:
6 +// 1. Provides a method channel ("com.ondeinference.onde_inference") so Dart
7 +// can resolve the App Group shared container path at runtime.
8 +// 2. Acts as the CocoaPods framework host — the Rust static library
9 +// (libonde_inference_dart.a) is force-loaded into this framework via
10 +// OTHER_LDFLAGS in the podspec.
11
12 import Flutter
13 +import UIKit
14
15 public class OndeInferencePlugin: NSObject, FlutterPlugin {
16 +
17 + /// The App Group identifier shared across all Onde-powered apps.
18 + /// Must match the value in Runner.entitlements.
19 + private static let appGroupId = "group.com.ondeinference.apps"
20 +
21 public static func register(with registrar: FlutterPluginRegistrar) {
12 - // No-op — all FFI symbols come from the Rust static library.
22 + let channel = FlutterMethodChannel(
23 + name: "com.ondeinference.onde_inference",
24 + binaryMessenger: registrar.messenger()
25 + )
26 + let instance = OndeInferencePlugin()
27 + registrar.addMethodCallDelegate(instance, channel: channel)
28 + }
29 +
30 + public func handle(
31 + _ call: FlutterMethodCall,
32 + result: @escaping FlutterResult
33 + ) {
34 + switch call.method {
35 + case "getAppGroupContainerPath":
36 + if let url = FileManager.default.containerURL(
37 + forSecurityApplicationGroupIdentifier: Self.appGroupId
38 + ) {
39 + result(url.path)
40 + } else {
41 + // App Group not configured or not entitled — fall back to nil
42 + // so the Dart side can use getApplicationSupportDirectory() instead.
43 + result(nil)
44 + }
45 + default:
46 + result(FlutterMethodNotImplemented)
47 + }
48 }
14 -}
49 +}
\ No newline at end of file
sdk/dart/lib/src/engine.dart
+74 -10
@@ -3,6 +3,7 @@
3 import 'dart:async';
4 import 'dart:io' show Platform;
5
6 +import 'package:flutter/services.dart';
7 import 'package:flutter_rust_bridge/flutter_rust_bridge_for_generated.dart'
8 show ExternalLibrary;
9
@@ -20,7 +21,8 @@ export 'frb_generated.dart/api.dart'
21 OndeError_ModelBuild,
22 OndeError_Inference,
23 OndeError_Cancelled,
23 - OndeError_Other;
24 + OndeError_Other,
25 + configureCacheDir;
26
27 // ---------------------------------------------------------------------------
28 // OndeChatEngine convenience wrapper
@@ -135,24 +137,86 @@ abstract final class OndeInference {
137 /// Call this in `main()` or in a Flutter `initState` override before any
138 /// user interaction that could trigger model loading.
139 ///
138 - /// On macOS and iOS the Rust static library is force-loaded into the
139 - /// application process by the CocoaPods podspec (`-force_load`), so the
140 - /// symbols are available via [ffi.DynamicLibrary.process] rather than a
141 - /// separate `.framework` bundle.
140 + /// On macOS the Rust static library is force-loaded into the
141 + /// `onde_inference.framework` CocoaPods dynamic framework — we must
142 + /// `dlopen` that framework by name.
143 + ///
144 + /// On iOS the pod is statically linked into the Runner executable
145 + /// (no separate `.framework` file exists at runtime), so we use
146 + /// `DynamicLibrary.process()` (`RTLD_DEFAULT`) which searches the
147 + /// main executable and all loaded images.
148 + ///
149 + /// On Android / Linux / Windows the default FRB stem-based loader
150 + /// finds the `.so` / `.dll` automatically — no override needed.
151 static Future<void> init() async {
152 ExternalLibrary? lib;
144 - if (Platform.isMacOS || Platform.isIOS) {
145 - // The Rust static library is force-loaded into the onde_inference
146 - // CocoaPods framework via OTHER_LDFLAGS in the podspec. We must open
147 - // that framework explicitly — DynamicLibrary.process() (RTLD_DEFAULT)
148 - // cannot resolve symbols inside a separately-loaded framework.
153 + if (Platform.isMacOS) {
154 + // CocoaPods + use_frameworks! → pod is a dynamic framework.
155 + // DynamicLibrary.process() can't resolve symbols inside it.
156 lib = ExternalLibrary.open(
157 'onde_inference.framework/onde_inference',
158 );
159 + } else if (Platform.isIOS) {
160 + // CocoaPods on iOS statically links the pod into the Runner
161 + // binary via -force_load. Symbols live in the main executable.
162 + lib = ExternalLibrary.process(iKnowHowToUseIt: true);
163 }
164 await frb.RustLib.init(externalLibrary: lib);
165 }
166
167 + // -------------------------------------------------------------------------
168 + // Sandbox / cache setup
169 + // -------------------------------------------------------------------------
170 +
171 + /// Method channel shared with the native iOS/macOS plugin.
172 + static const _channel = MethodChannel('com.ondeinference.onde_inference');
173 +
174 + /// Configures the HuggingFace model cache for sandboxed platforms.
175 + ///
176 + /// On **iOS and macOS** this first tries to resolve the App Group shared
177 + /// container (`group.com.ondeinference.apps`) via a native method channel
178 + /// so all Onde-powered apps share downloaded models. If the App Group is
179 + /// unavailable (entitlement missing, group not registered), it falls back
180 + /// to [fallbackDir] if provided.
181 + ///
182 + /// On **Android** there is no App Group equivalent — pass [fallbackDir]
183 + /// (e.g. from `getApplicationSupportDirectory().path`).
184 + ///
185 + /// On **desktop Linux / Windows** this is a no-op — the default
186 + /// `~/.cache/huggingface` works without intervention.
187 + ///
188 + /// Call **once** at startup, after [init] and before any model load:
189 + ///
190 + /// ```dart
191 + /// await OndeInference.init();
192 + /// await OndeInference.setupCacheDir();
193 + /// // or with explicit fallback:
194 + /// final dir = await getApplicationSupportDirectory();
195 + /// await OndeInference.setupCacheDir(fallbackDir: dir.path);
196 + /// ```
197 + static Future<void> setupCacheDir({String? fallbackDir}) async {
198 + // 1. Apple platforms — try the shared App Group container.
199 + if (Platform.isIOS || Platform.isMacOS) {
200 + try {
201 + final String? groupPath = await _channel.invokeMethod<String>(
202 + 'getAppGroupContainerPath',
203 + );
204 + if (groupPath != null && groupPath.isNotEmpty) {
205 + api.configureCacheDir(appDataDir: groupPath);
206 + return;
207 + }
208 + } catch (_) {
209 + // Method channel not available (e.g. plugin not registered).
210 + // Fall through to fallbackDir.
211 + }
212 + }
213 +
214 + // 2. Fallback — use the caller-provided directory (app sandbox).
215 + if (fallbackDir != null && fallbackDir.isNotEmpty) {
216 + api.configureCacheDir(appDataDir: fallbackDir);
217 + }
218 + }
219 +
220 // -------------------------------------------------------------------------
221 // Model config factories
222 // -------------------------------------------------------------------------
sdk/dart/lib/src/frb_generated.dart/api.dart
+25
@@ -47,6 +47,31 @@ SamplingConfig deterministicSamplingConfig() =>
47 SamplingConfig mobileSamplingConfig() =>
48 RustLib.instance.api.crateApiMobileSamplingConfig();
49
50 +/// Seed the HuggingFace cache environment for sandboxed platforms.
51 +///
52 +/// On iOS (and Android), the default `~/.cache/huggingface/hub` path is outside
53 +/// the app sandbox and writes fail with `Operation not permitted (os error 1)`.
54 +///
55 +/// Call this **once** at startup — before any `load_gguf_model` or
56 +/// `download_model` — with the app's writable data directory.
57 +///
58 +/// # Dart usage
59 +///
60 +/// ```dart
61 +/// import 'package:path_provider/path_provider.dart';
62 +///
63 +/// final dir = await getApplicationSupportDirectory();
64 +/// configureCacheDir(appDataDir: dir.path);
65 +/// ```
66 +///
67 +/// Internally this:
68 +/// 1. Creates `<app_data_dir>/huggingface/hub/` if it doesn't exist.
69 +/// 2. Sets the `HF_HOME` and `HF_HUB_CACHE` environment variables.
70 +/// 3. Seeds `mistralrs_core::GLOBAL_HF_CACHE` so the model loader never
71 +/// falls back to `Cache::default()`.
72 +void configureCacheDir({required String appDataDir}) =>
73 + RustLib.instance.api.crateApiConfigureCacheDir(appDataDir: appDataDir);
74 +
75 // Rust type: RustOpaqueMoi<flutter_rust_bridge::for_generated::RustAutoOpaqueInner<OndeChatEngine>>
76 abstract class OndeChatEngine implements RustOpaqueInterface {
77 /// Clear the conversation history while keeping the model loaded.
sdk/dart/lib/src/frb_generated.dart/frb_generated.dart
+34 -9
@@ -70,7 +70,7 @@ class RustLib extends BaseEntrypoint<RustLibApi, RustLibApiImpl, RustLibWire> {
70 String get codegenVersion => '2.12.0';
71
72 @override
73 - int get rustContentHash => -2058074152;
73 + int get rustContentHash => -2049740590;
74
75 static const kDefaultExternalLibraryLoaderConfig =
76 ExternalLibraryLoaderConfig(
@@ -126,6 +126,8 @@ abstract class RustLibApi extends BaseApi {
126 Future<String?> crateApiOndeChatEngineUnloadModel(
127 {required OndeChatEngine that});
128
129 + void crateApiConfigureCacheDir({required String appDataDir});
130 +
131 GgufModelConfig crateApiDefaultModelConfig();
132
133 SamplingConfig crateApiDefaultSamplingConfig();
@@ -551,12 +553,35 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
553 );
554
555 @override
554 - GgufModelConfig crateApiDefaultModelConfig() {
556 + void crateApiConfigureCacheDir({required String appDataDir}) {
557 return handler.executeSync(SyncTask(
558 callFfi: () {
559 final serializer = SseSerializer(generalizedFrbRustBinding);
560 + sse_encode_String(appDataDir, serializer);
561 return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 15)!;
562 },
563 + codec: SseCodec(
564 + decodeSuccessData: sse_decode_unit,
565 + decodeErrorData: sse_decode_onde_error,
566 + ),
567 + constMeta: kCrateApiConfigureCacheDirConstMeta,
568 + argValues: [appDataDir],
569 + apiImpl: this,
570 + ));
571 + }
572 +
573 + TaskConstMeta get kCrateApiConfigureCacheDirConstMeta => const TaskConstMeta(
574 + debugName: "configure_cache_dir",
575 + argNames: ["appDataDir"],
576 + );
577 +
578 + @override
579 + GgufModelConfig crateApiDefaultModelConfig() {
580 + return handler.executeSync(SyncTask(
581 + callFfi: () {
582 + final serializer = SseSerializer(generalizedFrbRustBinding);
583 + return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 16)!;
584 + },
585 codec: SseCodec(
586 decodeSuccessData: sse_decode_gguf_model_config,
587 decodeErrorData: null,
@@ -577,7 +602,7 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
602 return handler.executeSync(SyncTask(
603 callFfi: () {
604 final serializer = SseSerializer(generalizedFrbRustBinding);
580 - return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 16)!;
605 + return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 17)!;
606 },
607 codec: SseCodec(
608 decodeSuccessData: sse_decode_sampling_config,
@@ -600,7 +625,7 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
625 return handler.executeSync(SyncTask(
626 callFfi: () {
627 final serializer = SseSerializer(generalizedFrbRustBinding);
603 - return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 17)!;
628 + return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 18)!;
629 },
630 codec: SseCodec(
631 decodeSuccessData: sse_decode_sampling_config,
@@ -623,7 +648,7 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
648 return handler.executeSync(SyncTask(
649 callFfi: () {
650 final serializer = SseSerializer(generalizedFrbRustBinding);
626 - return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 18)!;
651 + return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 19)!;
652 },
653 codec: SseCodec(
654 decodeSuccessData: sse_decode_sampling_config,
@@ -646,7 +671,7 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
671 return handler.executeSync(SyncTask(
672 callFfi: () {
673 final serializer = SseSerializer(generalizedFrbRustBinding);
649 - return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 19)!;
674 + return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 20)!;
675 },
676 codec: SseCodec(
677 decodeSuccessData: sse_decode_gguf_model_config,
@@ -668,7 +693,7 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
693 return handler.executeSync(SyncTask(
694 callFfi: () {
695 final serializer = SseSerializer(generalizedFrbRustBinding);
671 - return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 20)!;
696 + return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 21)!;
697 },
698 codec: SseCodec(
699 decodeSuccessData: sse_decode_gguf_model_config,
@@ -690,7 +715,7 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
715 return handler.executeSync(SyncTask(
716 callFfi: () {
717 final serializer = SseSerializer(generalizedFrbRustBinding);
693 - return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 21)!;
718 + return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 22)!;
719 },
720 codec: SseCodec(
721 decodeSuccessData: sse_decode_gguf_model_config,
@@ -713,7 +738,7 @@ class RustLibApiImpl extends RustLibApiImplPlatform implements RustLibApi {
738 return handler.executeSync(SyncTask(
739 callFfi: () {
740 final serializer = SseSerializer(generalizedFrbRustBinding);
716 - return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 22)!;
741 + return pdeCallFfi(generalizedFrbRustBinding, serializer, funcId: 23)!;
742 },
743 codec: SseCodec(
744 decodeSuccessData: sse_decode_gguf_model_config,
sdk/dart/macos/Classes/OndeInferencePlugin.swift
+42 -7
@@ -1,14 +1,49 @@
1 -// This file exists so CocoaPods creates a real framework target for the
2 -// onde_inference pod. The actual native code is the Rust static library
3 -// (libonde_inference_dart.a) which is force-loaded into this framework
4 -// via OTHER_LDFLAGS in the podspec.
1 +// OndeInferencePlugin.swift
2 //
6 -// Do not add any logic here.
3 +// Native macOS plugin for onde_inference.
4 +//
5 +// Responsibilities:
6 +// 1. Provides a method channel ("com.ondeinference.onde_inference") so Dart
7 +// can resolve the App Group shared container path at runtime.
8 +// 2. Acts as the CocoaPods framework host — the Rust static library
9 +// (libonde_inference_dart.a) is force-loaded into this framework via
10 +// OTHER_LDFLAGS in the podspec.
11
12 import FlutterMacOS
13 +import Foundation
14
15 public class OndeInferencePlugin: NSObject, FlutterPlugin {
16 +
17 + /// The App Group identifier shared across all Onde-powered apps.
18 + /// Must match the value in DebugProfile.entitlements / Release.entitlements.
19 + private static let appGroupId = "group.com.ondeinference.apps"
20 +
21 public static func register(with registrar: FlutterPluginRegistrar) {
12 - // No-op — all FFI symbols come from the Rust static library.
22 + let channel = FlutterMethodChannel(
23 + name: "com.ondeinference.onde_inference",
24 + binaryMessenger: registrar.messenger
25 + )
26 + let instance = OndeInferencePlugin()
27 + registrar.addMethodCallDelegate(instance, channel: channel)
28 + }
29 +
30 + public func handle(
31 + _ call: FlutterMethodCall,
32 + result: @escaping FlutterResult
33 + ) {
34 + switch call.method {
35 + case "getAppGroupContainerPath":
36 + if let url = FileManager.default.containerURL(
37 + forSecurityApplicationGroupIdentifier: Self.appGroupId
38 + ) {
39 + result(url.path)
40 + } else {
41 + // App Group not configured or not entitled — fall back to nil
42 + // so the Dart side can use getApplicationSupportDirectory() instead.
43 + result(nil)
44 + }
45 + default:
46 + result(FlutterMethodNotImplemented)
47 + }
48 }
14 -}
49 +}
\ No newline at end of file
sdk/dart/pubspec.yaml
+2
@@ -40,8 +40,10 @@ flutter:
40 ffiPlugin: true
41 ios:
42 ffiPlugin: true
43 + pluginClass: OndeInferencePlugin
44 macos:
45 ffiPlugin: true
46 + pluginClass: OndeInferencePlugin
47 linux:
48 ffiPlugin: true
49 windows:
sdk/dart/rust/src/api.rs
+60
@@ -510,3 +510,63 @@ pub fn deterministic_sampling_config() -> SamplingConfig {
510 pub fn mobile_sampling_config() -> SamplingConfig {
511 OndeSamplingConfig::mobile().into()
512 }
513 +
514 +/// Seed the HuggingFace cache environment for sandboxed platforms.
515 +///
516 +/// On iOS (and Android), the default `~/.cache/huggingface/hub` path is outside
517 +/// the app sandbox and writes fail with `Operation not permitted (os error 1)`.
518 +///
519 +/// Call this **once** at startup — before any `load_gguf_model` or
520 +/// `download_model` — with the app's writable data directory.
521 +///
522 +/// # Dart usage
523 +///
524 +/// ```dart
525 +/// import 'package:path_provider/path_provider.dart';
526 +///
527 +/// final dir = await getApplicationSupportDirectory();
528 +/// configureCacheDir(appDataDir: dir.path);
529 +/// ```
530 +///
531 +/// Internally this:
532 +/// 1. Creates `<app_data_dir>/huggingface/hub/` if it doesn't exist.
533 +/// 2. Sets the `HF_HOME` and `HF_HUB_CACHE` environment variables.
534 +/// 3. Seeds `mistralrs_core::GLOBAL_HF_CACHE` so the model loader never
535 +/// falls back to `Cache::default()`.
536 +#[frb(sync)]
537 +pub fn configure_cache_dir(app_data_dir: String) -> Result<(), OndeError> {
538 + use std::fs;
539 + use std::path::PathBuf;
540 +
541 + let data_dir = PathBuf::from(&app_data_dir);
542 + let hf_home = data_dir.join("models");
543 + let hf_hub_cache = hf_home.join("hub");
544 +
545 + fs::create_dir_all(&hf_hub_cache).map_err(|e| OndeError::Other {
546 + reason: format!(
547 + "Cannot create HF cache dir {}: {}",
548 + hf_hub_cache.display(),
549 + e
550 + ),
551 + })?;
552 +
553 + // Set env vars BEFORE any hf-hub or mistralrs call. Both libraries
554 + // read these to resolve the cache directory — no need to touch
555 + // GLOBAL_HF_CACHE directly (that's seeded lazily by load_gguf_model
556 + // on Android/iOS when it reads HF_HOME back).
557 + std::env::set_var("HF_HOME", &hf_home);
558 + std::env::set_var("HF_HUB_CACHE", &hf_hub_cache);
559 +
560 + // Pre-create tmp dir — iOS sandbox also restricts the system TMPDIR.
561 + let tmp_dir = data_dir.join("tmp");
562 + let _ = fs::create_dir_all(&tmp_dir);
563 + std::env::set_var("TMPDIR", &tmp_dir);
564 +
565 + log::info!(
566 + "HF cache configured: HF_HOME={}, HF_HUB_CACHE={}",
567 + hf_home.display(),
568 + hf_hub_cache.display(),
569 + );
570 +
571 + Ok(())
572 +}
sdk/dart/rust/src/frb_generated.rs
+40 -9
@@ -39,7 +39,7 @@ flutter_rust_bridge::frb_generated_boilerplate!(
39 default_rust_auto_opaque = RustAutoOpaqueMoi,
40 );
41 pub(crate) const FLUTTER_RUST_BRIDGE_CODEGEN_VERSION: &str = "2.12.0";
42 -pub(crate) const FLUTTER_RUST_BRIDGE_CODEGEN_CONTENT_HASH: i32 = -2058074152;
42 +pub(crate) const FLUTTER_RUST_BRIDGE_CODEGEN_CONTENT_HASH: i32 = -2049740590;
43
44 // Section: executor
45
@@ -850,6 +850,36 @@ fn wire__crate__api__OndeChatEngine_unload_model_impl(
850 },
851 )
852 }
853 +fn wire__crate__api__configure_cache_dir_impl(
854 + ptr_: flutter_rust_bridge::for_generated::PlatformGeneralizedUint8ListPtr,
855 + rust_vec_len_: i32,
856 + data_len_: i32,
857 +) -> flutter_rust_bridge::for_generated::WireSyncRust2DartSse {
858 + FLUTTER_RUST_BRIDGE_HANDLER.wrap_sync::<flutter_rust_bridge::for_generated::SseCodec, _>(
859 + flutter_rust_bridge::for_generated::TaskInfo {
860 + debug_name: "configure_cache_dir",
861 + port: None,
862 + mode: flutter_rust_bridge::for_generated::FfiCallMode::Sync,
863 + },
864 + move || {
865 + let message = unsafe {
866 + flutter_rust_bridge::for_generated::Dart2RustMessageSse::from_wire(
867 + ptr_,
868 + rust_vec_len_,
869 + data_len_,
870 + )
871 + };
872 + let mut deserializer =
873 + flutter_rust_bridge::for_generated::SseDeserializer::new(message);
874 + let api_app_data_dir = <String>::sse_decode(&mut deserializer);
875 + deserializer.end();
876 + transform_result_sse::<_, crate::api::OndeError>((move || {
877 + let output_ok = crate::api::configure_cache_dir(api_app_data_dir)?;
878 + Ok(output_ok)
879 + })())
880 + },
881 + )
882 +}
883 fn wire__crate__api__default_model_config_impl(
884 ptr_: flutter_rust_bridge::for_generated::PlatformGeneralizedUint8ListPtr,
885 rust_vec_len_: i32,
@@ -1493,14 +1523,15 @@ fn pde_ffi_dispatcher_sync_impl(
1523 // Codec=Pde (Serialization + dispatch), see doc to use other codecs
1524 match func_id {
1525 8 => wire__crate__api__OndeChatEngine_new_impl(ptr, rust_vec_len, data_len),
1496 - 15 => wire__crate__api__default_model_config_impl(ptr, rust_vec_len, data_len),
1497 - 16 => wire__crate__api__default_sampling_config_impl(ptr, rust_vec_len, data_len),
1498 - 17 => wire__crate__api__deterministic_sampling_config_impl(ptr, rust_vec_len, data_len),
1499 - 18 => wire__crate__api__mobile_sampling_config_impl(ptr, rust_vec_len, data_len),
1500 - 19 => wire__crate__api__qwen25_1_5b_config_impl(ptr, rust_vec_len, data_len),
1501 - 20 => wire__crate__api__qwen25_3b_config_impl(ptr, rust_vec_len, data_len),
1502 - 21 => wire__crate__api__qwen25_coder_1_5b_config_impl(ptr, rust_vec_len, data_len),
1503 - 22 => wire__crate__api__qwen25_coder_3b_config_impl(ptr, rust_vec_len, data_len),
1526 + 15 => wire__crate__api__configure_cache_dir_impl(ptr, rust_vec_len, data_len),
1527 + 16 => wire__crate__api__default_model_config_impl(ptr, rust_vec_len, data_len),
1528 + 17 => wire__crate__api__default_sampling_config_impl(ptr, rust_vec_len, data_len),
1529 + 18 => wire__crate__api__deterministic_sampling_config_impl(ptr, rust_vec_len, data_len),
1530 + 19 => wire__crate__api__mobile_sampling_config_impl(ptr, rust_vec_len, data_len),
1531 + 20 => wire__crate__api__qwen25_1_5b_config_impl(ptr, rust_vec_len, data_len),
1532 + 21 => wire__crate__api__qwen25_3b_config_impl(ptr, rust_vec_len, data_len),
1533 + 22 => wire__crate__api__qwen25_coder_1_5b_config_impl(ptr, rust_vec_len, data_len),
1534 + 23 => wire__crate__api__qwen25_coder_3b_config_impl(ptr, rust_vec_len, data_len),
1535 _ => unreachable!(),
1536 }
1537 }
src/hf_cache.rs
+2 -2
@@ -907,7 +907,7 @@ where
907 // It is harmless on non-sandboxed desktop builds (the cache simply
908 // lives under the app data dir instead of ~/.cache).
909 if let Some(ref data_dir) = app_data_dir {
910 - let hf_home = data_dir.join("huggingface");
910 + let hf_home = data_dir.join("models");
911 let hf_hub_cache = hf_home.join("hub");
912 fs::create_dir_all(&hf_hub_cache)
913 .map_err(|e| format!("Cannot create HF cache dir: {e}"))?;
@@ -1084,7 +1084,7 @@ where
1084 let resolved_app_data = app_data_dir
1085 .ok_or_else(|| "app_data_dir is required on Android for HF cache resolution".to_string())?;
1086
1087 - let hf_home = resolved_app_data.join("huggingface");
1087 + let hf_home = resolved_app_data.join("models");
1088 let hf_hub_cache = hf_home.join("hub");
1089
1090 fs::create_dir_all(&hf_hub_cache)
src/inference/engine.rs
+14 -10
@@ -237,23 +237,27 @@ impl ChatEngine {
237 config.model_id, config.files
238 );
239
240 - // ── Android: seed GLOBAL_HF_CACHE ────────────────────────────────
240 + // ── Sandboxed platforms: seed GLOBAL_HF_CACHE ────────────────────
241 //
242 - // On Android, `dirs::home_dir()` returns `None`, so
243 - // `hf_hub::Cache::default()` panics with "Cache directory cannot be
244 - // found". The `get_paths_gguf!` macro in mistralrs falls back to
242 + // On sandboxed platforms the default `~/.cache/huggingface/hub` path
243 + // is either inaccessible or non-existent:
244 + // - Android: `dirs::home_dir()` returns `None` → `Cache::default()` panics.
245 + // - iOS/tvOS: `~/.cache` is outside the app container → os error 1.
246 + //
247 + // The `get_paths_gguf!` macro in mistralrs falls back to
248 // `Cache::default()` when `GLOBAL_HF_CACHE` (a OnceLock) is empty.
249 //
247 - // `HF_HOME` is always set by `setup_hf_home()` before any command
248 - // runs, so we read it here. `get_or_init` is a no-op if already
249 - // seeded by `download_model` — safe to call repeatedly.
250 - #[cfg(target_os = "android")]
250 + // `HF_HOME` must be set by the host app (via `configure_cache_dir`
251 + // or `download_model(app_data_dir:)`) before any model load.
252 + // `get_or_init` is a no-op if already seeded — safe to call repeatedly.
253 + #[cfg(any(target_os = "android", target_os = "ios", target_os = "tvos"))]
254 {
255 let hf_home = std::env::var("HF_HOME")
256 .map(std::path::PathBuf::from)
257 .map_err(|_| InferenceError::ModelBuild {
255 - reason: "HF_HOME is not set — cannot initialise HF cache on Android. \
256 - Ensure setup_hf_home() ran before load_gguf_model()."
258 + reason: "HF_HOME is not set — cannot initialise HF cache. \
259 + On iOS/tvOS/Android, call configure_cache_dir() or \
260 + download_model(app_data_dir:) before load_gguf_model()."
261 .to_string(),
262 })?;
263 let hf_hub_cache = hf_home.join("hub");
src/lib.rs
+5 -3
@@ -59,8 +59,10 @@ uniffi::setup_scaffolding!();
59 ))]
60 pub use mistralrs;
61
62 -// Android-only re-exports needed for the GLOBAL_HF_CACHE workaround.
63 -#[cfg(target_os = "android")]
62 +// Re-exports needed for the GLOBAL_HF_CACHE workaround on sandboxed platforms.
63 +// On iOS/tvOS `~/.cache` is outside the container; on Android `dirs::home_dir()`
64 +// panics. All three need `hf_hub::Cache` + `mistralrs_core::GLOBAL_HF_CACHE`.
65 +#[cfg(any(target_os = "android", target_os = "ios", target_os = "tvos"))]
66 pub use hf_hub;
65 -#[cfg(target_os = "android")]
67 +#[cfg(any(target_os = "android", target_os = "ios", target_os = "tvos"))]
68 pub use mistralrs_core;