Rewrite docs for 1.0: clarify platform support, registry links, and SDKs
keypair34 committed
Apr 26, 2026 at 22:13 UTC
88eae921bca78c77deb24ea997d59fe94964d2fe
10 files changed
+273
-180
.agents/AGENTS.md
+131
-56
@@ -7,11 +7,14 @@
7
8
## What Is Onde?
9
10
-Onde is an **on-device LLM inference SDK** targeting Apple silicon (iOS, tvOS, macOS) plus Android, Windows, and Linux. It wraps [mistral.rs](https://github.com/EricLBuehler/mistral.rs) behind a clean, platform-aware API — automatic model selection, HuggingFace Hub downloads, cache management, and GPU acceleration via Metal.
10
+Onde is an **on-device LLM inference SDK family** built around a shared Rust core. It targets Apple silicon (iOS, tvOS, macOS) plus Android, Windows, and Linux, and ships SDKs for Rust, Swift, Kotlin Multiplatform, Flutter, and React Native. It wraps [mistral.rs](https://github.com/EricLBuehler/mistral.rs) behind platform-friendly APIs with automatic model selection, HuggingFace Hub downloads, cache management, and Metal acceleration where available.
11
12
- **Website:** https://ondeinference.com
13
- **Rust crate:** https://crates.io/crates/onde
14
- **Swift package:** https://github.com/ondeinference/onde-swift
15
+- **Maven Central:** https://central.sonatype.com/artifact/com.ondeinference/onde-inference
16
+- **Flutter package:** https://pub.dev/packages/onde_inference
17
+- **React Native package:** https://www.npmjs.com/package/@ondeinference/react-native
18
- **In production:** [Splitfire AB apps on the Apple AppStore](https://apps.apple.com/se/developer/splitfire-ab/id1831430993)
19
20
---
@@ -25,21 +28,26 @@ onde/
28
│ ├── hf_cache.rs # HuggingFace Hub cache: list, download, delete, diagnose, repair
29
│ └── inference/
30
│ ├── mod.rs # Module exports + top-level re-exports
28
-│ ├── engine.rs # ChatEngine — Rust-native API (generics, mpsc)
31
+│ ├── engine.rs # ChatEngine — Rust-native API (generics, mpsc, tool calling)
32
│ ├── ffi.rs # OndeChatEngine — UniFFI Object (FFI-safe, Arc-wrapped)
33
│ ├── models.rs # Model ID constants + SupportedModelInfo metadata
34
│ ├── token.rs # HF token resolution: build-time literal vs cache file
32
-│ └── types.rs # All shared types: ChatMessage, SamplingConfig, InferenceResult, etc.
35
+│ └── types.rs # Shared types: ChatMessage, SamplingConfig, InferenceResult, ToolCallInfo, etc.
36
├── sdk/
34
-│ ├── Onde/ # Generated Swift package output (git-ignored; built by CI)
37
+│ ├── dart/ # Flutter/Dart package + example app + FRB Rust bridge
38
│ ├── gem/ # Ruby native extension (Magnus)
36
-│ ├── kotlin/ # Kotlin/Android bindings
39
+│ ├── kotlin/ # Kotlin Multiplatform package (Android + JVM)
40
+│ ├── react-native/ # Expo module wrapping the Rust core for iOS/Android
41
│ └── python/ # Python bindings (maturin + uniffi)
42
├── generated/ # UniFFI-generated headers and Swift glue (git-ignored)
43
├── uniffi-bindgen/ # Standalone bindgen binary crate (pinned uniffi =0.31.0)
44
├── .github/
45
│ ├── workflows/
42
-│ │ └── release-sdk-swift.yml # CI: tag push → build XCFramework → GitHub Release → update onde-swift
46
+│ │ ├── release-sdk-swift.yml # CI: tag push → build XCFramework → GitHub Release → update onde-swift
47
+│ │ ├── release-sdk-kotlin.yml # CI: tag push → build Android/JVM artifacts → publish to Maven Central
48
+│ │ ├── release-sdk-dart.yml # CI: tag push → publish Flutter package to pub.dev
49
+│ │ ├── release-sdk-npm.yml # CI: tag push → publish React Native package to npm
50
+│ │ └── release-sdk-rust.yml # CI: tag push → publish Rust crate to crates.io
51
│ └── scripts/
52
│ └── build-swift-xcframework.sh # Local/CI XCFramework assembly script
53
├── .cargo/config.toml # Target-specific rustflags (fp16, linker overrides)
@@ -103,18 +111,19 @@ onde-swift/ # Swift Package Manager wrapper repo
111
112
## Platform Support
113
106
-| Platform | `target_os` | GPU Backend | Default Model | mistralrs features |
107
-| -------- | ----------- | ----------------- | --------------------- | -------------------|
108
-| macOS | `macos` | Metal | Qwen 2.5 3B (~1.93 GB) | `["metal"]` |
109
-| iOS | `ios` | Metal | Qwen 2.5 1.5B (~941 MB)| `["metal"]` |
110
-| tvOS | `tvos` | Metal | Qwen 2.5 1.5B (~941 MB)| `["metal"]` |
111
-| Android | `android` | CPU (candle) | Qwen 2.5 1.5B (~941 MB)| `[]` + hf-hub |
112
-| Windows | `windows` | CPU (CUDA in CI) | Qwen 2.5 3B (~1.93 GB) | `[]` |
113
-| Linux | `linux` | CPU (CUDA in CI) | Qwen 2.5 3B (~1.93 GB) | `[]` |
114
-
115
-- **iOS / tvOS memory constraint:** iOS gives apps ~2–3 GB; the 3B model (~1.93 GB) causes OOM on iPhone 16e (8 GB RAM). Always default to the 1.5B on iOS/tvOS/Android.
114
+| Platform | `target_os` | GPU Backend | Default Model | mistralrs features |
115
+| -------- | ----------- | ----------------- | ------------- | ------------------ |
116
+| macOS | `macos` | Metal | Qwen 2.5 Coder 3B (~1.93 GB) | `["metal"]` |
117
+| iOS | `ios` | Metal | Qwen 2.5 Coder 1.5B (~941 MB) | `["metal"]` |
118
+| tvOS | `tvos` | Metal | Qwen 2.5 Coder 1.5B (~941 MB) | `["metal"]` |
119
+| Android | `android` | CPU (candle) | Qwen 2.5 Coder 1.5B (~941 MB) | `[]` + hf-hub |
120
+| Windows | `windows` | CPU (CUDA in CI) | Qwen 2.5 Coder 3B (~1.93 GB) | `[]` |
121
+| Linux | `linux` | CPU (CUDA in CI) | Qwen 2.5 Coder 3B (~1.93 GB) | `[]` |
122
+
123
+- **Current default routing:** `platform_default()` now prefers the Coder variants, 1.5B on mobile and 3B on desktop.
124
+- **iOS / tvOS memory constraint:** iOS gives apps about 2–3 GB. The 3B model (~1.93 GB) can still cause OOM on constrained devices, so mobile defaults stay on the 1.5B model.
125
- **tvOS tier-3 target:** requires `cargo +nightly -Z build-std`. Stable toolchain cannot build tvOS targets.
117
-- **Android `hf_hub`:** `dirs::home_dir()` panics in the Android sandbox. `hf-hub` is added as an explicit dep on Android so `HF_HOME` can be seeded programmatically via `hf_hub::api::tokio::ApiBuilder`.
126
+- **Android `hf_hub`:** `dirs::home_dir()` panics in the Android sandbox. `hf-hub` is added as an explicit dependency on Android so `HF_HOME` can be seeded programmatically via `hf_hub::api::tokio::ApiBuilder`.
127
128
---
129
@@ -122,21 +131,28 @@ onde-swift/ # Swift Package Manager wrapper repo
131
132
All model constants live in `src/inference/models.rs`. When adding a new model:
133
125
-1. Add `pub const` entries for the HF repo ID, GGUF filename, and (on Android) `TOK_MODEL_ID`.
134
+1. Add `pub const` entries for the HF repo ID, GGUF filename, and, on Android, `TOK_MODEL_ID` when needed.
135
2. Add the repo ID to `SUPPORTED_MODELS` so `list_local_hf_models` filters it.
127
-3. Add a `SupportedModelInfo` entry to `SUPPORTED_MODEL_INFO` with accurate `expected_size_bytes` (from HF API `siblings[].size`).
128
-4. Add a constructor to `GgufModelConfig` in `engine.rs` (see `qwen25_1_5b`, `qwen25_3b` for pattern).
129
-5. Export a free function in `ffi.rs` (see `qwen25_1_5b_config`, `qwen25_3b_config`).
136
+3. Add a `SupportedModelInfo` entry to `SUPPORTED_MODEL_INFO` with accurate `expected_size_bytes` from the HF API `siblings[].size`.
137
+4. Add a constructor to `GgufModelConfig` in `engine.rs`.
138
+5. Export a free function in `ffi.rs` for UniFFI consumers when the model should be reachable from Swift and Kotlin.
139
+6. If the GGUF does not ship with a built-in chat template, set `chat_template` explicitly. DeepSeek Coder is the current example.
140
141
### Current Models
142
143
| Model | Repo | File | Size | Platforms |
144
|-------|------|------|------|-----------|
135
-| Qwen 2.5 1.5B (GGUF Q4_K_M) | `bartowski/Qwen2.5-1.5B-Instruct-GGUF` | `Qwen2.5-1.5B-Instruct-Q4_K_M.gguf` | ~941 MB | iOS, tvOS, Android (default) |
136
-| Qwen 2.5 3B (GGUF Q4_K_M) | `bartowski/Qwen2.5-3B-Instruct-GGUF` | `Qwen2.5-3B-Instruct-Q4_K_M.gguf` | ~1.93 GB | macOS, Windows, Linux (default) |
137
-| Qwen 2.5 Coder 1.5B (GGUF) | `bartowski/Qwen2.5-Coder-1.5B-Instruct-GGUF` | `Qwen2.5-Coder-1.5B-Instruct-Q4_K_M.gguf` | ~941 MB | All platforms |
138
-| Qwen 2.5 Coder 3B (GGUF) | `bartowski/Qwen2.5-Coder-3B-Instruct-GGUF` | `Qwen2.5-Coder-3B-Instruct-Q4_K_M.gguf` | ~1.93 GB | macOS |
139
-| Qwen 2.5 Coder 7B (ISQ) | `Qwen/Qwen2.5-Coder-7B-Instruct` | safetensors (ISQ in-situ) | ~8 GB | macOS (ISQ pipeline) |
145
+| Qwen 2.5 1.5B Instruct (GGUF Q4_K_M) | `bartowski/Qwen2.5-1.5B-Instruct-GGUF` | `Qwen2.5-1.5B-Instruct-Q4_K_M.gguf` | ~941 MB | All platforms |
146
+| Qwen 2.5 3B Instruct (GGUF Q4_K_M) | `bartowski/Qwen2.5-3B-Instruct-GGUF` | `Qwen2.5-3B-Instruct-Q4_K_M.gguf` | ~1.93 GB | All platforms |
147
+| Qwen 2.5 Coder 1.5B Instruct (GGUF Q4_K_M) | `bartowski/Qwen2.5-Coder-1.5B-Instruct-GGUF` | `Qwen2.5-Coder-1.5B-Instruct-Q4_K_M.gguf` | ~941 MB | All platforms (mobile default) |
148
+| Qwen 2.5 Coder 3B Instruct (GGUF Q4_K_M) | `bartowski/Qwen2.5-Coder-3B-Instruct-GGUF` | `Qwen2.5-Coder-3B-Instruct-Q4_K_M.gguf` | ~1.93 GB | All platforms (desktop default) |
149
+| Qwen 2.5 Coder 7B Instruct (GGUF Q4_K_M) | `bartowski/Qwen2.5-Coder-7B-Instruct-GGUF` | `Qwen2.5-Coder-7B-Instruct-Q4_K_M.gguf` | ~4.4 GB | Higher-memory devices |
150
+| Qwen 3 1.7B (GGUF Q4_K_M) | `bartowski/Qwen3-1.7B-GGUF` | `Qwen3-1.7B-Q4_K_M.gguf` | ~1.3 GB | All platforms |
151
+| Qwen 3 4B (GGUF Q4_K_M) | `bartowski/Qwen3-4B-GGUF` | `Qwen3-4B-Q4_K_M.gguf` | ~2.7 GB | All platforms |
152
+| Qwen 3 8B (GGUF Q4_K_M) | `bartowski/Qwen3-8B-GGUF` | `Qwen3-8B-Q4_K_M.gguf` | ~5 GB | Higher-memory devices |
153
+| Qwen 3 14B (GGUF Q4_K_M) | `bartowski/Qwen3-14B-GGUF` | `Qwen3-14B-Q4_K_M.gguf` | ~8.4 GB | Higher-memory devices |
154
+| DeepSeek Coder 6.7B Instruct (GGUF Q4_K_M) | `bartowski/deepseek-coder-6.7b-instruct-GGUF` | `deepseek-coder-6.7b-instruct-Q4_K_M.gguf` | ~3.8 GB | Higher-memory devices, custom chat template |
155
+| Qwen 2.5 Coder 7B Instruct (ISQ) | `Qwen/Qwen2.5-Coder-7B-Instruct` | safetensors (ISQ in-situ) | ~8 GB | macOS (ISQ pipeline) |
156
157
---
158
@@ -147,9 +163,10 @@ All model constants live in `src/inference/models.rs`. When adding a new model:
163
| `ChatRole` | `enum ChatRole` | `.system`, `.user`, `.assistant` |
164
| `ChatMessage` | `struct ChatMessage` | `role: ChatRole`, `content: String` |
165
| `SamplingConfig` | `struct SamplingConfig` | All fields `Optional` |
150
-| `GgufModelConfig` | `struct GgufModelConfig` | `modelId`, `files`, `tokModelId?`, `displayName`, `approxMemory` |
166
+| `GgufModelConfig` | `struct GgufModelConfig` | `modelId`, `files`, `tokModelId?`, `displayName`, `approxMemory`, `chatTemplate?` |
167
| `IsqModelConfig` | `struct IsqModelConfig` | `modelId`, `isqBits: UInt8`, `displayName`, `approxMemory` |
152
-| `InferenceResult` | `struct InferenceResult` | `text`, `durationSecs`, `durationDisplay`, `finishReason` |
168
+| `InferenceResult` | `struct InferenceResult` | `text`, `durationSecs`, `durationDisplay`, `finishReason`, `toolCalls` |
169
+| `ToolCallInfo` | `struct ToolCallInfo` | Structured tool call request emitted by the model |
170
| `StreamChunk` | `struct StreamChunk` | `delta`, `done`, `finishReason?` |
171
| `EngineStatus` | `enum EngineStatus` | `.unloaded`, `.loading`, `.ready`, `.generating`, `.error` |
172
| `EngineInfo` | `struct EngineInfo` | `status`, `modelName?`, `approxMemory?`, `historyLength: UInt64` |
@@ -304,15 +321,24 @@ import Onde
321
// Create engine
322
let engine = OndeChatEngine()
323
307
-// Load model (platform-aware default)
324
+// Load the platform default model
325
let elapsed = try await engine.loadDefaultModel(
326
systemPrompt: "You are a helpful assistant.",
327
sampling: nil
328
)
329
330
+// Or load the model assigned to your app in the Onde dashboard
331
+let assignedElapsed = try await engine.loadAssignedModel(
332
+ appId: "your-app-id",
333
+ appSecret: "your-app-secret",
334
+ systemPrompt: "You are a helpful assistant.",
335
+ sampling: nil
336
+)
337
+
338
// Multi-turn chat
339
let result = try await engine.sendMessage(message: "Hello!")
340
print(result.text)
341
+print(result.toolCalls)
342
343
// Streaming (free function — callback_interface constraint in UniFFI 0.31)
344
class Handler: StreamChunkListener {
@@ -345,9 +371,9 @@ await engine.unloadModel()
371
372
| Function | Returns | Notes |
373
|----------|---------|-------|
348
-| `defaultModelConfig()` | `GgufModelConfig` | Platform-aware (1.5B on iOS/tvOS, 3B on macOS) |
349
-| `qwen251_5bConfig()` | `GgufModelConfig` | Forces 1.5B regardless of platform |
350
-| `qwen253bConfig()` | `GgufModelConfig` | Forces 3B regardless of platform |
374
+| `defaultModelConfig()` | `GgufModelConfig` | Platform-aware Coder default (1.5B on iOS/tvOS/Android, 3B on desktop) |
375
+| `qwen251_5bConfig()` | `GgufModelConfig` | Forces Qwen 2.5 1.5B regardless of platform |
376
+| `qwen253bConfig()` | `GgufModelConfig` | Forces Qwen 2.5 3B regardless of platform |
377
| `defaultSamplingConfig()` | `SamplingConfig` | temp=0.7, top_p=0.95, max_tokens=512 |
378
| `deterministicSamplingConfig()` | `SamplingConfig` | temp=0.0, greedy |
379
| `mobileSamplingConfig()` | `SamplingConfig` | temp=0.7, max_tokens=128 |
@@ -476,39 +502,86 @@ cargo check
502
503
### `release-sdk-swift.yml`
504
479
-Triggered on **semver tag push** (e.g. `git tag 0.2.0 && git push origin 0.2.0`) or `workflow_dispatch`. Runs on `macos-15`.
505
+Triggered on **semver tag push** or `workflow_dispatch`. Runs on `macos-15`.
506
507
Steps:
482
-1. Install stable Rust (`1.92.0`) with iOS/macOS targets.
483
-2. Install nightly Rust with `rust-src` component (tvOS uses `-Z build-std`, no `rustup target add` needed).
508
+1. Install stable Rust with iOS/macOS targets.
509
+2. Install nightly Rust with `rust-src` for tvOS `-Z build-std`.
510
3. Run `build-swift-xcframework.sh`.
511
4. Read `version.txt` and `OndeFramework.checksum.txt` into step outputs.
486
-5. Validate tag matches `Cargo.toml` version (fails fast on mismatch).
512
+5. Validate that the tag matches `Cargo.toml`.
513
6. Upload CI artifacts: zip, checksum, version, generated `onde.swift`.
488
-7. On tag push: create a GitHub Release with the zip and checksum attached.
489
-8. On tag push: check out `ondeinference/onde-swift` using `ONDE_SWIFT_PAT`.
490
-9. On tag push: rewrite the `.binaryTarget` in `onde-swift/Package.swift` with the release download URL and computed checksum.
491
-10. On tag push: copy the freshly generated `onde.swift` into `onde-swift/Sources/Onde/`.
492
-11. On tag push: commit both files, tag the commit with the version, and push to `ondeinference/onde-swift`.
514
+7. On tag push, create a GitHub Release with the zip and checksum attached.
515
+8. On tag push, check out `ondeinference/onde-swift` using `ONDE_SWIFT_PAT`.
516
+9. On tag push, rewrite the `.binaryTarget` in `onde-swift/Package.swift` with the release URL and checksum.
517
+10. On tag push, copy the generated `onde.swift` into `onde-swift/Sources/Onde/`.
518
+11. On tag push, commit both files, tag the commit with the version, and push to `ondeinference/onde-swift`.
519
520
#### Required secret
521
496
-`ONDE_SWIFT_PAT` — a GitHub Personal Access Token with **`contents: write`** scope on the `ondeinference/onde-swift` repository. Create it at <https://github.com/settings/tokens> and add it as a repository secret on the `onde` repo.
522
+`ONDE_SWIFT_PAT` — a GitHub Personal Access Token with **`contents: write`** scope on the `ondeinference/onde-swift` repository.
523
498
-### Release Process (end-to-end)
524
+### `release-sdk-kotlin.yml`
525
+
526
+Triggered on **semver tag push** or `workflow_dispatch`. Runs on `macos-15` because it builds both Android artifacts and the macOS Apple Silicon JVM native library.
527
+
528
+Steps:
529
+1. Install stable Rust and the `aarch64-apple-darwin` target.
530
+2. Install Android SDK / NDK and auto-discover the latest installed 29.x NDK for `cargo-ndk`.
531
+3. Build Android JNI libraries for all supported ABIs.
532
+4. Build the macOS Apple Silicon `libonde.dylib` used by the JVM target.
533
+5. Run Gradle publication tasks for the Kotlin Multiplatform package.
534
+6. Validate that the tag matches `sdk/kotlin/gradle.properties` `VERSION_NAME`.
535
+7. On tag push, publish `com.ondeinference:onde-inference` to Maven Central.
536
+
537
+#### Required secrets
538
+
539
+- `ORG_GRADLE_PROJECT_MAVENCENTRALUSERNAME`
540
+- `ORG_GRADLE_PROJECT_MAVENCENTRALPASSWORD`
541
+- `ORG_GRADLE_PROJECT_SIGNINGKEYID`
542
+- `ORG_GRADLE_PROJECT_SIGNINGKEY`
543
+- `ORG_GRADLE_PROJECT_SIGNINGPASSWORD`
544
500
-1. Bump version in `onde/Cargo.toml`.
501
-2. Commit, tag, push:
502
- ```
503
- git add Cargo.toml
504
- git commit -m "0.2.0"
505
- git tag 0.2.0 && git push origin main 0.2.0
506
- ```
507
-3. CI builds the XCFramework, creates a GitHub Release, attaches assets, then **automatically** rewrites `onde-swift/Package.swift` with the correct `url:` + `checksum:`, commits, tags, and pushes `onde-swift`.
508
-4. Swift Package Index picks up the new `onde-swift` tag automatically — no manual step needed.
509
-5. `cargo publish` to crates.io.
545
+### `release-sdk-dart.yml`
546
+
547
+Triggered on **semver tag push** or `workflow_dispatch`.
548
+
549
+- Publishes `onde_inference` to pub.dev.
550
+- Version must match the Dart package version.
551
+- The example app is part of release validation.
552
+
553
+### `release-sdk-npm.yml`
554
+
555
+Triggered on **semver tag push** or `workflow_dispatch`.
556
+
557
+- Publishes `@ondeinference/react-native` to npm.
558
+- Builds the Expo module package and validates the published version.
559
+
560
+### `release-sdk-rust.yml`
561
+
562
+Triggered on **semver tag push** or `workflow_dispatch`.
563
+
564
+- Publishes `onde` to crates.io.
565
+- Uses registry dependencies, not git refs, because `cargo publish` strips git patches.
566
+
567
+### Release Process (end-to-end)
568
511
-> **Note:** Steps 3–4 are fully automated. Manual edits to `onde-swift/Package.swift` are never needed. Only touch it when doing local development (use the `path:` form documented in the file header).
569
+1. Bump versions in every affected package manifest:
570
+ - `Cargo.toml`
571
+ - `sdk/kotlin/gradle.properties`
572
+ - `sdk/dart/pubspec.yaml`
573
+ - `sdk/react-native/package.json`
574
+2. Merge release branches with `--no-ff`. Do not fast-forward multi-commit feature branches.
575
+3. Commit, tag, and push the release.
576
+4. CI publishes all five SDKs:
577
+ - crates.io (`onde`)
578
+ - Swift Package Index / GitHub Releases (`onde-swift`)
579
+ - Maven Central (`com.ondeinference:onde-inference`)
580
+ - pub.dev (`onde_inference`)
581
+ - npm (`@ondeinference/react-native`)
582
+5. Verify the registries after publish, especially Maven Central indexing and the generated Swift release assets.
583
+
584
+> **Note:** `onde-swift/Package.swift` is updated automatically by CI. For local development only, switch it to the `path:` form documented in the file header, then switch it back before committing.
585
586
---
587
@@ -516,11 +589,13 @@ Steps:
589
590
| Registry | Name | Import |
591
|----------|------|--------|
519
-| crates.io | `onde` | `onde = "0.x"` |
592
+| crates.io | `onde` | `onde = "1.x"` |
593
| Swift Package Index | `onde-swift` (org: `ondeinference`) | `import Onde` |
594
+| Maven Central | `com.ondeinference:onde-inference` | Gradle `implementation("com.ondeinference:onde-inference:<version>")` |
595
+| pub.dev | `onde_inference` | `import 'package:onde_inference/onde_inference.dart';` |
596
+| npm | `@ondeinference/react-native` | `import { OndeChatEngine } from "@ondeinference/react-native"` |
597
| PyPI | `onde-inference` | `import onde_inference` |
598
| RubyGems | `onde-inference` | `require 'onde'` |
523
-| Maven Central | `com.ondeinference:onde-inference` | Gradle `implementation(...)` |
599
600
---
601
.agents/skills/humanizer/SKILL.md
+15
-1
@@ -195,7 +195,9 @@ Avoiding AI patterns is only half the job. Sterile, voiceless writing is just as
195
196
### Style Patterns
197
198
-#### 13. Em Dash Overuse
198
+#### 13. No Em Dashes
199
+
200
+Em dashes are the single biggest AI tell. Never use them. Replace with commas, parentheses, colons, or just split into two sentences.
201
202
**Before:**
203
> The term is promoted by institutions—not the people themselves—yet this continues—even in documents.
@@ -203,6 +205,18 @@ Avoiding AI patterns is only half the job. Sterile, voiceless writing is just as
205
**After:**
206
> The term is promoted by institutions, not the people themselves, yet this continues in official documents.
207
208
+**Before:**
209
+> The tools are useful—especially for boilerplate—but they have limits.
210
+
211
+**After:**
212
+> The tools are useful (especially for boilerplate), but they have limits.
213
+
214
+**Before:**
215
+> There's one problem—nobody tested it.
216
+
217
+**After:**
218
+> There's one problem: nobody tested it.
219
+
220
#### 14. Boldface Overuse
221
222
**Before:**
CHANGELOG.md
+10
-10
@@ -1,10 +1,10 @@
1
## 1.0.0
2
3
-Onde has been running in production across multiple Splitfire AB apps on the App Store for months. The `0.x` versioning wasn't reflecting reality anymore.
3
+Onde has already been running in real Splitfire AB apps on the App Store for months, so keeping it on `0.x` no longer felt accurate.
4
5
### Assigned model loading (FFI)
6
7
-`load_assigned_model` is now exposed through the UniFFI layer. Swift and Kotlin consumers can call it directly — the method hardcodes `Environment::Production` so SDK consumers don't need to know about the internal environment enum. Pass your `app_id` and `app_secret` from [ondeinference.com](https://ondeinference.com), and the SDK fetches whatever model you've assigned in the dashboard. Falls back to the platform default if nothing's assigned.
7
+`load_assigned_model` is now exposed through the UniFFI layer. Swift and Kotlin consumers can call it directly. The method hardcodes `Environment::Production`, so SDK consumers do not need to know about the internal environment enum. Pass your `app_id` and `app_secret` from [ondeinference.com](https://ondeinference.com), and the SDK will fetch the model assigned in the dashboard. If nothing has been assigned yet, it falls back to the platform default.
8
9
### New models
10
@@ -14,26 +14,26 @@ Onde has been running in production across multiple Splitfire AB apps on the App
14
15
### Type changes
16
17
-* `GgufModelConfig` has a new `chat_template: Option<String>` field for models that need a custom template.
18
-* `InferenceResult` now carries `tool_calls: Vec<ToolCallInfo>`. Empty for most responses. When a model requests a tool call, you get structured data instead of raw markup in `text`.
19
-* New types: `ToolCallInfo`, `ToolDefinition`, `ToolResult`. The last two are Rust-only for now — they'll hit the FFI surface when `send_message_with_tools` is wired up.
17
+* `GgufModelConfig` now has a `chat_template: Option<String>` field for models that need a custom template.
18
+* `InferenceResult` now carries `tool_calls: Vec<ToolCallInfo>`. It will be empty for most responses, but when a model requests a tool call, you now get structured data instead of raw markup in `text`.
19
+* New types: `ToolCallInfo`, `ToolDefinition`, and `ToolResult`. The last two are Rust-only for now. They will reach the FFI surface once `send_message_with_tools` is wired up.
20
21
### Engine
22
23
-* Model weights are dropped outside the write lock during model swaps. Status queries no longer block while the old model's memory is being freed.
24
-* `platform_default()` now returns Coder variants (Qwen 2.5 Coder 1.5B on mobile, Coder 3B on desktop).
23
+* Model weights are now dropped outside the write lock during model swaps, so status queries no longer block while the old model's memory is being freed.
24
+* `platform_default()` now returns Coder variants, Qwen 2.5 Coder 1.5B on mobile and Coder 3B on desktop.
25
26
### Tool calling (Rust-native only)
27
28
-* `send_message_with_tools`, `send_tool_results`, and `stream_tool_results` on `ChatEngine`. These are available to direct Rust consumers (like [siGit](https://github.com/smbcloud/sigit)) but not yet exposed through UniFFI. The types cross the FFI boundary on the output side (`InferenceResult.tool_calls`), just not the input side yet.
28
+* `send_message_with_tools`, `send_tool_results`, and `stream_tool_results` are now available on `ChatEngine`. These are available to direct Rust consumers, like [siGit](https://github.com/smbcloud/sigit), but are not exposed through UniFFI yet. The types already cross the FFI boundary on the output side through `InferenceResult.tool_calls`, just not on the input side yet.
29
30
### Dependencies
31
32
-* Switched from git-based `mistralrs` to published `onde-mistralrs 0.8.2` crates on crates.io. No more `[patch.crates-io]` gymnastics for `cargo publish`.
32
+* Switched from the git-based `mistralrs` dependency to the published `onde-mistralrs 0.8.2` crates on crates.io. That means no more `[patch.crates-io]` gymnastics for `cargo publish`.
33
34
### Cross-platform
35
36
-* Linux and Windows CPU builds work properly now — fixed `TokenSource` for non-Darwin platforms.
36
+* Linux and Windows CPU builds now work properly too, thanks to a `TokenSource` fix for non-Darwin platforms.
37
* Added cross-compile CI for all supported targets.
38
39
## 0.1.8
README.md
+4
-3
@@ -5,11 +5,12 @@
5
<h1 align="center">Onde Inference</h1>
6
7
<p align="center">
8
- <strong>On-device LLM inference — optimized for <a href="https://en.wikipedia.org/wiki/Apple_silicon">Apple silicon</a>.</strong>
8
+ <strong>Run LLMs on-device, with first-class support for <a href="https://en.wikipedia.org/wiki/Apple_silicon">Apple silicon</a>.</strong>
9
</p>
10
11
<p align="center">
12
<a href="https://crates.io/crates/onde"><img src="https://img.shields.io/crates/v/onde?style=flat-square&color=235843&labelColor=17211D&label=crates.io" alt="crates.io"></a>
13
+ <a href="https://central.sonatype.com/artifact/com.ondeinference/onde-inference"><img src="https://img.shields.io/maven-central/v/com.ondeinference/onde-inference?style=flat-square&color=235843&labelColor=17211D&label=maven" alt="Maven Central"></a>
14
<a href="https://swiftpackageindex.com/ondeinference/onde-swift"><img src="https://img.shields.io/badge/Swift%20Package%20Index-onde--swift-235843?style=flat-square&labelColor=17211D" alt="Swift Package Index"></a>
15
<a href="https://pub.dev/packages/onde_inference"><img src="https://img.shields.io/pub/v/onde_inference?style=flat-square&color=235843&labelColor=17211D&label=pub.dev" alt="pub.dev"></a>
16
<a href="https://www.npmjs.com/package/@ondeinference/react-native"><img src="https://img.shields.io/npm/v/@ondeinference/react-native?style=flat-square&color=235843&labelColor=17211D&label=npm" alt="npm"></a>
@@ -18,14 +19,14 @@
19
</p>
20
21
<p align="center">
21
- <a href="https://github.com/ondeinference/onde-swift">Swift SDK</a> · <a href="https://pub.dev/packages/onde_inference">Flutter SDK</a> · <a href="https://www.npmjs.com/package/@ondeinference/react-native">React Native SDK</a> · <a href="https://ondeinference.com">Website</a>
22
+ <a href="https://github.com/ondeinference/onde-swift">Swift SDK</a> · <a href="https://central.sonatype.com/artifact/com.ondeinference/onde-inference">Kotlin Multiplatform SDK</a> · <a href="https://pub.dev/packages/onde_inference">Flutter SDK</a> · <a href="https://www.npmjs.com/package/@ondeinference/react-native">React Native SDK</a> · <a href="https://ondeinference.com">Website</a>
23
</p>
24
25
---
26
27
## In production
28
28
-Onde powers live App Store apps with fully on-device chat — no server, no latency, no data leaving the device.
29
+Onde is already shipping in real App Store apps. Chat runs fully on-device, so there is no server round trip and no user data leaving the device.
30
31
<p align="center">
32
<a href="https://apps.apple.com/se/developer/splitfire-ab/id1831430993" target="_blank">
sdk/dart/CHANGELOG.md
+22
-22
@@ -1,12 +1,12 @@
1
## 1.0.0
2
3
-This is the first stable release. Onde has been running in production across multiple Splitfire AB apps on the App Store for months now — the `0.x` versioning stopped making sense a while ago.
3
+This is the first stable release. Onde has already been running in real App Store apps for months, so keeping it on `0.x` no longer felt right.
4
5
### New: assigned model loading
6
7
-`loadAssignedModel(appId:appSecret:)` fetches whatever model you've assigned to your app in the [ondeinference.com](https://ondeinference.com) dashboard. If you haven't assigned anything yet, it falls back to the platform default. This is the recommended path for production apps — `loadDefaultModel` is still there for prototyping.
7
+`loadAssignedModel(appId:appSecret:)` fetches the model you've assigned to your app in the [ondeinference.com](https://ondeinference.com) dashboard. If nothing has been assigned yet, it falls back to the platform default. This is the path we recommend for production apps. `loadDefaultModel` is still there if you just want to prototype quickly.
8
9
-The example app picks up credentials via `--dart-define=ONDE_APP_ID=...` and `--dart-define=ONDE_APP_SECRET=...`. Without them, it falls back to the default model like before.
9
+The example app reads credentials from `--dart-define=ONDE_APP_ID=...` and `--dart-define=ONDE_APP_SECRET=...`. If you do not pass them, it falls back to the default model like before.
10
11
### New models
12
@@ -16,39 +16,39 @@ The example app picks up credentials via `--dart-define=ONDE_APP_ID=...` and `--
16
17
### Type changes
18
19
-* `GgufModelConfig` has a new optional `chatTemplate` field for models that need a custom chat template (DeepSeek Coder uses this).
20
-* `InferenceResult` now carries a `toolCalls` list (`List<ToolCallInfo>`). Most responses return an empty list — but if the model decides to request a tool call, the structured data is there instead of raw markup in `text`. The `InferenceResultToolsX` extension adds a `hasToolCalls` convenience getter.
19
+* `GgufModelConfig` now has an optional `chatTemplate` field for models that need a custom chat template, like DeepSeek Coder.
20
+* `InferenceResult` now carries a `toolCalls` list (`List<ToolCallInfo>`). Most responses will still return an empty list, but if the model asks for a tool call, you now get structured data instead of raw markup in `text`. The `InferenceResultToolsX` extension also adds a `hasToolCalls` convenience getter.
21
22
### Engine
23
24
-* Old model weights are now dropped outside the lock when loading a new model. Previously the drop happened while holding the write lock, which blocked status queries during the memory release.
24
+* Old model weights are now dropped outside the lock when loading a new model. Before this, the drop happened while the write lock was still held, which meant status queries could stall while memory was being released.
25
26
### Dependencies
27
28
-* Switched from git-based `mistralrs` to the published `onde-mistralrs 0.8.2` crates on crates.io. Builds are faster and `cargo publish` works without `[patch.crates-io]` gymnastics.
28
+* Switched from the git-based `mistralrs` dependency to the published `onde-mistralrs 0.8.2` crates on crates.io. Builds are faster, and `cargo publish` no longer needs `[patch.crates-io]` gymnastics.
29
30
### Cross-platform
31
32
-* Linux and Windows builds with CPU inference now work out of the box (TokenSource fix for non-Darwin platforms).
32
+* Linux and Windows builds with CPU inference now work out of the box, thanks to a `TokenSource` fix for non-Darwin platforms.
33
34
## 0.1.7
35
36
-* **Fix:** Removed `example/android/app/src/main/java/io/flutter/plugins/GeneratedPluginRegistrant.java` from git tracking and added it to `.gitignore`. This file is regenerated by `flutter pub get` on every CI run, causing a dirty working tree that blocked `pub publish`. Added a CI restore step as an additional safety net.
36
+* **Fix:** Removed `example/android/app/src/main/java/io/flutter/plugins/GeneratedPluginRegistrant.java` from git tracking and added it to `.gitignore`. Flutter regenerates this file on every CI run, which kept leaving the working tree dirty and blocked `pub publish`. Added a CI restore step as an extra safety net.
37
38
## 0.1.6
39
40
-* **Fix:** Replaced the composite `LICENSE` file with the canonical MIT license text so pub.dev's `pana` tool correctly recognises the OSI-approved license and awards the full license score.
40
+* **Fix:** Replaced the composite `LICENSE` file with the standard MIT license text so pub.dev's `pana` tool correctly recognises the OSI-approved license and gives the package full license credit.
41
42
## 0.1.5
43
44
-* **Engine:** Added `load_assigned_model()` — fetches the operator-assigned model config from the Onde SDK backend using app credentials (no user JWT required); falls back gracefully to the platform default when no model is assigned yet.
45
-* **Telemetry:** Added GresIQ pulse telemetry client. The engine now reports usage events to the GresIQ dashboard. Configure via `GRESIQ_ENVIRONMENT` and `ONDE_EDGE_ID` env vars before the engine initialises.
46
-* **Build:** GresIQ API credentials (`GRESIQ_API_KEY`, `GRESIQ_API_SECRET`, `GRESIQ_APP_ID`) are now embedded at build time via `dotenvy`. CI can inject secrets via env vars without modifying source.
44
+* **Engine:** Added `load_assigned_model()`. It fetches the model config assigned to your app from the Onde SDK backend using app credentials, with no user JWT required. If no model has been assigned yet, it falls back to the platform default.
45
+* **Telemetry:** Added the GresIQ pulse telemetry client. The engine now reports usage events to the GresIQ dashboard. Configure it with the `GRESIQ_ENVIRONMENT` and `ONDE_EDGE_ID` environment variables before the engine starts.
46
+* **Build:** GresIQ API credentials (`GRESIQ_API_KEY`, `GRESIQ_API_SECRET`, `GRESIQ_APP_ID`) are now embedded at build time through `dotenvy`, so CI can inject secrets through environment variables without changing source files.
47
48
## 0.1.4
49
50
-* Added Qwen 3 4B GGUF model (`bartowski/Qwen_Qwen3-4B-GGUF`) with full OpenAI-compatible tool calling support.
51
-* Added `GgufModelConfig.qwen3_4b()` constructor and registered it in the supported model list.
50
+* Added the Qwen 3 4B GGUF model (`bartowski/Qwen_Qwen3-4B-GGUF`) with full OpenAI-compatible tool calling support.
51
+* Added the `GgufModelConfig.qwen3_4b()` constructor and registered it in the supported model list.
52
53
## 0.1.3
54
@@ -56,23 +56,23 @@ The example app picks up credentials via `--dart-define=ONDE_APP_ID=...` and `--
56
57
## 0.1.2
58
59
-* **Engine:** Switched all platform-specific `mistralrs` and `mistralrs-core` dependencies to the `setoelkahfi/mistral.rs` fork (branch `fix/all-platform-fixes`), picking up cross-platform stability fixes ahead of the upstream merge.
60
-* **License:** Dual-licensed under MIT OR Apache-2.0; added `LICENSE-APACHE` alongside the existing `LICENSE-MIT` for pub.dev compliance.
59
+* **Engine:** Switched all platform-specific `mistralrs` and `mistralrs-core` dependencies to the `setoelkahfi/mistral.rs` fork (branch `fix/all-platform-fixes`) to pick up cross-platform stability fixes before they landed upstream.
60
+* **License:** Moved to dual licensing under MIT OR Apache-2.0, and added `LICENSE-APACHE` alongside the existing `LICENSE-MIT` for pub.dev compliance.
61
* **Dependencies:** Upgraded `freezed_annotation` to `^3.1.0` and `freezed` to `^3.2.5`.
62
* Removed a stale `ignore_for_file` directive from the generated Flutter Rust Bridge glue code.
63
64
## 0.1.1
65
66
-* CI/CD: `release-sdk-dart.yml` publishes `onde_inference` to pub.dev on tag push.
67
-* Copyright headers on all hand-written source files (`engine.dart`, `types.dart`, `dart_test.dart`, iOS and macOS Swift plugin classes).
68
-* Example app README rewritten — branding, platform notes, and an SDK quick reference.
69
-* `android/local.properties` gitignored; local SDK paths no longer pollute diffs.
66
+* CI/CD: `release-sdk-dart.yml` now publishes `onde_inference` to pub.dev on tag push.
67
+* Added copyright headers to all hand-written source files (`engine.dart`, `types.dart`, `dart_test.dart`, and the iOS and macOS Swift plugin classes).
68
+* Rewrote the example app README with updated branding, platform notes, and an SDK quick reference.
69
+* Added `android/local.properties` to `.gitignore`, so local SDK paths no longer show up in diffs.
70
71
## 0.1.0
72
73
* Initial MVP release.
74
* Multi-turn chat inference with Qwen 2.5 1.5B and 3B GGUF Q4_K_M models.
75
-* Streaming token delivery via Dart `Stream<StreamChunk>` — display tokens as they are generated.
75
+* Streaming token delivery via Dart `Stream<StreamChunk>`, so you can display tokens as they are generated.
76
* Metal acceleration on iOS and macOS (Apple silicon and Intel).
77
* CPU inference on Android, Linux, and Windows.
78
* Platform-aware default model selection (1.5B on iOS / Android, 3B on macOS / Linux / Windows).
sdk/dart/README.md
+12
-12
@@ -5,7 +5,7 @@
5
<h1 align="center">Onde Inference</h1>
6
7
<p align="center">
8
- <strong>On-device LLM inference for Flutter & Dart — Metal on iOS and macOS, CPU everywhere else.</strong>
8
+ <strong>Run LLMs on-device from Flutter and Dart. Metal on iOS and macOS, CPU everywhere else.</strong>
9
</p>
10
11
<p align="center">
@@ -23,9 +23,9 @@
23
24
---
25
26
-Run Qwen 2.5 models inside your Flutter app. The model downloads from HuggingFace on first launch, then everything runs locally — no server, no API key, nothing leaves the device. Metal gives you ~15 tok/s on an iPhone 15 Pro; Android and desktop run on CPU, slower but it works.
26
+Run Qwen 2.5 models directly inside your Flutter app. The model downloads from Hugging Face on first launch, then everything runs locally. No server, no API key, and no user data leaving the device. On an iPhone 15 Pro, Metal gets you around 15 tok/s. Android and desktop run on CPU, so they are slower, but they work well for local inference.
27
28
-Multi-turn chat, streaming, one-shot generation, configurable sampling — the full API is one import away.
28
+You get multi-turn chat, streaming, one-shot generation, and configurable sampling in a single package.
29
30
## Platform support
31
@@ -48,11 +48,11 @@ dependencies:
48
onde_inference: ^0.1.0
49
```
50
51
-The inference engine is Rust compiled via [flutter_rust_bridge](https://pub.dev/packages/flutter_rust_bridge). You need a working [Rust toolchain](https://rustup.rs). First build is slow (~5–10 min, compiling the full dep tree); incremental builds are under a minute.
51
+The inference engine is written in Rust and wired into Dart through [flutter_rust_bridge](https://pub.dev/packages/flutter_rust_bridge). You will need a working [Rust toolchain](https://rustup.rs). The first build is usually slow, around 5 to 10 minutes, because it has to compile the full dependency tree. After that, incremental builds are much faster.
52
53
### Initialize
54
55
-Call once at startup, before anything else:
55
+Call this once at startup, before anything else:
56
57
```dart
58
import 'package:onde_inference/onde_inference.dart';
@@ -69,7 +69,7 @@ void main() async {
69
```dart
70
final engine = await OndeChatEngine.create();
71
72
-// Picks the right model for the device:
72
+// Picks the default model for the device:
73
// iOS / Android → Qwen 2.5 1.5B (~941 MB)
74
// macOS / Linux / Windows → Qwen 2.5 3B (~1.93 GB)
75
final elapsed = await engine.loadDefaultModel(
@@ -128,7 +128,7 @@ await engine.pushHistory(ChatMessage.assistant('Hi! How can I help today?'));
128
129
### One-shot generation
130
131
-Runs inference without touching conversation history. Good for prompt enhancement, classification, formatting.
131
+This runs inference without changing conversation history. It is useful for things like prompt enhancement, classification, or formatting.
132
133
```dart
134
final result = await engine.generate(
@@ -212,13 +212,13 @@ try {
212
}
213
```
214
215
-Common causes: calling `sendMessage` before loading a model, no internet on first run (the model needs to download), or out of memory (the 3B model needs ~2 GB free — use 1.5B on constrained devices).
215
+Common causes include calling `sendMessage` before loading a model, having no internet connection on first run while the model still needs to download, or running out of memory. The 3B model needs about 2 GB free, so on tighter devices you will usually want the 1.5B model instead.
216
217
---
218
219
## Sandboxed app setup (iOS / macOS)
220
221
-On iOS and sandboxed macOS, the default HuggingFace cache path is outside the app container. Call `setupCacheDir()` once at startup to point it somewhere accessible:
221
+On iOS and sandboxed macOS, the default Hugging Face cache path sits outside the app container. Call `setupCacheDir()` once at startup so Onde uses a location your app can actually access:
222
223
```dart
224
import 'package:onde_inference/onde_inference.dart';
@@ -239,13 +239,13 @@ void main() async {
239
}
240
```
241
242
-This first tries the App Group shared container (`group.com.ondeinference.apps`) so all Onde-powered apps share downloaded models. Falls back to the app's private directory if the App Group isn't configured.
242
+Onde first tries the App Group shared container (`group.com.ondeinference.apps`) so Onde-powered apps can share downloaded models. If that App Group is not configured, it falls back to the app's private directory.
243
244
---
245
246
## Contributing
247
248
-Source lives at [github.com/ondeinference/onde](https://github.com/ondeinference/onde):
248
+The source lives at [github.com/ondeinference/onde](https://github.com/ondeinference/onde):
249
250
- Rust core: `src/`
251
- Dart bridge crate: `sdk/dart/rust/`
@@ -256,7 +256,7 @@ Open an issue before sending large PRs.
256
257
## License
258
259
-Dual-licensed under **MIT** and **Apache 2.0**. Pick whichever works for you.
259
+Onde is dual-licensed under **MIT** and **Apache 2.0**. You can use either one.
260
261
- [MIT License](https://github.com/ondeinference/onde/blob/main/LICENSE-MIT)
262
- [Apache License 2.0](https://github.com/ondeinference/onde/blob/main/LICENSE-APACHE)
sdk/dart/example/README.md
+11
-11
@@ -5,7 +5,7 @@
5
<h1 align="center">onde_inference — Example App</h1>
6
7
<p align="center">
8
- <strong>A complete Flutter chat app running fully on-device LLM inference.</strong><br>
8
+ <strong>A Flutter chat app that runs LLM inference fully on-device.</strong><br>
9
No server. No API key. No data leaving the device.
10
</p>
11
@@ -17,9 +17,9 @@
17
18
---
19
20
-## What this example demonstrates
20
+## What this example shows
21
22
-This is a production-quality Material 3 chat app built on the `onde_inference` Flutter SDK. It covers every major SDK feature in a single, self-contained file ([`lib/main.dart`](lib/main.dart)):
22
+This example is a full Material 3 chat app built with the `onde_inference` Flutter SDK. It puts the main SDK features in one place, inside a single file, [`lib/main.dart`](lib/main.dart):
23
24
| Feature | Where |
25
|---|---|
@@ -40,10 +40,10 @@ This is a production-quality Material 3 chat app built on the `onde_inference` F
40
### Prerequisites
41
42
- [Flutter SDK](https://docs.flutter.dev/get-started/install) ≥ 3.10
43
-- [Rust toolchain](https://rustup.rs) (stable) — required to compile the native inference engine
43
+- [Rust toolchain](https://rustup.rs) (stable), required to compile the native inference engine
44
- A physical device or simulator/emulator for your target platform
45
46
-> **First build takes 5–10 minutes** to compile the full Rust dependency tree (mistral.rs + candle). Subsequent builds are incremental and fast.
46
+> **The first build usually takes 5 to 10 minutes** because it has to compile the full Rust dependency tree (`mistral.rs` + `candle`). After that, builds are much faster.
47
48
### Steps
49
@@ -58,7 +58,7 @@ flutter pub get
58
flutter run
59
```
60
61
-On **iOS / macOS** the Rust engine is compiled via the CocoaPods podspec and linked automatically by Xcode. On **Android** it is compiled via CMake/Gradle. On **Linux / Windows** it is linked as a shared library.
61
+On **iOS / macOS**, the Rust engine is built through the CocoaPods podspec and linked automatically by Xcode. On **Android**, it is built through CMake and Gradle. On **Linux / Windows**, it is linked as a shared library.
62
63
### Platform notes
64
@@ -69,7 +69,7 @@ On **iOS / macOS** the Rust engine is compiled via the CocoaPods podspec and lin
69
| Android (arm64) | CPU | Qwen 2.5 1.5B (~941 MB) |
70
| Linux / Windows | CPU | Qwen 2.5 3B (~1.93 GB) |
71
72
-> The model is downloaded from HuggingFace Hub on first launch and cached locally — an internet connection is required once.
72
+> The model is downloaded from Hugging Face Hub on first launch and cached locally, so you only need an internet connection once.
73
74
---
75
@@ -114,12 +114,12 @@ example/
114
115
## Learn more
116
117
-- **[onde_inference on pub.dev](https://pub.dev/packages/onde_inference)** — full API reference
118
-- **[Onde Inference docs](https://ondeinference.com)** — guides, model catalogue, platform setup
119
-- **[onde on GitHub](https://github.com/ondeinference/onde)** — Rust engine source, Swift SDK, issue tracker
117
+- **[onde_inference on pub.dev](https://pub.dev/packages/onde_inference)**, for the full API reference
118
+- **[Onde Inference docs](https://ondeinference.com)**, for guides, the model catalog, and platform setup
119
+- **[onde on GitHub](https://github.com/ondeinference/onde)**, for the Rust engine source, Swift SDK, and issue tracker
120
121
---
122
123
<p align="center">
124
- <sub>© 2026 <a href="https://ondeinference.com">Onde Inference</a> — MIT License</sub>
124
+ <sub>© 2026 <a href="https://ondeinference.com">Onde Inference</a>, MIT License</sub>
125
</p>
\ No newline at end of file
sdk/kotlin/README.md
+33
-31
@@ -2,28 +2,30 @@
2
<img src="https://raw.githubusercontent.com/ondeinference/onde/main/assets/onde-inference-logo.svg" alt="Onde Inference" width="96">
3
</p>
4
5
-<h1 align="center">Onde Inference — Kotlin / Android SDK</h1>
5
+<h1 align="center">Onde Inference Kotlin SDK</h1>
6
7
<p align="center">
8
- <strong>On-device LLM inference for Android. Run Qwen 2.5 models locally — no cloud, no API key, no data leaving the device.</strong>
8
+ <strong>Run LLMs on-device from Kotlin. No cloud, no API key, and no user data leaving the device.</strong>
9
</p>
10
11
<p align="center">
12
- <a href="https://central.sonatype.com/artifact/com.ondeinference/onde-inference"><img src="https://img.shields.io/maven-central/v/com.ondeinference/onde-inference" alt="Maven Central"></a>
13
- <a href="https://ondeinference.com"><img src="https://img.shields.io/badge/website-ondeinference.com-blue" alt="Website"></a>
14
- <a href="https://github.com/ondeinference/onde/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-blue" alt="License"></a>
12
+ <a href="https://central.sonatype.com/artifact/com.ondeinference/onde-inference"><img src="https://img.shields.io/maven-central/v/com.ondeinference/onde-inference?style=flat-square&color=235843&labelColor=17211D&label=Maven%20Central" alt="Maven Central"></a>
13
+ <a href="https://ondeinference.com"><img src="https://img.shields.io/badge/ondeinference.com-235843?style=flat-square&labelColor=17211D" alt="Website"></a>
14
+ <a href="https://github.com/ondeinference/onde/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-235843?style=flat-square&labelColor=17211D" alt="License"></a>
15
</p>
16
17
---
18
19
## What is Onde?
20
21
-Onde is an on-device LLM inference SDK for Android. It wraps [mistral.rs](https://github.com/EricLBuehler/mistral.rs) behind a clean Kotlin API — automatic model downloading from HuggingFace Hub, cache management, and CPU inference via the Candle backend.
21
+Onde is an on-device LLM inference SDK for Kotlin apps. It wraps [mistral.rs](https://github.com/EricLBuehler/mistral.rs) in a Kotlin-friendly API, with model downloads from Hugging Face, local cache management, and native inference under the hood.
22
23
-- **On-device** — no cloud, no API key, no data leaving the device
24
-- **Kotlin-first** — suspend functions and `Flow<StreamChunk>` streaming
25
-- **UniFFI-powered** — the Rust engine is bound to Kotlin via [uniffi-rs](https://github.com/mozilla/uniffi-rs)
26
-- **Maven Central** — single Gradle dependency, no manual `.so` management
23
+- **Runs locally**: no cloud, no API key, no user data leaving the device
24
+- **Kotlin-friendly**: suspend functions and `Flow<StreamChunk>` for streaming
25
+- **Rust underneath**: powered by [uniffi-rs](https://github.com/mozilla/uniffi-rs)
26
+- **Published on Maven Central**: add one Gradle dependency and you're done
27
+
28
+This package is built as a Kotlin Multiplatform library. Today that means Android and JVM support, with the same Rust core underneath both.
29
30
---
31
@@ -45,11 +47,11 @@ Add `INTERNET` permission to `AndroidManifest.xml` (required for the initial mod
47
48
**Minimum requirements**
49
- Android 8.0 (API 26) or higher
48
-- ~1.1 GB free storage for the default model (Qwen 2.5 1.5B Q4_K_M)
50
+- About 1.1 GB of free storage for the default model (Qwen 2.5 1.5B Q4_K_M)
51
52
---
53
52
-## Quick start
54
+## Quick start on Android
55
56
```kotlin
57
import com.ondeinference.onde.OndeInference
@@ -64,7 +66,7 @@ class MainActivity : AppCompatActivity() {
66
67
lifecycleScope.launch {
68
// 1. Load the default model (Qwen 2.5 1.5B on Android, ~941 MB)
67
- // Downloads from HuggingFace Hub on first run (~1–5 min on Wi-Fi).
69
+ // It downloads from Hugging Face the first time you run it.
70
val elapsed = onde.loadDefaultModel(
71
systemPrompt = "You are a helpful, concise assistant."
72
)
@@ -90,7 +92,7 @@ class MainActivity : AppCompatActivity() {
92
93
## Streaming
94
93
-Receive tokens as they are generated using `Flow`:
95
+If you want tokens as they arrive, use `Flow`:
96
97
```kotlin
98
lifecycleScope.launch {
@@ -106,7 +108,7 @@ lifecycleScope.launch {
108
109
## Multi-turn conversation
110
109
-The engine maintains conversation history automatically:
111
+Onde keeps conversation history for you:
112
113
```kotlin
114
lifecycleScope.launch {
@@ -131,7 +133,7 @@ lifecycleScope.launch {
133
import com.ondeinference.onde.OndeModels
134
135
lifecycleScope.launch {
134
- // Qwen 2.5 1.5B — default on Android
136
+ // Qwen 2.5 1.5B, the default on Android
137
val elapsed = onde.loadModel(
138
config = OndeModels.qwen25_1_5b(),
139
systemPrompt = "You are a coding assistant."
@@ -147,20 +149,20 @@ lifecycleScope.launch {
149
import com.ondeinference.onde.OndeSampling
150
151
lifecycleScope.launch {
150
- // Default creative chat — temperature 0.7, top_p 0.95, max 512 tokens
152
+ // Default chat settings: temperature 0.7, top_p 0.95, max 512 tokens
153
onde.loadDefaultModel(sampling = OndeSampling.default())
154
153
- // Greedy / deterministic — best for factual Q&A and code generation
155
+ // Deterministic output, useful for factual answers and code
156
onde.loadDefaultModel(sampling = OndeSampling.deterministic())
157
156
- // Mobile-conservative — max 128 tokens, faster on low-end devices
158
+ // Shorter responses, usually better for lower-end devices
159
onde.loadDefaultModel(sampling = OndeSampling.mobile())
160
}
161
```
162
163
---
164
163
-## One-shot generation (no history side-effects)
165
+## One-shot generation (without changing chat history)
166
167
```kotlin
168
import com.ondeinference.onde.OndeMessage
@@ -195,8 +197,8 @@ lifecycleScope.launch {
197
198
## File system layout
199
198
-Onde stores model files in the app's internal `filesDir`. No external storage or
199
-special permissions are required beyond `INTERNET`.
200
+Onde stores model files in the app's internal `filesDir`. You do not need external
201
+storage permissions. `INTERNET` is only needed for the initial download.
202
203
```
204
<filesDir>/
@@ -215,7 +217,7 @@ println(onde.modelCacheDir.absolutePath)
217
218
---
219
218
-## Architecture
220
+## How it fits together
221
222
```
223
┌───────────────────────────────────────┐
@@ -238,7 +240,7 @@ println(onde.modelCacheDir.absolutePath)
240
└───────────────────────────────────────┘
241
```
242
241
-The Rust engine is compiled for four ABIs and bundled as JNI libs:
243
+On Android, the Rust engine is compiled for four ABIs and bundled as JNI libs:
244
245
| ABI | Architecture |
246
|-----|-------------|
@@ -271,7 +273,7 @@ export ANDROID_NDK_HOME=$HOME/Library/Android/sdk/ndk/26.1.10909125
273
./sdk/kotlin/scripts/build-android.sh
274
```
275
274
-This produces `.so` files in `sdk/kotlin/lib/src/main/jniLibs/`.
276
+This produces `.so` files in `sdk/kotlin/lib/src/androidMain/jniLibs/`.
277
278
### Generate Kotlin UniFFI bindings
279
@@ -280,9 +282,9 @@ This produces `.so` files in `sdk/kotlin/lib/src/main/jniLibs/`.
282
```
283
284
This generates `onde.kt` in `sdk/kotlin/lib/src/generated/kotlin/`. The generated file
283
-is gitignored — regenerate it whenever the Rust API changes.
285
+is gitignored, so regenerate it whenever the Rust API changes.
286
285
-### Build the AAR
287
+### Build the Android artifact
288
289
```bash
290
cd sdk/kotlin
@@ -294,7 +296,7 @@ cd sdk/kotlin
296
297
## Publishing to Maven Central
298
297
-Publishing requires:
299
+Publishing is handled by CI. It requires:
300
301
| Secret | Description |
302
|--------|-------------|
@@ -311,13 +313,13 @@ Tag a release to trigger the workflow:
313
git tag 0.1.4 && git push origin 0.1.4
314
```
315
314
-The CI workflow builds all ABIs, generates bindings, packages the AAR, signs it, and
315
-publishes to Maven Central automatically.
316
+The CI workflow builds the native libraries, generates bindings, packages the Android
317
+artifact, signs it, and publishes to Maven Central automatically.
318
319
---
320
321
## License
322
321
-MIT OR Apache-2.0 — see [LICENSE](../../LICENSE).
323
+MIT OR Apache-2.0. See [LICENSE](../../LICENSE).
324
325
Built by [Onde Inference / Splitfire AB](https://ondeinference.com).
sdk/react-native/CHANGELOG.md
+23
-23
@@ -1,12 +1,12 @@
1
## 1.0.0
2
3
-First stable release. Onde has been in production across multiple App Store apps for a while now, so we're dropping the `0.x`.
3
+This is the first stable release. Onde has already been running in real App Store apps for a while, so it felt like time to leave `0.x` behind.
4
5
### Assigned model loading
6
7
-New `loadAssignedModel(appId, appSecret)` method on `OndeChatEngine`. Register your app at [ondeinference.com](https://ondeinference.com), assign a model in the dashboard, and the SDK fetches it at runtime. Falls back to the platform default if nothing's assigned yet. `loadDefaultModel` still works for quick prototyping.
7
+There is a new `loadAssignedModel(appId, appSecret)` method on `OndeChatEngine`. You register your app at [ondeinference.com](https://ondeinference.com), pick a model in the dashboard, and the SDK loads it at runtime. If nothing has been assigned yet, it falls back to the platform default. `loadDefaultModel` still works fine if you just want to get started quickly.
8
9
-The example app has `ONDE_APP_ID` and `ONDE_APP_SECRET` constants at the top — fill them in and it switches to the assigned model path automatically.
9
+The example app includes `ONDE_APP_ID` and `ONDE_APP_SECRET` constants near the top. Fill those in and it will switch to the assigned-model flow automatically.
10
11
### New models
12
@@ -16,50 +16,50 @@ The example app has `ONDE_APP_ID` and `ONDE_APP_SECRET` constants at the top —
16
17
### Type changes
18
19
-* `GgufModelConfig` has a new optional `chatTemplate` field. Models that ship without a built-in template (like DeepSeek Coder) use this.
20
-* `InferenceResult` now includes a `toolCalls` array (`ToolCallInfo[]`). Usually empty — but when a model decides to request a tool call, you get structured data instead of raw markup in `text`.
21
-* Added `ToolCallInfo` type export.
19
+* `GgufModelConfig` now has an optional `chatTemplate` field. This is for models that do not ship with a built-in template, like DeepSeek Coder.
20
+* `InferenceResult` now includes a `toolCalls` array (`ToolCallInfo[]`). It will usually be empty, but when a model asks for a tool call, you now get structured data instead of raw markup in `text`.
21
+* Added a `ToolCallInfo` type export.
22
23
### Engine
24
25
-* Old model weights are dropped outside the lock during model swaps, so status queries don't block while memory is being freed.
25
+* Old model weights are now dropped outside the lock during model swaps, so status queries do not get stuck while memory is being freed.
26
27
### Dependencies
28
29
-* Switched from git-based `mistralrs` to published `onde-mistralrs 0.8.2` crates on crates.io.
29
+* Switched from the git-based `mistralrs` dependency to the published `onde-mistralrs 0.8.2` crates on crates.io.
30
31
### Cross-platform
32
33
-* Linux and Windows CPU inference builds work properly now (TokenSource fix for non-Darwin platforms).
33
+* Linux and Windows CPU inference builds now work properly too, thanks to a `TokenSource` fix for non-Darwin platforms.
34
35
## 0.1.7
36
37
-* **CI:** No user-visible changes. Internal release to keep version numbers in sync across all SDK distributions.
37
+* **CI:** No user-facing changes here. This was an internal release to keep version numbers aligned across the SDKs.
38
39
## 0.1.6
40
41
-* **Fix:** Replaced the composite `LICENSE` file with the canonical MIT license text so registries correctly recognise the OSI-approved license.
41
+* **Fix:** Replaced the composite `LICENSE` file with the standard MIT license text so package registries correctly recognise the OSI-approved license.
42
43
## 0.1.5
44
45
-* **Engine:** Added `loadAssignedModel()` — fetches the operator-assigned model config from the Onde SDK backend using app credentials (no user JWT required); falls back gracefully to the platform default when no model is assigned yet.
46
-* **Telemetry:** Added GresIQ pulse telemetry client. The engine now reports usage events to the GresIQ dashboard. Configure via `GRESIQ_ENVIRONMENT` and `ONDE_EDGE_ID` env vars before the engine initialises.
47
-* **Build:** GresIQ API credentials (`GRESIQ_API_KEY`, `GRESIQ_API_SECRET`, `GRESIQ_APP_ID`) are now embedded at build time via `dotenvy`. CI can inject secrets via env vars without modifying source.
45
+* **Engine:** Added `loadAssignedModel()`. It fetches the model config assigned to your app from the Onde SDK backend using app credentials, with no user JWT required. If no model has been assigned yet, it falls back to the platform default.
46
+* **Telemetry:** Added the GresIQ pulse telemetry client. The engine now reports usage events to the GresIQ dashboard. Configure it with the `GRESIQ_ENVIRONMENT` and `ONDE_EDGE_ID` environment variables before the engine starts.
47
+* **Build:** GresIQ API credentials (`GRESIQ_API_KEY`, `GRESIQ_API_SECRET`, `GRESIQ_APP_ID`) are now embedded at build time through `dotenvy`, so CI can inject secrets through environment variables without changing source files.
48
49
## 0.1.4
50
51
-* Added Qwen 3 4B GGUF model (`bartowski/Qwen_Qwen3-4B-GGUF`) with full OpenAI-compatible tool calling support.
52
-* Added `qwen3_4b()` model config and registered it in the supported model list.
51
+* Added the Qwen 3 4B GGUF model (`bartowski/Qwen_Qwen3-4B-GGUF`) with full OpenAI-compatible tool calling support.
52
+* Added the `qwen3_4b()` model config and registered it in the supported model list.
53
54
## 0.1.3
55
56
-* Initial release of the React Native Expo module.
56
+* First release of the React Native Expo module.
57
* Multi-turn chat inference with Qwen 2.5 1.5B and 3B GGUF models.
58
* Metal acceleration on iOS (Apple silicon).
59
* CPU inference on Android.
60
-* Platform-aware default model selection (1.5B on iOS / Android).
61
-* Conversation history management: `history()`, `clearHistory()`, `pushHistory()`.
62
-* One-shot `generate()` API that does not affect conversation history.
63
-* Configurable sampling: temperature, top-p, top-k, min-p, max tokens, penalties.
64
-* Built-in sampling presets: `defaultSamplingConfig()`, `deterministicSamplingConfig()`, `mobileSamplingConfig()`.
65
-* Engine status via `info()`: status, loaded model name, memory, history length.
60
+* Platform-aware default model selection, with 1.5B on iOS and Android.
61
+* Conversation history management through `history()`, `clearHistory()`, and `pushHistory()`.
62
+* A one-shot `generate()` API that does not change conversation history.
63
+* Configurable sampling, including temperature, top-p, top-k, min-p, max tokens, and penalties.
64
+* Built-in sampling presets: `defaultSamplingConfig()`, `deterministicSamplingConfig()`, and `mobileSamplingConfig()`.
65
+* Engine status through `info()`, including status, loaded model name, memory, and history length.
sdk/react-native/README.md
+12
-11
@@ -5,26 +5,27 @@
5
<h1 align="center">Onde Inference</h1>
6
7
<p align="center">
8
- <strong>On-device LLM inference for React Native — Metal on iOS, CPU on Android.</strong>
8
+ <strong>Run LLMs on-device from React Native. Metal on iOS, CPU on Android, and no user data leaving the device.</strong>
9
</p>
10
11
<p align="center">
12
<a href="https://www.npmjs.com/package/@ondeinference/react-native"><img src="https://img.shields.io/npm/v/@ondeinference/react-native?style=flat-square&color=235843&labelColor=17211D&label=npm" alt="npm"></a>
13
<a href="https://crates.io/crates/onde"><img src="https://img.shields.io/crates/v/onde?style=flat-square&color=235843&labelColor=17211D&label=crates.io" alt="crates.io"></a>
14
+ <a href="https://central.sonatype.com/artifact/com.ondeinference/onde-inference"><img src="https://img.shields.io/maven-central/v/com.ondeinference/onde-inference?style=flat-square&color=235843&labelColor=17211D&label=Maven%20Central" alt="Maven Central"></a>
15
<a href="https://swiftpackageindex.com/ondeinference/onde-swift"><img src="https://img.shields.io/badge/Swift%20Package%20Index-onde--swift-235843?style=flat-square&labelColor=17211D" alt="Swift Package Index"></a>
16
<a href="https://pub.dev/packages/onde_inference"><img src="https://img.shields.io/pub/v/onde_inference?style=flat-square&color=235843&labelColor=17211D&label=pub.dev" alt="pub.dev"></a>
17
<a href="https://ondeinference.com"><img src="https://img.shields.io/badge/ondeinference.com-235843?style=flat-square&labelColor=17211D" alt="Website"></a>
18
</p>
19
20
<p align="center">
20
- <a href="https://github.com/ondeinference/onde">Rust SDK</a> · <a href="https://swiftpackageindex.com/ondeinference/onde-swift">Swift SDK</a> · <a href="https://pub.dev/packages/onde_inference">Flutter SDK</a> · <a href="https://ondeinference.com">Website</a>
21
+ <a href="https://github.com/ondeinference/onde">Rust SDK</a> · <a href="https://swiftpackageindex.com/ondeinference/onde-swift">Swift SDK</a> · <a href="https://central.sonatype.com/artifact/com.ondeinference/onde-inference">Kotlin Multiplatform SDK</a> · <a href="https://pub.dev/packages/onde_inference">Flutter SDK</a> · <a href="https://ondeinference.com">Website</a>
22
</p>
23
24
---
25
25
-Run Qwen 2.5 models on the phone. No server, no API key, nothing leaves the device.
26
+Run Qwen 2.5 models directly on the device. No server, no API key, and no user data leaving the phone.
27
27
-The model downloads from HuggingFace on first load, then runs locally. ~941 MB for the 1.5B variant. Metal gives you ~15 tok/s on an iPhone 15; Android runs on CPU, slower but works.
28
+The model downloads from Hugging Face the first time you load it, then runs locally after that. The 1.5B model is about 941 MB. On iPhone, Metal makes it feel surprisingly fast. On Android, it runs on CPU, so it is slower, but it still works well enough for local chat.
29
30
## Installation
31
@@ -37,7 +38,7 @@ npx expo install @ondeinference/react-native
38
```typescript
39
import { OndeChatEngine, userMessage } from "@ondeinference/react-native";
40
40
-// Picks the right model for the device:
41
+// Picks the default model for the device:
42
// iOS → Qwen 2.5 1.5B (~941 MB, Metal)
43
// Android → Qwen 2.5 1.5B (~941 MB, CPU)
44
const seconds = await OndeChatEngine.loadDefaultModel(
@@ -101,7 +102,7 @@ import {
102
103
## Example app
104
104
-There's a working chat app in [`example/`](./example):
105
+There is a working chat app in [`example/`](./example):
106
107
```bash
108
cd example
@@ -109,7 +110,7 @@ npm install
110
npx expo run:ios
111
```
112
112
-Single file, ~290 lines. Shows loading, chat, status, history management, and error handling.
113
+It is a single-file example, about 290 lines, and it covers loading, chat, status, history management, and error handling.
114
115
## Building from source
116
@@ -125,9 +126,9 @@ rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-and
126
./scripts/build-rust.sh android
127
```
128
128
-The script compiles the Rust FFI bridge in `rust/`, then copies the static lib (iOS) or shared libs (Android) into the right places under `ios/` and `android/`.
129
+The script builds the Rust FFI bridge in `rust/`, then copies the static library for iOS or the shared libraries for Android into the right places under `ios/` and `android/`.
130
130
-## How it works
131
+## How it fits together
132
133
```
134
TypeScript → Expo Module (Swift / Kotlin) → Rust C FFI → onde crate → mistral.rs
@@ -135,11 +136,11 @@ TypeScript → Expo Module (Swift / Kotlin) → Rust C FFI → onde crate
136
JNI external (Android) Metal / CPU
137
```
138
138
-The native module talks to Rust through `extern "C"` functions. Complex types cross the boundary as JSON strings — the TypeScript layer handles camelCase ↔ snake_case conversion. A global `tokio::Runtime` (created once) runs the async inference.
139
+The native module talks to Rust through `extern "C"` functions. Complex types cross the boundary as JSON strings, and the TypeScript layer handles camelCase ↔ snake_case conversion. A global `tokio::Runtime`, created once, runs the async inference work.
140
141
## License
142
142
-Dual-licensed under **MIT** and **Apache 2.0**. Pick whichever works for you.
143
+Onde is dual-licensed under **MIT** and **Apache 2.0**. You can use either one.
144
145
- [MIT License](https://github.com/ondeinference/onde/blob/main/LICENSE-MIT)
146
- [Apache License 2.0](https://github.com/ondeinference/onde/blob/main/LICENSE-APACHE)