Add Kotlin/Android SDK: UniFFI bindings, Gradle build, and example app
- Adds `sdk/kotlin/` with Gradle-based Android library project for Onde Inference - Implements Android idiomatic wrapper (`OndeInference.kt`) around UniFFI-generated bindings - Provides scripts for building Rust `.so` for all ABIs and generating Kotlin bindings - Includes Maven Central publishing setup with Vanniktech plugin and ProGuard rules - Adds Jetpack Compose example app (`sdk/kotlin/example/`) demonstrating streaming chat UI - GitHub Actions workflow for CI build and Maven Central release - Documentation and README for SDK usage, architecture, and troubleshooting
keypair34 committed
Apr 10, 2026 at 13:38 UTC
6d8d8b37eb9156123326dd7c9294ae1abd18e47f
24 files changed
+2847
.agents/skills/sdk-kotlin/SKILL.md
new
+557
@@ -0,0 +1,557 @@
1
+---
2
+name: sdk-kotlin
3
+description: Build, generate, and publish the Onde Inference Kotlin/Android SDK. Covers UniFFI Kotlin binding generation from the Rust onde crate, Android library project structure, cargo-ndk cross-compilation for all Android ABIs, Maven Central publishing via Vanniktech plugin, Android filesystem sandbox setup (HF_HOME / TMPDIR via Os.setenv), ProGuard consumer rules, GitHub Actions CI release workflow, and the Jetpack Compose example app.
4
+allowed-tools: Read, Write, Edit, Glob, Grep, Bash
5
+user-invocable: true
6
+---
7
+
8
+# Skill: Kotlin / Android SDK
9
+
10
+## What This Skill Covers
11
+
12
+Building the `onde-inference` Android AAR from Rust source using UniFFI, distributing it via Maven Central, and maintaining the Kotlin idiomatic wrapper on top of the generated bindings.
13
+
14
+---
15
+
16
+## Repository Layout
17
+
18
+```
19
+onde/
20
+├── src/ # Rust source — UniFFI-annotated types and OndeChatEngine
21
+├── uniffi-bindgen/ # Standalone uniffi-bindgen CLI binary (pinned =0.31.0)
22
+├── sdk/kotlin/
23
+│ ├── settings.gradle.kts # Gradle root — includes :lib and :example
24
+│ ├── build.gradle.kts # root — plugin declarations only (all versions live here)
25
+│ ├── gradle.properties # GROUP, POM_ARTIFACT_ID, VERSION_NAME, signing/OSSRH
26
+│ ├── gradle/wrapper/
27
+│ │ └── gradle-wrapper.properties # pinned Gradle 8.9
28
+│ ├── lib/
29
+│ │ ├── build.gradle.kts # Android library + Vanniktech maven-publish
30
+│ │ ├── consumer-rules.pro # ProGuard: preserve UniFFI JNI bridge symbols
31
+│ │ └── src/
32
+│ │ ├── generated/kotlin/ # UniFFI-generated onde.kt ← GITIGNORED
33
+│ │ └── main/
34
+│ │ ├── AndroidManifest.xml
35
+│ │ ├── jniLibs/ # compiled .so per ABI ← GITIGNORED
36
+│ │ └── kotlin/com/ondeinference/onde/
37
+│ │ └── OndeInference.kt # Android idiomatic wrapper
38
+│ ├── example/
39
+│ │ ├── build.gradle.kts # com.android.application — no plugin versions here
40
+│ │ ├── proguard-rules.pro
41
+│ │ ├── README.md
42
+│ │ └── src/main/
43
+│ │ ├── AndroidManifest.xml # INTERNET permission
44
+│ │ ├── res/values/
45
+│ │ │ ├── strings.xml
46
+│ │ │ └── themes.xml # Material3 NoActionBar activity theme
47
+│ │ └── kotlin/com/ondeinference/example/
48
+│ │ ├── MainActivity.kt # ComponentActivity — enableEdgeToEdge + setContent
49
+│ │ ├── ChatViewModel.kt # model lifecycle, streaming, history
50
+│ │ └── ui/
51
+│ │ ├── ChatScreen.kt # full Compose UI
52
+│ │ └── theme/Theme.kt # dynamic colour + dark mode
53
+│ └── scripts/
54
+│ ├── build-android.sh # Rust .so → jniLibs/ via cargo-ndk
55
+│ └── generate-bindings.sh # uniffi-bindgen → src/generated/kotlin/onde.kt
56
+└── .github/workflows/
57
+ └── release-sdk-kotlin.yml # tag push → build → Maven Central publish
58
+```
59
+
60
+---
61
+
62
+## Key Design Decisions
63
+
64
+| Decision | Rationale |
65
+|---|---|
66
+| UniFFI Kotlin bindings — not hand-written JNI | Same Rust source as Swift/Dart, zero API drift |
67
+| `src/generated/kotlin/` is gitignored | Regenerated from Rust on every release |
68
+| `cargo-ndk` for cross-compilation | Handles NDK toolchain wiring for all 4 ABIs transparently |
69
+| Vanniktech `maven-publish` plugin | Simplest path to Maven Central + signing with one plugin |
70
+| `minSdk 26` | Required for `android.system.Os.setenv` to set `HF_HOME` |
71
+| `OndeInference` wraps `OndeChatEngine` | Adds Android filesystem init, `Dispatchers.IO`, and `Flow` streaming |
72
+| ProGuard consumer rules bundled in AAR | Prevents R8 from stripping UniFFI JNI bridge in consumer apps |
73
+| JNA dependency required (`net.java.dev.jna:jna:5.14.0@aar`) | UniFFI 0.31.0 generates Kotlin bindings that use `com.sun.jna.*` for the FFI bridge. Despite the `com.sun` package name, JNA works on Android when pulled as `@aar`. Without it, you get `Unresolved reference 'sun'` on every import. |
74
+| `:example` is a Gradle submodule, not a standalone project | Plugin versions are declared once in the root `build.gradle.kts` and inherited by all submodules — a standalone project would need versions on every plugin, and `project(":lib")` would not resolve |
75
+
76
+---
77
+
78
+## UniFFI Kotlin Type Map
79
+
80
+| Rust type | UniFFI derive | Kotlin type |
81
+|---|---|---|
82
+| `ChatRole` | `uniffi::Enum` | `sealed class ChatRole` |
83
+| `ChatMessage` | `uniffi::Record` | `data class ChatMessage` |
84
+| `SamplingConfig` | `uniffi::Record` | `data class SamplingConfig` |
85
+| `GgufModelConfig` | `uniffi::Record` | `data class GgufModelConfig` |
86
+| `InferenceResult` | `uniffi::Record` | `data class InferenceResult` |
87
+| `StreamChunk` | `uniffi::Record` | `data class StreamChunk` |
88
+| `EngineStatus` | `uniffi::Enum` | `sealed class EngineStatus` |
89
+| `EngineInfo` | `uniffi::Record` | `data class EngineInfo` |
90
+| `InferenceError` | `uniffi::Error` | `sealed class InferenceException : Exception` |
91
+| `OndeChatEngine` | `uniffi::Object` | `class OndeChatEngine` |
92
+| `StreamChunkListener` | `callback_interface` | `interface StreamChunkListener` |
93
+
94
+The generated Kotlin package is `uniffi.onde.*`. The `OndeInference.kt` wrapper
95
+re-exports these as `typealias` under `com.ondeinference.onde.*` so callers only
96
+need one import.
97
+
98
+---
99
+
100
+## Android ABI Targets
101
+
102
+| ABI | Rust triple | Typical device |
103
+|---|---|---|
104
+| `arm64-v8a` | `aarch64-linux-android` | Modern Android phones (primary) |
105
+| `armeabi-v7a` | `armv7-linux-androideabi` | Older 32-bit ARM phones |
106
+| `x86_64` | `x86_64-linux-android` | Android emulators (Intel/AMD) |
107
+| `x86` | `i686-linux-android` | 32-bit emulators |
108
+
109
+---
110
+
111
+## Build Sequence (Manual / CI)
112
+
113
+### 1. Prerequisites
114
+
115
+```bash
116
+# Rust Android targets
117
+rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android i686-linux-android
118
+
119
+# cargo-ndk
120
+cargo install cargo-ndk
121
+
122
+# Android NDK
123
+export ANDROID_NDK_HOME=$HOME/Library/Android/sdk/ndk/26.1.10909125
124
+```
125
+
126
+### 2. Build .so files for all ABIs
127
+
128
+```bash
129
+./sdk/kotlin/scripts/build-android.sh # release, all ABIs
130
+./sdk/kotlin/scripts/build-android.sh --debug # debug build
131
+```
132
+
133
+Internally uses:
134
+
135
+```bash
136
+cargo ndk -t arm64-v8a -t armeabi-v7a -t x86_64 -t x86 \
137
+ --manifest-path Cargo.toml --release
138
+```
139
+
140
+Outputs land in `sdk/kotlin/lib/src/main/jniLibs/<ABI>/libonde.so`.
141
+
142
+### 3. Build the uniffi-bindgen CLI
143
+
144
+```bash
145
+cargo build --manifest-path uniffi-bindgen/Cargo.toml --release
146
+# Binary: uniffi-bindgen/target/release/uniffi-bindgen
147
+```
148
+
149
+### 4. Generate Kotlin UniFFI bindings
150
+
151
+```bash
152
+./sdk/kotlin/scripts/generate-bindings.sh
153
+```
154
+
155
+Internally runs:
156
+
157
+```bash
158
+uniffi-bindgen/target/release/uniffi-bindgen generate \
159
+ target/aarch64-linux-android/release/libonde.so \
160
+ --language kotlin \
161
+ --out-dir sdk/kotlin/lib/src/generated/kotlin
162
+```
163
+
164
+Produces `sdk/kotlin/lib/src/generated/kotlin/onde.kt` — the complete
165
+generated binding including all types, the `OndeChatEngine` class, free
166
+functions, and `System.loadLibrary("onde")` in a companion object static init.
167
+
168
+### 5. Build the AAR
169
+
170
+```bash
171
+cd sdk/kotlin
172
+./gradlew :lib:assembleRelease
173
+# Output: lib/build/outputs/aar/lib-release.aar
174
+```
175
+
176
+---
177
+
178
+## Android Filesystem Sandbox Setup
179
+
180
+On Android, `dirs::home_dir()` (used by `hf-hub` under the hood) panics because
181
+there is no home directory in the Android sandbox. The Rust engine requires
182
+`HF_HOME` to be set **before** any model load on Android.
183
+
184
+The `OndeInference` Kotlin wrapper handles this automatically in `setup()`:
185
+
186
+```kotlin
187
+fun setup() {
188
+ if (configured) return
189
+
190
+ val hfHome = File(dataDir, "models").also { it.mkdirs() }
191
+ val hfHubCache = File(hfHome, "hub").also { it.mkdirs() }
192
+ val tmpDir = File(dataDir, "tmp").also { it.mkdirs() }
193
+
194
+ Os.setenv("HF_HOME", hfHome.absolutePath, true)
195
+ Os.setenv("HF_HUB_CACHE", hfHubCache.absolutePath, true)
196
+ Os.setenv("HUGGINGFACE_HUB_CACHE", hfHubCache.absolutePath, true)
197
+ Os.setenv("TMPDIR", tmpDir.absolutePath, true)
198
+
199
+ configured = true
200
+}
201
+```
202
+
203
+`setup()` is called automatically by `loadDefaultModel()` and `loadModel()`.
204
+Callers can call it explicitly at app startup for tighter control.
205
+
206
+### Filesystem layout inside filesDir
207
+
208
+```
209
+<filesDir>/
210
+├── models/ ← HF_HOME
211
+│ └── hub/ ← HF_HUB_CACHE (GGUF blobs live here)
212
+└── tmp/ ← TMPDIR
213
+```
214
+
215
+**This layout matches the shared convention used by the Tauri and Swift SDKs.**
216
+On iOS/macOS, the equivalent paths are inside the App Group container at
217
+`<group.com.ondeinference.apps>/models/hub/`. On Android, there is no app group
218
+mechanism, so each app gets its own `filesDir`.
219
+
220
+---
221
+
222
+## Kotlin Public API
223
+
224
+### `OndeInference` — primary entry point
225
+
226
+```kotlin
227
+import com.ondeinference.onde.OndeInference
228
+import com.ondeinference.onde.OndeSampling
229
+import com.ondeinference.onde.OndeModels
230
+import com.ondeinference.onde.OndeMessage
231
+
232
+val onde = OndeInference(context) // pass applicationContext
233
+
234
+// Load (downloads from HuggingFace Hub on first run)
235
+val elapsed: Double = onde.loadDefaultModel(
236
+ systemPrompt = "You are helpful.",
237
+ sampling = OndeSampling.mobile()
238
+)
239
+
240
+// Chat (suspend)
241
+val result = onde.chat("What is Rust?")
242
+println(result.text)
243
+
244
+// Streaming (Flow)
245
+onde.stream("Tell me a story.").collect { chunk ->
246
+ print(chunk.delta)
247
+}
248
+
249
+// One-shot (no history side-effects)
250
+val summary = onde.generate(
251
+ messages = listOf(
252
+ OndeMessage.system("Summarise this."),
253
+ OndeMessage.user(longText)
254
+ )
255
+)
256
+
257
+// Status
258
+val info = onde.info() // EngineInfo
259
+
260
+// History
261
+onde.clearHistory()
262
+onde.pushHistory(OndeMessage.user("Injected context"))
263
+
264
+// Cleanup
265
+onde.unload()
266
+onde.close() // decrements Rust Arc refcount
267
+```
268
+
269
+### `OndeSampling` — convenience constructors
270
+
271
+```kotlin
272
+OndeSampling.default() // temp=0.7, top_p=0.95, max=512 tokens
273
+OndeSampling.deterministic() // temp=0.0, greedy
274
+OndeSampling.mobile() // temp=0.7, max=128 tokens
275
+```
276
+
277
+### `OndeModels` — model config constructors
278
+
279
+```kotlin
280
+OndeModels.default() // platform default = Qwen 2.5 1.5B on Android
281
+OndeModels.qwen25_1_5b() // ~941 MB
282
+OndeModels.qwen25_3b() // ~1.93 GB
283
+```
284
+
285
+### `OndeMessage` — message constructors
286
+
287
+```kotlin
288
+OndeMessage.system("You are helpful.")
289
+OndeMessage.user("Hello!")
290
+OndeMessage.assistant("Hi there!")
291
+```
292
+
293
+---
294
+
295
+## lib/build.gradle.kts Key Sections
296
+
297
+```kotlin
298
+import com.vanniktech.maven.publish.AndroidSingleVariantLibrary
299
+import com.vanniktech.maven.publish.SonatypeHost
300
+
301
+android {
302
+ namespace = "com.ondeinference.onde"
303
+ compileSdk = 35
304
+ defaultConfig { minSdk = 26 }
305
+
306
+ sourceSets {
307
+ getByName("main") {
308
+ jniLibs.srcDirs("src/main/jniLibs")
309
+ kotlin.srcDirs("src/main/kotlin", "src/generated/kotlin")
310
+ }
311
+ }
312
+
313
+ // Do NOT add android { publishing { singleVariant("release") } } here.
314
+ // AndroidSingleVariantLibrary in mavenPublishing already calls it — doing
315
+ // both causes "singleVariant publishing DSL multiple times" at sync time.
316
+}
317
+
318
+mavenPublishing {
319
+ // This is the correct way to configure variant + sources/javadoc for an
320
+ // Android library with Vanniktech 0.25+. It registers singleVariant internally.
321
+ configure(
322
+ AndroidSingleVariantLibrary(
323
+ variant = "release",
324
+ sourcesJar = true,
325
+ publishJavadocJar = true,
326
+ )
327
+ )
328
+ publishToMavenCentral(SonatypeHost.CENTRAL_PORTAL)
329
+ signAllPublications()
330
+ coordinates("com.ondeinference", "onde-inference", "<version>")
331
+ pom {
332
+ name.set("Onde Inference")
333
+ // ... licenses, developers, scm
334
+ }
335
+}
336
+```
337
+
338
+The Vanniktech plugin reads signing and OSSRH credentials from environment
339
+variables prefixed with `ORG_GRADLE_PROJECT_`:
340
+
341
+| Gradle property | CI env var |
342
+|---|---|
343
+| `mavenCentralUsername` | `ORG_GRADLE_PROJECT_mavenCentralUsername` |
344
+| `mavenCentralPassword` | `ORG_GRADLE_PROJECT_mavenCentralPassword` |
345
+| `signingKeyId` | `ORG_GRADLE_PROJECT_signingKeyId` |
346
+| `signingKey` | `ORG_GRADLE_PROJECT_signingKey` |
347
+| `signingPassword` | `ORG_GRADLE_PROJECT_signingPassword` |
348
+
349
+---
350
+
351
+## ProGuard Consumer Rules
352
+
353
+The `consumer-rules.pro` bundled in the AAR instructs R8/ProGuard in consumer
354
+apps to preserve:
355
+
356
+- `uniffi.onde.**` — all generated binding classes
357
+- `com.ondeinference.onde.**` — the public wrapper
358
+- `uniffi.onde.StreamChunkListener` — callback interface called from Rust
359
+- Native method declarations (`keepclasseswithmembernames`)
360
+- Static initializers in `uniffi.onde.*` (contains `System.loadLibrary("onde")`)
361
+
362
+Always update `consumer-rules.pro` when adding new top-level UniFFI-exported
363
+types or callback interfaces.
364
+
365
+---
366
+
367
+## Maven Central Publishing (CI)
368
+
369
+Trigger: push a semver tag matching `[0-9]+.[0-9]+.[0-9]+`.
370
+
371
+```bash
372
+# 1. Bump VERSION_NAME in sdk/kotlin/gradle.properties
373
+# 2. Bump version in Cargo.toml [package]
374
+# 3. Commit, tag, push
375
+git tag 0.1.4 && git push origin 0.1.4
376
+```
377
+
378
+CI (`release-sdk-kotlin.yml`) steps:
379
+1. Install Rust stable with all 4 Android targets
380
+2. Cache NDK 26.1.10909125
381
+3. Install `cargo-ndk`
382
+4. `build-android.sh --release` → `.so` files
383
+5. Build `uniffi-bindgen`
384
+6. `generate-bindings.sh` → `onde.kt`
385
+7. Validate tag == `Cargo.toml` version
386
+8. `./gradlew :lib:publishAndReleaseToMavenCentral`
387
+9. Upload AAR as GitHub Release asset
388
+
389
+---
390
+
391
+## Version Synchronisation Rule
392
+
393
+`VERSION_NAME` in `sdk/kotlin/gradle.properties` MUST match `version` in the
394
+root `Cargo.toml`. The CI workflow enforces this and fails fast on mismatch.
395
+Never bump one without the other.
396
+
397
+---
398
+
399
+## Common Pitfalls
400
+
401
+| Pitfall | Fix |
402
+|---|---|
403
+| `dirs::home_dir()` panics on Android | `setup()` must be called before any Rust model load. `OndeInference.loadDefaultModel()` calls it automatically. |
404
+| Generated `onde.kt` not found at compile time | Run `scripts/generate-bindings.sh`. The file is gitignored and must be regenerated after every Rust API change. |
405
+| `.so` files missing from AAR | Run `scripts/build-android.sh`. The `jniLibs/` directory is gitignored. |
406
+| UniFFI version mismatch | Keep `uniffi = "=0.31.0"` identical in `Cargo.toml`, `build-deps`, and `uniffi-bindgen/Cargo.toml`. |
407
+| R8 strips UniFFI bridge at runtime | Ensure `consumer-rules.pro` is present in the AAR and includes `-keep class uniffi.onde.**`. |
408
+| `System.loadLibrary("onde")` fails | The `.so` files must match the ABI of the device. Check that all 4 ABIs are built and included in `jniLibs/`. |
409
+| `Os.setenv` not available | `minSdk` must be 26+. `android.system.Os` is available from API 21 but `setenv` specifically needs 26. |
410
+| Gradle fails with `VERSION_NAME` missing | Ensure `gradle.properties` contains `VERSION_NAME=x.y.z`. |
411
+| `cargo-ndk` not found in CI | Add `cargo install cargo-ndk --locked` step before the build step. |
412
+| `ANDROID_NDK_HOME` not set | Set it explicitly: `echo "ANDROID_NDK_HOME=$ANDROID_SDK_ROOT/ndk/26.1.10909125" >> $GITHUB_ENV` |
413
+| `build-android.sh: line XX: arm64: unbound variable` on macOS | Do not use Bash associative arrays in `sdk/kotlin/scripts/build-android.sh`. macOS often ships an older Bash where associative arrays are unreliable or unavailable. Use plain indexed arrays (or another portable structure) for ABI → Rust target mapping. |
414
+| `Unresolved reference 'sun'` / `Unresolved reference 'Structure'` in generated `onde.kt` | UniFFI 0.31.0 generates Kotlin bindings that depend on JNA (`com.sun.jna.*`). Add `implementation("net.java.dev.jna:jna:5.14.0@aar")` to `lib/build.gradle.kts`. The `@aar` classifier is required on Android — a plain `.jar` will not work. Despite the `com.sun` package name, JNA is a third-party library, not a JDK internal, and works fine on Android. |
415
+| HuggingFace download fails on first load | The app must declare `<uses-permission android:name="android.permission.INTERNET" />`. |
416
+| `EngineStatus` values are lowercase (`"ready"`) | The Rust `Display` impl emits lowercase. Normalize in the UI layer: `info.status.toString().lowercase()` or compare with the generated sealed class variants directly. |
417
+| `Plugin [id: 'com.android.application'] was not found` in `:example` | The example has no plugin versions of its own — they are inherited from the root `build.gradle.kts`. Open `sdk/kotlin/` (not `sdk/kotlin/example/`) in Android Studio. Never add a `settings.gradle.kts` inside `example/` — doing so makes Gradle treat it as a standalone root project, breaking plugin resolution and `project(":lib")`. |
418
+| `Task 'wrapper' not found in project ':example'` | Same cause as above — Android Studio was opened at the wrong directory. The `wrapper` task only exists on the root project. Open `sdk/kotlin/`. |
419
+| `Using singleVariant publishing DSL multiple times to publish variant "release"` | Vanniktech 0.25+ registers `singleVariant("release")` internally via `AndroidSingleVariantLibrary`. Do **not** also add `android { publishing { singleVariant("release") { ... } } }` — that registers it a second time and causes this error. Remove the `android { publishing }` block entirely and configure sources/javadoc through `mavenPublishing` instead: `configure(AndroidSingleVariantLibrary(variant = "release", sourcesJar = true, publishJavadocJar = true))`. |
420
+
421
+---
422
+
423
+## Testing
424
+
425
+Unit tests can be run on the host JVM (no Android device needed) for pure Kotlin
426
+logic, but any test that calls into Rust via the generated bindings requires a
427
+connected Android device or emulator because `System.loadLibrary("onde")` only
428
+works with an Android runtime.
429
+
430
+```bash
431
+# Kotlin-only unit tests (host JVM)
432
+cd sdk/kotlin
433
+./gradlew :lib:test
434
+
435
+# Instrumented tests (requires connected device or emulator)
436
+./gradlew :lib:connectedAndroidTest
437
+```
438
+
439
+For CI, use an emulator (`x86_64` ABI) spun up with `reactivecircus/android-emulator-runner`.
440
+
441
+---
442
+
443
+## Dependency Graph
444
+
445
+```
446
+OndeInference.kt
447
+ └── uniffi.onde.OndeChatEngine (generated onde.kt)
448
+ └── uniffi.onde.StreamChunkListener (generated onde.kt)
449
+ └── uniffi.onde.*Config / *Message (generated onde.kt)
450
+ └── com.sun.jna.Native (JNA — net.java.dev.jna:jna:5.14.0@aar)
451
+ └── libonde.so (Rust, per ABI)
452
+ └── mistralrs (Candle CPU backend on Android)
453
+ └── hf-hub (model download)
454
+```
455
+
456
+---
457
+
458
+## Distribution Registry
459
+
460
+| Registry | Artifact | Import |
461
+|---|---|---|
462
+| Maven Central | `com.ondeinference:onde-inference` | `implementation("com.ondeinference:onde-inference:0.1.3")` |
463
+| GitHub Releases | `onde-inference-<version>.aar` | Direct AAR download |
464
+
465
+---
466
+
467
+## Example App
468
+
469
+A working Jetpack Compose chat app lives at `sdk/kotlin/example/`. It is the canonical way to see the SDK in action and is the first thing to open when verifying a new build.
470
+
471
+### Opening in Android Studio
472
+
473
+Always open **`sdk/kotlin/`** — the Gradle root. Do not open `sdk/kotlin/example/` directly. Android Studio will see both `:lib` and `:example` as modules and sync correctly.
474
+
475
+### What the example covers
476
+
477
+- Loading the default model with `loadDefaultModel()` and showing load time
478
+- Streaming replies token-by-token via `onde.stream()` and `Flow<StreamChunk>`
479
+- Multi-turn conversation history maintained in the Rust engine
480
+- Clearing history without reloading the model
481
+- Material3 dynamic colour theme with dark mode support
482
+- Proper IME / navigation-bar inset handling with `imePadding()` + `navigationBarsPadding()`
483
+
484
+### Gradle submodule rules
485
+
486
+`example/` is a Gradle submodule of the `sdk/kotlin/` root project. The rules that must always hold:
487
+
488
+1. **No `settings.gradle.kts` inside `example/`** — if one exists, Gradle stops its upward search there and treats `example/` as a standalone root, breaking plugin resolution and `project(":lib")`.
489
+2. **No plugin versions in `example/build.gradle.kts`** — versions are declared once in the root `build.gradle.kts` with `apply false` and inherited by submodules.
490
+3. **`include(":example")` in root `settings.gradle.kts`** — required for Gradle to recognise `:example` as a subproject.
491
+4. **`project(":lib")` as the SDK dependency** — the example always builds against local source, not the published Maven artifact. This means the `.so` files and `onde.kt` must be generated before the example will compile.
492
+
493
+The root `build.gradle.kts` must declare all plugins used by any submodule:
494
+
495
+```kotlin
496
+plugins {
497
+ id("com.android.library") version "8.5.2" apply false // :lib
498
+ id("com.android.application") version "8.5.2" apply false // :example
499
+ id("org.jetbrains.kotlin.android") version "2.0.21" apply false
500
+ id("org.jetbrains.kotlin.plugin.compose") version "2.0.21" apply false // :example
501
+ id("com.vanniktech.maven.publish") version "0.28.0" apply false // :lib
502
+}
503
+```
504
+
505
+### Example architecture
506
+
507
+```
508
+ChatScreen (Compose — pure function of ChatUiState)
509
+ ↓ collectAsStateWithLifecycle
510
+ChatViewModel (AndroidViewModel + viewModelScope)
511
+ ↓ loadModel() → onde.loadDefaultModel(...)
512
+ ↓ sendMessage() → onde.stream(...).collect { ... }
513
+ ↓ clearChat() → onde.clearHistory()
514
+OndeInference ← project(":lib")
515
+ ↓ UniFFI / libonde.so
516
+mistral.rs — Qwen 2.5 1.5B, Candle CPU backend
517
+```
518
+
519
+### Streaming pattern
520
+
521
+Tokens are accumulated into a `StringBuilder` and the last item in the message list is replaced in-place on every chunk, so Compose only recomposes the one bubble that is changing:
522
+
523
+```kotlin
524
+val placeholderId = System.nanoTime()
525
+// insert empty bubble once
526
+_uiState.update { it.copy(messages = it.messages + UiMessage(id = placeholderId, ...)) }
527
+
528
+val buffer = StringBuilder()
529
+onde.stream(text).collect { chunk ->
530
+ buffer.append(chunk.delta)
531
+ // swap only the tail — no full list rebuild
532
+ _uiState.update { state ->
533
+ state.copy(messages = state.messages.replaceLast(
534
+ UiMessage(id = placeholderId, content = buffer.toString(), isStreaming = !chunk.done)
535
+ ))
536
+ }
537
+}
538
+```
539
+
540
+### Running the example
541
+
542
+`build-android.sh` must stay portable on macOS. Avoid Bash associative arrays in that script — use indexed arrays instead.
543
+
544
+```bash
545
+cd sdk/kotlin
546
+
547
+# 1. Build the .so files (needed by :lib, which :example depends on)
548
+./scripts/build-android.sh
549
+
550
+# 2. Generate the Kotlin bindings
551
+./scripts/generate-bindings.sh
552
+
553
+# 3. Install on a connected device
554
+./gradlew :example:installDebug
555
+```
556
+
557
+Or open `sdk/kotlin/` in Android Studio and hit Run on the `:example` configuration.
\ No newline at end of file
.github/workflows/release-sdk-kotlin.yml
new
+172
@@ -0,0 +1,172 @@
1
+name: Release Kotlin SDK to Maven Central
2
+
3
+on:
4
+ push:
5
+ tags:
6
+ - '[0-9]+.[0-9]+.[0-9]+'
7
+ workflow_dispatch:
8
+ inputs:
9
+ version:
10
+ description: 'Version to publish (e.g. 0.1.3)'
11
+ required: true
12
+ type: string
13
+
14
+jobs:
15
+ publish:
16
+ name: Build and publish AAR to Maven Central
17
+ runs-on: ubuntu-latest
18
+
19
+ steps:
20
+ - name: Checkout repository
21
+ uses: actions/checkout@v4
22
+
23
+ - name: Resolve version
24
+ id: version
25
+ run: |
26
+ if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
27
+ VERSION="${{ inputs.version }}"
28
+ else
29
+ VERSION="${GITHUB_REF_NAME}"
30
+ fi
31
+ echo "version=$VERSION" >> "$GITHUB_OUTPUT"
32
+ echo "Publishing version: $VERSION"
33
+
34
+ - name: Set up JDK 17
35
+ uses: actions/setup-java@v4
36
+ with:
37
+ java-version: '17'
38
+ distribution: 'temurin'
39
+
40
+ - name: Set up Gradle
41
+ uses: gradle/actions/setup-gradle@v3
42
+ with:
43
+ gradle-version: wrapper
44
+ build-root-directory: sdk/kotlin
45
+
46
+ - name: Install Rust stable toolchain
47
+ uses: dtolnay/rust-toolchain@stable
48
+ with:
49
+ targets: >-
50
+ aarch64-linux-android,
51
+ armv7-linux-androideabi,
52
+ x86_64-linux-android,
53
+ i686-linux-android
54
+
55
+ - name: Cache Rust build artifacts
56
+ uses: actions/cache@v4
57
+ with:
58
+ path: |
59
+ ~/.cargo/registry
60
+ ~/.cargo/git
61
+ target/
62
+ key: rust-android-${{ hashFiles('Cargo.toml', 'Cargo.lock', 'src/**/*.rs') }}
63
+ restore-keys: rust-android-
64
+
65
+ - name: Cache NDK
66
+ uses: actions/cache@v4
67
+ id: ndk-cache
68
+ with:
69
+ path: ${{ runner.tool_cache }}/ndk
70
+ key: ndk-26.1.10909125
71
+
72
+ - name: Install Android NDK
73
+ if: steps.ndk-cache.outputs.cache-hit != 'true'
74
+ run: |
75
+ ANDROID_SDK_ROOT="$ANDROID_SDK_ROOT"
76
+ echo "y" | $ANDROID_SDK_ROOT/cmdline-tools/latest/bin/sdkmanager \
77
+ "ndk;26.1.10909125"
78
+
79
+ - name: Set ANDROID_NDK_HOME
80
+ run: |
81
+ echo "ANDROID_NDK_HOME=$ANDROID_SDK_ROOT/ndk/26.1.10909125" >> "$GITHUB_ENV"
82
+
83
+ - name: Install cargo-ndk
84
+ run: cargo install cargo-ndk --locked
85
+
86
+ - name: Build Rust .so files for all Android ABIs
87
+ run: |
88
+ chmod +x sdk/kotlin/scripts/build-android.sh
89
+ sdk/kotlin/scripts/build-android.sh --release
90
+
91
+ - name: Build uniffi-bindgen
92
+ run: cargo build --manifest-path uniffi-bindgen/Cargo.toml --release
93
+
94
+ - name: Generate Kotlin UniFFI bindings
95
+ run: |
96
+ chmod +x sdk/kotlin/scripts/generate-bindings.sh
97
+ sdk/kotlin/scripts/generate-bindings.sh
98
+
99
+ - name: Verify generated bindings exist
100
+ run: |
101
+ ls -lh sdk/kotlin/lib/src/generated/kotlin/
102
+ echo "JNI libs:"
103
+ find sdk/kotlin/lib/src/main/jniLibs -name "*.so" | sort
104
+
105
+ - name: Validate version matches Cargo.toml
106
+ run: |
107
+ CARGO_VERSION=$(grep '^version' Cargo.toml | head -1 | sed 's/version = "\(.*\)"/\1/')
108
+ TAG_VERSION="${{ steps.version.outputs.version }}"
109
+ echo "Cargo.toml version: $CARGO_VERSION"
110
+ echo "Tag/input version: $TAG_VERSION"
111
+ if [ "$CARGO_VERSION" != "$TAG_VERSION" ]; then
112
+ echo "Error: version mismatch — bump Cargo.toml before tagging."
113
+ exit 1
114
+ fi
115
+
116
+ - name: Update VERSION_NAME in gradle.properties
117
+ run: |
118
+ sed -i "s/^VERSION_NAME=.*/VERSION_NAME=${{ steps.version.outputs.version }}/" \
119
+ sdk/kotlin/gradle.properties
120
+
121
+ - name: Publish AAR to Maven Central
122
+ working-directory: sdk/kotlin
123
+ env:
124
+ ORG_GRADLE_PROJECT_mavenCentralUsername: ${{ secrets.MAVEN_CENTRAL_USERNAME }}
125
+ ORG_GRADLE_PROJECT_mavenCentralPassword: ${{ secrets.MAVEN_CENTRAL_PASSWORD }}
126
+ ORG_GRADLE_PROJECT_signingKeyId: ${{ secrets.SIGNING_KEY_ID }}
127
+ ORG_GRADLE_PROJECT_signingKey: ${{ secrets.SIGNING_KEY }}
128
+ ORG_GRADLE_PROJECT_signingPassword: ${{ secrets.SIGNING_PASSWORD }}
129
+ run: ./gradlew :lib:publishAndReleaseToMavenCentral --no-configuration-cache
130
+
131
+ - name: Collect AAR artifact
132
+ run: |
133
+ find sdk/kotlin/lib/build/outputs/aar -name "*.aar" | sort
134
+ mkdir -p dist/kotlin
135
+ cp sdk/kotlin/lib/build/outputs/aar/*.aar dist/kotlin/
136
+
137
+ - name: Upload AAR as workflow artifact
138
+ uses: actions/upload-artifact@v4
139
+ with:
140
+ name: onde-inference-${{ steps.version.outputs.version }}.aar
141
+ path: dist/kotlin/*.aar
142
+ retention-days: 90
143
+
144
+ - name: Create GitHub Release
145
+ if: github.event_name == 'push'
146
+ uses: softprops/action-gh-release@v2
147
+ with:
148
+ tag_name: ${{ github.ref_name }}
149
+ name: Kotlin SDK ${{ steps.version.outputs.version }}
150
+ body: |
151
+ ## Onde Inference Kotlin SDK ${{ steps.version.outputs.version }}
152
+
153
+ ### Maven Central
154
+
155
+ ```kotlin
156
+ // build.gradle.kts
157
+ dependencies {
158
+ implementation("com.ondeinference:onde-inference:${{ steps.version.outputs.version }}")
159
+ }
160
+ ```
161
+
162
+ ### What's included
163
+
164
+ - Rust inference engine compiled for: `arm64-v8a`, `armeabi-v7a`, `x86_64`, `x86`
165
+ - UniFFI-generated Kotlin bindings
166
+ - Coroutine-first public API (`OndeInference`)
167
+ - Flow-based token streaming
168
+
169
+ See the [README](https://github.com/ondeinference/onde/tree/main/sdk/kotlin/README.md) for full usage docs.
170
+ files: dist/kotlin/*.aar
171
+ draft: false
172
+ prerelease: false
sdk/kotlin/.gitignore
new
+36
@@ -0,0 +1,36 @@
1
+# Dependencies
2
+.gradle/
3
+build/
4
+
5
+# Generated UniFFI Kotlin bindings — regenerated by scripts/generate-bindings.sh
6
+lib/src/generated/
7
+
8
+# Compiled Rust native libraries — regenerated by scripts/build-android.sh
9
+lib/src/main/jniLibs/
10
+
11
+# Android Studio / IntelliJ
12
+.idea/
13
+*.iml
14
+*.ipr
15
+*.iws
16
+local.properties
17
+captures/
18
+
19
+# macOS
20
+.DS_Store
21
+
22
+# Gradle wrapper binary — too large and platform-specific
23
+gradle/wrapper/gradle-wrapper.jar
24
+
25
+# Signing keys — never commit private keys
26
+*.jks
27
+*.keystore
28
+*.p12
29
+keystore.properties
30
+
31
+# ProGuard output
32
+*.map
33
+
34
+# Logs
35
+*.log
36
+npm-debug.log*
sdk/kotlin/README.md
new
+323
@@ -0,0 +1,323 @@
1
+<p align="center">
2
+ <img src="https://raw.githubusercontent.com/ondeinference/onde/main/assets/onde-inference-logo.svg" alt="Onde Inference" width="96">
3
+</p>
4
+
5
+<h1 align="center">Onde Inference — Kotlin / Android SDK</h1>
6
+
7
+<p align="center">
8
+ <strong>On-device LLM inference for Android. Run Qwen 2.5 models locally — no cloud, no API key, no data leaving the device.</strong>
9
+</p>
10
+
11
+<p align="center">
12
+ <a href="https://central.sonatype.com/artifact/com.ondeinference/onde-inference"><img src="https://img.shields.io/maven-central/v/com.ondeinference/onde-inference" alt="Maven Central"></a>
13
+ <a href="https://ondeinference.com"><img src="https://img.shields.io/badge/website-ondeinference.com-blue" alt="Website"></a>
14
+ <a href="https://github.com/ondeinference/onde/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-blue" alt="License"></a>
15
+</p>
16
+
17
+---
18
+
19
+## What is Onde?
20
+
21
+Onde is an on-device LLM inference SDK for Android. It wraps [mistral.rs](https://github.com/EricLBuehler/mistral.rs) behind a clean Kotlin API — automatic model downloading from HuggingFace Hub, cache management, and CPU inference via the Candle backend.
22
+
23
+- **On-device** — no cloud, no API key, no data leaving the device
24
+- **Kotlin-first** — suspend functions and `Flow<StreamChunk>` streaming
25
+- **UniFFI-powered** — the Rust engine is bound to Kotlin via [uniffi-rs](https://github.com/mozilla/uniffi-rs)
26
+- **Maven Central** — single Gradle dependency, no manual `.so` management
27
+
28
+---
29
+
30
+## Installation
31
+
32
+Add to your app's `build.gradle.kts`:
33
+
34
+```kotlin
35
+dependencies {
36
+ implementation("com.ondeinference:onde-inference:0.1.3")
37
+}
38
+```
39
+
40
+Add `INTERNET` permission to `AndroidManifest.xml` (required for the initial model download):
41
+
42
+```xml
43
+<uses-permission android:name="android.permission.INTERNET" />
44
+```
45
+
46
+**Minimum requirements**
47
+- Android 8.0 (API 26) or higher
48
+- ~1.1 GB free storage for the default model (Qwen 2.5 1.5B Q4_K_M)
49
+
50
+---
51
+
52
+## Quick start
53
+
54
+```kotlin
55
+import com.ondeinference.onde.OndeInference
56
+import kotlinx.coroutines.launch
57
+
58
+class MainActivity : AppCompatActivity() {
59
+
60
+ private val onde by lazy { OndeInference(applicationContext) }
61
+
62
+ override fun onCreate(savedInstanceState: Bundle?) {
63
+ super.onCreate(savedInstanceState)
64
+
65
+ lifecycleScope.launch {
66
+ // 1. Load the default model (Qwen 2.5 1.5B on Android, ~941 MB)
67
+ // Downloads from HuggingFace Hub on first run (~1–5 min on Wi-Fi).
68
+ val elapsed = onde.loadDefaultModel(
69
+ systemPrompt = "You are a helpful, concise assistant."
70
+ )
71
+ println("Model loaded in ${elapsed}s")
72
+
73
+ // 2. Chat
74
+ val result = onde.chat("What is the capital of Sweden?")
75
+ println(result.text) // → "The capital of Sweden is Stockholm."
76
+
77
+ // 3. Clean up
78
+ onde.unload()
79
+ }
80
+ }
81
+
82
+ override fun onDestroy() {
83
+ super.onDestroy()
84
+ onde.close()
85
+ }
86
+}
87
+```
88
+
89
+---
90
+
91
+## Streaming
92
+
93
+Receive tokens as they are generated using `Flow`:
94
+
95
+```kotlin
96
+lifecycleScope.launch {
97
+ onde.stream("Write a haiku about the ocean.").collect { chunk ->
98
+ // Append each token delta to your UI
99
+ textView.append(chunk.delta)
100
+ if (chunk.done) println("\n[done]")
101
+ }
102
+}
103
+```
104
+
105
+---
106
+
107
+## Multi-turn conversation
108
+
109
+The engine maintains conversation history automatically:
110
+
111
+```kotlin
112
+lifecycleScope.launch {
113
+ onde.loadDefaultModel(systemPrompt = "You are a Rust tutor.")
114
+
115
+ val r1 = onde.chat("What is ownership?")
116
+ println(r1.text)
117
+
118
+ val r2 = onde.chat("Can you give me a code example?")
119
+ println(r2.text) // knows the context from r1
120
+
121
+ // Clear history without unloading the model
122
+ onde.clearHistory()
123
+}
124
+```
125
+
126
+---
127
+
128
+## Load a specific model
129
+
130
+```kotlin
131
+import com.ondeinference.onde.OndeModels
132
+
133
+lifecycleScope.launch {
134
+ // Qwen 2.5 1.5B — default on Android
135
+ val elapsed = onde.loadModel(
136
+ config = OndeModels.qwen25_1_5b(),
137
+ systemPrompt = "You are a coding assistant."
138
+ )
139
+}
140
+```
141
+
142
+---
143
+
144
+## Sampling configuration
145
+
146
+```kotlin
147
+import com.ondeinference.onde.OndeSampling
148
+
149
+lifecycleScope.launch {
150
+ // Default creative chat — temperature 0.7, top_p 0.95, max 512 tokens
151
+ onde.loadDefaultModel(sampling = OndeSampling.default())
152
+
153
+ // Greedy / deterministic — best for factual Q&A and code generation
154
+ onde.loadDefaultModel(sampling = OndeSampling.deterministic())
155
+
156
+ // Mobile-conservative — max 128 tokens, faster on low-end devices
157
+ onde.loadDefaultModel(sampling = OndeSampling.mobile())
158
+}
159
+```
160
+
161
+---
162
+
163
+## One-shot generation (no history side-effects)
164
+
165
+```kotlin
166
+import com.ondeinference.onde.OndeMessage
167
+
168
+lifecycleScope.launch {
169
+ val result = onde.generate(
170
+ messages = listOf(
171
+ OndeMessage.system("You are a summariser."),
172
+ OndeMessage.user("Summarise this text: $longArticle"),
173
+ ),
174
+ sampling = OndeSampling.deterministic()
175
+ )
176
+ println(result.text)
177
+}
178
+```
179
+
180
+---
181
+
182
+## Engine status
183
+
184
+```kotlin
185
+lifecycleScope.launch {
186
+ val info = onde.info()
187
+ println("Status: ${info.status}") // Ready / Loading / Unloaded / Error
188
+ println("Model: ${info.modelName}")
189
+ println("Memory: ${info.approxMemory}")
190
+ println("History: ${info.historyLength} turns")
191
+}
192
+```
193
+
194
+---
195
+
196
+## File system layout
197
+
198
+Onde stores model files in the app's internal `filesDir`. No external storage or
199
+special permissions are required beyond `INTERNET`.
200
+
201
+```
202
+<filesDir>/
203
+├── models/
204
+│ └── hub/
205
+│ └── models--bartowski--Qwen2.5-1.5B-Instruct-GGUF/
206
+│ └── ...
207
+└── tmp/
208
+```
209
+
210
+You can inspect the cache location at runtime:
211
+
212
+```kotlin
213
+println(onde.modelCacheDir.absolutePath)
214
+```
215
+
216
+---
217
+
218
+## Architecture
219
+
220
+```
221
+┌───────────────────────────────────────┐
222
+│ Kotlin app / Android activity │
223
+│ import com.ondeinference.onde.* │
224
+└──────────────┬────────────────────────┘
225
+ │ suspend / Flow
226
+ ▼
227
+┌───────────────────────────────────────┐
228
+│ OndeInference.kt (this SDK) │
229
+│ Android filesystem setup │
230
+│ Dispatchers.IO dispatch │
231
+└──────────────┬────────────────────────┘
232
+ │ UniFFI-generated Kotlin (onde.kt)
233
+ ▼
234
+┌───────────────────────────────────────┐
235
+│ libonde.so (Rust, per ABI) │
236
+│ mistral.rs + Candle (CPU) │
237
+│ HuggingFace Hub download │
238
+└───────────────────────────────────────┘
239
+```
240
+
241
+The Rust engine is compiled for four ABIs and bundled as JNI libs:
242
+
243
+| ABI | Architecture |
244
+|-----|-------------|
245
+| `arm64-v8a` | 64-bit ARM (modern Android phones) |
246
+| `armeabi-v7a` | 32-bit ARM (older phones) |
247
+| `x86_64` | Android emulators (Intel/AMD) |
248
+| `x86` | 32-bit emulators |
249
+
250
+---
251
+
252
+## Building from source
253
+
254
+### Prerequisites
255
+
256
+```bash
257
+# Rust Android targets
258
+rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android i686-linux-android
259
+
260
+# cargo-ndk — manages NDK toolchain for cross-compilation
261
+cargo install cargo-ndk
262
+
263
+# Android NDK (via Android Studio or SDK manager)
264
+export ANDROID_NDK_HOME=$HOME/Library/Android/sdk/ndk/26.1.10909125
265
+```
266
+
267
+### Build the native libraries
268
+
269
+```bash
270
+# From the repo root
271
+./sdk/kotlin/scripts/build-android.sh
272
+```
273
+
274
+This produces `.so` files in `sdk/kotlin/lib/src/main/jniLibs/`.
275
+
276
+### Generate Kotlin UniFFI bindings
277
+
278
+```bash
279
+./sdk/kotlin/scripts/generate-bindings.sh
280
+```
281
+
282
+This generates `onde.kt` in `sdk/kotlin/lib/src/generated/kotlin/`. The generated file
283
+is gitignored — regenerate it whenever the Rust API changes.
284
+
285
+### Build the AAR
286
+
287
+```bash
288
+cd sdk/kotlin
289
+./gradlew :lib:assembleRelease
290
+# Output: lib/build/outputs/aar/lib-release.aar
291
+```
292
+
293
+---
294
+
295
+## Publishing to Maven Central
296
+
297
+Publishing requires:
298
+
299
+| Secret | Description |
300
+|--------|-------------|
301
+| `MAVEN_CENTRAL_USERNAME` | Sonatype Central Portal username |
302
+| `MAVEN_CENTRAL_PASSWORD` | Sonatype Central Portal password |
303
+| `SIGNING_KEY_ID` | PGP key ID (last 8 hex chars) |
304
+| `SIGNING_KEY` | ASCII-armored PGP private key |
305
+| `SIGNING_PASSWORD` | PGP key passphrase |
306
+
307
+Tag a release to trigger the workflow:
308
+
309
+```bash
310
+# Bump version in Cargo.toml and gradle.properties first
311
+git tag 0.1.4 && git push origin 0.1.4
312
+```
313
+
314
+The CI workflow builds all ABIs, generates bindings, packages the AAR, signs it, and
315
+publishes to Maven Central automatically.
316
+
317
+---
318
+
319
+## License
320
+
321
+MIT OR Apache-2.0 — see [LICENSE](../../LICENSE).
322
+
323
+Built by [Onde Inference / Splitfire AB](https://ondeinference.com).
sdk/kotlin/build.gradle.kts
new
+8
@@ -0,0 +1,8 @@
1
+// Root build file — declares plugins available to subprojects.
2
+plugins {
3
+ id("com.android.library") version "8.5.2" apply false
4
+ id("com.android.application") version "8.5.2" apply false
5
+ id("org.jetbrains.kotlin.android") version "2.0.21" apply false
6
+ id("org.jetbrains.kotlin.plugin.compose") version "2.0.21" apply false
7
+ id("com.vanniktech.maven.publish") version "0.28.0" apply false
8
+}
sdk/kotlin/example/README.md
new
+101
@@ -0,0 +1,101 @@
1
+# Onde Chat
2
+
3
+A simple Android chat app built with the [Onde Inference](https://ondeinference.com) SDK. It runs Qwen 2.5 1.5B entirely on your phone — no server, no API key, nothing leaves the device.
4
+
5
+---
6
+
7
+## What it does
8
+
9
+- Downloads the model on first launch (~941 MB, cached after that)
10
+- Streams replies token by token as the model generates them
11
+- Keeps a full conversation history across turns
12
+- Works offline once the model is cached
13
+
14
+---
15
+
16
+## Getting started
17
+
18
+Open `onde/sdk/kotlin/example/` as the project root in Android Studio (not the repo root — the `settings.gradle.kts` lives here). Gradle will pull everything from Maven Central on first sync.
19
+
20
+You'll need a device or emulator running **Android 8.0 (API 26) or higher**. An x86_64 emulator works fine, though a real ARM phone will feel noticeably faster.
21
+
22
+Hit Run. The first launch will download the model over your internet connection — after that the app is fully offline.
23
+
24
+---
25
+
26
+## Project layout
27
+
28
+```
29
+example/
30
+├── settings.gradle.kts standalone project root
31
+├── build.gradle.kts deps and build config
32
+└── src/main/
33
+ ├── AndroidManifest.xml INTERNET permission lives here
34
+ └── kotlin/com/ondeinference/example/
35
+ ├── MainActivity.kt one activity, sets up the theme and hands off to Compose
36
+ ├── ChatViewModel.kt model loading, streaming, history — all the logic
37
+ └── ui/
38
+ ├── ChatScreen.kt everything you see on screen
39
+ └── theme/Theme.kt Material3 colours, dynamic on Android 12+
40
+```
41
+
42
+---
43
+
44
+## How the streaming works
45
+
46
+When you send a message, the app drops an empty assistant bubble into the list immediately, then fills it in character by character as tokens arrive from the Rust engine. The ViewModel holds a `StringBuilder` and swaps the tail of the message list on every chunk — so Compose only recomposes the one bubble that's changing, not the whole conversation.
47
+
48
+```kotlin
49
+onde.stream(text).collect { chunk ->
50
+ buffer.append(chunk.delta)
51
+ // update only the last bubble in the list
52
+}
53
+```
54
+
55
+---
56
+
57
+## Swapping the model
58
+
59
+The app loads the platform default (Qwen 2.5 1.5B) via `loadDefaultModel()`. To use a different model, call `loadModel()` with a config instead:
60
+
61
+```kotlin
62
+onde.loadModel(
63
+ config = OndeModels.qwen25_3b(), // ~1.93 GB — not great on phones
64
+ systemPrompt = "You are a creative writing assistant.",
65
+ sampling = OndeSampling.deterministic(),
66
+)
67
+```
68
+
69
+Available sampling presets:
70
+
71
+| Preset | Temp | Max tokens | Good for |
72
+|---|---|---|---|
73
+| `OndeSampling.mobile()` | 0.7 | 128 | everyday chat |
74
+| `OndeSampling.default()` | 0.7 | 512 | longer responses |
75
+| `OndeSampling.deterministic()` | 0.0 | 512 | coding, facts |
76
+
77
+---
78
+
79
+## Developing against a local SDK build
80
+
81
+By default the app pulls `onde-inference` from Maven Central. If you want to test changes to the SDK itself, open the parent directory (`onde/sdk/kotlin/`) in Android Studio instead and swap the dependency in `build.gradle.kts`:
82
+
83
+```kotlin
84
+// instead of:
85
+implementation("com.ondeinference:onde-inference:0.1.3")
86
+
87
+// use:
88
+implementation(project(":lib"))
89
+```
90
+
91
+---
92
+
93
+## Troubleshooting
94
+
95
+**Download never finishes** — check your internet connection. HuggingFace Hub occasionally rate-limits anonymous downloads; setting an `HF_TOKEN` env variable before building can help.
96
+
97
+**App crashes immediately** — make sure you're on API 26+. The SDK uses `Os.setenv` to configure the HuggingFace cache paths, which requires Android 8.0.
98
+
99
+**Inference is very slow** — CPU inference on an emulator is slow by design. Try a physical ARM device for a much better experience.
100
+
101
+**Out of memory on load** — the 1.5B model needs roughly 1–1.5 GB of free RAM. Close other apps and try again, or restart the device.
\ No newline at end of file
sdk/kotlin/example/build.gradle.kts
new
+75
@@ -0,0 +1,75 @@
1
+plugins {
2
+ id("com.android.application")
3
+ id("org.jetbrains.kotlin.android")
4
+ id("org.jetbrains.kotlin.plugin.compose")
5
+}
6
+
7
+android {
8
+ namespace = "com.ondeinference.example"
9
+ compileSdk = 35
10
+
11
+ defaultConfig {
12
+ applicationId = "com.ondeinference.example"
13
+ minSdk = 26 // Os.setenv needs API 26+
14
+ targetSdk = 35
15
+ versionCode = 1
16
+ versionName = "1.0"
17
+ }
18
+
19
+ buildTypes {
20
+ release {
21
+ isMinifyEnabled = true
22
+ proguardFiles(
23
+ getDefaultProguardFile("proguard-android-optimize.txt"),
24
+ "proguard-rules.pro",
25
+ )
26
+ }
27
+ }
28
+
29
+ compileOptions {
30
+ sourceCompatibility = JavaVersion.VERSION_17
31
+ targetCompatibility = JavaVersion.VERSION_17
32
+ }
33
+
34
+ kotlinOptions {
35
+ jvmTarget = "17"
36
+ }
37
+
38
+ buildFeatures {
39
+ compose = true
40
+ }
41
+
42
+ packaging {
43
+ resources {
44
+ excludes += "/META-INF/{AL2.0,LGPL2.1}"
45
+ }
46
+ }
47
+}
48
+
49
+dependencies {
50
+ // The local SDK — built straight from source, no publish step needed.
51
+ // To test against the published release instead, swap this for:
52
+ // implementation("com.ondeinference:onde-inference:0.1.3")
53
+ implementation(project(":lib"))
54
+
55
+ // Compose
56
+ implementation(platform("androidx.compose:compose-bom:2024.10.01"))
57
+ implementation("androidx.compose.ui:ui")
58
+ implementation("androidx.compose.ui:ui-graphics")
59
+ implementation("androidx.compose.ui:ui-tooling-preview")
60
+ implementation("androidx.compose.material3:material3")
61
+ implementation("androidx.compose.material:material-icons-core")
62
+
63
+ // AndroidX
64
+ implementation("androidx.core:core-ktx:1.15.0")
65
+ implementation("androidx.activity:activity-compose:1.9.3")
66
+ implementation("androidx.lifecycle:lifecycle-runtime-ktx:2.8.7")
67
+ implementation("androidx.lifecycle:lifecycle-runtime-compose:2.8.7")
68
+ implementation("androidx.lifecycle:lifecycle-viewmodel-compose:2.8.7")
69
+
70
+ // Coroutines
71
+ implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.8.1")
72
+
73
+ debugImplementation("androidx.compose.ui:ui-tooling")
74
+ debugImplementation("androidx.compose.ui:ui-test-manifest")
75
+}
sdk/kotlin/example/proguard-rules.pro
new
+56
@@ -0,0 +1,56 @@
1
+# ProGuard rules for the Onde Chat example app (release build).
2
+#
3
+# The Onde SDK AAR ships its own consumer-rules.pro that preserves all UniFFI
4
+# JNI bridge symbols automatically — no SDK-specific rules are needed here.
5
+#
6
+# These rules cover the example app itself and common Android/Kotlin patterns.
7
+
8
+# ── Kotlin ────────────────────────────────────────────────────────────────────
9
+
10
+# Keep Kotlin metadata so reflection and serialisation work correctly at runtime.
11
+-keepattributes *Annotation*, Signature, InnerClasses, EnclosingMethod
12
+
13
+# Kotlin coroutines — keep internal machinery used by the coroutine debugger
14
+# and exception handling paths.
15
+-keepnames class kotlinx.coroutines.internal.MainDispatcherFactory {}
16
+-keepnames class kotlinx.coroutines.CoroutineExceptionHandler {}
17
+-keepclassmembernames class kotlinx.coroutines.** {
18
+ volatile <fields>;
19
+}
20
+
21
+# ── Jetpack Compose ───────────────────────────────────────────────────────────
22
+
23
+# Compose relies heavily on reflection for @Composable slot-table lookups.
24
+# The compose-bom ships its own rules but this ensures nothing is stripped
25
+# during incremental R8 optimisation passes.
26
+-keepclassmembers class * {
27
+ @androidx.compose.runtime.Composable <methods>;
28
+}
29
+
30
+# ── AndroidX ──────────────────────────────────────────────────────────────────
31
+
32
+# ViewModel — keep all subclasses so the ViewModelProvider can instantiate them.
33
+-keep public class * extends androidx.lifecycle.ViewModel {
34
+ public <init>(...);
35
+}
36
+
37
+# ── General Android ───────────────────────────────────────────────────────────
38
+
39
+# Preserve classes that implement Parcelable so the OS can reconstruct them
40
+# across process boundaries (e.g. saved instance state).
41
+-keep class * implements android.os.Parcelable {
42
+ public static final android.os.Parcelable$Creator *;
43
+}
44
+
45
+# Keep enum members used in switch expressions generated by kotlinc.
46
+-keepclassmembers enum * {
47
+ public static **[] values();
48
+ public static ** valueOf(java.lang.String);
49
+}
50
+
51
+# Remove verbose logging in release builds to reduce APK size and avoid
52
+# leaking internal state to logcat on production devices.
53
+-assumenosideeffects class android.util.Log {
54
+ public static int v(...);
55
+ public static int d(...);
56
+}
sdk/kotlin/example/src/main/AndroidManifest.xml
new
+28
@@ -0,0 +1,28 @@
1
+<?xml version="1.0" encoding="utf-8"?>
2
+<manifest xmlns:android="http://schemas.android.com/apk/res/android">
3
+
4
+ <!--
5
+ Required: Onde downloads GGUF model weights from HuggingFace Hub on first launch.
6
+ Subsequent runs use the local cache in filesDir/models/hub/ — no network needed.
7
+ -->
8
+ <uses-permission android:name="android.permission.INTERNET" />
9
+
10
+ <application
11
+ android:allowBackup="true"
12
+ android:label="@string/app_name"
13
+ android:supportsRtl="true"
14
+ android:theme="@style/Theme.OndeExample">
15
+
16
+ <activity
17
+ android:name=".MainActivity"
18
+ android:exported="true"
19
+ android:windowSoftInputMode="adjustResize">
20
+ <intent-filter>
21
+ <action android:name="android.intent.action.MAIN" />
22
+ <category android:name="android.intent.category.LAUNCHER" />
23
+ </intent-filter>
24
+ </activity>
25
+
26
+ </application>
27
+
28
+</manifest>
sdk/kotlin/example/src/main/kotlin/com/ondeinference/example/ChatViewModel.kt
new
+173
@@ -0,0 +1,173 @@
1
+package com.ondeinference.example
2
+
3
+import android.app.Application
4
+import androidx.lifecycle.AndroidViewModel
5
+import androidx.lifecycle.viewModelScope
6
+import com.ondeinference.onde.OndeInference
7
+import com.ondeinference.onde.OndeSampling
8
+import kotlinx.coroutines.flow.MutableStateFlow
9
+import kotlinx.coroutines.flow.StateFlow
10
+import kotlinx.coroutines.flow.asStateFlow
11
+import kotlinx.coroutines.flow.catch
12
+import kotlinx.coroutines.flow.update
13
+import kotlinx.coroutines.launch
14
+
15
+// One message in the chat list.
16
+data class UiMessage(
17
+ val id: Long = System.nanoTime(), // nanoTime as a stable list key — collisions are basically impossible
18
+ val content: String,
19
+ val isUser: Boolean,
20
+ val isStreaming: Boolean = false, // true while the model is still typing
21
+)
22
+
23
+// Where the model is in its lifecycle.
24
+sealed class ModelState {
25
+ object Unloaded : ModelState()
26
+ object Loading : ModelState()
27
+ data class Ready(val name: String, val loadTimeSeconds: Double) : ModelState()
28
+ data class Error(val cause: String) : ModelState()
29
+}
30
+
31
+// Everything the UI needs, in one place. Compose observes this and re-renders when it changes.
32
+data class ChatUiState(
33
+ val modelState: ModelState = ModelState.Unloaded,
34
+ val messages: List<UiMessage> = emptyList(),
35
+ val isGenerating: Boolean = false, // blocks the send button while a reply is in flight
36
+ val input: String = "",
37
+)
38
+
39
+class ChatViewModel(app: Application) : AndroidViewModel(app) {
40
+
41
+ // The Rust engine, wrapped by the Onde SDK. We pass applicationContext to avoid leaking the Activity.
42
+ private val onde = OndeInference(app.applicationContext)
43
+
44
+ private val _uiState = MutableStateFlow(ChatUiState())
45
+ val uiState: StateFlow<ChatUiState> = _uiState.asStateFlow()
46
+
47
+ // Pull down the model and get it ready to chat.
48
+ // On first launch this downloads ~941 MB from HuggingFace; after that it loads straight from cache.
49
+ fun loadModel() {
50
+ viewModelScope.launch {
51
+ _uiState.update { it.copy(modelState = ModelState.Loading) }
52
+
53
+ runCatching {
54
+ onde.loadDefaultModel(
55
+ systemPrompt = "You are a helpful, concise assistant " +
56
+ "running entirely on-device. No data leaves this phone.",
57
+ sampling = OndeSampling.mobile(), // shorter replies feel snappier on mobile
58
+ )
59
+ }.onSuccess { elapsedSeconds ->
60
+ val info = onde.info()
61
+ _uiState.update {
62
+ it.copy(
63
+ modelState = ModelState.Ready(
64
+ name = info.modelName ?: "Qwen 2.5 1.5B",
65
+ loadTimeSeconds = elapsedSeconds,
66
+ ),
67
+ )
68
+ }
69
+ }.onFailure { error ->
70
+ _uiState.update {
71
+ it.copy(
72
+ modelState = ModelState.Error(
73
+ error.message ?: "Something went wrong — check logcat for details",
74
+ ),
75
+ )
76
+ }
77
+ }
78
+ }
79
+ }
80
+
81
+ // Send the current input and stream the reply back token by token.
82
+ fun sendMessage() {
83
+ val text = _uiState.value.input.trim()
84
+ if (text.isBlank() || _uiState.value.isGenerating) return
85
+
86
+ // Show the user's message right away so the UI feels snappy.
87
+ _uiState.update {
88
+ it.copy(
89
+ messages = it.messages + UiMessage(content = text, isUser = true),
90
+ input = "",
91
+ isGenerating = true,
92
+ )
93
+ }
94
+
95
+ viewModelScope.launch {
96
+ // Drop in an empty assistant bubble and fill it in as tokens arrive,
97
+ // rather than appending a new item on every chunk.
98
+ val placeholderId = System.nanoTime()
99
+ _uiState.update {
100
+ it.copy(
101
+ messages = it.messages + UiMessage(
102
+ id = placeholderId,
103
+ content = "",
104
+ isUser = false,
105
+ isStreaming = true,
106
+ ),
107
+ )
108
+ }
109
+
110
+ val buffer = StringBuilder()
111
+
112
+ onde.stream(text)
113
+ .catch { error ->
114
+ // Something broke mid-stream — show the error inside the bubble instead of crashing.
115
+ _uiState.update { state ->
116
+ state.copy(
117
+ messages = state.messages.replaceLast(
118
+ UiMessage(
119
+ id = placeholderId,
120
+ content = "⚠ ${error.message ?: "Inference failed"}",
121
+ isUser = false,
122
+ isStreaming = false,
123
+ ),
124
+ ),
125
+ isGenerating = false,
126
+ )
127
+ }
128
+ }
129
+ .collect { chunk ->
130
+ buffer.append(chunk.delta)
131
+ _uiState.update { state ->
132
+ state.copy(
133
+ messages = state.messages.replaceLast(
134
+ UiMessage(
135
+ id = placeholderId,
136
+ content = buffer.toString(),
137
+ isUser = false,
138
+ isStreaming = !chunk.done,
139
+ ),
140
+ ),
141
+ isGenerating = !chunk.done,
142
+ )
143
+ }
144
+ }
145
+ }
146
+ }
147
+
148
+ fun updateInput(text: String) {
149
+ _uiState.update { it.copy(input = text) }
150
+ }
151
+
152
+ // Wipe the screen and reset the model's memory of the conversation.
153
+ // The model stays loaded — no re-download needed.
154
+ fun clearChat() {
155
+ viewModelScope.launch {
156
+ onde.clearHistory()
157
+ _uiState.update { it.copy(messages = emptyList()) }
158
+ }
159
+ }
160
+
161
+ // Clean up when the screen is gone for good.
162
+ override fun onCleared() {
163
+ super.onCleared()
164
+ viewModelScope.launch {
165
+ onde.unload()
166
+ onde.close()
167
+ }
168
+ }
169
+
170
+ // Swap the last item in the list in-place — keeps the streaming bubble updating smoothly.
171
+ private fun List<UiMessage>.replaceLast(replacement: UiMessage): List<UiMessage> =
172
+ if (isEmpty()) listOf(replacement) else dropLast(1) + replacement
173
+}
sdk/kotlin/example/src/main/kotlin/com/ondeinference/example/MainActivity.kt
new
+24
@@ -0,0 +1,24 @@
1
+package com.ondeinference.example
2
+
3
+import android.os.Bundle
4
+import androidx.activity.ComponentActivity
5
+import androidx.activity.compose.setContent
6
+import androidx.activity.enableEdgeToEdge
7
+import com.ondeinference.example.ui.ChatScreen
8
+import com.ondeinference.example.ui.theme.OndeExampleTheme
9
+
10
+class MainActivity : ComponentActivity() {
11
+
12
+ override fun onCreate(savedInstanceState: Bundle?) {
13
+ super.onCreate(savedInstanceState)
14
+
15
+ // Draw behind the status and nav bars so the layout owns the full screen.
16
+ enableEdgeToEdge()
17
+
18
+ setContent {
19
+ OndeExampleTheme {
20
+ ChatScreen()
21
+ }
22
+ }
23
+ }
24
+}
sdk/kotlin/example/src/main/kotlin/com/ondeinference/example/ui/ChatScreen.kt
new
+405
@@ -0,0 +1,405 @@
1
+// The main screen. One state object drives everything — no local state beyond scroll position.
2
+
3
+package com.ondeinference.example.ui
4
+
5
+import androidx.compose.animation.AnimatedContent
6
+import androidx.compose.animation.core.LinearEasing
7
+import androidx.compose.animation.core.RepeatMode
8
+import androidx.compose.animation.core.animateFloat
9
+import androidx.compose.animation.core.infiniteRepeatable
10
+import androidx.compose.animation.core.rememberInfiniteTransition
11
+import androidx.compose.animation.core.tween
12
+import androidx.compose.animation.fadeIn
13
+import androidx.compose.animation.fadeOut
14
+import androidx.compose.animation.togetherWith
15
+import androidx.compose.foundation.layout.Arrangement
16
+import androidx.compose.foundation.layout.Box
17
+import androidx.compose.foundation.layout.Column
18
+import androidx.compose.foundation.layout.PaddingValues
19
+import androidx.compose.foundation.layout.Row
20
+import androidx.compose.foundation.layout.Spacer
21
+import androidx.compose.foundation.layout.fillMaxSize
22
+import androidx.compose.foundation.layout.fillMaxWidth
23
+import androidx.compose.foundation.layout.height
24
+import androidx.compose.foundation.layout.imePadding
25
+import androidx.compose.foundation.layout.navigationBarsPadding
26
+import androidx.compose.foundation.layout.padding
27
+import androidx.compose.foundation.layout.size
28
+import androidx.compose.foundation.layout.width
29
+import androidx.compose.foundation.layout.widthIn
30
+import androidx.compose.foundation.lazy.LazyColumn
31
+import androidx.compose.foundation.lazy.items
32
+import androidx.compose.foundation.lazy.rememberLazyListState
33
+import androidx.compose.foundation.shape.CircleShape
34
+import androidx.compose.foundation.shape.RoundedCornerShape
35
+import androidx.compose.foundation.text.KeyboardActions
36
+import androidx.compose.foundation.text.KeyboardOptions
37
+import androidx.compose.material.icons.Icons
38
+import androidx.compose.material.icons.automirrored.filled.Send
39
+import androidx.compose.material.icons.filled.Delete
40
+import androidx.compose.material3.Button
41
+import androidx.compose.material3.CircularProgressIndicator
42
+import androidx.compose.material3.ExperimentalMaterial3Api
43
+import androidx.compose.material3.FilledIconButton
44
+import androidx.compose.material3.Icon
45
+import androidx.compose.material3.IconButton
46
+import androidx.compose.material3.LinearProgressIndicator
47
+import androidx.compose.material3.MaterialTheme
48
+import androidx.compose.material3.OutlinedTextField
49
+import androidx.compose.material3.Scaffold
50
+import androidx.compose.material3.Surface
51
+import androidx.compose.material3.Text
52
+import androidx.compose.material3.TextButton
53
+import androidx.compose.material3.TopAppBar
54
+import androidx.compose.material3.TopAppBarDefaults
55
+import androidx.compose.runtime.Composable
56
+import androidx.compose.runtime.LaunchedEffect
57
+import androidx.compose.runtime.getValue
58
+import androidx.compose.ui.Alignment
59
+import androidx.compose.ui.Modifier
60
+import androidx.compose.ui.draw.alpha
61
+import androidx.compose.ui.text.font.FontWeight
62
+import androidx.compose.ui.text.input.ImeAction
63
+import androidx.compose.ui.unit.dp
64
+import androidx.lifecycle.compose.collectAsStateWithLifecycle
65
+import androidx.lifecycle.viewmodel.compose.viewModel
66
+import com.ondeinference.example.ChatViewModel
67
+import com.ondeinference.example.ModelState
68
+import com.ondeinference.example.UiMessage
69
+
70
+@OptIn(ExperimentalMaterial3Api::class)
71
+@Composable
72
+fun ChatScreen(viewModel: ChatViewModel = viewModel()) {
73
+ val state by viewModel.uiState.collectAsStateWithLifecycle()
74
+ val listState = rememberLazyListState()
75
+
76
+ // Jump to the latest message whenever a new one arrives.
77
+ LaunchedEffect(state.messages.size) {
78
+ if (state.messages.isNotEmpty()) {
79
+ listState.animateScrollToItem(state.messages.lastIndex)
80
+ }
81
+ }
82
+
83
+ Scaffold(
84
+ topBar = {
85
+ TopAppBar(
86
+ title = {
87
+ Column {
88
+ Text(
89
+ text = "Onde Chat",
90
+ style = MaterialTheme.typography.titleMedium,
91
+ fontWeight = FontWeight.SemiBold,
92
+ )
93
+ Text(
94
+ text = "On-device LLM — no cloud, no API key",
95
+ style = MaterialTheme.typography.labelSmall,
96
+ color = MaterialTheme.colorScheme.onPrimaryContainer.copy(alpha = 0.7f),
97
+ )
98
+ }
99
+ },
100
+ actions = {
101
+ // Trash icon only appears once there's something to clear.
102
+ if (state.messages.isNotEmpty()) {
103
+ IconButton(onClick = viewModel::clearChat) {
104
+ Icon(
105
+ imageVector = Icons.Default.Delete,
106
+ contentDescription = "Clear conversation",
107
+ )
108
+ }
109
+ }
110
+ },
111
+ colors = TopAppBarDefaults.topAppBarColors(
112
+ containerColor = MaterialTheme.colorScheme.primaryContainer,
113
+ titleContentColor = MaterialTheme.colorScheme.onPrimaryContainer,
114
+ actionIconContentColor = MaterialTheme.colorScheme.onPrimaryContainer,
115
+ ),
116
+ )
117
+ },
118
+ bottomBar = {
119
+ InputBar(
120
+ text = state.input,
121
+ onTextChange = viewModel::updateInput,
122
+ onSend = viewModel::sendMessage,
123
+ canSend = state.modelState is ModelState.Ready && !state.isGenerating,
124
+ )
125
+ },
126
+ ) { innerPadding ->
127
+ Column(
128
+ modifier = Modifier
129
+ .fillMaxSize()
130
+ .padding(innerPadding),
131
+ ) {
132
+ ModelStatusBanner(
133
+ state = state.modelState,
134
+ onLoad = viewModel::loadModel,
135
+ )
136
+
137
+ if (state.messages.isEmpty()) {
138
+ EmptyState(
139
+ isReady = state.modelState is ModelState.Ready,
140
+ modifier = Modifier.weight(1f),
141
+ )
142
+ } else {
143
+ LazyColumn(
144
+ state = listState,
145
+ modifier = Modifier.weight(1f),
146
+ contentPadding = PaddingValues(horizontal = 16.dp, vertical = 12.dp),
147
+ verticalArrangement = Arrangement.spacedBy(12.dp),
148
+ ) {
149
+ items(state.messages, key = { it.id }) { message ->
150
+ MessageBubble(message = message)
151
+ }
152
+ }
153
+ }
154
+ }
155
+ }
156
+}
157
+
158
+// Coloured strip below the top bar that crossfades between load states.
159
+@Composable
160
+private fun ModelStatusBanner(state: ModelState, onLoad: () -> Unit) {
161
+ AnimatedContent(
162
+ targetState = state,
163
+ transitionSpec = {
164
+ fadeIn(animationSpec = tween(durationMillis = 250)) togetherWith
165
+ fadeOut(animationSpec = tween(durationMillis = 250))
166
+ },
167
+ label = "model_status_banner",
168
+ ) { currentState ->
169
+ val bannerColor = when (currentState) {
170
+ is ModelState.Unloaded -> MaterialTheme.colorScheme.surfaceVariant
171
+ is ModelState.Loading -> MaterialTheme.colorScheme.tertiaryContainer
172
+ is ModelState.Ready -> MaterialTheme.colorScheme.secondaryContainer
173
+ is ModelState.Error -> MaterialTheme.colorScheme.errorContainer
174
+ }
175
+
176
+ Surface(color = bannerColor, modifier = Modifier.fillMaxWidth()) {
177
+ Row(
178
+ modifier = Modifier
179
+ .padding(horizontal = 16.dp, vertical = 10.dp)
180
+ .fillMaxWidth(),
181
+ verticalAlignment = Alignment.CenterVertically,
182
+ horizontalArrangement = Arrangement.SpaceBetween,
183
+ ) {
184
+ when (currentState) {
185
+ is ModelState.Unloaded -> {
186
+ Text(
187
+ text = "Tap Load to download & run the model on-device",
188
+ style = MaterialTheme.typography.bodySmall,
189
+ color = MaterialTheme.colorScheme.onSurfaceVariant,
190
+ modifier = Modifier.weight(1f),
191
+ )
192
+ Spacer(modifier = Modifier.width(8.dp))
193
+ Button(onClick = onLoad) {
194
+ Text("Load Model")
195
+ }
196
+ }
197
+
198
+ is ModelState.Loading -> {
199
+ Text(
200
+ text = "Downloading model (~941 MB on first run)…",
201
+ style = MaterialTheme.typography.bodySmall,
202
+ modifier = Modifier.weight(1f),
203
+ )
204
+ CircularProgressIndicator(
205
+ modifier = Modifier.size(20.dp),
206
+ strokeWidth = 2.dp,
207
+ )
208
+ }
209
+
210
+ is ModelState.Ready -> {
211
+ Column(modifier = Modifier.weight(1f)) {
212
+ Text(
213
+ text = currentState.name,
214
+ style = MaterialTheme.typography.labelMedium,
215
+ fontWeight = FontWeight.SemiBold,
216
+ color = MaterialTheme.colorScheme.onSecondaryContainer,
217
+ )
218
+ Text(
219
+ text = "Ready · loaded in ${"%.1f".format(currentState.loadTimeSeconds)}s · 100% on-device",
220
+ style = MaterialTheme.typography.labelSmall,
221
+ color = MaterialTheme.colorScheme.onSecondaryContainer.copy(alpha = 0.75f),
222
+ )
223
+ }
224
+ }
225
+
226
+ is ModelState.Error -> {
227
+ Text(
228
+ text = currentState.cause,
229
+ style = MaterialTheme.typography.bodySmall,
230
+ color = MaterialTheme.colorScheme.onErrorContainer,
231
+ modifier = Modifier.weight(1f),
232
+ )
233
+ Spacer(modifier = Modifier.width(8.dp))
234
+ TextButton(onClick = onLoad) {
235
+ Text(
236
+ "Retry",
237
+ color = MaterialTheme.colorScheme.onErrorContainer,
238
+ )
239
+ }
240
+ }
241
+ }
242
+ }
243
+ }
244
+ }
245
+}
246
+
247
+@Composable
248
+private fun EmptyState(isReady: Boolean, modifier: Modifier = Modifier) {
249
+ Box(
250
+ modifier = modifier.fillMaxWidth(),
251
+ contentAlignment = Alignment.Center,
252
+ ) {
253
+ Column(horizontalAlignment = Alignment.CenterHorizontally) {
254
+ Text(
255
+ text = if (isReady) "🤖" else "📦",
256
+ style = MaterialTheme.typography.displayLarge,
257
+ )
258
+ Spacer(modifier = Modifier.height(16.dp))
259
+ Text(
260
+ text = if (isReady) "Model ready — start chatting!" else "Load a model to begin",
261
+ style = MaterialTheme.typography.bodyLarge,
262
+ color = MaterialTheme.colorScheme.onSurfaceVariant,
263
+ )
264
+ if (!isReady) {
265
+ Spacer(modifier = Modifier.height(4.dp))
266
+ Text(
267
+ text = "Runs 100% on-device · no data leaves your phone",
268
+ style = MaterialTheme.typography.bodySmall,
269
+ color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.6f),
270
+ )
271
+ }
272
+ }
273
+ }
274
+}
275
+
276
+// Your messages go right; the AI's go left with a small avatar badge.
277
+// A pulsing bar under the bubble shows when the model is still typing.
278
+@Composable
279
+private fun MessageBubble(message: UiMessage) {
280
+ val isUser = message.isUser
281
+
282
+ Row(
283
+ modifier = Modifier.fillMaxWidth(),
284
+ horizontalArrangement = if (isUser) Arrangement.End else Arrangement.Start,
285
+ verticalAlignment = Alignment.Bottom,
286
+ ) {
287
+ if (!isUser) {
288
+ Surface(
289
+ shape = CircleShape,
290
+ color = MaterialTheme.colorScheme.primary,
291
+ modifier = Modifier.size(28.dp),
292
+ ) {
293
+ Box(contentAlignment = Alignment.Center) {
294
+ Text(
295
+ text = "AI",
296
+ style = MaterialTheme.typography.labelSmall,
297
+ fontWeight = FontWeight.ExtraBold,
298
+ color = MaterialTheme.colorScheme.onPrimary,
299
+ )
300
+ }
301
+ }
302
+ Spacer(modifier = Modifier.width(8.dp))
303
+ }
304
+
305
+ Column(modifier = Modifier.widthIn(max = 300.dp)) {
306
+ Surface(
307
+ shape = RoundedCornerShape(
308
+ topStart = 18.dp,
309
+ topEnd = 18.dp,
310
+ bottomStart = if (isUser) 18.dp else 4.dp,
311
+ bottomEnd = if (isUser) 4.dp else 18.dp,
312
+ ),
313
+ color = if (isUser) MaterialTheme.colorScheme.primary
314
+ else MaterialTheme.colorScheme.surfaceVariant,
315
+ shadowElevation = 1.dp,
316
+ ) {
317
+ Text(
318
+ // Non-breaking space gives the bubble height before the first token arrives.
319
+ text = message.content.ifEmpty { "\u00A0" },
320
+ modifier = Modifier.padding(horizontal = 14.dp, vertical = 10.dp),
321
+ style = MaterialTheme.typography.bodyMedium,
322
+ color = if (isUser) MaterialTheme.colorScheme.onPrimary
323
+ else MaterialTheme.colorScheme.onSurfaceVariant,
324
+ )
325
+ }
326
+
327
+ if (message.isStreaming) {
328
+ StreamingIndicator(modifier = Modifier.fillMaxWidth().padding(top = 2.dp))
329
+ }
330
+ }
331
+ }
332
+}
333
+
334
+// Thin progress bar that pulses while tokens are coming in.
335
+@Composable
336
+private fun StreamingIndicator(modifier: Modifier = Modifier) {
337
+ val infiniteTransition = rememberInfiniteTransition(label = "streaming_pulse")
338
+ val alpha by infiniteTransition.animateFloat(
339
+ initialValue = 0.4f,
340
+ targetValue = 1.0f,
341
+ animationSpec = infiniteRepeatable(
342
+ animation = tween(durationMillis = 600, easing = LinearEasing),
343
+ repeatMode = RepeatMode.Reverse,
344
+ ),
345
+ label = "streaming_alpha",
346
+ )
347
+ LinearProgressIndicator(
348
+ modifier = modifier
349
+ .height(2.dp)
350
+ .alpha(alpha),
351
+ )
352
+}
353
+
354
+// Sticks to the bottom and slides up with the keyboard.
355
+@Composable
356
+private fun InputBar(
357
+ text: String,
358
+ onTextChange: (String) -> Unit,
359
+ onSend: () -> Unit,
360
+ canSend: Boolean,
361
+) {
362
+ Surface(
363
+ shadowElevation = 8.dp,
364
+ color = MaterialTheme.colorScheme.surface,
365
+ ) {
366
+ Row(
367
+ modifier = Modifier
368
+ .fillMaxWidth()
369
+ .padding(horizontal = 12.dp, vertical = 8.dp)
370
+ .navigationBarsPadding()
371
+ .imePadding(),
372
+ verticalAlignment = Alignment.Bottom,
373
+ ) {
374
+ OutlinedTextField(
375
+ value = text,
376
+ onValueChange = onTextChange,
377
+ modifier = Modifier.weight(1f),
378
+ placeholder = {
379
+ Text(
380
+ text = if (canSend) "Ask anything…" else "Load a model to chat",
381
+ style = MaterialTheme.typography.bodyMedium,
382
+ )
383
+ },
384
+ enabled = canSend,
385
+ maxLines = 5,
386
+ keyboardOptions = KeyboardOptions(imeAction = ImeAction.Send),
387
+ keyboardActions = KeyboardActions(
388
+ onSend = { if (text.isNotBlank()) onSend() },
389
+ ),
390
+ shape = RoundedCornerShape(24.dp),
391
+ )
392
+ Spacer(modifier = Modifier.width(8.dp))
393
+ FilledIconButton(
394
+ onClick = onSend,
395
+ enabled = canSend && text.isNotBlank(),
396
+ modifier = Modifier.size(52.dp),
397
+ ) {
398
+ Icon(
399
+ imageVector = Icons.AutoMirrored.Filled.Send,
400
+ contentDescription = "Send message",
401
+ )
402
+ }
403
+ }
404
+ }
405
+}
sdk/kotlin/example/src/main/kotlin/com/ondeinference/example/ui/theme/Theme.kt
new
+86
@@ -0,0 +1,86 @@
1
+package com.ondeinference.example.ui.theme
2
+
3
+import android.os.Build
4
+import androidx.compose.foundation.isSystemInDarkTheme
5
+import androidx.compose.material3.MaterialTheme
6
+import androidx.compose.material3.darkColorScheme
7
+import androidx.compose.material3.dynamicDarkColorScheme
8
+import androidx.compose.material3.dynamicLightColorScheme
9
+import androidx.compose.material3.lightColorScheme
10
+import androidx.compose.runtime.Composable
11
+import androidx.compose.ui.graphics.Color
12
+import androidx.compose.ui.platform.LocalContext
13
+
14
+// On Android 12+ we pick up the user's wallpaper colours automatically.
15
+// On older devices we fall back to this hand-picked teal palette.
16
+
17
+private val FallbackLightScheme = lightColorScheme(
18
+ primary = Color(0xFF006874),
19
+ onPrimary = Color(0xFFFFFFFF),
20
+ primaryContainer = Color(0xFF97F0FF),
21
+ onPrimaryContainer = Color(0xFF001F24),
22
+ secondary = Color(0xFF4A6267),
23
+ onSecondary = Color(0xFFFFFFFF),
24
+ secondaryContainer = Color(0xFFCDE7EC),
25
+ onSecondaryContainer = Color(0xFF051F23),
26
+ tertiary = Color(0xFF525E7D),
27
+ onTertiary = Color(0xFFFFFFFF),
28
+ tertiaryContainer = Color(0xFFD9E2FF),
29
+ onTertiaryContainer = Color(0xFF0E1B37),
30
+ error = Color(0xFFBA1A1A),
31
+ onError = Color(0xFFFFFFFF),
32
+ errorContainer = Color(0xFFFFDAD6),
33
+ onErrorContainer = Color(0xFF410002),
34
+ background = Color(0xFFFAFDFD),
35
+ onBackground = Color(0xFF191C1D),
36
+ surface = Color(0xFFFAFDFD),
37
+ onSurface = Color(0xFF191C1D),
38
+ surfaceVariant = Color(0xFFDBE4E6),
39
+ onSurfaceVariant = Color(0xFF3F484A),
40
+)
41
+
42
+private val FallbackDarkScheme = darkColorScheme(
43
+ primary = Color(0xFF4FD8EB),
44
+ onPrimary = Color(0xFF00363D),
45
+ primaryContainer = Color(0xFF004F58),
46
+ onPrimaryContainer = Color(0xFF97F0FF),
47
+ secondary = Color(0xFFB1CBD0),
48
+ onSecondary = Color(0xFF1C3438),
49
+ secondaryContainer = Color(0xFF334B4F),
50
+ onSecondaryContainer = Color(0xFFCDE7EC),
51
+ tertiary = Color(0xFFBAC6E8),
52
+ onTertiary = Color(0xFF24304D),
53
+ tertiaryContainer = Color(0xFF3A4664),
54
+ onTertiaryContainer = Color(0xFFD9E2FF),
55
+ error = Color(0xFFFFB4AB),
56
+ onError = Color(0xFF690005),
57
+ errorContainer = Color(0xFF93000A),
58
+ onErrorContainer = Color(0xFFFFDAD6),
59
+ background = Color(0xFF191C1D),
60
+ onBackground = Color(0xFFE1E3E3),
61
+ surface = Color(0xFF191C1D),
62
+ onSurface = Color(0xFFE1E3E3),
63
+ surfaceVariant = Color(0xFF3F484A),
64
+ onSurfaceVariant = Color(0xFFBFC8CA),
65
+)
66
+
67
+@Composable
68
+fun OndeExampleTheme(
69
+ darkTheme: Boolean = isSystemInDarkTheme(),
70
+ content: @Composable () -> Unit,
71
+) {
72
+ val colorScheme = when {
73
+ Build.VERSION.SDK_INT >= Build.VERSION_CODES.S -> {
74
+ val context = LocalContext.current
75
+ if (darkTheme) dynamicDarkColorScheme(context)
76
+ else dynamicLightColorScheme(context)
77
+ }
78
+ darkTheme -> FallbackDarkScheme
79
+ else -> FallbackLightScheme
80
+ }
81
+
82
+ MaterialTheme(
83
+ colorScheme = colorScheme,
84
+ content = content,
85
+ )
86
+}
sdk/kotlin/example/src/main/res/values/strings.xml
new
+4
@@ -0,0 +1,4 @@
1
+<?xml version="1.0" encoding="utf-8"?>
2
+<resources>
3
+ <string name="app_name">Onde Chat</string>
4
+</resources>
sdk/kotlin/example/src/main/res/values/themes.xml
new
+10
@@ -0,0 +1,10 @@
1
+<?xml version="1.0" encoding="utf-8" ?>
2
+<resources>
3
+ <!-- Compose owns all theming at runtime via OndeExampleTheme.
4
+ This just gives the Activity a bare window with no action bar
5
+ while Compose sets up. No MDC or Material3 XML dep needed. -->
6
+ <style
7
+ name="Theme.OndeExample"
8
+ parent="android:Theme.Material.Light.NoActionBar"
9
+ />
10
+</resources>
sdk/kotlin/gradle.properties
new
+28
@@ -0,0 +1,28 @@
1
+# Project-wide Gradle settings
2
+
3
+# Android
4
+android.useAndroidX=true
5
+android.nonTransitiveRClass=true
6
+
7
+# Kotlin
8
+kotlin.code.style=official
9
+
10
+# Maven publishing coordinates — keep in sync with Cargo.toml [package] version
11
+GROUP=com.ondeinference
12
+POM_ARTIFACT_ID=onde-inference
13
+VERSION_NAME=0.1.3
14
+
15
+# Sonatype credentials — set in ~/.gradle/gradle.properties or via CI secrets
16
+# ossrhUsername=
17
+# ossrhPassword=
18
+
19
+# PGP signing — set in ~/.gradle/gradle.properties or via CI secrets
20
+# signingKeyId=
21
+# signingKey= (ASCII-armored private key)
22
+# signingPassword=
23
+
24
+# Performance
25
+org.gradle.caching=true
26
+org.gradle.parallel=true
27
+org.gradle.configureondemand=true
28
+org.gradle.jvmargs=-Xmx4g -XX:MaxMetaspaceSize=512m -XX:+HeapDumpOnOutOfMemoryError
sdk/kotlin/gradle/wrapper/gradle-wrapper.properties
new
+7
@@ -0,0 +1,7 @@
1
+distributionBase=GRADLE_USER_HOME
2
+distributionPath=wrapper/dists
3
+distributionUrl=https\://services.gradle.org/distributions/gradle-8.9-bin.zip
4
+networkTimeout=10000
5
+validateDistributionUrl=true
6
+zipStoreBase=GRADLE_USER_HOME
7
+zipStorePath=wrapper/dists
sdk/kotlin/lib/build.gradle.kts
new
+113
@@ -0,0 +1,113 @@
1
+import com.vanniktech.maven.publish.AndroidSingleVariantLibrary
2
+import com.vanniktech.maven.publish.SonatypeHost
3
+
4
+plugins {
5
+ id("com.android.library")
6
+ id("org.jetbrains.kotlin.android")
7
+ id("com.vanniktech.maven.publish")
8
+}
9
+
10
+android {
11
+ namespace = "com.ondeinference.onde"
12
+ compileSdk = 35
13
+
14
+ defaultConfig {
15
+ minSdk = 26 // Os.setenv needs API 26+
16
+ targetSdk = 35
17
+ consumerProguardFiles("consumer-rules.pro")
18
+ }
19
+
20
+ compileOptions {
21
+ sourceCompatibility = JavaVersion.VERSION_17
22
+ targetCompatibility = JavaVersion.VERSION_17
23
+ }
24
+
25
+ kotlinOptions {
26
+ jvmTarget = "17"
27
+ }
28
+
29
+ sourceSets {
30
+ getByName("main") {
31
+ jniLibs.srcDirs("src/main/jniLibs")
32
+ // UniFFI-generated onde.kt lands in src/generated/kotlin/ after
33
+ // running scripts/generate-bindings.sh — gitignored, regenerate after Rust API changes.
34
+ kotlin.srcDirs("src/main/kotlin", "src/generated/kotlin")
35
+ }
36
+ }
37
+
38
+ // No android { publishing { singleVariant(...) } } block here.
39
+ // Vanniktech's AndroidSingleVariantLibrary (below) registers it — adding
40
+ // it twice causes "singleVariant publishing DSL multiple times" error.
41
+}
42
+
43
+dependencies {
44
+ // JNA — the UniFFI-generated onde.kt uses com.sun.jna.* for its FFI bridge.
45
+ // The @aar suffix pulls the Android-specific build with native .so files included.
46
+ implementation("net.java.dev.jna:jna:5.14.0@aar")
47
+
48
+ // Coroutines — needed by the suspend funs and Flow streaming in OndeInference.kt
49
+ implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.8.1")
50
+}
51
+
52
+// Required secrets (in ~/.gradle/gradle.properties or CI env vars):
53
+// ORG_GRADLE_PROJECT_mavenCentralUsername
54
+// ORG_GRADLE_PROJECT_mavenCentralPassword
55
+// ORG_GRADLE_PROJECT_signingKeyId
56
+// ORG_GRADLE_PROJECT_signingKey (ASCII-armored PGP private key)
57
+// ORG_GRADLE_PROJECT_signingPassword
58
+mavenPublishing {
59
+ // AndroidSingleVariantLibrary is the correct way to configure variant + jars
60
+ // for an Android library with Vanniktech 0.25+. It calls singleVariant("release")
61
+ // internally, so the android { publishing { } } block must NOT be used alongside it.
62
+ configure(
63
+ AndroidSingleVariantLibrary(
64
+ variant = "release",
65
+ sourcesJar = true,
66
+ publishJavadocJar = true,
67
+ )
68
+ )
69
+
70
+ publishToMavenCentral(SonatypeHost.CENTRAL_PORTAL)
71
+ signAllPublications()
72
+
73
+ coordinates(
74
+ groupId = "com.ondeinference",
75
+ artifactId = "onde-inference",
76
+ version = project.property("VERSION_NAME") as String,
77
+ )
78
+
79
+ pom {
80
+ name.set("Onde Inference")
81
+ description.set(
82
+ "On-device LLM inference for Android. " +
83
+ "Run Qwen 2.5 models locally — no cloud, no API key, no data leaving the device."
84
+ )
85
+ url.set("https://ondeinference.com")
86
+ inceptionYear.set("2025")
87
+
88
+ licenses {
89
+ license {
90
+ name.set("MIT OR Apache-2.0")
91
+ url.set("https://github.com/ondeinference/onde/blob/main/LICENSE")
92
+ distribution.set("repo")
93
+ }
94
+ }
95
+
96
+ developers {
97
+ developer {
98
+ id.set("ondeinference")
99
+ name.set("Onde Inference")
100
+ email.set("hello@ondeinference.com")
101
+ url.set("https://ondeinference.com")
102
+ organization.set("Splitfire AB")
103
+ organizationUrl.set("https://splitfire.se")
104
+ }
105
+ }
106
+
107
+ scm {
108
+ url.set("https://github.com/ondeinference/onde")
109
+ connection.set("scm:git:github.com/ondeinference/onde.git")
110
+ developerConnection.set("scm:git:ssh://github.com/ondeinference/onde.git")
111
+ }
112
+ }
113
+}
sdk/kotlin/lib/consumer-rules.pro
new
+29
@@ -0,0 +1,29 @@
1
+# Onde Inference — ProGuard consumer rules
2
+#
3
+# These rules are applied to any app that depends on the Onde Inference AAR.
4
+# They preserve the UniFFI JNI bridge symbols that the Rust native library
5
+# resolves at runtime via System.loadLibrary("onde").
6
+
7
+# Keep all UniFFI-generated classes and their native method declarations
8
+-keep class uniffi.onde.** { *; }
9
+
10
+# Keep the OndeInference public API and companion objects
11
+-keep class com.ondeinference.onde.** { *; }
12
+
13
+# Keep the StreamChunkListener callback interface (called from Rust)
14
+-keep interface uniffi.onde.StreamChunkListener { *; }
15
+
16
+# Prevent stripping of native method registrations
17
+-keepclasseswithmembernames class * {
18
+ native <methods>;
19
+}
20
+
21
+# Keep Kotlin coroutine internal classes referenced by the Flow streaming API
22
+-keepnames class kotlinx.coroutines.internal.MainDispatcherFactory {}
23
+-keepnames class kotlinx.coroutines.CoroutineExceptionHandler {}
24
+
25
+# Prevent R8 from removing the load-library static initializer in the
26
+# UniFFI-generated onde.kt (it calls System.loadLibrary in a companion object)
27
+-keepclassmembers class uniffi.onde.* {
28
+ static <clinit>();
29
+}
sdk/kotlin/lib/src/main/AndroidManifest.xml
new
+7
@@ -0,0 +1,7 @@
1
+<?xml version="1.0" encoding="utf-8"?>
2
+<manifest xmlns:android="http://schemas.android.com/apk/res/android">
3
+ <!-- Onde Inference Android SDK
4
+ Library manifest — no activities, services, or permissions are declared
5
+ here. The host application must declare INTERNET permission to allow
6
+ model downloads from Hugging Face Hub. -->
7
+</manifest>
sdk/kotlin/lib/src/main/kotlin/com/ondeinference/onde/OndeInference.kt
new
+399
@@ -0,0 +1,399 @@
1
+// Copyright 2025 Onde Inference (Splitfire AB). All rights reserved.
2
+// SPDX-License-Identifier: MIT OR Apache-2.0
3
+//
4
+// OndeInference.kt
5
+//
6
+// Android-idiomatic wrapper around the UniFFI-generated OndeChatEngine.
7
+//
8
+// This file is the primary public API surface for the Onde Inference Android
9
+// SDK. It wraps the low-level UniFFI bindings (generated by
10
+// scripts/generate-bindings.sh into src/generated/kotlin/onde.kt) with:
11
+//
12
+// 1. Android-specific filesystem initialisation (HF_HOME, TMPDIR, etc.)
13
+// 2. suspend functions backed by Dispatchers.IO
14
+// 3. Flow<StreamChunk> for token streaming
15
+// 4. Sensible Kotlin defaults so callers need minimal boilerplate
16
+//
17
+// Usage
18
+// -----
19
+// val onde = OndeInference(context)
20
+//
21
+// // Load the platform default model (Qwen 2.5 1.5B on Android)
22
+// val elapsed = onde.loadDefaultModel(systemPrompt = "You are helpful.")
23
+// println("Loaded in ${elapsed}s")
24
+//
25
+// // Multi-turn chat
26
+// val result = onde.chat("Hello!")
27
+// println(result.text)
28
+//
29
+// // Streaming
30
+// onde.stream("Tell me a story.").collect { chunk ->
31
+// print(chunk.delta)
32
+// }
33
+//
34
+// // Cleanup
35
+// onde.unload()
36
+
37
+package com.ondeinference.onde
38
+
39
+import android.content.Context
40
+import android.system.Os
41
+import kotlinx.coroutines.Dispatchers
42
+import kotlinx.coroutines.channels.awaitClose
43
+import kotlinx.coroutines.flow.Flow
44
+import kotlinx.coroutines.flow.callbackFlow
45
+import kotlinx.coroutines.withContext
46
+import java.io.File
47
+
48
+// ── Public re-exports ─────────────────────────────────────────────────────────
49
+//
50
+// Re-export UniFFI-generated types so callers only need to import from
51
+// com.ondeinference.onde.* — no direct dependency on onde.kt generated types.
52
+
53
+// These are generated by UniFFI from the Rust crate and land in
54
+// src/generated/kotlin/onde.kt. They are re-exported here as typealiases
55
+// so the public API namespace stays clean.
56
+typealias ChatMessage = uniffi.onde.ChatMessage
57
+typealias ChatRole = uniffi.onde.ChatRole
58
+typealias SamplingConfig = uniffi.onde.SamplingConfig
59
+typealias GgufModelConfig = uniffi.onde.GgufModelConfig
60
+typealias InferenceResult = uniffi.onde.InferenceResult
61
+typealias StreamChunk = uniffi.onde.StreamChunk
62
+typealias EngineInfo = uniffi.onde.EngineInfo
63
+typealias EngineStatus = uniffi.onde.EngineStatus
64
+typealias InferenceError = uniffi.onde.InferenceException
65
+
66
+// ── OndeInference ─────────────────────────────────────────────────────────────
67
+
68
+/**
69
+ * On-device LLM inference engine for Android.
70
+ *
71
+ * Wraps the Rust/UniFFI [uniffi.onde.OndeChatEngine] with Android-specific
72
+ * filesystem initialisation and idiomatic Kotlin coroutine APIs.
73
+ *
74
+ * The engine is thread-safe. All suspending functions dispatch to
75
+ * [Dispatchers.IO] internally — call them from any coroutine scope.
76
+ *
77
+ * @param context Android application or activity context used to resolve
78
+ * the app's internal storage directory for the HuggingFace model cache.
79
+ * Pass `applicationContext` to avoid leaking activity references.
80
+ *
81
+ * @param dataDir Optional override for the root cache directory. Defaults to
82
+ * `context.filesDir`. Useful in tests or when the app wants to use a
83
+ * different storage location (e.g. external storage on devices with limited
84
+ * internal space).
85
+ */
86
+class OndeInference(
87
+ private val context: Context,
88
+ private val dataDir: File = context.filesDir,
89
+) : AutoCloseable {
90
+
91
+ // The underlying UniFFI engine — thread-safe Arc<OndeChatEngine> under the hood.
92
+ private val engine = uniffi.onde.OndeChatEngine()
93
+
94
+ // Tracks whether the Android filesystem environment has been seeded.
95
+ @Volatile private var configured = false
96
+
97
+ // ── Filesystem setup ──────────────────────────────────────────────────────
98
+
99
+ /**
100
+ * Seed the HuggingFace cache and temp directory environment variables.
101
+ *
102
+ * Must be called before any model load. [loadDefaultModel] and [loadModel]
103
+ * call this automatically, so explicit calls are only needed if you want
104
+ * to control timing (e.g. at app startup).
105
+ *
106
+ * Sets:
107
+ * - `HF_HOME` → `<dataDir>/models`
108
+ * - `HF_HUB_CACHE` → `<dataDir>/models/hub`
109
+ * - `HUGGINGFACE_HUB_CACHE`→ `<dataDir>/models/hub`
110
+ * - `TMPDIR` → `<dataDir>/tmp`
111
+ *
112
+ * This is idempotent — safe to call multiple times.
113
+ */
114
+ fun setup() {
115
+ if (configured) return
116
+
117
+ val hfHome = File(dataDir, "models").also { it.mkdirs() }
118
+ val hfHubCache = File(hfHome, "hub").also { it.mkdirs() }
119
+ val tmpDir = File(dataDir, "tmp").also { it.mkdirs() }
120
+
121
+ Os.setenv("HF_HOME", hfHome.absolutePath, true)
122
+ Os.setenv("HF_HUB_CACHE", hfHubCache.absolutePath, true)
123
+ Os.setenv("HUGGINGFACE_HUB_CACHE", hfHubCache.absolutePath, true)
124
+ Os.setenv("TMPDIR", tmpDir.absolutePath, true)
125
+
126
+ configured = true
127
+ }
128
+
129
+ /**
130
+ * Resolve the directory where HuggingFace model files are stored.
131
+ *
132
+ * Returns the path even if setup has not been called yet.
133
+ */
134
+ val modelCacheDir: File
135
+ get() = File(File(dataDir, "models"), "hub")
136
+
137
+ // ── Model lifecycle ───────────────────────────────────────────────────────
138
+
139
+ /**
140
+ * Load the platform-appropriate default model.
141
+ *
142
+ * On Android this is **Qwen 2.5 1.5B (GGUF Q4_K_M, ~941 MB)**,
143
+ * running on CPU via the Candle backend.
144
+ *
145
+ * If the model is not already cached locally it will be downloaded from
146
+ * HuggingFace Hub on first call — ensure the app has INTERNET permission
147
+ * and sufficient storage space (~1 GB free).
148
+ *
149
+ * @param systemPrompt Optional system prompt injected before the first turn.
150
+ * @param sampling Optional sampling configuration. Defaults to
151
+ * [SamplingConfig.defaultSamplingConfig] when null.
152
+ * @return Wall-clock model loading time in seconds.
153
+ * @throws InferenceError.ModelBuild if the model cannot be loaded.
154
+ */
155
+ suspend fun loadDefaultModel(
156
+ systemPrompt: String? = null,
157
+ sampling: SamplingConfig? = null,
158
+ ): Double = withContext(Dispatchers.IO) {
159
+ ensureConfigured()
160
+ engine.loadDefaultModel(
161
+ systemPrompt = systemPrompt,
162
+ sampling = sampling,
163
+ )
164
+ }
165
+
166
+ /**
167
+ * Load a specific GGUF model.
168
+ *
169
+ * @param config Model repository and filename configuration.
170
+ * @param systemPrompt Optional system prompt.
171
+ * @param sampling Optional sampling configuration.
172
+ * @return Wall-clock model loading time in seconds.
173
+ * @throws InferenceError.ModelBuild if the model cannot be loaded.
174
+ * @throws InferenceError.AlreadyLoaded if a model is already loaded —
175
+ * call [unload] first.
176
+ */
177
+ suspend fun loadModel(
178
+ config: GgufModelConfig,
179
+ systemPrompt: String? = null,
180
+ sampling: SamplingConfig? = null,
181
+ ): Double = withContext(Dispatchers.IO) {
182
+ ensureConfigured()
183
+ engine.loadGgufModel(
184
+ config = config,
185
+ systemPrompt = systemPrompt,
186
+ sampling = sampling,
187
+ )
188
+ }
189
+
190
+ /**
191
+ * Unload the current model and free all associated memory.
192
+ *
193
+ * @return The display name of the model that was unloaded, or null if no
194
+ * model was loaded.
195
+ */
196
+ suspend fun unload(): String? = withContext(Dispatchers.IO) {
197
+ engine.unloadModel()
198
+ }
199
+
200
+ /**
201
+ * Check whether a model is currently loaded and ready.
202
+ */
203
+ suspend fun isLoaded(): Boolean = withContext(Dispatchers.IO) {
204
+ engine.isLoaded()
205
+ }
206
+
207
+ // ── Inference ─────────────────────────────────────────────────────────────
208
+
209
+ /**
210
+ * Send a user message and receive a complete assistant reply.
211
+ *
212
+ * The user turn and assistant reply are automatically appended to the
213
+ * engine's conversation history.
214
+ *
215
+ * @param message The user's message text.
216
+ * @return The completed inference result including generated text and timing.
217
+ * @throws InferenceError.NoModelLoaded if no model has been loaded.
218
+ * @throws InferenceError.Inference if inference fails at runtime.
219
+ */
220
+ suspend fun chat(message: String): InferenceResult =
221
+ withContext(Dispatchers.IO) { engine.sendMessage(message) }
222
+
223
+ /**
224
+ * Stream a user message, emitting each token delta as it is generated.
225
+ *
226
+ * The user turn and assembled assistant reply are automatically appended
227
+ * to the conversation history when streaming completes.
228
+ *
229
+ * Collect the returned [Flow] from a coroutine scope tied to a UI
230
+ * lifecycle to ensure automatic cancellation:
231
+ *
232
+ * ```kotlin
233
+ * lifecycleScope.launch {
234
+ * onde.stream("Tell me a joke.").collect { chunk ->
235
+ * textView.append(chunk.delta)
236
+ * }
237
+ * }
238
+ * ```
239
+ *
240
+ * @param message The user's message text.
241
+ * @return A cold [Flow] that emits [StreamChunk] values including a final
242
+ * chunk with [StreamChunk.done] set to true.
243
+ */
244
+ fun stream(message: String): Flow<StreamChunk> = callbackFlow {
245
+ val listener = object : uniffi.onde.StreamChunkListener {
246
+ override fun onChunk(chunk: StreamChunk): Boolean {
247
+ val result = trySend(chunk)
248
+ // Stop if the downstream collector has cancelled.
249
+ return result.isSuccess && !chunk.done
250
+ }
251
+ }
252
+ withContext(Dispatchers.IO) {
253
+ uniffi.onde.streamChatMessage(
254
+ engine = engine,
255
+ message = message,
256
+ listener = listener,
257
+ )
258
+ }
259
+ awaitClose()
260
+ }
261
+
262
+ /**
263
+ * Run a one-shot inference on an explicit message list WITHOUT modifying
264
+ * the engine's conversation history.
265
+ *
266
+ * Useful for parallel prompt enhancement, summarisation, or any task where
267
+ * you want to keep the main chat history clean.
268
+ *
269
+ * @param messages Explicit list of messages to run inference on.
270
+ * @param sampling Optional sampling configuration override.
271
+ */
272
+ suspend fun generate(
273
+ messages: List<ChatMessage>,
274
+ sampling: SamplingConfig? = null,
275
+ ): InferenceResult = withContext(Dispatchers.IO) {
276
+ engine.generate(messages = messages, sampling = sampling)
277
+ }
278
+
279
+ // ── Engine state ──────────────────────────────────────────────────────────
280
+
281
+ /**
282
+ * Get a snapshot of the engine's current state.
283
+ *
284
+ * @return [EngineInfo] describing load status, model name, approximate
285
+ * memory, and conversation history length.
286
+ */
287
+ suspend fun info(): EngineInfo = withContext(Dispatchers.IO) { engine.info() }
288
+
289
+ // ── System prompt ─────────────────────────────────────────────────────────
290
+
291
+ /**
292
+ * Set or replace the system prompt without reloading the model.
293
+ *
294
+ * Takes effect on the next inference call.
295
+ */
296
+ suspend fun setSystemPrompt(prompt: String) = withContext(Dispatchers.IO) {
297
+ engine.setSystemPrompt(prompt)
298
+ }
299
+
300
+ /**
301
+ * Clear the system prompt.
302
+ */
303
+ suspend fun clearSystemPrompt() = withContext(Dispatchers.IO) {
304
+ engine.clearSystemPrompt()
305
+ }
306
+
307
+ // ── Sampling ──────────────────────────────────────────────────────────────
308
+
309
+ /**
310
+ * Replace the sampling configuration without reloading the model.
311
+ *
312
+ * Takes effect on the next inference call.
313
+ */
314
+ suspend fun setSampling(sampling: SamplingConfig) = withContext(Dispatchers.IO) {
315
+ engine.setSampling(sampling)
316
+ }
317
+
318
+ // ── Conversation history ──────────────────────────────────────────────────
319
+
320
+ /**
321
+ * Get a copy of the full conversation history.
322
+ */
323
+ suspend fun history(): List<ChatMessage> = withContext(Dispatchers.IO) {
324
+ engine.history()
325
+ }
326
+
327
+ /**
328
+ * Clear the conversation history while keeping the model loaded.
329
+ *
330
+ * @return The number of turns that were removed.
331
+ */
332
+ suspend fun clearHistory(): ULong = withContext(Dispatchers.IO) {
333
+ engine.clearHistory()
334
+ }
335
+
336
+ /**
337
+ * Append a message to the conversation history without running inference.
338
+ *
339
+ * Useful for injecting context or restoring a previous session.
340
+ */
341
+ suspend fun pushHistory(message: ChatMessage) = withContext(Dispatchers.IO) {
342
+ engine.pushHistory(message)
343
+ }
344
+
345
+ // ── AutoCloseable ─────────────────────────────────────────────────────────
346
+
347
+ /**
348
+ * Release all native resources held by this engine.
349
+ *
350
+ * The underlying Rust engine is reference-counted via [Arc] — this call
351
+ * decrements the count and frees memory when the count reaches zero.
352
+ * Safe to call multiple times.
353
+ */
354
+ override fun close() {
355
+ engine.destroy()
356
+ }
357
+
358
+ // ── Internal helpers ──────────────────────────────────────────────────────
359
+
360
+ private fun ensureConfigured() {
361
+ if (!configured) setup()
362
+ }
363
+}
364
+
365
+// ── Convenience constructors ──────────────────────────────────────────────────
366
+
367
+/**
368
+ * Construct preconfigured [SamplingConfig] instances without importing the
369
+ * generated uniffi package directly.
370
+ */
371
+object OndeSampling {
372
+ /** Creative chat defaults — temperature 0.7, top_p 0.95, max 512 tokens. */
373
+ fun default(): SamplingConfig = uniffi.onde.defaultSamplingConfig()
374
+ /** Greedy/deterministic — temperature 0.0, max 512 tokens. */
375
+ fun deterministic(): SamplingConfig = uniffi.onde.deterministicSamplingConfig()
376
+ /** Conservative mobile — temperature 0.7, max 128 tokens. */
377
+ fun mobile(): SamplingConfig = uniffi.onde.mobileSamplingConfig()
378
+}
379
+
380
+/**
381
+ * Construct [GgufModelConfig] instances for supported Onde models.
382
+ */
383
+object OndeModels {
384
+ /** Platform-appropriate default — Qwen 2.5 1.5B on Android (~941 MB). */
385
+ fun default(): GgufModelConfig = uniffi.onde.defaultModelConfig()
386
+ /** Qwen 2.5 1.5B Instruct GGUF Q4_K_M (~941 MB). */
387
+ fun qwen25_1_5b(): GgufModelConfig = uniffi.onde.qwen2515bConfig()
388
+ /** Qwen 2.5 3B Instruct GGUF Q4_K_M (~1.93 GB). */
389
+ fun qwen25_3b(): GgufModelConfig = uniffi.onde.qwen253bConfig()
390
+}
391
+
392
+/**
393
+ * Construct [ChatMessage] values without importing the generated package.
394
+ */
395
+object OndeMessage {
396
+ fun system(content: String): ChatMessage = uniffi.onde.systemMessage(content)
397
+ fun user(content: String): ChatMessage = uniffi.onde.userMessage(content)
398
+ fun assistant(content: String): ChatMessage = uniffi.onde.assistantMessage(content)
399
+}
sdk/kotlin/scripts/build-android.sh
new
+135
@@ -0,0 +1,135 @@
1
+#!/usr/bin/env bash
2
+# build-android.sh
3
+#
4
+# Builds the Onde Rust crate as an Android shared library (.so) for all ABIs
5
+# and copies the outputs into lib/src/main/jniLibs/ for the Gradle build.
6
+#
7
+# Why this script uses plain indexed arrays instead of associative arrays:
8
+# some environments still ship older Bash versions, and associative arrays can
9
+# fail there. Indexed arrays are much more portable.
10
+#
11
+# Prerequisites
12
+# -------------
13
+# 1. Rust Android targets installed:
14
+# rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android i686-linux-android
15
+#
16
+# 2. Android NDK installed and ANDROID_NDK_HOME set:
17
+# export ANDROID_NDK_HOME=$HOME/Library/Android/sdk/ndk/26.1.10909125
18
+#
19
+# 3. cargo-ndk installed:
20
+# cargo install cargo-ndk
21
+#
22
+# Usage
23
+# -----
24
+# ./scripts/build-android.sh
25
+# ./scripts/build-android.sh --debug
26
+# FILTER_ABI=arm64-v8a ./scripts/build-android.sh
27
+
28
+set -euo pipefail
29
+
30
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
31
+SDK_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
32
+REPO_ROOT="$(cd "$SDK_DIR/../.." && pwd)"
33
+JNILIBS_DIR="$SDK_DIR/lib/src/main/jniLibs"
34
+
35
+PROFILE="release"
36
+for arg in "$@"; do
37
+ if [[ "$arg" == "--debug" ]]; then
38
+ PROFILE="debug"
39
+ fi
40
+done
41
+
42
+# Keep these arrays in the same order.
43
+ABIS=(
44
+ "arm64-v8a"
45
+ "armeabi-v7a"
46
+ "x86_64"
47
+ "x86"
48
+)
49
+
50
+RUST_TARGETS=(
51
+ "aarch64-linux-android"
52
+ "armv7-linux-androideabi"
53
+ "x86_64-linux-android"
54
+ "i686-linux-android"
55
+)
56
+
57
+if [[ "${#ABIS[@]}" -ne "${#RUST_TARGETS[@]}" ]]; then
58
+ echo "Error: ABI and Rust target arrays are out of sync."
59
+ exit 1
60
+fi
61
+
62
+if [[ -z "${ANDROID_NDK_HOME:-}" ]]; then
63
+ echo "Error: ANDROID_NDK_HOME is not set."
64
+ echo "Example: export ANDROID_NDK_HOME=\$HOME/Library/Android/sdk/ndk/26.1.10909125"
65
+ exit 1
66
+fi
67
+
68
+if ! command -v cargo-ndk >/dev/null 2>&1; then
69
+ echo "Error: cargo-ndk not found."
70
+ echo "Install with: cargo install cargo-ndk"
71
+ exit 1
72
+fi
73
+
74
+SELECTED_ABIS=()
75
+SELECTED_TARGETS=()
76
+
77
+if [[ -n "${FILTER_ABI:-}" ]]; then
78
+ FOUND="false"
79
+ for i in "${!ABIS[@]}"; do
80
+ if [[ "${ABIS[$i]}" == "$FILTER_ABI" ]]; then
81
+ SELECTED_ABIS+=("${ABIS[$i]}")
82
+ SELECTED_TARGETS+=("${RUST_TARGETS[$i]}")
83
+ FOUND="true"
84
+ break
85
+ fi
86
+ done
87
+
88
+ if [[ "$FOUND" != "true" ]]; then
89
+ echo "Error: FILTER_ABI='$FILTER_ABI' is not supported."
90
+ echo "Supported ABIs: ${ABIS[*]}"
91
+ exit 1
92
+ fi
93
+else
94
+ SELECTED_ABIS=("${ABIS[@]}")
95
+ SELECTED_TARGETS=("${RUST_TARGETS[@]}")
96
+fi
97
+
98
+echo "=== Building Onde for Android (profile: $PROFILE) ==="
99
+echo "Repository root: $REPO_ROOT"
100
+echo "Output jniLibs: $JNILIBS_DIR"
101
+
102
+cd "$REPO_ROOT"
103
+
104
+CARGO_NDK_ARGS=(--manifest-path Cargo.toml)
105
+if [[ "$PROFILE" == "release" ]]; then
106
+ CARGO_NDK_ARGS+=(--release)
107
+fi
108
+
109
+ABI_FLAGS=()
110
+for abi in "${SELECTED_ABIS[@]}"; do
111
+ ABI_FLAGS+=(-t "$abi")
112
+done
113
+
114
+echo "→ cargo ndk ${ABI_FLAGS[*]} build ${CARGO_NDK_ARGS[*]}"
115
+cargo ndk "${ABI_FLAGS[@]}" build "${CARGO_NDK_ARGS[@]}"
116
+
117
+for i in "${!SELECTED_ABIS[@]}"; do
118
+ abi="${SELECTED_ABIS[$i]}"
119
+ rust_target="${SELECTED_TARGETS[$i]}"
120
+ src="$REPO_ROOT/target/$rust_target/$PROFILE/libonde.so"
121
+ dst="$JNILIBS_DIR/$abi/libonde.so"
122
+
123
+ if [[ ! -f "$src" ]]; then
124
+ echo "Error: expected output not found: $src"
125
+ exit 1
126
+ fi
127
+
128
+ mkdir -p "$(dirname "$dst")"
129
+ cp "$src" "$dst"
130
+ echo "✓ $abi -> $dst"
131
+done
132
+
133
+echo
134
+echo "Android build complete."
135
+echo "Shared libraries copied to: $JNILIBS_DIR"
sdk/kotlin/scripts/generate-bindings.sh
new
+52
@@ -0,0 +1,52 @@
1
+#!/usr/bin/env bash
2
+# generate-bindings.sh
3
+#
4
+# Generates Kotlin UniFFI bindings from the compiled Android arm64 library.
5
+# Output goes to lib/src/generated/kotlin/ and is gitignored.
6
+#
7
+# Run this after build-android.sh whenever the Rust API changes.
8
+#
9
+# Prerequisites
10
+# -------------
11
+# cargo-ndk and Android targets installed (see build-android.sh)
12
+# uniffi-bindgen binary built (pinned to =0.31.0):
13
+# cargo build --manifest-path ../../uniffi-bindgen/Cargo.toml --release
14
+#
15
+# Usage
16
+# ./scripts/generate-bindings.sh
17
+
18
+set -euo pipefail
19
+
20
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
21
+SDK_DIR="$SCRIPT_DIR/.."
22
+REPO_ROOT="$(cd "$SDK_DIR/../.." && pwd)"
23
+OUT_DIR="$SDK_DIR/lib/src/generated/kotlin"
24
+BINDGEN="$REPO_ROOT/uniffi-bindgen/target/release/uniffi-bindgen"
25
+LIBRARY="$REPO_ROOT/target/aarch64-linux-android/release/libonde.so"
26
+
27
+echo "=== Generating Kotlin UniFFI bindings ==="
28
+
29
+# Build the bindgen binary if it doesn't exist or is stale
30
+if [[ ! -f "$BINDGEN" ]]; then
31
+ echo " → Building uniffi-bindgen…"
32
+ cargo build --manifest-path "$REPO_ROOT/uniffi-bindgen/Cargo.toml" --release
33
+fi
34
+
35
+# The library must exist — run build-android.sh first
36
+if [[ ! -f "$LIBRARY" ]]; then
37
+ echo "Error: $LIBRARY not found."
38
+ echo "Run ./scripts/build-android.sh first to compile the Rust library."
39
+ exit 1
40
+fi
41
+
42
+mkdir -p "$OUT_DIR"
43
+
44
+echo " → Generating from: $LIBRARY"
45
+"$BINDGEN" generate "$LIBRARY" \
46
+ --language kotlin \
47
+ --out-dir "$OUT_DIR"
48
+
49
+echo ""
50
+echo "Generated bindings: $OUT_DIR"
51
+echo "Generated files:"
52
+ls -lh "$OUT_DIR"
sdk/kotlin/settings.gradle.kts
new
+19
@@ -0,0 +1,19 @@
1
+pluginManagement {
2
+ repositories {
3
+ google()
4
+ mavenCentral()
5
+ gradlePluginPortal()
6
+ }
7
+}
8
+
9
+dependencyResolutionManagement {
10
+ repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
11
+ repositories {
12
+ google()
13
+ mavenCentral()
14
+ }
15
+}
16
+
17
+rootProject.name = "onde-inference"
18
+include(":lib")
19
+include(":example")