Migrate Kotlin SDK to multiplatform (Android + JVM)
- Switch to Kotlin Multiplatform project structure - Add JVM target with native library extraction and loading - Share engine wrapper and UniFFI bindings via shared source sets - Add platform-specific environment setup for HuggingFace cache - Update build scripts for JVM native library bundling - Bump version to 1.0.0 and update Maven publish plugin - Add Gradle wrapper for reproducible builds
keypair34 committed
Apr 26, 2026 at 19:47 UTC
ef806b8243de9778c206310d0a6bf225eab913e2
17 files changed
+1263
-454
.agents/skills/sdk-kotlin/SKILL.md
+411
-227
@@ -1,15 +1,31 @@
1
---
2
name: sdk-kotlin
3
-description: Build, generate, and publish the Onde Inference Kotlin/Android SDK. Covers UniFFI Kotlin binding generation from the Rust onde crate, Android library project structure, cargo-ndk cross-compilation for all Android ABIs, Maven Central publishing via Vanniktech plugin, Android filesystem sandbox setup (HF_HOME / TMPDIR via Os.setenv), ProGuard consumer rules, GitHub Actions CI release workflow, and the Jetpack Compose example app.
3
+description: Build, generate, and publish the Onde Inference Kotlin Multiplatform SDK (Android + JVM). Covers UniFFI Kotlin binding generation, KMP source set layout with shared srcDir pattern, cargo-ndk cross-compilation for Android ABIs, JVM native library bundling for macOS Apple Silicon, Maven Central publishing via Vanniktech plugin, klibs.io discovery requirements, ProGuard consumer rules, and the Jetpack Compose example app.
4
allowed-tools: Read, Write, Edit, Glob, Grep, Bash
5
user-invocable: true
6
---
7
8
-# Skill: Kotlin / Android SDK
8
+# Skill: Kotlin Multiplatform SDK
9
10
## What This Skill Covers
11
12
-Building the `onde-inference` Android AAR from Rust source using UniFFI, distributing it via Maven Central, and maintaining the Kotlin idiomatic wrapper on top of the generated bindings.
12
+Building the `onde-inference` Kotlin Multiplatform library from Rust source
13
+using UniFFI, distributing it via Maven Central, and getting it listed on
14
+[klibs.io](https://klibs.io) — the JetBrains-maintained discovery platform for
15
+KMP libraries.
16
+
17
+**Targets:**
18
+
19
+| Target | GPU Backend | Default Model | Primary Use Case |
20
+|--------|-------------|---------------|------------------|
21
+| `androidTarget()` | CPU (Candle) | Qwen 2.5 1.5B (~941 MB) | Mobile apps |
22
+| `jvm()` | Metal (macOS) / CPU (Linux) | Qwen 2.5 3B (~1.93 GB) | Desktop / server on Apple Silicon |
23
+
24
+**Context:** Onde's tagline is "AI for Apple silicon devices." The primary SDKs
25
+are **Swift** (iOS, tvOS, macOS) and **Dart** (Flutter, cross-platform). The
26
+Kotlin SDK extends Onde to Android and JVM developers — particularly those
27
+building KMP apps that target both Android and macOS, or running inference on
28
+Apple Silicon Macs from Kotlin/JVM.
29
30
---
31
@@ -17,61 +33,142 @@ Building the `onde-inference` Android AAR from Rust source using UniFFI, distrib
33
34
```
35
onde/
20
-├── src/ # Rust source — UniFFI-annotated types and OndeChatEngine
21
-├── uniffi-bindgen/ # Standalone uniffi-bindgen CLI binary (pinned =0.31.0)
36
+├── src/ # Rust source — UniFFI-annotated types and OndeChatEngine
37
+├── uniffi-bindgen/ # Standalone uniffi-bindgen CLI binary (pinned =0.31.0)
38
├── sdk/kotlin/
23
-│ ├── settings.gradle.kts # Gradle root — includes :lib and :example
24
-│ ├── build.gradle.kts # root — plugin declarations only (all versions live here)
25
-│ ├── gradle.properties # GROUP, POM_ARTIFACT_ID, VERSION_NAME, signing/OSSRH
39
+│ ├── settings.gradle.kts # Gradle root — includes :lib and :example
40
+│ ├── build.gradle.kts # root — plugin declarations (all versions live here)
41
+│ ├── gradle.properties # GROUP, POM_ARTIFACT_ID, VERSION_NAME, signing/OSSRH
42
│ ├── gradle/wrapper/
27
-│ │ └── gradle-wrapper.properties # pinned Gradle 8.9
43
+│ │ └── gradle-wrapper.properties # pinned Gradle 8.9
44
│ ├── lib/
29
-│ │ ├── build.gradle.kts # Android library + Vanniktech maven-publish
30
-│ │ ├── consumer-rules.pro # ProGuard: preserve UniFFI JNI bridge symbols
45
+│ │ ├── build.gradle.kts # KMP library + Vanniktech maven-publish
46
+│ │ ├── consumer-rules.pro # ProGuard: preserve UniFFI JNI bridge symbols
47
│ │ └── src/
32
-│ │ ├── generated/kotlin/ # UniFFI-generated onde.kt ← GITIGNORED
33
-│ │ └── main/
34
-│ │ ├── AndroidManifest.xml
35
-│ │ ├── jniLibs/ # compiled .so per ABI ← GITIGNORED
36
-│ │ └── kotlin/com/ondeinference/onde/
37
-│ │ └── OndeInference.kt # Android idiomatic wrapper
48
+│ │ ├── commonMain/kotlin/.gitkeep # empty — KMP requires this source set to exist
49
+│ │ ├── shared/kotlin/ # shared srcDir added to BOTH androidMain and jvmMain
50
+│ │ │ └── com/ondeinference/onde/
51
+│ │ │ ├── PlatformSupport.kt # internal interface — setEnv + ensureNativeLoaded
52
+│ │ │ ├── OndeInference.kt # engine wrapper (internal constructor)
53
+│ │ │ └── Convenience.kt # OndeSampling, OndeModels, OndeMessage
54
+│ │ ├── generated/kotlin/ # UniFFI-generated onde.kt ← GITIGNORED
55
+│ │ ├── androidMain/
56
+│ │ │ ├── AndroidManifest.xml
57
+│ │ │ ├── jniLibs/ # compiled .so per ABI ← GITIGNORED
58
+│ │ │ └── kotlin/com/ondeinference/onde/
59
+│ │ │ └── Platform.android.kt # AndroidPlatform + factory fun OndeInference(context)
60
+│ │ └── jvmMain/
61
+│ │ ├── kotlin/com/ondeinference/onde/
62
+│ │ │ ├── Platform.jvm.kt # JvmPlatform + factory fun OndeInference(dataDir)
63
+│ │ │ └── NativeLoader.kt # extract libonde from JAR resources
64
+│ │ └── resources/native/ # bundled .dylib/.so ← GITIGNORED
65
+│ │ ├── macos-aarch64/libonde.dylib
66
+│ │ ├── macos-x86_64/libonde.dylib
67
+│ │ ├── linux-x86_64/libonde.so
68
+│ │ └── linux-aarch64/libonde.so
69
│ ├── example/
39
-│ │ ├── build.gradle.kts # com.android.application — no plugin versions here
70
+│ │ ├── build.gradle.kts # com.android.application — no plugin versions here
71
│ │ ├── proguard-rules.pro
72
│ │ ├── README.md
73
│ │ └── src/main/
43
-│ │ ├── AndroidManifest.xml # INTERNET permission
74
+│ │ ├── AndroidManifest.xml # INTERNET permission
75
│ │ ├── res/values/
76
│ │ │ ├── strings.xml
46
-│ │ │ └── themes.xml # Material3 NoActionBar activity theme
77
+│ │ │ └── themes.xml # Material3 NoActionBar activity theme
78
│ │ └── kotlin/com/ondeinference/example/
48
-│ │ ├── MainActivity.kt # ComponentActivity — enableEdgeToEdge + setContent
49
-│ │ ├── ChatViewModel.kt # model lifecycle, streaming, history
79
+│ │ ├── MainActivity.kt # ComponentActivity — enableEdgeToEdge + setContent
80
+│ │ ├── ChatViewModel.kt # model lifecycle, streaming, history
81
│ │ └── ui/
51
-│ │ ├── ChatScreen.kt # full Compose UI
52
-│ │ └── theme/Theme.kt # dynamic colour + dark mode
82
+│ │ ├── ChatScreen.kt # full Compose UI
83
+│ │ └── theme/Theme.kt # dynamic colour + dark mode
84
│ └── scripts/
54
-│ ├── build-android.sh # Rust .so → jniLibs/ via cargo-ndk
55
-│ └── generate-bindings.sh # uniffi-bindgen → src/generated/kotlin/onde.kt
85
+│ ├── build-android.sh # Rust .so → androidMain/jniLibs/ via cargo-ndk
86
+│ ├── build-jvm.sh # Rust .dylib/.so → jvmMain/resources/native/
87
+│ └── generate-bindings.sh # uniffi-bindgen → src/generated/kotlin/onde.kt
88
└── .github/workflows/
57
- └── release-sdk-kotlin.yml # tag push → build → Maven Central publish
89
+ └── release-sdk-kotlin.yml # tag push → build → Maven Central publish
90
+```
91
+
92
+---
93
+
94
+## KMP Architecture
95
+
96
+### Why KMP Instead of Android-Only
97
+
98
+1. **klibs.io listing** — the JetBrains KMP discovery platform requires
99
+ `kotlin-tooling-metadata.json`, which only the `kotlin.multiplatform` Gradle
100
+ plugin generates. An Android-only AAR will not appear on klibs.io.
101
+2. **JVM target** — Kotlin developers on macOS Apple Silicon can run inference
102
+ locally from desktop/server apps, Gradle plugins, or CLI tools.
103
+3. **Future-proof** — adding Kotlin/Native targets (iOS, macOS) later requires
104
+ no structural changes. The Swift SDK already covers Apple platforms natively,
105
+ but KMP apps that share Kotlin across Android + iOS could benefit.
106
+
107
+### Source Set Strategy: Shared srcDir
108
+
109
+Instead of KMP intermediate source sets (which add Gradle complexity and
110
+require careful dependency resolution for JNA), the SDK uses a **shared srcDir**
111
+pattern:
112
+
113
+```
114
+commonMain/ — empty (KMP requires it)
115
+shared/ — hand-written wrapper code (compiled for BOTH targets)
116
+generated/ — UniFFI-generated onde.kt (compiled for BOTH targets)
117
+androidMain/ — Android-specific: Os.setenv, Context factory
118
+jvmMain/ — JVM-specific: JNA setenv, NativeLoader, File factory
119
+```
120
+
121
+In `lib/build.gradle.kts`:
122
+
123
+```kotlin
124
+androidMain.get().kotlin.srcDir("src/shared/kotlin")
125
+androidMain.get().kotlin.srcDir("src/generated/kotlin")
126
+jvmMain.get().kotlin.srcDir("src/shared/kotlin")
127
+jvmMain.get().kotlin.srcDir("src/generated/kotlin")
128
+```
129
+
130
+Both targets compile the shared code independently with their own dependencies.
131
+Since both are JVM-based, `java.io.File`, JNA, and coroutines are available in
132
+the shared code.
133
+
134
+### Platform Abstraction: Interface Pattern
135
+
136
+The shared `OndeInference` class takes a `PlatformSupport` interface via an
137
+`internal` constructor. Each target provides a concrete implementation and a
138
+top-level factory function:
139
+
140
+```
141
+PlatformSupport (interface — shared/)
142
+├── AndroidPlatform (object — androidMain/)
143
+│ └── fun OndeInference(context, dataDir?) → OndeInference
144
+└── JvmPlatform (object — jvmMain/)
145
+ └── fun OndeInference(dataDir?) → OndeInference
146
```
147
148
+| Operation | Android | JVM |
149
+|-----------|---------|-----|
150
+| `setEnv(key, value)` | `Os.setenv(key, value, true)` | JNA → `libc.setenv(key, value, 1)` |
151
+| `ensureNativeLoaded()` | No-op (APK jniLibs) | `NativeLoader.ensureLoaded()` |
152
+| Default `dataDir` | `context.filesDir` | `~/.onde/` |
153
+
154
---
155
156
## Key Design Decisions
157
158
| Decision | Rationale |
159
|---|---|
160
+| KMP with `androidTarget()` + `jvm()` | Required for klibs.io listing; enables macOS Apple Silicon JVM users |
161
+| Shared srcDir instead of intermediate source sets | Avoids KMP dependency resolution complexity with JNA (`@aar` vs JAR) |
162
+| Interface-based platform abstraction | Simpler than `expect`/`actual` when constructor signatures differ between targets |
163
| UniFFI Kotlin bindings — not hand-written JNI | Same Rust source as Swift/Dart, zero API drift |
164
| `src/generated/kotlin/` is gitignored | Regenerated from Rust on every release |
68
-| `cargo-ndk` for cross-compilation | Handles NDK toolchain wiring for all 4 ABIs transparently |
69
-| Vanniktech `maven-publish` plugin | Simplest path to Maven Central + signing with one plugin |
165
+| `cargo-ndk` for Android cross-compilation | Handles NDK toolchain wiring for all 4 ABIs transparently |
166
+| `NativeLoader` extracts dylib from JAR resources | JVM users don't need to manage `java.library.path` manually |
167
+| Vanniktech `maven-publish` plugin with `KotlinMultiplatform` | Publishes KMP metadata + per-target artifacts to Maven Central |
168
| `minSdk 26` | Required for `android.system.Os.setenv` to set `HF_HOME` |
71
-| `OndeInference` wraps `OndeChatEngine` | Adds Android filesystem init, `Dispatchers.IO`, and `Flow` streaming |
169
+| JNA dependency (`net.java.dev.jna:jna:5.14.0`) | UniFFI 0.31.0 generates Kotlin bindings using `com.sun.jna.*`. Android uses `@aar` variant; JVM uses plain JAR. |
170
| ProGuard consumer rules bundled in AAR | Prevents R8 from stripping UniFFI JNI bridge in consumer apps |
73
-| JNA dependency required (`net.java.dev.jna:jna:5.14.0@aar`) | UniFFI 0.31.0 generates Kotlin bindings that use `com.sun.jna.*` for the FFI bridge. Despite the `com.sun` package name, JNA works on Android when pulled as `@aar`. Without it, you get `Unresolved reference 'sun'` on every import. |
74
-| `:example` is a Gradle submodule, not a standalone project | Plugin versions are declared once in the root `build.gradle.kts` and inherited by all submodules — a standalone project would need versions on every plugin, and `project(":lib")` would not resolve |
171
+| `:example` is a Gradle submodule | Plugin versions declared once in root `build.gradle.kts` and inherited |
172
173
---
174
@@ -91,13 +188,15 @@ onde/
188
| `OndeChatEngine` | `uniffi::Object` | `class OndeChatEngine` |
189
| `StreamChunkListener` | `callback_interface` | `interface StreamChunkListener` |
190
94
-The generated Kotlin package is `uniffi.onde.*`. The `OndeInference.kt` wrapper
191
+The generated Kotlin package is `uniffi.onde.*`. The shared `OndeInference.kt`
192
re-exports these as `typealias` under `com.ondeinference.onde.*` so callers only
193
need one import.
194
195
---
196
100
-## Android ABI Targets
197
+## Build Targets
198
+
199
+### Android ABI Targets
200
201
| ABI | Rust triple | Typical device |
202
|---|---|---|
@@ -106,6 +205,15 @@ need one import.
205
| `x86_64` | `x86_64-linux-android` | Android emulators (Intel/AMD) |
206
| `x86` | `i686-linux-android` | 32-bit emulators |
207
208
+### JVM Native Library Targets
209
+
210
+| Platform | Rust triple | Library | Primary Use Case |
211
+|---|---|---|---|
212
+| macOS Apple Silicon | `aarch64-apple-darwin` | `libonde.dylib` | **Primary** — Mac development + Metal GPU |
213
+| macOS Intel | `x86_64-apple-darwin` | `libonde.dylib` | Legacy Mac support |
214
+| Linux x86_64 | `x86_64-unknown-linux-gnu` | `libonde.so` | CI, servers |
215
+| Linux ARM64 | `aarch64-unknown-linux-gnu` | `libonde.so` | ARM servers |
216
+
217
---
218
219
## Build Sequence (Manual / CI)
@@ -113,64 +221,102 @@ need one import.
221
### 1. Prerequisites
222
223
```bash
116
-# Rust Android targets
117
-rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android i686-linux-android
224
+# Rust toolchain
225
+rustup toolchain install stable
226
119
-# cargo-ndk
227
+# Android targets + cargo-ndk
228
+rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android i686-linux-android
229
cargo install cargo-ndk
230
231
# Android NDK
232
export ANDROID_NDK_HOME=$HOME/Library/Android/sdk/ndk/26.1.10909125
233
+
234
+# macOS/JVM target (usually pre-installed on Apple Silicon)
235
+rustup target add aarch64-apple-darwin
236
```
237
126
-### 2. Build .so files for all ABIs
238
+### 2. Build Android .so files
239
240
```bash
241
./sdk/kotlin/scripts/build-android.sh # release, all ABIs
242
./sdk/kotlin/scripts/build-android.sh --debug # debug build
243
+FILTER_ABI=arm64-v8a ./sdk/kotlin/scripts/build-android.sh # single ABI
244
```
245
133
-Internally uses:
246
+Outputs land in `sdk/kotlin/lib/src/androidMain/jniLibs/<ABI>/libonde.so`.
247
+
248
+### 3. Build JVM native library
249
250
```bash
136
-cargo ndk -t arm64-v8a -t armeabi-v7a -t x86_64 -t x86 \
137
- --manifest-path Cargo.toml --release
251
+./sdk/kotlin/scripts/build-jvm.sh # host platform, release
252
+./sdk/kotlin/scripts/build-jvm.sh --debug # debug build
253
+ONDE_TARGET_TRIPLE=x86_64-apple-darwin ./sdk/kotlin/scripts/build-jvm.sh # cross-compile
254
```
255
140
-Outputs land in `sdk/kotlin/lib/src/main/jniLibs/<ABI>/libonde.so`.
256
+Outputs land in `sdk/kotlin/lib/src/jvmMain/resources/native/<os-arch>/libonde.<ext>`.
257
142
-### 3. Build the uniffi-bindgen CLI
258
+### 4. Build the uniffi-bindgen CLI
259
260
```bash
261
cargo build --manifest-path uniffi-bindgen/Cargo.toml --release
262
# Binary: uniffi-bindgen/target/release/uniffi-bindgen
263
```
264
149
-### 4. Generate Kotlin UniFFI bindings
265
+### 5. Generate Kotlin UniFFI bindings
266
267
```bash
268
./sdk/kotlin/scripts/generate-bindings.sh
269
```
270
155
-Internally runs:
156
-
157
-```bash
158
-uniffi-bindgen/target/release/uniffi-bindgen generate \
159
- target/aarch64-linux-android/release/libonde.so \
160
- --language kotlin \
161
- --out-dir sdk/kotlin/lib/src/generated/kotlin
162
-```
163
-
271
Produces `sdk/kotlin/lib/src/generated/kotlin/onde.kt` — the complete
272
generated binding including all types, the `OndeChatEngine` class, free
273
functions, and `System.loadLibrary("onde")` in a companion object static init.
274
168
-### 5. Build the AAR
275
+### 6. Build the library
276
277
```bash
278
cd sdk/kotlin
279
+
280
+# Android AAR
281
./gradlew :lib:assembleRelease
173
-# Output: lib/build/outputs/aar/lib-release.aar
282
+
283
+# JVM JAR (with bundled native lib)
284
+./gradlew :lib:jvmJar
285
+
286
+# Both
287
+./gradlew :lib:build
288
+```
289
+
290
+---
291
+
292
+## JVM Native Library Loading
293
+
294
+The JVM target bundles native libraries inside the JAR under
295
+`/native/<os>-<arch>/libonde.<ext>`. The `NativeLoader` object handles
296
+extraction and loading at runtime:
297
+
298
+```
299
+OndeInference() factory (jvmMain)
300
+ └── JvmPlatform.ensureNativeLoaded()
301
+ └── NativeLoader.ensureLoaded()
302
+ ├── Detect OS + arch from system properties
303
+ ├── Extract /native/<os>-<arch>/libonde.<ext> from JAR resources
304
+ ├── Write to temp file: <java.io.tmpdir>/onde-native/libonde.<ext>
305
+ └── System.load(tempFile.absolutePath)
306
+```
307
+
308
+If the bundled library is not found (e.g. development mode without running
309
+`build-jvm.sh`), `NativeLoader` falls back to `System.loadLibrary("onde")`
310
+which searches `java.library.path`.
311
+
312
+**Important:** `NativeLoader.ensureLoaded()` must run BEFORE any UniFFI type is
313
+accessed. The `OndeInference` constructor handles this via the `init` block:
314
+
315
+```kotlin
316
+class OndeInference internal constructor(...) : AutoCloseable {
317
+ init { platform.ensureNativeLoaded() } // runs before engine = OndeChatEngine()
318
+ private val engine = uniffi.onde.OndeChatEngine()
319
+}
320
```
321
322
---
@@ -181,7 +327,7 @@ On Android, `dirs::home_dir()` (used by `hf-hub` under the hood) panics because
327
there is no home directory in the Android sandbox. The Rust engine requires
328
`HF_HOME` to be set **before** any model load on Android.
329
184
-The `OndeInference` Kotlin wrapper handles this automatically in `setup()`:
330
+The `OndeInference` wrapper handles this automatically in `setup()`:
331
332
```kotlin
333
fun setup() {
@@ -191,100 +337,56 @@ fun setup() {
337
val hfHubCache = File(hfHome, "hub").also { it.mkdirs() }
338
val tmpDir = File(dataDir, "tmp").also { it.mkdirs() }
339
194
- Os.setenv("HF_HOME", hfHome.absolutePath, true)
195
- Os.setenv("HF_HUB_CACHE", hfHubCache.absolutePath, true)
196
- Os.setenv("HUGGINGFACE_HUB_CACHE", hfHubCache.absolutePath, true)
197
- Os.setenv("TMPDIR", tmpDir.absolutePath, true)
340
+ platform.setEnv("HF_HOME", hfHome.absolutePath)
341
+ platform.setEnv("HF_HUB_CACHE", hfHubCache.absolutePath)
342
+ platform.setEnv("HUGGINGFACE_HUB_CACHE", hfHubCache.absolutePath)
343
+ platform.setEnv("TMPDIR", tmpDir.absolutePath)
344
345
configured = true
346
}
347
```
348
349
`setup()` is called automatically by `loadDefaultModel()` and `loadModel()`.
204
-Callers can call it explicitly at app startup for tighter control.
350
206
-### Filesystem layout inside filesDir
351
+### Filesystem layout
352
353
```
354
+# Android (context.filesDir)
355
<filesDir>/
356
├── models/ ← HF_HOME
211
-│ └── hub/ ← HF_HUB_CACHE (GGUF blobs live here)
357
+│ └── hub/ ← HF_HUB_CACHE
358
└── tmp/ ← TMPDIR
213
-```
359
215
-**This layout matches the shared convention used by the Tauri and Swift SDKs.**
216
-On iOS/macOS, the equivalent paths are inside the App Group container at
217
-`<group.com.ondeinference.apps>/models/hub/`. On Android, there is no app group
218
-mechanism, so each app gets its own `filesDir`.
360
+# JVM (defaults to ~/.onde/)
361
+~/.onde/
362
+├── models/ ← HF_HOME
363
+│ └── hub/ ← HF_HUB_CACHE
364
+└── tmp/ ← TMPDIR
365
+```
366
220
-### Cross-platform model sharing comparison
367
+### Cross-platform model cache comparison
368
222
-| Platform | Mechanism | Shared across apps? | How it works |
369
+| Platform | SDK | Cache Location | Shared across apps? |
370
|---|---|---|---|
224
-| iOS / tvOS / macOS | App Groups (`group.com.ondeinference.apps`) | Yes — all apps signed by the same team that declare the same App Group entitlement share a single container directory | OS provides a shared `containerURL(forSecurityApplicationGroupIdentifier:)` path; all Onde apps read and write the same `models/hub/` within it |
225
-| Android | Per-app `filesDir` sandbox | No — each app gets its own isolated copy | Linux UID isolation prevents one app from reading another's internal storage; no entitlement or manifest flag can opt in to sharing |
371
+| iOS / tvOS / macOS | Swift | App Group container | Yes (same team + entitlement) |
372
+| macOS (JVM) | Kotlin | `~/.onde/models/hub/` | Yes (all JVM apps on same user) |
373
+| Android | Kotlin | `<filesDir>/models/hub/` | No (per-app sandbox) |
374
+| Flutter | Dart | Platform-dependent | Follows platform conventions |
375
227
-### Why Android can't do App Groups
376
+### Why Android can't share models across apps
377
378
Android's security model assigns each app a unique Linux UID at install time.
230
-The kernel enforces file permissions — app A literally cannot open a file
231
-descriptor inside app B's `/data/data/com.B/files/`. There is no platform
232
-primitive to override this the way iOS App Groups do. The approaches that
233
-_don't_ work:
234
-
235
-- **`android:sharedUserId`** — deprecated in API 29, removed in API 33. Dead end.
236
-- **`MANAGE_EXTERNAL_STORAGE`** — lets you write to `/sdcard/onde/models/` but Google Play rejects apps that use it unless they are file managers or antivirus tools.
237
-- **SAF (Storage Access Framework)** — requires the user to manually pick a directory every time. Not viable for a background model load.
238
-
239
-### What we do today
240
-
241
-Each Onde-powered Android app downloads its own copy of the model (~941 MB for
242
-Qwen 2.5 1.5B). This is what every major Android ML SDK does (TensorFlow Lite,
243
-MediaPipe, ML Kit). Android devices typically ship with 128–256 GB storage, so
244
-1 GB per app is tolerable. The SDK caches after first download, so the user only
245
-pays the bandwidth cost once per app.
246
-
247
-### Future: ContentProvider model manager
248
-
249
-The Android-idiomatic way to share data across apps without special permissions
250
-is a `ContentProvider`. The plan (not yet implemented):
251
-
252
-1. The first Onde-powered app installed acts as the **model host**. It downloads
253
- models into its own `filesDir` as it does today.
254
-2. It registers a `ContentProvider` at a well-known authority
255
- (e.g. `com.ondeinference.models`) that serves model files via
256
- `ParcelFileDescriptor`.
257
-3. Other Onde-powered apps query the provider first. If a model is already
258
- cached by any sibling app, they get a read-only file descriptor — no
259
- download, no copy.
260
-4. If the provider is not installed (no sibling app on the device), the app
261
- falls back to downloading its own copy.
379
+The kernel enforces file permissions — app A cannot open a file descriptor
380
+inside app B's `/data/data/com.B/files/`. There is no platform primitive to
381
+override this the way iOS App Groups do. The approaches that don't work:
382
263
-```kotlin
264
-// In the host app — exposes cached models to sibling apps
265
-class OndeModelProvider : ContentProvider() {
266
- override fun openFile(uri: Uri, mode: String): ParcelFileDescriptor? {
267
- // uri = content://com.ondeinference.models/qwen25-1.5b
268
- val modelFile = File(context!!.filesDir, "models/hub/${uri.lastPathSegment}")
269
- if (!modelFile.exists()) return null
270
- return ParcelFileDescriptor.open(modelFile, ParcelFileDescriptor.MODE_READ_ONLY)
271
- }
272
-}
383
+- **`android:sharedUserId`** — deprecated API 29, removed API 33
384
+- **`MANAGE_EXTERNAL_STORAGE`** — Google Play rejects unless file manager
385
+- **SAF (Storage Access Framework)** — requires user interaction every time
386
274
-// In any Onde-powered app — checks for a cached model before downloading
275
-val uri = Uri.parse("content://com.ondeinference.models/qwen25-1.5b")
276
-val pfd = contentResolver.openFileDescriptor(uri, "r")
277
-if (pfd != null) {
278
- // model already downloaded by a sibling app — read from file descriptor
279
-} else {
280
- // no sibling app has the model — download our own copy
281
-}
282
-```
283
-
284
-No special permissions required, works on all API levels, no Google Play policy
285
-issues. The only requirement is that at least one Onde app is installed on the
286
-device. This is tracked but not yet implemented — ship with per-app caching for
287
-now and add the provider when multiple production apps exist.
387
+Each Onde-powered Android app downloads its own copy (~941 MB). This matches
388
+how TensorFlow Lite, MediaPipe, and ML Kit handle model caching. The SDK caches
389
+after first download — bandwidth cost is one-time per app.
390
391
---
392
@@ -298,8 +400,15 @@ import com.ondeinference.onde.OndeSampling
400
import com.ondeinference.onde.OndeModels
401
import com.ondeinference.onde.OndeMessage
402
403
+// ── Android ──────────────────────────────────────────────────
404
val onde = OndeInference(context) // pass applicationContext
405
406
+// ── JVM (macOS / Linux) ──────────────────────────────────────
407
+val onde = OndeInference() // uses ~/.onde/
408
+val onde = OndeInference(File("/custom")) // custom cache dir
409
+
410
+// ── Shared API (both platforms) ──────────────────────────────
411
+
412
// Load (downloads from HuggingFace Hub on first run)
413
val elapsed: Double = onde.loadDefaultModel(
414
systemPrompt = "You are helpful.",
@@ -346,7 +455,7 @@ OndeSampling.mobile() // temp=0.7, max=128 tokens
455
### `OndeModels` — model config constructors
456
457
```kotlin
349
-OndeModels.default() // platform default = Qwen 2.5 1.5B on Android
458
+OndeModels.default() // platform default (1.5B on Android, 3B on macOS JVM)
459
OndeModels.qwen25_1_5b() // ~941 MB
460
OndeModels.qwen25_3b() // ~1.93 GB
461
```
@@ -364,43 +473,51 @@ OndeMessage.assistant("Hi there!")
473
## lib/build.gradle.kts Key Sections
474
475
```kotlin
367
-import com.vanniktech.maven.publish.AndroidSingleVariantLibrary
476
+import com.vanniktech.maven.publish.KotlinMultiplatform
477
+import com.vanniktech.maven.publish.JavadocJar
478
import com.vanniktech.maven.publish.SonatypeHost
479
370
-android {
371
- namespace = "com.ondeinference.onde"
372
- compileSdk = 35
373
- defaultConfig { minSdk = 26 }
480
+plugins {
481
+ id("org.jetbrains.kotlin.multiplatform")
482
+ id("com.android.library")
483
+ id("com.vanniktech.maven.publish")
484
+}
485
+
486
+kotlin {
487
+ androidTarget {
488
+ publishLibraryVariants("release")
489
+ compilations.all { kotlinOptions { jvmTarget = "17" } }
490
+ }
491
+ jvm {
492
+ compilations.all { kotlinOptions { jvmTarget = "17" } }
493
+ }
494
495
sourceSets {
376
- getByName("main") {
377
- jniLibs.srcDirs("src/main/jniLibs")
378
- kotlin.srcDirs("src/main/kotlin", "src/generated/kotlin")
496
+ androidMain.dependencies {
497
+ implementation("net.java.dev.jna:jna:5.14.0@aar")
498
+ implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.8.1")
499
+ }
500
+ jvmMain.dependencies {
501
+ implementation("net.java.dev.jna:jna:5.14.0")
502
+ implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.8.1")
503
}
380
- }
504
382
- // Do NOT add android { publishing { singleVariant("release") } } here.
383
- // AndroidSingleVariantLibrary in mavenPublishing already calls it — doing
384
- // both causes "singleVariant publishing DSL multiple times" at sync time.
505
+ // Shared srcDirs — compiled independently for each target
506
+ androidMain.get().kotlin.srcDir("src/shared/kotlin")
507
+ androidMain.get().kotlin.srcDir("src/generated/kotlin")
508
+ jvmMain.get().kotlin.srcDir("src/shared/kotlin")
509
+ jvmMain.get().kotlin.srcDir("src/generated/kotlin")
510
+ }
511
}
512
513
mavenPublishing {
388
- // This is the correct way to configure variant + sources/javadoc for an
389
- // Android library with Vanniktech 0.25+. It registers singleVariant internally.
390
- configure(
391
- AndroidSingleVariantLibrary(
392
- variant = "release",
393
- sourcesJar = true,
394
- publishJavadocJar = true,
395
- )
396
- )
514
+ configure(KotlinMultiplatform(
515
+ javadocJar = JavadocJar.Empty(),
516
+ sourcesJar = true,
517
+ ))
518
publishToMavenCentral(SonatypeHost.CENTRAL_PORTAL)
519
signAllPublications()
520
coordinates("com.ondeinference", "onde-inference", "<version>")
400
- pom {
401
- name.set("Onde Inference")
402
- // ... licenses, developers, scm
403
- }
521
}
522
```
523
@@ -433,6 +550,29 @@ types or callback interfaces.
550
551
---
552
553
+## klibs.io Discovery
554
+
555
+[klibs.io](https://klibs.io) is a JetBrains-maintained search platform for
556
+Kotlin Multiplatform libraries. It auto-indexes from Maven Central.
557
+
558
+### Requirements for listing
559
+
560
+| Requirement | How we satisfy it |
561
+|---|---|
562
+| Open source on GitHub | `ondeinference/onde` is public |
563
+| At least one artifact on Maven Central | Published via Vanniktech plugin |
564
+| Artifact has `kotlin-tooling-metadata.json` | KMP plugin generates this automatically |
565
+| POM has valid GitHub `url` or `scm.url` | `scm { url.set("https://github.com/ondeinference/onde") }` |
566
+
567
+### Timeline
568
+
569
+- New libraries appear **within one month** of first Maven Central publish
570
+ (public index update frequency).
571
+- New versions of already-indexed libraries appear **the next day**.
572
+- No manual submission needed — discovery is fully automatic.
573
+
574
+---
575
+
576
## Maven Central Publishing (CI)
577
578
Trigger: push a semver tag matching `[0-9]+.[0-9]+.[0-9]+`.
@@ -441,19 +581,20 @@ Trigger: push a semver tag matching `[0-9]+.[0-9]+.[0-9]+`.
581
# 1. Bump VERSION_NAME in sdk/kotlin/gradle.properties
582
# 2. Bump version in Cargo.toml [package]
583
# 3. Commit, tag, push
444
-git tag 0.1.4 && git push origin 0.1.4
584
+git tag 1.0.1 && git push origin 1.0.1
585
```
586
587
CI (`release-sdk-kotlin.yml`) steps:
448
-1. Install Rust stable with all 4 Android targets
588
+1. Install Rust stable with all 4 Android targets + host target
589
2. Cache NDK 26.1.10909125
590
3. Install `cargo-ndk`
591
4. `build-android.sh --release` → `.so` files
452
-5. Build `uniffi-bindgen`
453
-6. `generate-bindings.sh` → `onde.kt`
454
-7. Validate tag == `Cargo.toml` version
455
-8. `./gradlew :lib:publishAndReleaseToMavenCentral`
456
-9. Upload AAR as GitHub Release asset
592
+5. `build-jvm.sh --release` → `.dylib` / `.so`
593
+6. Build `uniffi-bindgen`
594
+7. `generate-bindings.sh` → `onde.kt`
595
+8. Validate tag == `Cargo.toml` version
596
+9. `./gradlew :lib:publishAndReleaseToMavenCentral`
597
+10. Upload artifacts as GitHub Release assets
598
599
---
600
@@ -471,75 +612,113 @@ Never bump one without the other.
612
|---|---|
613
| `dirs::home_dir()` panics on Android | `setup()` must be called before any Rust model load. `OndeInference.loadDefaultModel()` calls it automatically. |
614
| Generated `onde.kt` not found at compile time | Run `scripts/generate-bindings.sh`. The file is gitignored and must be regenerated after every Rust API change. |
474
-| `.so` files missing from AAR | Run `scripts/build-android.sh`. The `jniLibs/` directory is gitignored. |
615
+| `.so` files missing from AAR | Run `scripts/build-android.sh`. The `androidMain/jniLibs/` directory is gitignored. |
616
+| JVM `UnsatisfiedLinkError` at runtime | Run `scripts/build-jvm.sh` to bundle the native lib in JAR resources, or add `libonde.dylib` to `java.library.path`. |
617
| UniFFI version mismatch | Keep `uniffi = "=0.31.0"` identical in `Cargo.toml`, `build-deps`, and `uniffi-bindgen/Cargo.toml`. |
618
| R8 strips UniFFI bridge at runtime | Ensure `consumer-rules.pro` is present in the AAR and includes `-keep class uniffi.onde.**`. |
477
-| `System.loadLibrary("onde")` fails | The `.so` files must match the ABI of the device. Check that all 4 ABIs are built and included in `jniLibs/`. |
619
+| `System.loadLibrary("onde")` fails | The `.so` files must match the ABI of the device. Check that all 4 ABIs are built and included in `androidMain/jniLibs/`. |
620
| `Os.setenv` not available | `minSdk` must be 26+. `android.system.Os` is available from API 21 but `setenv` specifically needs 26. |
621
| Gradle fails with `VERSION_NAME` missing | Ensure `gradle.properties` contains `VERSION_NAME=x.y.z`. |
622
| `cargo-ndk` not found in CI | Add `cargo install cargo-ndk --locked` step before the build step. |
623
| `ANDROID_NDK_HOME` not set | Set it explicitly: `echo "ANDROID_NDK_HOME=$ANDROID_SDK_ROOT/ndk/26.1.10909125" >> $GITHUB_ENV` |
482
-| `build-android.sh: line XX: arm64: unbound variable` on macOS | Do not use Bash associative arrays in `sdk/kotlin/scripts/build-android.sh`. macOS often ships an older Bash where associative arrays are unreliable or unavailable. Use plain indexed arrays (or another portable structure) for ABI → Rust target mapping. |
483
-| `Unresolved reference 'sun'` / `Unresolved reference 'Structure'` in generated `onde.kt` | UniFFI 0.31.0 generates Kotlin bindings that depend on JNA (`com.sun.jna.*`). Add `implementation("net.java.dev.jna:jna:5.14.0@aar")` to `lib/build.gradle.kts`. The `@aar` classifier is required on Android — a plain `.jar` will not work. Despite the `com.sun` package name, JNA is a third-party library, not a JDK internal, and works fine on Android. |
624
+| `build-android.sh: line XX: arm64: unbound variable` on macOS | Do not use Bash associative arrays. The script uses portable indexed arrays for ABI → Rust target mapping. |
625
+| `Unresolved reference 'sun'` / `Unresolved reference 'Structure'` in `onde.kt` | UniFFI 0.31.0 depends on JNA (`com.sun.jna.*`). Add `implementation("net.java.dev.jna:jna:5.14.0@aar")` for Android or `implementation("net.java.dev.jna:jna:5.14.0")` for JVM. |
626
| HuggingFace download fails on first load | The app must declare `<uses-permission android:name="android.permission.INTERNET" />`. |
485
-| `EngineStatus` values are lowercase (`"ready"`) | The Rust `Display` impl emits lowercase. Normalize in the UI layer: `info.status.toString().lowercase()` or compare with the generated sealed class variants directly. |
486
-| `Plugin [id: 'com.android.application'] was not found` in `:example` | The example has no plugin versions of its own — they are inherited from the root `build.gradle.kts`. Open `sdk/kotlin/` (not `sdk/kotlin/example/`) in Android Studio. Never add a `settings.gradle.kts` inside `example/` — doing so makes Gradle treat it as a standalone root project, breaking plugin resolution and `project(":lib")`. |
487
-| `Task 'wrapper' not found in project ':example'` | Same cause as above — Android Studio was opened at the wrong directory. The `wrapper` task only exists on the root project. Open `sdk/kotlin/`. |
488
-| `Using singleVariant publishing DSL multiple times to publish variant "release"` | Vanniktech 0.25+ registers `singleVariant("release")` internally via `AndroidSingleVariantLibrary`. Do **not** also add `android { publishing { singleVariant("release") { ... } } }` — that registers it a second time and causes this error. Remove the `android { publishing }` block entirely and configure sources/javadoc through `mavenPublishing` instead: `configure(AndroidSingleVariantLibrary(variant = "release", sourcesJar = true, publishJavadocJar = true))`. |
627
+| `Plugin [id: 'com.android.application'] was not found` in `:example` | Open `sdk/kotlin/` (not `sdk/kotlin/example/`) in Android Studio. Plugin versions are inherited from the root `build.gradle.kts`. |
628
+| `Using singleVariant publishing DSL multiple times` | This error means the old `AndroidSingleVariantLibrary` config is still present. Use `KotlinMultiplatform(...)` instead — the KMP plugin handles variant publishing internally. |
629
+| Convenience functions (`OndeSampling.default()` etc.) crash before `OndeInference()` created | These functions call into the native library via UniFFI. Always create an `OndeInference` instance first — its `init` block ensures the native library is loaded. |
630
631
---
632
633
## Testing
634
494
-Unit tests can be run on the host JVM (no Android device needed) for pure Kotlin
495
-logic, but any test that calls into Rust via the generated bindings requires a
496
-connected Android device or emulator because `System.loadLibrary("onde")` only
497
-works with an Android runtime.
635
+### Unit tests (host JVM — no Android device needed)
636
+
637
+Pure Kotlin logic tests that don't call into Rust:
638
639
```bash
500
-# Kotlin-only unit tests (host JVM)
640
cd sdk/kotlin
502
-./gradlew :lib:test
641
+./gradlew :lib:test # all JVM unit tests
642
+./gradlew :lib:jvmTest # JVM target only
643
+```
644
504
-# Instrumented tests (requires connected device or emulator)
645
+### Instrumented tests (requires Android device or emulator)
646
+
647
+Tests that call into Rust via the generated bindings:
648
+
649
+```bash
650
./gradlew :lib:connectedAndroidTest
651
```
652
508
-For CI, use an emulator (`x86_64` ABI) spun up with `reactivecircus/android-emulator-runner`.
653
+For CI, use an emulator (`x86_64` ABI) with `reactivecircus/android-emulator-runner`.
654
+
655
+### JVM integration tests
656
+
657
+Tests that load the native library and run inference:
658
+
659
+```bash
660
+# Build the native lib first
661
+./sdk/kotlin/scripts/build-jvm.sh
662
+
663
+# Run JVM tests
664
+cd sdk/kotlin && ./gradlew :lib:jvmTest
665
+```
666
667
---
668
669
## Dependency Graph
670
671
```
515
-OndeInference.kt
516
- └── uniffi.onde.OndeChatEngine (generated onde.kt)
517
- └── uniffi.onde.StreamChunkListener (generated onde.kt)
518
- └── uniffi.onde.*Config / *Message (generated onde.kt)
519
- └── com.sun.jna.Native (JNA — net.java.dev.jna:jna:5.14.0@aar)
520
- └── libonde.so (Rust, per ABI)
521
- └── mistralrs (Candle CPU backend on Android)
522
- └── hf-hub (model download)
672
+OndeInference.kt (shared)
673
+├── PlatformSupport (interface, shared)
674
+│ ├── AndroidPlatform (androidMain)
675
+│ │ └── android.system.Os.setenv
676
+│ └── JvmPlatform (jvmMain)
677
+│ ├── JNA → libc.setenv
678
+│ └── NativeLoader → extracts libonde from JAR resources
679
+├── uniffi.onde.OndeChatEngine (generated onde.kt)
680
+├── uniffi.onde.StreamChunkListener (generated onde.kt)
681
+├── uniffi.onde.*Config / *Message (generated onde.kt)
682
+│ └── com.sun.jna.Native (JNA)
683
+│ ├── libonde.so (Android, per ABI)
684
+│ └── libonde.dylib (macOS, from JAR resources)
685
+│ └── mistralrs (Metal on macOS, Candle CPU on Android)
686
+│ └── hf-hub (model download)
687
+└── Convenience.kt (shared)
688
+ └── OndeSampling, OndeModels, OndeMessage
689
```
690
691
---
692
693
## Distribution Registry
694
529
-| Registry | Artifact | Import |
530
-|---|---|---|
531
-| Maven Central | `com.ondeinference:onde-inference` | `implementation("com.ondeinference:onde-inference:0.1.3")` |
532
-| GitHub Releases | `onde-inference-<version>.aar` | Direct AAR download |
695
+| Registry | Artifact | Import | Discovery |
696
+|---|---|---|---|
697
+| Maven Central | `com.ondeinference:onde-inference` | `implementation("com.ondeinference:onde-inference:1.0.0")` | [search.maven.org](https://search.maven.org) |
698
+| klibs.io | `onde-inference` | Same Gradle coordinate | [klibs.io](https://klibs.io) (auto-indexed) |
699
+| GitHub Releases | `onde-inference-<version>.aar` + `.jar` | Direct download | [github.com/ondeinference/onde/releases](https://github.com/ondeinference/onde/releases) |
700
+
701
+### End-game for Kotlin developers
702
+
703
+One line in `build.gradle.kts`:
704
+
705
+```kotlin
706
+implementation("com.ondeinference:onde-inference:1.0.0")
707
+```
708
709
---
710
711
## Example App
712
538
-A working Jetpack Compose chat app lives at `sdk/kotlin/example/`. It is the canonical way to see the SDK in action and is the first thing to open when verifying a new build.
713
+A working Jetpack Compose chat app lives at `sdk/kotlin/example/`. It is the
714
+canonical way to see the SDK in action and the first thing to open when
715
+verifying a new build.
716
717
### Opening in Android Studio
718
542
-Always open **`sdk/kotlin/`** — the Gradle root. Do not open `sdk/kotlin/example/` directly. Android Studio will see both `:lib` and `:example` as modules and sync correctly.
719
+Always open **`sdk/kotlin/`** — the Gradle root. Do not open
720
+`sdk/kotlin/example/` directly. Android Studio will see both `:lib` and
721
+`:example` as modules and sync correctly.
722
723
### What the example covers
724
@@ -548,56 +727,38 @@ Always open **`sdk/kotlin/`** — the Gradle root. Do not open `sdk/kotlin/examp
727
- Multi-turn conversation history maintained in the Rust engine
728
- Clearing history without reloading the model
729
- Material3 dynamic colour theme with dark mode support
551
-- Proper IME / navigation-bar inset handling with `imePadding()` + `navigationBarsPadding()`
730
+- Proper IME / navigation-bar inset handling
731
732
### Gradle submodule rules
733
555
-`example/` is a Gradle submodule of the `sdk/kotlin/` root project. The rules that must always hold:
556
-
557
-1. **No `settings.gradle.kts` inside `example/`** — if one exists, Gradle stops its upward search there and treats `example/` as a standalone root, breaking plugin resolution and `project(":lib")`.
558
-2. **No plugin versions in `example/build.gradle.kts`** — versions are declared once in the root `build.gradle.kts` with `apply false` and inherited by submodules.
559
-3. **`include(":example")` in root `settings.gradle.kts`** — required for Gradle to recognise `:example` as a subproject.
560
-4. **`project(":lib")` as the SDK dependency** — the example always builds against local source, not the published Maven artifact. This means the `.so` files and `onde.kt` must be generated before the example will compile.
734
+1. **No `settings.gradle.kts` inside `example/`** — Gradle would treat it as
735
+ standalone, breaking plugin resolution and `project(":lib")`.
736
+2. **No plugin versions in `example/build.gradle.kts`** — inherited from root.
737
+3. **`include(":example")` in root `settings.gradle.kts`** — required for Gradle.
738
+4. **`project(":lib")` as the SDK dependency** — builds against local source.
739
562
-The root `build.gradle.kts` must declare all plugins used by any submodule:
740
+The root `build.gradle.kts` declares all plugins used by any submodule:
741
742
```kotlin
743
plugins {
566
- id("com.android.library") version "8.5.2" apply false // :lib
567
- id("com.android.application") version "8.5.2" apply false // :example
568
- id("org.jetbrains.kotlin.android") version "2.0.21" apply false
569
- id("org.jetbrains.kotlin.plugin.compose") version "2.0.21" apply false // :example
570
- id("com.vanniktech.maven.publish") version "0.28.0" apply false // :lib
744
+ id("com.android.library") version "8.5.2" apply false
745
+ id("com.android.application") version "8.5.2" apply false
746
+ id("org.jetbrains.kotlin.android") version "2.0.21" apply false
747
+ id("org.jetbrains.kotlin.multiplatform") version "2.0.21" apply false
748
+ id("org.jetbrains.kotlin.plugin.compose") version "2.0.21" apply false
749
+ id("com.vanniktech.maven.publish") version "0.28.0" apply false
750
}
751
```
752
574
-### Example architecture
575
-
576
-```
577
-ChatScreen (Compose — pure function of ChatUiState)
578
- ↓ collectAsStateWithLifecycle
579
-ChatViewModel (AndroidViewModel + viewModelScope)
580
- ↓ loadModel() → onde.loadDefaultModel(...)
581
- ↓ sendMessage() → onde.stream(...).collect { ... }
582
- ↓ clearChat() → onde.clearHistory()
583
-OndeInference ← project(":lib")
584
- ↓ UniFFI / libonde.so
585
-mistral.rs — Qwen 2.5 1.5B, Candle CPU backend
586
-```
587
-
753
### Streaming pattern
754
590
-Tokens are accumulated into a `StringBuilder` and the last item in the message list is replaced in-place on every chunk, so Compose only recomposes the one bubble that is changing:
591
-
755
```kotlin
756
val placeholderId = System.nanoTime()
594
-// insert empty bubble once
757
_uiState.update { it.copy(messages = it.messages + UiMessage(id = placeholderId, ...)) }
758
759
val buffer = StringBuilder()
760
onde.stream(text).collect { chunk ->
761
buffer.append(chunk.delta)
600
- // swap only the tail — no full list rebuild
762
_uiState.update { state ->
763
state.copy(messages = state.messages.replaceLast(
764
UiMessage(id = placeholderId, content = buffer.toString(), isStreaming = !chunk.done)
@@ -608,12 +769,10 @@ onde.stream(text).collect { chunk ->
769
770
### Running the example
771
611
-`build-android.sh` must stay portable on macOS. Avoid Bash associative arrays in that script — use indexed arrays instead.
612
-
772
```bash
773
cd sdk/kotlin
774
616
-# 1. Build the .so files (needed by :lib, which :example depends on)
775
+# 1. Build the .so files
776
./scripts/build-android.sh
777
778
# 2. Generate the Kotlin bindings
@@ -623,4 +782,29 @@ cd sdk/kotlin
782
./gradlew :example:installDebug
783
```
784
626
-Or open `sdk/kotlin/` in Android Studio and hit Run on the `:example` configuration.
\ No newline at end of file
785
+Or open `sdk/kotlin/` in Android Studio and hit Run on the `:example` configuration.
786
+
787
+---
788
+
789
+## Cross-Platform SDK Comparison
790
+
791
+Onde ships SDKs for four platforms. The Kotlin SDK is one piece of the
792
+distribution strategy:
793
+
794
+| SDK | Primary Platform | GPU Backend | Package Manager | Discovery |
795
+|-----|-----------------|-------------|-----------------|-----------|
796
+| **Swift** | iOS, tvOS, macOS | Metal | Swift Package Manager | Swift Package Index |
797
+| **Dart** | Android, iOS (Flutter) | Platform-dependent | pub.dev | pub.dev search |
798
+| **Kotlin** | Android, JVM (macOS) | Metal (JVM) / CPU (Android) | Maven Central | klibs.io |
799
+| **React Native** | Android, iOS | Platform-dependent | npm | npmjs.com |
800
+
801
+The Swift SDK is the primary distribution — it runs natively on Apple silicon
802
+with Metal acceleration. The Kotlin JVM target provides the same Metal-backed
803
+inference for Kotlin developers on macOS, while the Android target extends
804
+reach to the Android ecosystem.
805
+
806
+---
807
+
808
+*Update this skill when adding KMP targets (e.g. Kotlin/Native for iOS),
809
+changing the source set structure, or modifying the Maven Central publishing
810
+pipeline.*
\ No newline at end of file
sdk/kotlin/.gitignore
+4
-1
@@ -6,7 +6,7 @@ build/
6
lib/src/generated/
7
8
# Compiled Rust native libraries — regenerated by scripts/build-android.sh
9
-lib/src/main/jniLibs/
9
+lib/src/androidMain/jniLibs/
10
11
# Android Studio / IntelliJ
12
.idea/
@@ -34,3 +34,6 @@ keystore.properties
34
# Logs
35
*.log
36
npm-debug.log*
37
+
38
+# JVM native libraries — regenerated by scripts/build-jvm.sh
39
+lib/src/jvmMain/resources/native/
sdk/kotlin/build.gradle.kts
+2
-1
@@ -3,6 +3,7 @@ plugins {
3
id("com.android.library") version "8.5.2" apply false
4
id("com.android.application") version "8.5.2" apply false
5
id("org.jetbrains.kotlin.android") version "2.0.21" apply false
6
+ id("org.jetbrains.kotlin.multiplatform") version "2.0.21" apply false
7
id("org.jetbrains.kotlin.plugin.compose") version "2.0.21" apply false
7
- id("com.vanniktech.maven.publish") version "0.28.0" apply false
8
+ id("com.vanniktech.maven.publish") version "0.34.0" apply false
9
}
sdk/kotlin/gradle.properties
+3
-1
@@ -10,7 +10,9 @@ kotlin.code.style=official
10
# Maven publishing coordinates — keep in sync with Cargo.toml [package] version
11
GROUP=com.ondeinference
12
POM_ARTIFACT_ID=onde-inference
13
-VERSION_NAME=0.1.3
13
+VERSION_NAME=1.0.0
14
+
15
+
16
17
# Sonatype credentials — set in ~/.gradle/gradle.properties or via CI secrets
18
# ossrhUsername=
sdk/kotlin/gradlew
new
+252
@@ -0,0 +1,252 @@
1
+#!/bin/sh
2
+
3
+#
4
+# Copyright © 2015-2021 the original authors.
5
+#
6
+# Licensed under the Apache License, Version 2.0 (the "License");
7
+# you may not use this file except in compliance with the License.
8
+# You may obtain a copy of the License at
9
+#
10
+# https://www.apache.org/licenses/LICENSE-2.0
11
+#
12
+# Unless required by applicable law or agreed to in writing, software
13
+# distributed under the License is distributed on an "AS IS" BASIS,
14
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15
+# See the License for the specific language governing permissions and
16
+# limitations under the License.
17
+#
18
+# SPDX-License-Identifier: Apache-2.0
19
+#
20
+
21
+##############################################################################
22
+#
23
+# Gradle start up script for POSIX generated by Gradle.
24
+#
25
+# Important for running:
26
+#
27
+# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
28
+# noncompliant, but you have some other compliant shell such as ksh or
29
+# bash, then to run this script, type that shell name before the whole
30
+# command line, like:
31
+#
32
+# ksh Gradle
33
+#
34
+# Busybox and similar reduced shells will NOT work, because this script
35
+# requires all of these POSIX shell features:
36
+# * functions;
37
+# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
38
+# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
39
+# * compound commands having a testable exit status, especially «case»;
40
+# * various built-in commands including «command», «set», and «ulimit».
41
+#
42
+# Important for patching:
43
+#
44
+# (2) This script targets any POSIX shell, so it avoids extensions provided
45
+# by Bash, Ksh, etc; in particular arrays are avoided.
46
+#
47
+# The "traditional" practice of packing multiple parameters into a
48
+# space-separated string is a well documented source of bugs and security
49
+# problems, so this is (mostly) avoided, by progressively accumulating
50
+# options in "$@", and eventually passing that to Java.
51
+#
52
+# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
53
+# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
54
+# see the in-line comments for details.
55
+#
56
+# There are tweaks for specific operating systems such as AIX, CygWin,
57
+# Darwin, MinGW, and NonStop.
58
+#
59
+# (3) This script is generated from the Groovy template
60
+# https://github.com/gradle/gradle/blob/HEAD/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
61
+# within the Gradle project.
62
+#
63
+# You can find Gradle at https://github.com/gradle/gradle/.
64
+#
65
+##############################################################################
66
+
67
+# Attempt to set APP_HOME
68
+
69
+# Resolve links: $0 may be a link
70
+app_path=$0
71
+
72
+# Need this for daisy-chained symlinks.
73
+while
74
+ APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
75
+ [ -h "$app_path" ]
76
+do
77
+ ls=$( ls -ld "$app_path" )
78
+ link=${ls#*' -> '}
79
+ case $link in #(
80
+ /*) app_path=$link ;; #(
81
+ *) app_path=$APP_HOME$link ;;
82
+ esac
83
+done
84
+
85
+# This is normally unused
86
+# shellcheck disable=SC2034
87
+APP_BASE_NAME=${0##*/}
88
+# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
89
+APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s
90
+' "$PWD" ) || exit
91
+
92
+# Use the maximum available, or set MAX_FD != -1 to use that value.
93
+MAX_FD=maximum
94
+
95
+warn () {
96
+ echo "$*"
97
+} >&2
98
+
99
+die () {
100
+ echo
101
+ echo "$*"
102
+ echo
103
+ exit 1
104
+} >&2
105
+
106
+# OS specific support (must be 'true' or 'false').
107
+cygwin=false
108
+msys=false
109
+darwin=false
110
+nonstop=false
111
+case "$( uname )" in #(
112
+ CYGWIN* ) cygwin=true ;; #(
113
+ Darwin* ) darwin=true ;; #(
114
+ MSYS* | MINGW* ) msys=true ;; #(
115
+ NONSTOP* ) nonstop=true ;;
116
+esac
117
+
118
+CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
119
+
120
+
121
+# Determine the Java command to use to start the JVM.
122
+if [ -n "$JAVA_HOME" ] ; then
123
+ if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
124
+ # IBM's JDK on AIX uses strange locations for the executables
125
+ JAVACMD=$JAVA_HOME/jre/sh/java
126
+ else
127
+ JAVACMD=$JAVA_HOME/bin/java
128
+ fi
129
+ if [ ! -x "$JAVACMD" ] ; then
130
+ die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
131
+
132
+Please set the JAVA_HOME variable in your environment to match the
133
+location of your Java installation."
134
+ fi
135
+else
136
+ JAVACMD=java
137
+ if ! command -v java >/dev/null 2>&1
138
+ then
139
+ die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
140
+
141
+Please set the JAVA_HOME variable in your environment to match the
142
+location of your Java installation."
143
+ fi
144
+fi
145
+
146
+# Increase the maximum file descriptors if we can.
147
+if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
148
+ case $MAX_FD in #(
149
+ max*)
150
+ # In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
151
+ # shellcheck disable=SC2039,SC3045
152
+ MAX_FD=$( ulimit -H -n ) ||
153
+ warn "Could not query maximum file descriptor limit"
154
+ esac
155
+ case $MAX_FD in #(
156
+ '' | soft) :;; #(
157
+ *)
158
+ # In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
159
+ # shellcheck disable=SC2039,SC3045
160
+ ulimit -n "$MAX_FD" ||
161
+ warn "Could not set maximum file descriptor limit to $MAX_FD"
162
+ esac
163
+fi
164
+
165
+# Collect all arguments for the java command, stacking in reverse order:
166
+# * args from the command line
167
+# * the main class name
168
+# * -classpath
169
+# * -D...appname settings
170
+# * --module-path (only if needed)
171
+# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
172
+
173
+# For Cygwin or MSYS, switch paths to Windows format before running java
174
+if "$cygwin" || "$msys" ; then
175
+ APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
176
+ CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" )
177
+
178
+ JAVACMD=$( cygpath --unix "$JAVACMD" )
179
+
180
+ # Now convert the arguments - kludge to limit ourselves to /bin/sh
181
+ for arg do
182
+ if
183
+ case $arg in #(
184
+ -*) false ;; # don't mess with options #(
185
+ /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
186
+ [ -e "$t" ] ;; #(
187
+ *) false ;;
188
+ esac
189
+ then
190
+ arg=$( cygpath --path --ignore --mixed "$arg" )
191
+ fi
192
+ # Roll the args list around exactly as many times as the number of
193
+ # args, so each arg winds up back in the position where it started, but
194
+ # possibly modified.
195
+ #
196
+ # NB: a `for` loop captures its iteration list before it begins, so
197
+ # changing the positional parameters here affects neither the number of
198
+ # iterations, nor the values presented in `arg`.
199
+ shift # remove old arg
200
+ set -- "$@" "$arg" # push replacement arg
201
+ done
202
+fi
203
+
204
+
205
+# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
206
+DEFAULT_JVM_OPTS='-Dfile.encoding=UTF-8 "-Xmx64m" "-Xms64m"'
207
+
208
+# Collect all arguments for the java command:
209
+# * DEFAULT_JVM_OPTS, JAVA_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
210
+# and any embedded shellness will be escaped.
211
+# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
212
+# treated as '${Hostname}' itself on the command line.
213
+
214
+set -- \
215
+ "-Dorg.gradle.appname=$APP_BASE_NAME" \
216
+ -classpath "$CLASSPATH" \
217
+ org.gradle.wrapper.GradleWrapperMain \
218
+ "$@"
219
+
220
+# Stop when "xargs" is not available.
221
+if ! command -v xargs >/dev/null 2>&1
222
+then
223
+ die "xargs is not available"
224
+fi
225
+
226
+# Use "xargs" to parse quoted args.
227
+#
228
+# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
229
+#
230
+# In Bash we could simply go:
231
+#
232
+# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
233
+# set -- "${ARGS[@]}" "$@"
234
+#
235
+# but POSIX shell has neither arrays nor command substitution, so instead we
236
+# post-process each arg (as a line of input to sed) to backslash-escape any
237
+# character that might be a shell metacharacter, then use eval to reverse
238
+# that process (while maintaining the separation between arguments), and wrap
239
+# the whole thing up as a single "set" statement.
240
+#
241
+# This will of course break if any of these variables contains a newline or
242
+# an unmatched quote.
243
+#
244
+
245
+eval "set -- $(
246
+ printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
247
+ xargs -n1 |
248
+ sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
249
+ tr '\n' ' '
250
+ )" '"$@"'
251
+
252
+exec "$JAVACMD" "$@"
sdk/kotlin/gradlew.bat
new
+94
@@ -0,0 +1,94 @@
1
+@rem
2
+@rem Copyright 2015 the original author or authors.
3
+@rem
4
+@rem Licensed under the Apache License, Version 2.0 (the "License");
5
+@rem you may not use this file except in compliance with the License.
6
+@rem You may obtain a copy of the License at
7
+@rem
8
+@rem https://www.apache.org/licenses/LICENSE-2.0
9
+@rem
10
+@rem Unless required by applicable law or agreed to in writing, software
11
+@rem distributed under the License is distributed on an "AS IS" BASIS,
12
+@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+@rem See the License for the specific language governing permissions and
14
+@rem limitations under the License.
15
+@rem
16
+@rem SPDX-License-Identifier: Apache-2.0
17
+@rem
18
+
19
+@if "%DEBUG%"=="" @echo off
20
+@rem ##########################################################################
21
+@rem
22
+@rem Gradle startup script for Windows
23
+@rem
24
+@rem ##########################################################################
25
+
26
+@rem Set local scope for the variables with windows NT shell
27
+if "%OS%"=="Windows_NT" setlocal
28
+
29
+set DIRNAME=%~dp0
30
+if "%DIRNAME%"=="" set DIRNAME=.
31
+@rem This is normally unused
32
+set APP_BASE_NAME=%~n0
33
+set APP_HOME=%DIRNAME%
34
+
35
+@rem Resolve any "." and ".." in APP_HOME to make it shorter.
36
+for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
37
+
38
+@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
39
+set DEFAULT_JVM_OPTS=-Dfile.encoding=UTF-8 "-Xmx64m" "-Xms64m"
40
+
41
+@rem Find java.exe
42
+if defined JAVA_HOME goto findJavaFromJavaHome
43
+
44
+set JAVA_EXE=java.exe
45
+%JAVA_EXE% -version >NUL 2>&1
46
+if %ERRORLEVEL% equ 0 goto execute
47
+
48
+echo. 1>&2
49
+echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2
50
+echo. 1>&2
51
+echo Please set the JAVA_HOME variable in your environment to match the 1>&2
52
+echo location of your Java installation. 1>&2
53
+
54
+goto fail
55
+
56
+:findJavaFromJavaHome
57
+set JAVA_HOME=%JAVA_HOME:"=%
58
+set JAVA_EXE=%JAVA_HOME%/bin/java.exe
59
+
60
+if exist "%JAVA_EXE%" goto execute
61
+
62
+echo. 1>&2
63
+echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2
64
+echo. 1>&2
65
+echo Please set the JAVA_HOME variable in your environment to match the 1>&2
66
+echo location of your Java installation. 1>&2
67
+
68
+goto fail
69
+
70
+:execute
71
+@rem Setup the command line
72
+
73
+set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
74
+
75
+
76
+@rem Execute Gradle
77
+"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %*
78
+
79
+:end
80
+@rem End local scope for the variables with windows NT shell
81
+if %ERRORLEVEL% equ 0 goto mainEnd
82
+
83
+:fail
84
+rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
85
+rem the _cmd.exe /c_ return code!
86
+set EXIT_CODE=%ERRORLEVEL%
87
+if %EXIT_CODE% equ 0 set EXIT_CODE=1
88
+if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
89
+exit /b %EXIT_CODE%
90
+
91
+:mainEnd
92
+if "%OS%"=="Windows_NT" endlocal
93
+
94
+:omega
sdk/kotlin/lib/build.gradle.kts
+89
-38
@@ -1,10 +1,56 @@
1
-import com.vanniktech.maven.publish.AndroidSingleVariantLibrary
2
-import com.vanniktech.maven.publish.SonatypeHost
1
+import com.vanniktech.maven.publish.KotlinMultiplatform
2
+import com.vanniktech.maven.publish.JavadocJar
3
4
plugins {
5
+ id("org.jetbrains.kotlin.multiplatform")
6
id("com.android.library")
6
- id("org.jetbrains.kotlin.android")
7
id("com.vanniktech.maven.publish")
8
+ id("signing")
9
+}
10
+
11
+kotlin {
12
+ androidTarget {
13
+ publishLibraryVariants("release")
14
+ compilations.all {
15
+ kotlinOptions { jvmTarget = "17" }
16
+ }
17
+ }
18
+
19
+ jvm {
20
+ compilations.all {
21
+ kotlinOptions { jvmTarget = "17" }
22
+ }
23
+ }
24
+
25
+ sourceSets {
26
+ // Both androidMain and jvmMain share the code in src/shared/kotlin/
27
+ // and the UniFFI-generated bindings in src/generated/kotlin/.
28
+ // This avoids the complexity of intermediate source sets while keeping
29
+ // the shared engine wrapper code DRY.
30
+
31
+ androidMain.dependencies {
32
+ // JNA — UniFFI-generated onde.kt uses com.sun.jna.* for its FFI bridge.
33
+ // The @aar suffix pulls the Android-specific build with native .so files.
34
+ implementation("net.java.dev.jna:jna:5.14.0@aar")
35
+ // Coroutines — suspend funs and Flow streaming in OndeInference
36
+ implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.8.1")
37
+ }
38
+
39
+ jvmMain.dependencies {
40
+ // Desktop JNA with native libs for macOS/Linux/Windows
41
+ implementation("net.java.dev.jna:jna:5.14.0")
42
+ // Coroutines
43
+ implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.8.1")
44
+ }
45
+
46
+ // Add shared source directories to both JVM-based targets.
47
+ // src/shared/kotlin/ — hand-written OndeInference wrapper + convenience objects
48
+ // src/generated/kotlin/ — UniFFI-generated onde.kt (gitignored, regenerated)
49
+ androidMain.get().kotlin.srcDir("src/shared/kotlin")
50
+ androidMain.get().kotlin.srcDir("src/generated/kotlin")
51
+ jvmMain.get().kotlin.srcDir("src/shared/kotlin")
52
+ jvmMain.get().kotlin.srcDir("src/generated/kotlin")
53
+ }
54
}
55
56
android {
@@ -13,73 +59,78 @@ android {
59
60
defaultConfig {
61
minSdk = 26 // Os.setenv needs API 26+
16
- targetSdk = 35
62
consumerProguardFiles("consumer-rules.pro")
63
}
64
65
+ lint {
66
+ // The UniFFI-generated onde.kt uses java.lang.ref.Cleaner (API 33+)
67
+ // with an internal fallback for older API levels. We don't control
68
+ // the generated code, so suppress the lint error.
69
+ disable += "NewApi"
70
+ }
71
+
72
compileOptions {
73
sourceCompatibility = JavaVersion.VERSION_17
74
targetCompatibility = JavaVersion.VERSION_17
75
}
76
25
- kotlinOptions {
26
- jvmTarget = "17"
27
- }
28
-
77
sourceSets {
78
getByName("main") {
31
- jniLibs.srcDirs("src/main/jniLibs")
32
- // UniFFI-generated onde.kt lands in src/generated/kotlin/ after
33
- // running scripts/generate-bindings.sh — gitignored, regenerate after Rust API changes.
34
- kotlin.srcDirs("src/main/kotlin", "src/generated/kotlin")
79
+ // Android JNI shared libraries live under androidMain/jniLibs/
80
+ jniLibs.srcDirs("src/androidMain/jniLibs")
81
}
82
}
37
-
38
- // No android { publishing { singleVariant(...) } } block here.
39
- // Vanniktech's AndroidSingleVariantLibrary (below) registers it — adding
40
- // it twice causes "singleVariant publishing DSL multiple times" error.
41
-}
42
-
43
-dependencies {
44
- // JNA — the UniFFI-generated onde.kt uses com.sun.jna.* for its FFI bridge.
45
- // The @aar suffix pulls the Android-specific build with native .so files included.
46
- implementation("net.java.dev.jna:jna:5.14.0@aar")
47
-
48
- // Coroutines — needed by the suspend funs and Flow streaming in OndeInference.kt
49
- implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.8.1")
83
}
84
85
+// ── Maven Central publishing (KMP) ─────────────────────────────────────────
86
+//
87
// Required secrets (in ~/.gradle/gradle.properties or CI env vars):
88
// ORG_GRADLE_PROJECT_mavenCentralUsername
89
// ORG_GRADLE_PROJECT_mavenCentralPassword
90
// ORG_GRADLE_PROJECT_signingKeyId
91
// ORG_GRADLE_PROJECT_signingKey (ASCII-armored PGP private key)
92
// ORG_GRADLE_PROJECT_signingPassword
93
+// ── Signing ────────────────────────────────────────────────────────────────
94
+// Vanniktech 0.28 looks for signingInMemoryKey / signingInMemoryKeyId /
95
+// signingInMemoryKeyPassword. Local gradle.properties (and CI secrets) use
96
+// the shorter names signingKey / signingKeyId / signingPassword.
97
+// Bridge them here so both naming conventions work.
98
+val signingKeyId = findProperty("signingInMemoryKeyId") as String?
99
+ ?: findProperty("signingKeyId") as String?
100
+val signingKey = findProperty("signingInMemoryKey") as String?
101
+ ?: findProperty("signingKey") as String?
102
+val signingPassword = findProperty("signingInMemoryKeyPassword") as String?
103
+ ?: findProperty("signingPassword") as String?
104
+
105
+signing {
106
+ if (signingKey != null) {
107
+ useInMemoryPgpKeys(signingKeyId, signingKey, signingPassword)
108
+ }
109
+}
110
+
111
+
112
mavenPublishing {
59
- // AndroidSingleVariantLibrary is the correct way to configure variant + jars
60
- // for an Android library with Vanniktech 0.25+. It calls singleVariant("release")
61
- // internally, so the android { publishing { } } block must NOT be used alongside it.
113
configure(
63
- AndroidSingleVariantLibrary(
64
- variant = "release",
114
+ KotlinMultiplatform(
115
+ javadocJar = JavadocJar.Empty(),
116
sourcesJar = true,
66
- publishJavadocJar = true,
117
)
118
)
119
70
- publishToMavenCentral(SonatypeHost.CENTRAL_PORTAL)
120
+ publishToMavenCentral()
121
signAllPublications()
122
73
- coordinates(
74
- groupId = "com.ondeinference",
75
- artifactId = "onde-inference",
76
- version = project.property("VERSION_NAME") as String,
77
- )
123
+ // Coordinates are read automatically from gradle.properties:
124
+ // GROUP → groupId
125
+ // POM_ARTIFACT_ID → artifactId
126
+ // VERSION_NAME → version
127
+ // Do NOT call coordinates() explicitly — it conflicts with the
128
+ // properties-based approach and causes "value is final" errors.
129
130
pom {
131
name.set("Onde Inference")
132
description.set(
82
- "On-device LLM inference for Android. " +
133
+ "On-device LLM inference for Android and JVM (macOS Apple Silicon). " +
134
"Run Qwen 2.5 models locally — no cloud, no API key, no data leaving the device."
135
)
136
url.set("https://ondeinference.com")
sdk/kotlin/lib/src/androidMain/AndroidManifest.xml
renamed
sdk/kotlin/lib/src/androidMain/kotlin/com/ondeinference/onde/Platform.android.kt
new
+41
@@ -0,0 +1,41 @@
1
+// Copyright 2025 Onde Inference (Splitfire AB). All rights reserved.
2
+// SPDX-License-Identifier: MIT OR Apache-2.0
3
+
4
+package com.ondeinference.onde
5
+
6
+import android.content.Context
7
+import android.system.Os
8
+import java.io.File
9
+
10
+/**
11
+ * Android implementation of [PlatformSupport].
12
+ *
13
+ * Uses `android.system.Os.setenv()` (requires API 26+) to seed environment
14
+ * variables for the HuggingFace Hub cache. Native library loading is handled
15
+ * automatically by the APK's `jniLibs/` directory — no extraction needed.
16
+ */
17
+internal object AndroidPlatform : PlatformSupport {
18
+ override fun setEnv(key: String, value: String) {
19
+ Os.setenv(key, value, true)
20
+ }
21
+ // ensureNativeLoaded() is intentionally a no-op on Android.
22
+ // The UniFFI-generated companion object calls System.loadLibrary("onde"),
23
+ // which loads the correct ABI from the APK's lib/ directory automatically.
24
+}
25
+
26
+/**
27
+ * Create an [OndeInference] engine for Android.
28
+ *
29
+ * @param context Android application or activity context. Pass `applicationContext`
30
+ * to avoid leaking activity references.
31
+ * @param dataDir Optional override for the root cache directory. Defaults to
32
+ * `context.filesDir`. Useful when the app wants to use external storage.
33
+ * @return A ready-to-use [OndeInference] instance.
34
+ */
35
+fun OndeInference(
36
+ context: Context,
37
+ dataDir: File = context.filesDir,
38
+): OndeInference = OndeInference(
39
+ dataDir = dataDir,
40
+ platform = AndroidPlatform,
41
+)
sdk/kotlin/lib/src/commonMain/kotlin/.gitkeep
sdk/kotlin/lib/src/jvmMain/kotlin/com/ondeinference/onde/NativeLoader.kt
new
+83
@@ -0,0 +1,83 @@
1
+// Copyright 2025 Onde Inference (Splitfire AB). All rights reserved.
2
+// SPDX-License-Identifier: MIT OR Apache-2.0
3
+
4
+package com.ondeinference.onde
5
+
6
+import java.io.File
7
+import java.io.FileOutputStream
8
+
9
+/**
10
+ * Extracts and loads the native `libonde` library from JAR resources.
11
+ *
12
+ * The JVM target bundles native libraries under:
13
+ * `/native/<os>-<arch>/libonde.<ext>`
14
+ *
15
+ * Supported platforms:
16
+ * - `macos-aarch64/libonde.dylib` — macOS Apple Silicon
17
+ * - `macos-x86_64/libonde.dylib` — macOS Intel
18
+ * - `linux-x86_64/libonde.so` — Linux x86_64
19
+ * - `linux-aarch64/libonde.so` — Linux ARM64
20
+ *
21
+ * If the native library is not bundled (e.g. development mode), falls back to
22
+ * [System.loadLibrary] which searches `java.library.path`.
23
+ */
24
+internal object NativeLoader {
25
+ @Volatile private var loaded = false
26
+
27
+ fun ensureLoaded() {
28
+ if (loaded) return
29
+ synchronized(this) {
30
+ if (loaded) return
31
+ try {
32
+ loadFromResources()
33
+ } catch (_: Exception) {
34
+ // Fallback: library must be on java.library.path
35
+ System.loadLibrary("onde")
36
+ }
37
+ loaded = true
38
+ }
39
+ }
40
+
41
+ private fun loadFromResources() {
42
+ val osName = System.getProperty("os.name").lowercase()
43
+ val archName = System.getProperty("os.arch").lowercase()
44
+
45
+ val (os, ext) = when {
46
+ "mac" in osName || "darwin" in osName -> "macos" to "dylib"
47
+ "linux" in osName -> "linux" to "so"
48
+ "windows" in osName -> "windows" to "dll"
49
+ else -> throw UnsupportedOperationException(
50
+ "Onde Inference does not support this OS: $osName"
51
+ )
52
+ }
53
+
54
+ val arch = when (archName) {
55
+ "aarch64", "arm64" -> "aarch64"
56
+ "x86_64", "amd64" -> "x86_64"
57
+ else -> throw UnsupportedOperationException(
58
+ "Onde Inference does not support this architecture: $archName"
59
+ )
60
+ }
61
+
62
+ val resourcePath = "/native/$os-$arch/libonde.$ext"
63
+ val stream = NativeLoader::class.java.getResourceAsStream(resourcePath)
64
+ ?: throw UnsatisfiedLinkError(
65
+ "Native library not found in JAR resources: $resourcePath. " +
66
+ "Run scripts/build-jvm.sh to build and bundle the native library, " +
67
+ "or ensure libonde is on java.library.path."
68
+ )
69
+
70
+ val tempDir = File(System.getProperty("java.io.tmpdir"), "onde-native")
71
+ tempDir.mkdirs()
72
+ val tempFile = File(tempDir, "libonde.$ext")
73
+
74
+ // Overwrite on every load to handle version upgrades
75
+ stream.use { input ->
76
+ FileOutputStream(tempFile).use { output ->
77
+ input.copyTo(output)
78
+ }
79
+ }
80
+
81
+ System.load(tempFile.absolutePath)
82
+ }
83
+}
sdk/kotlin/lib/src/jvmMain/kotlin/com/ondeinference/onde/Platform.jvm.kt
new
+53
@@ -0,0 +1,53 @@
1
+// Copyright 2025 Onde Inference (Splitfire AB). All rights reserved.
2
+// SPDX-License-Identifier: MIT OR Apache-2.0
3
+
4
+package com.ondeinference.onde
5
+
6
+import java.io.File
7
+
8
+/**
9
+ * JVM (desktop) implementation of [PlatformSupport].
10
+ *
11
+ * Uses JNA to call the C library's `setenv()` function directly, since the
12
+ * JVM standard library does not provide a way to modify environment variables
13
+ * after process start. The Rust engine reads `HF_HOME` etc. via
14
+ * `std::env::var()`, which reads from the C-level environment.
15
+ *
16
+ * Native library loading extracts the bundled `libonde.dylib` (macOS) or
17
+ * `libonde.so` (Linux) from JAR resources and loads it via [System.load].
18
+ */
19
+internal object JvmPlatform : PlatformSupport {
20
+
21
+ private interface CLib : com.sun.jna.Library {
22
+ fun setenv(name: String, value: String, overwrite: Int): Int
23
+
24
+ companion object {
25
+ val INSTANCE: CLib = com.sun.jna.Native.load("c", CLib::class.java)
26
+ }
27
+ }
28
+
29
+ override fun setEnv(key: String, value: String) {
30
+ CLib.INSTANCE.setenv(key, value, 1)
31
+ }
32
+
33
+ override fun ensureNativeLoaded() {
34
+ NativeLoader.ensureLoaded()
35
+ }
36
+}
37
+
38
+/**
39
+ * Create an [OndeInference] engine for JVM (desktop).
40
+ *
41
+ * On macOS, the default data directory is `~/.onde/`. On Linux it is also
42
+ * `~/.onde/`. Model files are cached in `<dataDir>/models/hub/`.
43
+ *
44
+ * @param dataDir Root directory for the model cache and temp files.
45
+ * Defaults to `~/.onde/` in the user's home directory.
46
+ * @return A ready-to-use [OndeInference] instance.
47
+ */
48
+fun OndeInference(
49
+ dataDir: File = File(System.getProperty("user.home"), ".onde"),
50
+): OndeInference = OndeInference(
51
+ dataDir = dataDir,
52
+ platform = JvmPlatform,
53
+)
sdk/kotlin/lib/src/shared/kotlin/com/ondeinference/onde/Convenience.kt
new
+37
@@ -0,0 +1,37 @@
1
+// Copyright 2025 Onde Inference (Splitfire AB). All rights reserved.
2
+// SPDX-License-Identifier: MIT OR Apache-2.0
3
+
4
+package com.ondeinference.onde
5
+
6
+/**
7
+ * Construct preconfigured [SamplingConfig] instances.
8
+ */
9
+object OndeSampling {
10
+ /** Creative chat defaults — temperature 0.7, top_p 0.95, max 512 tokens. */
11
+ fun default(): SamplingConfig = uniffi.onde.defaultSamplingConfig()
12
+ /** Greedy/deterministic — temperature 0.0, max 512 tokens. */
13
+ fun deterministic(): SamplingConfig = uniffi.onde.deterministicSamplingConfig()
14
+ /** Conservative mobile — temperature 0.7, max 128 tokens. */
15
+ fun mobile(): SamplingConfig = uniffi.onde.mobileSamplingConfig()
16
+}
17
+
18
+/**
19
+ * Construct [GgufModelConfig] instances for supported Onde models.
20
+ */
21
+object OndeModels {
22
+ /** Platform-appropriate default — Qwen 2.5 1.5B on Android, 3B on macOS. */
23
+ fun default(): GgufModelConfig = uniffi.onde.defaultModelConfig()
24
+ /** Qwen 2.5 1.5B Instruct GGUF Q4_K_M (~941 MB). */
25
+ fun qwen25_1_5b(): GgufModelConfig = uniffi.onde.qwen2515bConfig()
26
+ /** Qwen 2.5 3B Instruct GGUF Q4_K_M (~1.93 GB). */
27
+ fun qwen25_3b(): GgufModelConfig = uniffi.onde.qwen253bConfig()
28
+}
29
+
30
+/**
31
+ * Construct [ChatMessage] values.
32
+ */
33
+object OndeMessage {
34
+ fun system(content: String): ChatMessage = uniffi.onde.systemMessage(content)
35
+ fun user(content: String): ChatMessage = uniffi.onde.userMessage(content)
36
+ fun assistant(content: String): ChatMessage = uniffi.onde.assistantMessage(content)
37
+}
sdk/kotlin/lib/src/shared/kotlin/com/ondeinference/onde/OndeInference.kt
renamed
+42
-185
@@ -3,41 +3,11 @@
3
//
4
// OndeInference.kt
5
//
6
-// Android-idiomatic wrapper around the UniFFI-generated OndeChatEngine.
7
-//
8
-// This file is the primary public API surface for the Onde Inference Android
9
-// SDK. It wraps the low-level UniFFI bindings (generated by
10
-// scripts/generate-bindings.sh into src/generated/kotlin/onde.kt) with:
11
-//
12
-// 1. Android-specific filesystem initialisation (HF_HOME, TMPDIR, etc.)
13
-// 2. suspend functions backed by Dispatchers.IO
14
-// 3. Flow<StreamChunk> for token streaming
15
-// 4. Sensible Kotlin defaults so callers need minimal boilerplate
16
-//
17
-// Usage
18
-// -----
19
-// val onde = OndeInference(context)
20
-//
21
-// // Load the platform default model (Qwen 2.5 1.5B on Android)
22
-// val elapsed = onde.loadDefaultModel(systemPrompt = "You are helpful.")
23
-// println("Loaded in ${elapsed}s")
24
-//
25
-// // Multi-turn chat
26
-// val result = onde.chat("Hello!")
27
-// println(result.text)
28
-//
29
-// // Streaming
30
-// onde.stream("Tell me a story.").collect { chunk ->
31
-// print(chunk.delta)
32
-// }
33
-//
34
-// // Cleanup
35
-// onde.unload()
6
+// Multiplatform wrapper around the UniFFI-generated OndeChatEngine.
7
+// This file is compiled for both Android and JVM targets via shared srcDir.
8
9
package com.ondeinference.onde
10
39
-import android.content.Context
40
-import android.system.Os
11
import kotlinx.coroutines.Dispatchers
12
import kotlinx.coroutines.channels.awaitClose
13
import kotlinx.coroutines.flow.Flow
@@ -46,13 +16,7 @@ import kotlinx.coroutines.withContext
16
import java.io.File
17
18
// ── Public re-exports ─────────────────────────────────────────────────────────
49
-//
50
-// Re-export UniFFI-generated types so callers only need to import from
51
-// com.ondeinference.onde.* — no direct dependency on onde.kt generated types.
19
53
-// These are generated by UniFFI from the Rust crate and land in
54
-// src/generated/kotlin/onde.kt. They are re-exported here as typealiases
55
-// so the public API namespace stays clean.
20
typealias ChatMessage = uniffi.onde.ChatMessage
21
typealias ChatRole = uniffi.onde.ChatRole
22
typealias SamplingConfig = uniffi.onde.SamplingConfig
@@ -66,32 +30,37 @@ typealias InferenceError = uniffi.onde.InferenceException
30
// ── OndeInference ─────────────────────────────────────────────────────────────
31
32
/**
69
- * On-device LLM inference engine for Android.
33
+ * On-device LLM inference engine.
34
*
71
- * Wraps the Rust/UniFFI [uniffi.onde.OndeChatEngine] with Android-specific
35
+ * Wraps the Rust/UniFFI [uniffi.onde.OndeChatEngine] with platform-specific
36
* filesystem initialisation and idiomatic Kotlin coroutine APIs.
37
*
38
* The engine is thread-safe. All suspending functions dispatch to
39
* [Dispatchers.IO] internally — call them from any coroutine scope.
40
*
77
- * @param context Android application or activity context used to resolve
78
- * the app's internal storage directory for the HuggingFace model cache.
79
- * Pass `applicationContext` to avoid leaking activity references.
41
+ * Create instances via the platform-specific factory functions:
42
+ * - **Android:** `OndeInference(context)` or `OndeInference(context, dataDir)`
43
+ * - **JVM:** `OndeInference()` or `OndeInference(dataDir)`
44
*
81
- * @param dataDir Optional override for the root cache directory. Defaults to
82
- * `context.filesDir`. Useful in tests or when the app wants to use a
83
- * different storage location (e.g. external storage on devices with limited
84
- * internal space).
45
+ * @param dataDir Root directory for the HuggingFace model cache and temp files.
46
+ * @param platform Platform-specific operations (env vars, native lib loading).
47
*/
86
-class OndeInference(
87
- private val context: Context,
88
- private val dataDir: File = context.filesDir,
48
+class OndeInference internal constructor(
49
+ private val dataDir: File,
50
+ private val platform: PlatformSupport,
51
) : AutoCloseable {
52
53
+ init {
54
+ // Load the native library before any UniFFI type is accessed.
55
+ // On Android this is a no-op (handled by System.loadLibrary in the APK).
56
+ // On JVM this extracts libonde from JAR resources.
57
+ platform.ensureNativeLoaded()
58
+ }
59
+
60
// The underlying UniFFI engine — thread-safe Arc<OndeChatEngine> under the hood.
61
private val engine = uniffi.onde.OndeChatEngine()
62
94
- // Tracks whether the Android filesystem environment has been seeded.
63
+ // Tracks whether the filesystem environment has been seeded.
64
@Volatile private var configured = false
65
66
// ── Filesystem setup ──────────────────────────────────────────────────────
@@ -118,18 +87,16 @@ class OndeInference(
87
val hfHubCache = File(hfHome, "hub").also { it.mkdirs() }
88
val tmpDir = File(dataDir, "tmp").also { it.mkdirs() }
89
121
- Os.setenv("HF_HOME", hfHome.absolutePath, true)
122
- Os.setenv("HF_HUB_CACHE", hfHubCache.absolutePath, true)
123
- Os.setenv("HUGGINGFACE_HUB_CACHE", hfHubCache.absolutePath, true)
124
- Os.setenv("TMPDIR", tmpDir.absolutePath, true)
90
+ platform.setEnv("HF_HOME", hfHome.absolutePath)
91
+ platform.setEnv("HF_HUB_CACHE", hfHubCache.absolutePath)
92
+ platform.setEnv("HUGGINGFACE_HUB_CACHE", hfHubCache.absolutePath)
93
+ platform.setEnv("TMPDIR", tmpDir.absolutePath)
94
95
configured = true
96
}
97
98
/**
99
* Resolve the directory where HuggingFace model files are stored.
131
- *
132
- * Returns the path even if setup has not been called yet.
100
*/
101
val modelCacheDir: File
102
get() = File(File(dataDir, "models"), "hub")
@@ -139,18 +106,16 @@ class OndeInference(
106
/**
107
* Load the platform-appropriate default model.
108
*
142
- * On Android this is **Qwen 2.5 1.5B (GGUF Q4_K_M, ~941 MB)**,
143
- * running on CPU via the Candle backend.
109
+ * On Android this is **Qwen 2.5 1.5B (GGUF Q4_K_M, ~941 MB)** on CPU.
110
+ * On JVM/macOS this is **Qwen 2.5 3B (GGUF Q4_K_M, ~1.93 GB)** on Metal.
111
*
112
* If the model is not already cached locally it will be downloaded from
146
- * HuggingFace Hub on first call — ensure the app has INTERNET permission
147
- * and sufficient storage space (~1 GB free).
113
+ * HuggingFace Hub on first call.
114
*
115
* @param systemPrompt Optional system prompt injected before the first turn.
150
- * @param sampling Optional sampling configuration. Defaults to
151
- * [SamplingConfig.defaultSamplingConfig] when null.
116
+ * @param sampling Optional sampling configuration.
117
* @return Wall-clock model loading time in seconds.
153
- * @throws InferenceError.ModelBuild if the model cannot be loaded.
118
+ * @throws InferenceError if the model cannot be loaded.
119
*/
120
suspend fun loadDefaultModel(
121
systemPrompt: String? = null,
@@ -170,9 +135,6 @@ class OndeInference(
135
* @param systemPrompt Optional system prompt.
136
* @param sampling Optional sampling configuration.
137
* @return Wall-clock model loading time in seconds.
173
- * @throws InferenceError.ModelBuild if the model cannot be loaded.
174
- * @throws InferenceError.AlreadyLoaded if a model is already loaded —
175
- * call [unload] first.
138
*/
139
suspend fun loadModel(
140
config: GgufModelConfig,
@@ -187,19 +149,12 @@ class OndeInference(
149
)
150
}
151
190
- /**
191
- * Unload the current model and free all associated memory.
192
- *
193
- * @return The display name of the model that was unloaded, or null if no
194
- * model was loaded.
195
- */
152
+ /** Unload the current model and free all associated memory. */
153
suspend fun unload(): String? = withContext(Dispatchers.IO) {
154
engine.unloadModel()
155
}
156
200
- /**
201
- * Check whether a model is currently loaded and ready.
202
- */
157
+ /** Check whether a model is currently loaded and ready. */
158
suspend fun isLoaded(): Boolean = withContext(Dispatchers.IO) {
159
engine.isLoaded()
160
}
@@ -208,14 +163,7 @@ class OndeInference(
163
164
/**
165
* Send a user message and receive a complete assistant reply.
211
- *
212
- * The user turn and assistant reply are automatically appended to the
213
- * engine's conversation history.
214
- *
215
- * @param message The user's message text.
216
- * @return The completed inference result including generated text and timing.
217
- * @throws InferenceError.NoModelLoaded if no model has been loaded.
218
- * @throws InferenceError.Inference if inference fails at runtime.
166
+ * The user turn and assistant reply are appended to conversation history.
167
*/
168
suspend fun chat(message: String): InferenceResult =
169
withContext(Dispatchers.IO) { engine.sendMessage(message) }
@@ -223,29 +171,13 @@ class OndeInference(
171
/**
172
* Stream a user message, emitting each token delta as it is generated.
173
*
226
- * The user turn and assembled assistant reply are automatically appended
227
- * to the conversation history when streaming completes.
228
- *
229
- * Collect the returned [Flow] from a coroutine scope tied to a UI
230
- * lifecycle to ensure automatic cancellation:
231
- *
232
- * ```kotlin
233
- * lifecycleScope.launch {
234
- * onde.stream("Tell me a joke.").collect { chunk ->
235
- * textView.append(chunk.delta)
236
- * }
237
- * }
238
- * ```
239
- *
174
* @param message The user's message text.
241
- * @return A cold [Flow] that emits [StreamChunk] values including a final
242
- * chunk with [StreamChunk.done] set to true.
175
+ * @return A cold [Flow] of [StreamChunk] values.
176
*/
177
fun stream(message: String): Flow<StreamChunk> = callbackFlow {
178
val listener = object : uniffi.onde.StreamChunkListener {
179
override fun onChunk(chunk: StreamChunk): Boolean {
180
val result = trySend(chunk)
248
- // Stop if the downstream collector has cancelled.
181
return result.isSuccess && !chunk.done
182
}
183
}
@@ -260,14 +192,7 @@ class OndeInference(
192
}
193
194
/**
263
- * Run a one-shot inference on an explicit message list WITHOUT modifying
264
- * the engine's conversation history.
265
- *
266
- * Useful for parallel prompt enhancement, summarisation, or any task where
267
- * you want to keep the main chat history clean.
268
- *
269
- * @param messages Explicit list of messages to run inference on.
270
- * @param sampling Optional sampling configuration override.
195
+ * Run a one-shot inference WITHOUT modifying conversation history.
196
*/
197
suspend fun generate(
198
messages: List<ChatMessage>,
@@ -278,122 +203,54 @@ class OndeInference(
203
204
// ── Engine state ──────────────────────────────────────────────────────────
205
281
- /**
282
- * Get a snapshot of the engine's current state.
283
- *
284
- * @return [EngineInfo] describing load status, model name, approximate
285
- * memory, and conversation history length.
286
- */
206
+ /** Get a snapshot of the engine's current state. */
207
suspend fun info(): EngineInfo = withContext(Dispatchers.IO) { engine.info() }
208
209
// ── System prompt ─────────────────────────────────────────────────────────
210
291
- /**
292
- * Set or replace the system prompt without reloading the model.
293
- *
294
- * Takes effect on the next inference call.
295
- */
211
+ /** Set or replace the system prompt without reloading the model. */
212
suspend fun setSystemPrompt(prompt: String) = withContext(Dispatchers.IO) {
213
engine.setSystemPrompt(prompt)
214
}
215
300
- /**
301
- * Clear the system prompt.
302
- */
216
+ /** Clear the system prompt. */
217
suspend fun clearSystemPrompt() = withContext(Dispatchers.IO) {
218
engine.clearSystemPrompt()
219
}
220
221
// ── Sampling ──────────────────────────────────────────────────────────────
222
309
- /**
310
- * Replace the sampling configuration without reloading the model.
311
- *
312
- * Takes effect on the next inference call.
313
- */
223
+ /** Replace the sampling configuration without reloading the model. */
224
suspend fun setSampling(sampling: SamplingConfig) = withContext(Dispatchers.IO) {
225
engine.setSampling(sampling)
226
}
227
228
// ── Conversation history ──────────────────────────────────────────────────
229
320
- /**
321
- * Get a copy of the full conversation history.
322
- */
230
+ /** Get a copy of the full conversation history. */
231
suspend fun history(): List<ChatMessage> = withContext(Dispatchers.IO) {
232
engine.history()
233
}
234
327
- /**
328
- * Clear the conversation history while keeping the model loaded.
329
- *
330
- * @return The number of turns that were removed.
331
- */
235
+ /** Clear the conversation history. Returns the number of turns removed. */
236
suspend fun clearHistory(): ULong = withContext(Dispatchers.IO) {
237
engine.clearHistory()
238
}
239
336
- /**
337
- * Append a message to the conversation history without running inference.
338
- *
339
- * Useful for injecting context or restoring a previous session.
340
- */
240
+ /** Append a message to conversation history without running inference. */
241
suspend fun pushHistory(message: ChatMessage) = withContext(Dispatchers.IO) {
242
engine.pushHistory(message)
243
}
244
245
// ── AutoCloseable ─────────────────────────────────────────────────────────
246
347
- /**
348
- * Release all native resources held by this engine.
349
- *
350
- * The underlying Rust engine is reference-counted via [Arc] — this call
351
- * decrements the count and frees memory when the count reaches zero.
352
- * Safe to call multiple times.
353
- */
247
override fun close() {
248
engine.destroy()
249
}
250
358
- // ── Internal helpers ──────────────────────────────────────────────────────
251
+ // ── Internal ──────────────────────────────────────────────────────────────
252
253
private fun ensureConfigured() {
254
if (!configured) setup()
255
}
256
}
364
-
365
-// ── Convenience constructors ──────────────────────────────────────────────────
366
-
367
-/**
368
- * Construct preconfigured [SamplingConfig] instances without importing the
369
- * generated uniffi package directly.
370
- */
371
-object OndeSampling {
372
- /** Creative chat defaults — temperature 0.7, top_p 0.95, max 512 tokens. */
373
- fun default(): SamplingConfig = uniffi.onde.defaultSamplingConfig()
374
- /** Greedy/deterministic — temperature 0.0, max 512 tokens. */
375
- fun deterministic(): SamplingConfig = uniffi.onde.deterministicSamplingConfig()
376
- /** Conservative mobile — temperature 0.7, max 128 tokens. */
377
- fun mobile(): SamplingConfig = uniffi.onde.mobileSamplingConfig()
378
-}
379
-
380
-/**
381
- * Construct [GgufModelConfig] instances for supported Onde models.
382
- */
383
-object OndeModels {
384
- /** Platform-appropriate default — Qwen 2.5 1.5B on Android (~941 MB). */
385
- fun default(): GgufModelConfig = uniffi.onde.defaultModelConfig()
386
- /** Qwen 2.5 1.5B Instruct GGUF Q4_K_M (~941 MB). */
387
- fun qwen25_1_5b(): GgufModelConfig = uniffi.onde.qwen2515bConfig()
388
- /** Qwen 2.5 3B Instruct GGUF Q4_K_M (~1.93 GB). */
389
- fun qwen25_3b(): GgufModelConfig = uniffi.onde.qwen253bConfig()
390
-}
391
-
392
-/**
393
- * Construct [ChatMessage] values without importing the generated package.
394
- */
395
-object OndeMessage {
396
- fun system(content: String): ChatMessage = uniffi.onde.systemMessage(content)
397
- fun user(content: String): ChatMessage = uniffi.onde.userMessage(content)
398
- fun assistant(content: String): ChatMessage = uniffi.onde.assistantMessage(content)
399
-}
sdk/kotlin/lib/src/shared/kotlin/com/ondeinference/onde/PlatformSupport.kt
new
+33
@@ -0,0 +1,33 @@
1
+// Copyright 2025 Onde Inference (Splitfire AB). All rights reserved.
2
+// SPDX-License-Identifier: MIT OR Apache-2.0
3
+
4
+package com.ondeinference.onde
5
+
6
+/**
7
+ * Platform-specific operations required by [OndeInference].
8
+ *
9
+ * Each platform target provides an implementation:
10
+ * - Android: [AndroidPlatform] — uses `android.system.Os.setenv`
11
+ * - JVM: [JvmPlatform] — uses JNA to call libc `setenv`
12
+ */
13
+internal interface PlatformSupport {
14
+ /**
15
+ * Set a process-level environment variable.
16
+ *
17
+ * This is used to configure `HF_HOME`, `HF_HUB_CACHE`, `TMPDIR` etc.
18
+ * before any Rust/HuggingFace Hub operation.
19
+ */
20
+ fun setEnv(key: String, value: String)
21
+
22
+ /**
23
+ * Ensure the native `libonde` library is loaded.
24
+ *
25
+ * On Android this is a no-op — `System.loadLibrary("onde")` is handled
26
+ * by the UniFFI-generated companion object init block, which loads from
27
+ * the APK's `jniLibs/` directory automatically.
28
+ *
29
+ * On JVM this extracts the bundled `.dylib`/`.so` from JAR resources
30
+ * and calls `System.load(path)` before UniFFI's init can run.
31
+ */
32
+ fun ensureNativeLoaded() {}
33
+}
sdk/kotlin/scripts/build-android.sh
+1
-1
@@ -30,7 +30,7 @@ set -euo pipefail
30
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
31
SDK_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
32
REPO_ROOT="$(cd "$SDK_DIR/../.." && pwd)"
33
-JNILIBS_DIR="$SDK_DIR/lib/src/main/jniLibs"
33
+JNILIBS_DIR="$SDK_DIR/lib/src/androidMain/jniLibs"
34
35
PROFILE="release"
36
for arg in "$@"; do
sdk/kotlin/scripts/build-jvm.sh
new
+118
@@ -0,0 +1,118 @@
1
+#!/usr/bin/env bash
2
+# build-jvm.sh
3
+#
4
+# Builds the Onde Rust crate as a shared library for the host JVM platform
5
+# and copies the output into lib/src/jvmMain/resources/native/<os-arch>/ for
6
+# bundling in the JVM JAR.
7
+#
8
+# This script detects the host OS and architecture automatically. For
9
+# cross-compilation, set ONDE_TARGET_TRIPLE explicitly.
10
+#
11
+# Prerequisites
12
+# -------------
13
+# 1. Rust stable toolchain installed
14
+# 2. On macOS: Xcode command line tools (for Metal backend)
15
+# 3. On Linux: standard build tools (gcc, etc.)
16
+#
17
+# Usage
18
+# -----
19
+# ./scripts/build-jvm.sh # host platform, release
20
+# ./scripts/build-jvm.sh --debug # host platform, debug
21
+# ONDE_TARGET_TRIPLE=x86_64-apple-darwin ./scripts/build-jvm.sh # cross-compile
22
+
23
+set -euo pipefail
24
+
25
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
26
+SDK_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
27
+REPO_ROOT="$(cd "$SDK_DIR/../.." && pwd)"
28
+RESOURCES_DIR="$SDK_DIR/lib/src/jvmMain/resources/native"
29
+
30
+PROFILE="release"
31
+for arg in "$@"; do
32
+ if [[ "$arg" == "--debug" ]]; then
33
+ PROFILE="debug"
34
+ fi
35
+done
36
+
37
+# ── Detect host platform ──────────────────────────────────────────────────────
38
+
39
+detect_platform() {
40
+ local os arch
41
+
42
+ case "$(uname -s)" in
43
+ Darwin) os="macos" ;;
44
+ Linux) os="linux" ;;
45
+ *) echo "Error: unsupported OS: $(uname -s)"; exit 1 ;;
46
+ esac
47
+
48
+ case "$(uname -m)" in
49
+ arm64|aarch64) arch="aarch64" ;;
50
+ x86_64|amd64) arch="x86_64" ;;
51
+ *) echo "Error: unsupported architecture: $(uname -m)"; exit 1 ;;
52
+ esac
53
+
54
+ echo "$os-$arch"
55
+}
56
+
57
+# Map our platform label to a Rust target triple
58
+platform_to_triple() {
59
+ case "$1" in
60
+ macos-aarch64) echo "aarch64-apple-darwin" ;;
61
+ macos-x86_64) echo "x86_64-apple-darwin" ;;
62
+ linux-aarch64) echo "aarch64-unknown-linux-gnu" ;;
63
+ linux-x86_64) echo "x86_64-unknown-linux-gnu" ;;
64
+ *) echo "Error: unknown platform: $1"; exit 1 ;;
65
+ esac
66
+}
67
+
68
+# Map our platform label to the shared library extension
69
+platform_ext() {
70
+ case "$1" in
71
+ macos-*) echo "dylib" ;;
72
+ linux-*) echo "so" ;;
73
+ *) echo "Error: unknown platform: $1"; exit 1 ;;
74
+ esac
75
+}
76
+
77
+PLATFORM="$(detect_platform)"
78
+TARGET_TRIPLE="${ONDE_TARGET_TRIPLE:-$(platform_to_triple "$PLATFORM")}"
79
+LIB_EXT="$(platform_ext "$PLATFORM")"
80
+
81
+echo "=== Building Onde for JVM (profile: $PROFILE) ==="
82
+echo "Platform: $PLATFORM"
83
+echo "Rust target: $TARGET_TRIPLE"
84
+echo "Repository root: $REPO_ROOT"
85
+
86
+# ── Build ──────────────────────────────────────────────────────────────────────
87
+
88
+cd "$REPO_ROOT"
89
+
90
+CARGO_ARGS=(build --manifest-path Cargo.toml --target "$TARGET_TRIPLE")
91
+if [[ "$PROFILE" == "release" ]]; then
92
+ CARGO_ARGS+=(--release)
93
+fi
94
+
95
+echo "→ cargo ${CARGO_ARGS[*]}"
96
+cargo "${CARGO_ARGS[@]}"
97
+
98
+# ── Copy to resources ──────────────────────────────────────────────────────────
99
+
100
+SRC="$REPO_ROOT/target/$TARGET_TRIPLE/$PROFILE/libonde.$LIB_EXT"
101
+DST_DIR="$RESOURCES_DIR/$PLATFORM"
102
+DST="$DST_DIR/libonde.$LIB_EXT"
103
+
104
+if [[ ! -f "$SRC" ]]; then
105
+ echo "Error: expected output not found: $SRC"
106
+ exit 1
107
+fi
108
+
109
+mkdir -p "$DST_DIR"
110
+cp "$SRC" "$DST"
111
+echo "✓ $PLATFORM -> $DST"
112
+
113
+echo
114
+echo "JVM build complete."
115
+echo "Native library copied to: $DST"
116
+echo
117
+echo "To build the JVM JAR with the bundled native library:"
118
+echo " cd $SDK_DIR && ./gradlew :lib:jvmJar"