| 1 | <p align="center"> |
| 2 | <img src="https://raw.githubusercontent.com/ondeinference/onde/main/assets/onde-inference-logo.svg" alt="Onde Inference" width="96"> |
| 3 | </p> |
| 4 | |
| 5 | <h1 align="center">Onde Inference</h1> |
| 6 | |
| 7 | <p align="center"> |
| 8 | <strong>Run LLMs on-device from Swift with <a href="https://ondeinference.com/">Onde Inference</a>, with first-class support for <a href="https://en.wikipedia.org/wiki/Apple_silicon">Apple silicon</a>.</strong> |
| 9 | </p> |
| 10 | |
| 11 | <p align="center"> |
| 12 | <a href="https://swiftpackageindex.com/ondeinference/onde-swift"><img src="https://img.shields.io/badge/Swift%20Package%20Index-onde--swift-235843?style=flat-square&labelColor=17211D" alt="Swift Package Index"></a> |
| 13 | <a href="https://ondeinference.com"><img src="https://img.shields.io/badge/ondeinference.com-235843?style=flat-square&labelColor=17211D" alt="Website"></a> |
| 14 | <a href="https://github.com/ondeinference/onde/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-235843?style=flat-square&labelColor=17211D" alt="License"></a> |
| 15 | </p> |
| 16 | |
| 17 | <p align="center"> |
| 18 | <a href="https://github.com/ondeinference/onde">Rust SDK</a> · <a href="https://central.sonatype.com/artifact/com.ondeinference/onde-inference">Kotlin Multiplatform SDK</a> · <a href="https://pub.dev/packages/onde_inference">Flutter SDK</a> · <a href="https://www.npmjs.com/package/@ondeinference/react-native">React Native SDK</a> · <a href="https://ondeinference.com">Website</a> |
| 19 | </p> |
| 20 | |
| 21 | ## Compatibility |
| 22 | |
| 23 | [](https://swiftpackageindex.com/ondeinference/onde-swift) |
| 24 | [](https://swiftpackageindex.com/ondeinference/onde-swift) |
| 25 | |
| 26 | --- |
| 27 | |
| 28 | ## Installation |
| 29 | |
| 30 | In Xcode, open **File → Add Package Dependencies** and enter: |
| 31 | |
| 32 | ``` |
| 33 | https://github.com/ondeinference/onde-swift |
| 34 | ``` |
| 35 | |
| 36 | Or add it to your `Package.swift`: |
| 37 | |
| 38 | ```swift |
| 39 | dependencies: [ |
| 40 | .package(url: "https://github.com/ondeinference/onde-swift", from: "0.1.0") |
| 41 | ], |
| 42 | targets: [ |
| 43 | .target(name: "MyApp", dependencies: [ |
| 44 | .product(name: "Onde", package: "onde-swift") |
| 45 | ]) |
| 46 | ] |
| 47 | ``` |
| 48 | |
| 49 | ## Platforms |
| 50 | |
| 51 | | Platform | Minimum | GPU Backend | |
| 52 | |----------|---------|-------------| |
| 53 | | iOS | 15.0 | Metal | |
| 54 | | macOS | 11.0 | Metal | |
| 55 | | tvOS | 15.0 | Metal | |
| 56 | |
| 57 | ## Quick Start |
| 58 | |
| 59 | ```swift |
| 60 | import Onde |
| 61 | |
| 62 | let engine = OndeChatEngine() |
| 63 | |
| 64 | // Load the default model for the current device: |
| 65 | // iOS / tvOS → Qwen 2.5 Coder 1.5B (~941 MB) |
| 66 | // macOS → Qwen 2.5 Coder 3B (~1.93 GB) |
| 67 | let elapsed = try await engine.loadDefaultModel( |
| 68 | systemPrompt: "You are a helpful assistant.", |
| 69 | sampling: nil |
| 70 | ) |
| 71 | |
| 72 | let result = try await engine.sendMessage(message: "Hello!") |
| 73 | print(result.text) |
| 74 | ``` |
| 75 | |
| 76 | ## Streaming |
| 77 | |
| 78 | ```swift |
| 79 | import Onde |
| 80 | |
| 81 | class StreamHandler: StreamChunkListener { |
| 82 | func onChunk(chunk: StreamChunk) -> Bool { |
| 83 | print(chunk.delta, terminator: "") |
| 84 | return !chunk.done |
| 85 | } |
| 86 | } |
| 87 | |
| 88 | try await streamChatMessage( |
| 89 | engine: engine, |
| 90 | message: "Tell me a story.", |
| 91 | listener: StreamHandler() |
| 92 | ) |
| 93 | ``` |
| 94 | |
| 95 | ## One-Shot Generation |
| 96 | |
| 97 | This runs inference without changing conversation history: |
| 98 | |
| 99 | ```swift |
| 100 | let result = try await engine.generate( |
| 101 | messages: [userMessage(content: "Expand: a cat in space")], |
| 102 | sampling: deterministicSamplingConfig() |
| 103 | ) |
| 104 | ``` |
| 105 | |
| 106 | ## Sandboxed App Setup |
| 107 | |
| 108 | On iOS and App Store macOS, the default `~/.cache/huggingface` path sits outside the app sandbox. This helper points `HF_HOME` at the App Group shared container (`group.com.ondeinference.apps`) so Onde-powered apps can share downloaded models. Call it once at app launch before creating an `OndeChatEngine`. |
| 109 | |
| 110 | ```swift |
| 111 | import Foundation |
| 112 | |
| 113 | func setupInferenceEnvironment() { |
| 114 | guard let container = FileManager.default.containerURL( |
| 115 | forSecurityApplicationGroupIdentifier: "group.com.ondeinference.apps" |
| 116 | ) else { |
| 117 | // Fall back to the app's private Application Support directory if the App Group is unavailable. |
| 118 | let appSupport = FileManager.default |
| 119 | .urls(for: .applicationSupportDirectory, in: .userDomainMask)[0] |
| 120 | setupHfCache(at: appSupport) |
| 121 | return |
| 122 | } |
| 123 | setupHfCache(at: container) |
| 124 | } |
| 125 | |
| 126 | private func setupHfCache(at base: URL) { |
| 127 | let hfHome = base.appendingPathComponent("models") |
| 128 | let hfHub = hfHome.appendingPathComponent("hub") |
| 129 | try? FileManager.default.createDirectory(at: hfHub, withIntermediateDirectories: true) |
| 130 | |
| 131 | setenv("HF_HOME", hfHome.path, 1) |
| 132 | setenv("HF_HUB_CACHE", hfHub.path, 1) |
| 133 | |
| 134 | let tmp = base.appendingPathComponent("tmp") |
| 135 | try? FileManager.default.createDirectory(at: tmp, withIntermediateDirectories: true) |
| 136 | setenv("TMPDIR", tmp.path, 1) |
| 137 | } |
| 138 | ``` |
| 139 | |
| 140 | ## Local Development |
| 141 | |
| 142 | For local SDK work, keep `onde-swift` next to the Rust repo: |
| 143 | |
| 144 | ```text |
| 145 | Repositories/ |
| 146 | ├── onde/ |
| 147 | └── onde-swift/ |
| 148 | ``` |
| 149 | |
| 150 | Then from the `onde-swift` root: |
| 151 | |
| 152 | ```bash |
| 153 | make ios |
| 154 | ``` |
| 155 | |
| 156 | That builds the local Rust core from `../onde`, regenerates `Sources/Onde/onde.swift`, and writes `OndeFramework.xcframework` into the package root so Xcode and SwiftPM can link it. |
| 157 | |
| 158 | To work on the example iOS app: |
| 159 | |
| 160 | ```bash |
| 161 | cd Examples/OndeExample |
| 162 | xcodegen generate |
| 163 | open OndeExample.xcodeproj |
| 164 | ``` |
| 165 | |
| 166 | ## Documentation |
| 167 | |
| 168 | - [on-device AI inference for iOS and macOS](https://ondeinference.com/sdk/swift) |
| 169 | - [Developer Guide](https://docs.rs/onde/latest/onde/) |
| 170 | - [Onde Inference Rust Crate](https://github.com/ondeinference/onde) |
| 171 | - [Onde CLI](https://github.com/ondeinference/onde-cli), for testing model downloads and GGUF export before wiring them into a Swift app |
| 172 | |
| 173 | ## License |
| 174 | |
| 175 | Onde is dual-licensed under **MIT** and **Apache 2.0**. You can use either one. |
| 176 | |
| 177 | - [MIT License](https://github.com/ondeinference/onde/blob/main/LICENSE-MIT) |
| 178 | - [Apache License 2.0](https://github.com/ondeinference/onde/blob/main/LICENSE-APACHE) |
| 179 | |
| 180 | --- |
| 181 | |
| 182 | ## Copyright |
| 183 | |
| 184 | © 2026 [Onde Inference](https://ondeinference.com) (Splitfire AB). |