onde_inference — Example App

A Flutter chat app that runs LLM inference fully on-device.
No server. No API key. No data leaving the device.

What this example shows

This example is a full Material 3 chat app built with the onde_inference Flutter SDK. It puts the main SDK features in one place, inside a single file, lib/main.dart:

Feature	Where
`OndeInference.init()` + sandbox cache setup	`main()`
Synchronous `OndeChatEngine()` factory (no `await`, no nulls)	`_ChatScreenState`
Platform-aware default model loading (`loadDefaultModel`)	`_loadModel()`
Multi-turn streaming chat via `streamMessage()`	`_sendMessage()`
Live `EngineInfo` status bar (model name, memory, history length)	`_EngineStatusBar`
`OndeError` sealed-class error handling	`_loadModel()`, `_sendMessage()`
Sampling preset switcher (creative / precise / fast)	`_SamplingPreset`
Unload and reload model at runtime	`_unloadModel()`, `_loadModel()`
Blinking cursor animation during streaming	`_BlinkingCursor`

Running the example

Prerequisites

Flutter SDK ≥ 3.10
Rust toolchain (stable), required to compile the native inference engine
A physical device or simulator/emulator for your target platform

The first build usually takes 5 to 10 minutes because it has to compile the full Rust dependency tree (mistral.rs + candle). After that, builds are much faster.

Steps

# From the repo root
cd onde/sdk/dart/example

# Install Dart dependencies
flutter pub get

# Run on your connected device or simulator
flutter run

On iOS / macOS, the Rust engine is built through the CocoaPods podspec and linked automatically by Xcode. On Android, a Gradle task cross-compiles it with cargo-ndk into jniLibs, so you need cargo install cargo-ndk plus the Android Rust targets. On Linux / Windows, it is linked as a shared library.

Building all three Android ABIs takes a long time. Pass the ABI you actually need while iterating:

flutter run -d <device> --android-project-arg=onde.androidAbis=arm64-v8a

Platform notes

Platform	GPU backend	Default model loaded
iOS (device)	Metal	Qwen 2.5 1.5B (~941 MB)
macOS	Metal	Qwen 2.5 3B (~1.93 GB)
Android (arm64)	CPU	Qwen 2.5 1.5B (~941 MB)
Linux / Windows	CPU	Qwen 2.5 3B (~1.93 GB)

The model is downloaded from Hugging Face Hub on first launch and cached locally, so you only need an internet connection once.

SDK quick reference

import 'package:onde_inference/onde_inference.dart';

// 1. Initialise once at startup
await OndeInference.init();

// 2. Create the engine and load the platform-appropriate model
final engine = OndeChatEngine();
await engine.loadDefaultModel(systemPrompt: 'You are a helpful assistant.');

// 3. Stream a reply token-by-token
await for (final chunk in engine.streamMessage('Hello!')) {
  stdout.write(chunk.delta);
  if (chunk.done) break;
}

// 4. Release memory when done
await engine.unloadModel();

Project structure

example/
├── lib/
│   └── main.dart        # Complete chat UI + SDK integration
├── android/             # Android Gradle build
├── ios/                 # iOS Xcode project + CocoaPods
├── macos/               # macOS Xcode project + CocoaPods
├── linux/               # Linux CMake build
└── windows/             # Windows CMake build

Learn more

onde_inference on pub.dev, for the full API reference
Onde Inference docs, for guides, the model catalog, and platform setup
onde on GitHub, for the Rust engine source, Swift SDK, and issue tracker

onde_inference — Example App

#What this example shows

#Running the example

#Prerequisites

#Steps

#Platform notes

#SDK quick reference

#Project structure

#Learn more

#Copyright