@ondeinference / cli
Code Commits Issues Pulls Cloud Agent CI/CD
main
development main
md 196 lines 6.9 KB
Rendered Raw
1 <p align="center">
2 <img src="https://raw.githubusercontent.com/ondeinference/onde/refs/heads/development/assets/onde-inference-logo.svg" alt="Onde Inference" width="96">
3 </p>
4
5 <h1 align="center">Onde Inference CLI</h1>
6
7 <p align="center">
8 Command-line interface for <a href="https://ondeinference.com/">Onde Inference</a>.
9 </p>
10
11 <p align="center">
12 <a href="https://ondeinference.com"><img src="https://img.shields.io/badge/ondeinference.com-235843?style=flat-square&labelColor=17211D" alt="Website"></a>
13 <a href="https://apps.apple.com/se/developer/splitfire-ab/id1831430993"><img src="https://img.shields.io/badge/App%20Store-live-235843?style=flat-square&labelColor=17211D" alt="App Store"></a>
14 <a href="https://www.npmjs.com/package/@ondeinference/cli"><img src="https://img.shields.io/npm/v/@ondeinference/cli?style=flat-square&labelColor=17211D&color=235843" alt="npm"></a>
15 <a href="https://pypi.org/project/onde-cli/"><img src="https://img.shields.io/pypi/v/onde-cli?style=flat-square&labelColor=17211D&color=235843" alt="PyPI"></a>
16 <a href="https://pub.dev/packages/onde_cli"><img src="https://img.shields.io/pub/v/onde_cli?style=flat-square&labelColor=17211D&color=235843" alt="pub.dev"></a>
17 <a href="https://www.nuget.org/packages/Onde.Cli"><img src="https://img.shields.io/nuget/v/Onde.Cli?style=flat-square&labelColor=17211D&color=235843" alt="NuGet"></a>
18 <a href="https://crates.io/crates/onde-cli"><img src="https://img.shields.io/crates/v/onde-cli?style=flat-square&labelColor=17211D&color=235843" alt="Crates.io"></a>
19 </p>
20
21 <p align="center">
22 <a href="https://github.com/ondeinference/onde-swift">Swift</a> · <a href="https://pub.dev/packages/onde_inference">Flutter</a> · <a href="https://www.npmjs.com/package/@ondeinference/react-native">React Native</a> · <a href="https://crates.io/crates/onde">Rust</a> · <a href="https://ondeinference.com">Website</a>
23 </p>
24
25 ---
26
27 Manage your Onde Inference account, fine-tune local models, and export them to GGUF, all from the terminal.
28
29 ## Install
30
31 [Install onde-cli](https://github.com/ondeinference/onde-cli) with your favorite tool. For package docs and the full install matrix, see <https://ondeinference.com/cli>.
32
33 ### npm
34
35 ```sh
36 npm install -g @ondeinference/cli
37 ```
38
39 ### Homebrew
40
41 ```sh
42 brew tap ondeinference/homebrew-tap && brew trust --tap ondeinference/homebrew-tap
43 brew install onde
44 ```
45
46 ### pip / uv / uvx
47
48 ```sh
49 pip install onde-cli
50 # or
51 uv tool install onde-cli
52 uv run onde --tui
53 # or with
54 uvx --from onde-cli onde --tui
55 ```
56
57 ### .NET tool
58
59 ```sh
60 dotnet tool install --global Onde.Cli
61 ```
62
63 ### Dart pub global
64
65 ```sh
66 dart pub global activate onde_cli
67 ```
68
69 The Dart package is a thin launcher. On first run it downloads the right native binary into `~/.onde/cli`, then reuses the local copy.
70
71 ### Pre-built binary
72
73 Download a release from [GitHub Releases](https://github.com/ondeinference/onde-cli/releases):
74
75 ```sh
76 # macOS Apple Silicon
77 curl -Lo onde https://github.com/ondeinference/onde-cli/releases/latest/download/onde-macos-arm64
78 chmod +x onde && mv onde /usr/local/bin/onde
79 ```
80
81 | Platform | File |
82 |---|---|
83 | macOS Apple Silicon | `onde-macos-arm64` |
84 | macOS Intel | `onde-macos-amd64` |
85 | Linux x64 | `onde-linux-amd64` |
86 | Linux arm64 | `onde-linux-arm64` |
87 | Windows x64 | `onde-win-amd64.exe` |
88 | Windows arm64 | `onde-win-arm64.exe` |
89
90 ---
91
92 ## Usage
93
94 ```sh
95 onde --tui
96 ```
97
98 This opens the TUI. Plain `onde` prints usage and exits. You can sign up or sign in right there.
99
100 | Key | What it does |
101 |---|---|
102 | `Tab` | Move between fields |
103 | `Enter` | Submit or sign out |
104 | `Ctrl+L` | Go to the sign-in screen |
105 | `Ctrl+N` | Go to the new account screen |
106 | `Ctrl+C` | Quit |
107
108 ### MCP server
109
110 Run `onde` as a [Model Context Protocol](https://modelcontextprotocol.io/) server over stdio instead of the TUI:
111
112 ```sh
113 onde --mcp
114 ```
115
116 This exposes Onde account and model-catalog operations as MCP tools — `login`, `me`, `apps_list`, `app_create`, `app_rename`, `models_list`, `model_register`, `model_assign`, `hf_search` — returning structured JSON. stdout is the JSON-RPC channel; tools run non-interactively and reuse the token from a TUI sign-in (or the `login` tool). Point any MCP client at the command `onde --mcp`.
117
118 ---
119
120 ## Fine-tuning
121
122 `onde` includes a LoRA fine-tuning pipeline for Qwen2, Qwen2.5, and Qwen3 models. It runs locally: Metal on Apple Silicon, CPU elsewhere. No cloud setup. No Python environment.
123
124 The flow is straightforward: download a safetensors base model, fine-tune it with LoRA, merge the adapter back into the base weights, then export to GGUF for use in the Onde SDK.
125
126 If you want a quick refresher on what the model is actually doing at inference time, Onde has a short note on the [forward pass](https://ondeinference.com/forward-pass).
127
128 ### Training data format
129
130 Each line should be one complete conversation in Qwen's chat template:
131
132 ```jsonl
133 {"text": "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nWhat is LoRA?<|im_end|>\n<|im_start|>assistant\nLoRA adds small trainable matrices to frozen layers, letting you fine-tune large models without updating all the weights.<|im_end|>"}
134 ```
135
136 Save the file wherever you want. The TUI lets you point to it directly.
137
138 ### Running it
139
140 ```
141 onde --tui
142 → Models tab (Tab from Apps)
143 → Select a safetensors model (↑↓, Enter)
144 → Press f
145 ```
146
147 Only safetensors models can be fine-tuned. GGUF models are already quantized, so their weights are not differentiable.
148
149 Configure the run:
150
151 | Field | Default | Notes |
152 |---|---|---|
153 | Training data | `~/.onde/finetune/train.jsonl` | Path to your JSONL file |
154 | LoRA rank | `8` | Higher means more capacity and more memory use |
155 | Epochs | `3` | Full passes over the dataset |
156 | Learning rate | `0.0001` | AdamW default |
157
158 Press `Enter` to start. In a healthy run, loss usually starts dropping by epoch 2. If it stays flat, try `0.0003`.
159
160 ### After training
161
162 For rank 8 on a 0.6B model, the adapter is about 1.5 MB. From the fine-tune complete screen:
163
164 - `m` to merge the adapter into the base model
165 - `g` to export the merged model to GGUF
166
167 The resulting GGUF loads directly in the [Onde SDK](https://ondeinference.com/sdk) for on-device AI inference.
168
169 ### Supported base models
170
171 | Model | Size | Notes |
172 |---|---|---|
173 | `Qwen/Qwen3-0.6B` | ~1.2 GB | Smallest and quickest to train |
174 | `Qwen/Qwen2.5-1.5B-Instruct` | ~3.0 GB | Good default for instruction tuning |
175 | `Qwen/Qwen3-1.7B` | ~3.4 GB | Newer small Qwen3 model |
176 | `Qwen/Qwen3-4B` | ~8.0 GB | Best quality, better suited to macOS |
177
178 You can search for any of these from the Models tab with `/`.
179
180 ---
181
182 ## Debug
183
184 Logs are written to `~/.cache/onde/debug.log`.
185
186 If you installed through pub.dev, the launcher cache lives under `~/.onde/cli`.
187
188 ---
189
190 ## License
191
192 Dual-licensed under [MIT](https://github.com/ondeinference/onde-cli/blob/main/LICENSE-MIT) and [Apache 2.0](https://github.com/ondeinference/onde-cli/blob/main/LICENSE-APACHE).
193
194 ## Copyright
195
196 © 2026 [Splitfire AB](https://5mb.app) ([Onde Inference](https://ondeinference.com)).