@ondeinference / onde
Code Commits Issues Pulls Cloud Agent CI/CD
feature/diffusion-vision
copilot/assess-inference-engine-complexity copilot/fix-onde-npm-sdk-v1-1-2-release development feature/diffusion-vision main
md 228 lines 7.02 KB
Rendered Raw
1 # Onde — Ruby Gem
2
3 Ruby bindings for the [Onde](https://ondeinference.com) on-device inference engine, powered by Rust via [magnus](https://github.com/matsadler/magnus).
4
5 Manage HuggingFace model caches, inspect supported models, and access inference configuration — all from Ruby, with zero C dependencies.
6
7 ## Requirements
8
9 - Ruby >= 3.0
10 - Rust >= 1.87 (installed via [rustup](https://rustup.rs))
11 - macOS, Linux, or Windows
12
13 ## Installation
14
15 Add to your Gemfile:
16
17 ```ruby
18 gem 'onde'
19 ```
20
21 Then:
22
23 ```bash
24 bundle install
25 ```
26
27 Or install directly:
28
29 ```bash
30 gem install onde
31 ```
32
33 The gem compiles a native Rust extension on install. Rust must be available on your `$PATH`.
34
35 ## Development Setup
36
37 ```bash
38 cd lib/crates/onde/gem
39
40 # Install dependencies and compile the native extension
41 bin/setup
42
43 # Or step by step:
44 bundle install
45 bundle exec rake compile
46
47 # Open an interactive console with the gem loaded
48 bin/console
49 ```
50
51 ## Building & Testing
52
53 ```bash
54 # Compile the Rust extension
55 bundle exec rake compile
56
57 # Build the gem package
58 bundle exec rake build
59
60 # Run the default task (compile)
61 bundle exec rake
62 ```
63
64 ## API Reference
65
66 ### `Onde.list_local_models` → Hash
67
68 Scans the local HuggingFace hub cache (`~/.cache/huggingface/hub/`) and returns all downloaded models that the inference engine supports.
69
70 ```ruby
71 response = Onde.list_local_models
72
73 response["cache_path"] # => "/Users/you/.cache/huggingface/hub"
74 response["total_size_bytes"] # => 24768454656
75 response["total_size_display"] # => "23.07 GB"
76 response["models"].each do |model|
77 puts "#{model["model_id"]} — #{model["size_display"]} (#{model["revisions"].length} revisions)"
78 puts " Path: #{model["path"]}"
79 end
80 ```
81
82 ### `Onde.list_supported_models` → Hash
83
84 Returns all models the engine supports with download status flags.
85
86 ```ruby
87 Onde.list_supported_models["models"].each do |m|
88 status = if m["is_downloaded"]
89 "✓ downloaded"
90 elsif m["is_incomplete"]
91 "⏳ incomplete (#{m["local_size_display"]} / #{m["expected_size_display"]})"
92 else
93 "✗ not downloaded"
94 end
95 puts "[#{status}] #{m["name"]} (#{m["org"]})"
96 puts " #{m["description"]}"
97 end
98 ```
99
100 Each model Hash contains:
101
102 | Key | Type | Description |
103 |-------------------------|---------|----------------------------------------------------|
104 | `model_id` | String | Full HuggingFace ID, e.g. `"bartowski/Qwen2.5-1.5B-Instruct-GGUF"` |
105 | `name` | String | Human-friendly display name |
106 | `org` | String | Organisation or publisher |
107 | `description` | String | Short description of the model |
108 | `is_downloaded` | Boolean | Fully downloaded locally |
109 | `is_incomplete` | Boolean | Partial download exists on disk |
110 | `local_size_bytes` | Integer | Bytes currently on disk |
111 | `local_size_display` | String | Human-readable local size |
112 | `expected_size_bytes` | Integer | Approximate total size when fully downloaded |
113 | `expected_size_display` | String | Human-readable expected size |
114
115 ### `Onde.delete_model(model_id)` → nil
116
117 Delete a locally cached model. Raises `RuntimeError` if the model is not found or deletion fails.
118
119 ```ruby
120 Onde.delete_model("bartowski/Qwen2.5-1.5B-Instruct-GGUF")
121 ```
122
123 ### `Onde.model_info(model_id)` → Hash or nil
124
125 Look up rich metadata for a single supported model. Returns `nil` if the model ID is not in the supported list.
126
127 ```ruby
128 info = Onde.model_info("bartowski/Qwen2.5-1.5B-Instruct-GGUF")
129 # => {
130 # "id" => "bartowski/Qwen2.5-1.5B-Instruct-GGUF",
131 # "name" => "Qwen 2.5 1.5B (GGUF)",
132 # "org" => "Qwen / Alibaba",
133 # "description" => "Lightest pre-quantized chat model — ideal for iOS & Android (~941 MB)...",
134 # "expected_size_bytes" => 986048768
135 # }
136 ```
137
138 ### `Onde.supported_model_ids` → Array
139
140 Returns the list of all supported model IDs as strings.
141
142 ```ruby
143 Onde.supported_model_ids
144 # => ["black-forest-labs/FLUX.1-schnell", "google/gemma-3n-E2B-it", ...]
145 ```
146
147 ### `Onde.cache_path` → String or nil
148
149 Returns the resolved HuggingFace cache directory path, or `nil` if it cannot be determined.
150
151 ```ruby
152 Onde.cache_path
153 # => "/Users/you/.cache/huggingface/hub"
154 ```
155
156 ### Sampling Config Helpers
157
158 Return sampling parameter Hashes suitable for passing to inference engines:
159
160 ```ruby
161 Onde.default_sampling_config
162 # => { "temperature" => 0.7, "top_p" => 0.95, "max_tokens" => 512, ... }
163
164 Onde.deterministic_sampling_config
165 # => { "temperature" => 0.0, "max_tokens" => 512, ... }
166
167 Onde.mobile_sampling_config
168 # => { "temperature" => 0.7, "top_p" => 0.95, "max_tokens" => 128, ... }
169 ```
170
171 ### Constants
172
173 ```ruby
174 Onde::VERSION # => "0.1.0" (gem version)
175 Onde::NATIVE_VERSION # => "0.1.0" (Rust crate version)
176 Onde::SUPPORTED_MODELS # => ["black-forest-labs/FLUX.1-schnell", ...] (frozen Array)
177 Onde::SUPPORTED_MODEL_INFO # => [{ "id" => ..., "name" => ..., ... }, ...] (frozen Array of frozen Hashes)
178 ```
179
180 ## Example: Rails Admin Dashboard
181
182 ```ruby
183 # app/controllers/admin/models_controller.rb
184 class Admin::ModelsController < ApplicationController
185 def index
186 @supported = Onde.list_supported_models["models"]
187 @local = Onde.list_local_models
188 end
189
190 def destroy
191 Onde.delete_model(params[:model_id])
192 redirect_to admin_models_path, notice: "Model deleted."
193 rescue RuntimeError => e
194 redirect_to admin_models_path, alert: e.message
195 end
196 end
197 ```
198
199 ## Architecture
200
201 ```
202 gem/
203 ├── onde.gemspec # Gem specification
204 ├── Gemfile # Dev dependencies (rake-compiler, rb_sys)
205 ├── Rakefile # Build tasks via RbSys::ExtensionTask
206 ├── rust-toolchain.toml # Pinned Rust toolchain version
207 ├── lib/
208 │ ├── onde.rb # Entry point — loads native ext + defines module
209 │ └── onde/
210 │ └── version.rb # Onde::VERSION
211 ├── ext/
212 │ └── onde/
213 │ ├── Cargo.toml # Rust crate — depends on `onde` + `magnus`
214 │ ├── extconf.rb # rb_sys build hook
215 │ └── src/
216 │ └── lib.rs # Magnus bindings — the bridge between Rust and Ruby
217 └── bin/
218 ├── console # IRB with gem loaded
219 └── setup # One-step dev setup
220 ```
221
222 The native extension compiles the `onde` Rust crate (which wraps [mistral.rs](https://github.com/EricLBuehler/mistral.rs) for LLM inference and HuggingFace Hub for model management) into a shared library (`.bundle` on macOS, `.so` on Linux) that Ruby loads at require time.
223
224 All Rust structs are converted to Ruby Hashes via `serde_json` serialization, keeping the binding layer thin and the Ruby API simple — no custom Ruby classes needed.
225
226 ## License
227
228 MIT OR Apache-2.0 — same as the Onde crate.