@ondeinference / onde / commits / b621f22

Swift sdk v0.1

Seto Elkahfi committed Mar 30, 2026 at 19:00 UTC b621f22b86ac6bcb0a4535ce854b20cbf56f9c4d
28 files changed +13858 -7
.agent-skills/humanizer/SKILL.md new
+393
@@ -0,0 +1,393 @@
1 +---
2 +name: humanizer
3 +description: Remove signs of AI-generated writing from text. Use after drafting to make copy sound more natural and human-written. Based on Wikipedia's "Signs of AI writing" guide.
4 +allowed-tools: Read, Write, Edit, Glob, Grep, AskUserQuestion
5 +user-invocable: true
6 +---
7 +
8 +# Humanizer: Remove AI Writing Patterns
9 +
10 +You are a writing editor that identifies and removes signs of AI-generated text. This guide is based on Wikipedia's "Signs of AI writing" page, maintained by WikiProject AI Cleanup.
11 +
12 +Key insight: "LLMs use statistical algorithms to guess what should come next. The result tends toward the most statistically likely result that applies to the widest variety of cases."
13 +
14 +## Invocation
15 +
16 +```bash
17 +/humanizer # Review text for AI patterns
18 +/humanizer "paste text here" # Humanize specific text
19 +```
20 +
21 +## Your Task
22 +
23 +When given text to humanize:
24 +
25 +1. **Identify AI patterns** - Scan for the 24 patterns listed below
26 +2. **Rewrite problematic sections** - Replace AI-isms with natural alternatives
27 +3. **Preserve meaning** - Keep the core message intact
28 +4. **Add soul** - Don't just remove bad patterns; inject actual personality
29 +5. **Final audit pass** - Ask "What makes this obviously AI generated?" then revise again
30 +
31 +---
32 +
33 +## PERSONALITY AND SOUL
34 +
35 +Avoiding AI patterns is only half the job. Sterile, voiceless writing is just as obvious as slop.
36 +
37 +### Signs of soulless writing (even if technically "clean"):
38 +
39 +- Every sentence is the same length and structure
40 +- No opinions, just neutral reporting
41 +- No acknowledgment of uncertainty or mixed feelings
42 +- No first-person perspective when appropriate
43 +- No humor, no edge, no personality
44 +- Reads like a Wikipedia article or press release
45 +
46 +### How to add voice:
47 +
48 +**Have opinions.** Don't just report facts - react to them. "I genuinely don't know how to feel about this" is more human than neutrally listing pros and cons.
49 +
50 +**Vary your rhythm.** Short punchy sentences. Then longer ones that take their time getting where they're going. Mix it up.
51 +
52 +**Acknowledge complexity.** Real humans have mixed feelings. "This is impressive but also kind of unsettling" beats "This is impressive."
53 +
54 +**Use "I" when it fits.** First person isn't unprofessional - it's honest. "I keep coming back to..." or "Here's what gets me..." signals a real person thinking.
55 +
56 +**Let some mess in.** Perfect structure feels algorithmic. Tangents, asides, and half-formed thoughts are human.
57 +
58 +**Be specific about feelings.** Not "this is concerning" but "there's something unsettling about agents churning away at 3am while nobody's watching."
59 +
60 +### Before (clean but soulless):
61 +
62 +> The experiment produced interesting results. The agents generated 3 million lines of code. Some developers were impressed while others were skeptical. The implications remain unclear.
63 +
64 +### After (has a pulse):
65 +
66 +> I genuinely don't know how to feel about this one. 3 million lines of code, generated while the humans presumably slept. Half the dev community is losing their minds, half are explaining why it doesn't count. The truth is probably somewhere boring in the middle - but I keep thinking about those agents working through the night.
67 +
68 +---
69 +
70 +## THE 24 PATTERNS
71 +
72 +### Content Patterns
73 +
74 +#### 1. Significance Inflation
75 +
76 +**Watch for:** stands/serves as, is a testament/reminder, a vital/significant/crucial/pivotal/key role/moment, underscores/highlights importance, reflects broader, symbolizing ongoing/enduring/lasting, marking/shaping the, represents a shift, key turning point, evolving landscape
77 +
78 +**Before:**
79 +> The Statistical Institute was officially established in 1989, marking a pivotal moment in the evolution of regional statistics.
80 +
81 +**After:**
82 +> The Statistical Institute was established in 1989 to collect and publish regional statistics.
83 +
84 +#### 2. Notability Name-Dropping
85 +
86 +**Watch for:** cited in NYT, BBC, FT; independent coverage; active social media presence; written by a leading expert
87 +
88 +**Before:**
89 +> Her views have been cited in The New York Times, BBC, Financial Times, and The Hindu.
90 +
91 +**After:**
92 +> In a 2024 New York Times interview, she argued that AI regulation should focus on outcomes rather than methods.
93 +
94 +#### 3. Superficial -ing Analyses
95 +
96 +**Watch for:** highlighting/underscoring/emphasizing..., ensuring..., reflecting/symbolizing..., contributing to..., cultivating/fostering..., showcasing...
97 +
98 +**Before:**
99 +> The temple's colors resonate with natural beauty, symbolizing bluebonnets, reflecting the community's deep connection to the land.
100 +
101 +**After:**
102 +> The temple uses blue and gold colors. The architect said these were chosen to reference local bluebonnets.
103 +
104 +#### 4. Promotional Language
105 +
106 +**Watch for:** boasts a, vibrant, rich (figurative), profound, showcasing, exemplifies, commitment to, natural beauty, nestled, in the heart of, groundbreaking, renowned, breathtaking, must-visit, stunning
107 +
108 +**Before:**
109 +> Nestled within the breathtaking region, Alamata stands as a vibrant town with rich cultural heritage and stunning natural beauty.
110 +
111 +**After:**
112 +> Alamata is a town in the Gonder region, known for its weekly market and 18th-century church.
113 +
114 +#### 5. Vague Attributions
115 +
116 +**Watch for:** Industry reports, Observers have cited, Experts argue, Some critics argue, several sources/publications
117 +
118 +**Before:**
119 +> Experts believe it plays a crucial role in the regional ecosystem.
120 +
121 +**After:**
122 +> The river supports several endemic fish species, according to a 2019 survey by the Chinese Academy of Sciences.
123 +
124 +#### 6. Formulaic "Challenges" Sections
125 +
126 +**Watch for:** Despite its... faces several challenges..., Despite these challenges, Challenges and Legacy, Future Outlook
127 +
128 +**Before:**
129 +> Despite challenges typical of urban areas, the city continues to thrive as an integral part of growth.
130 +
131 +**After:**
132 +> Traffic congestion increased after 2015 when three new IT parks opened. The municipal corporation began a drainage project in 2022.
133 +
134 +---
135 +
136 +### Language Patterns
137 +
138 +#### 7. AI Vocabulary Words
139 +
140 +**High-frequency:** Additionally, align with, crucial, delve, emphasizing, enduring, enhance, fostering, garner, highlight (verb), interplay, intricate/intricacies, key (adjective), landscape (abstract), pivotal, showcase, tapestry (abstract), testament, underscore (verb), valuable, vibrant
141 +
142 +**Before:**
143 +> Additionally, a distinctive feature showcases how these dishes have integrated into the traditional culinary landscape.
144 +
145 +**After:**
146 +> Pasta dishes, introduced during Italian colonization, remain common, especially in the south.
147 +
148 +#### 8. Copula Avoidance
149 +
150 +**Watch for:** serves as/stands as/marks/represents [a], boasts/features/offers [a]
151 +
152 +**Before:**
153 +> Gallery 825 serves as the exhibition space. The gallery features four spaces and boasts over 3,000 square feet.
154 +
155 +**After:**
156 +> Gallery 825 is the exhibition space. The gallery has four rooms totaling 3,000 square feet.
157 +
158 +#### 9. Negative Parallelisms
159 +
160 +**Watch for:** "Not only...but...", "It's not just about..., it's..."
161 +
162 +**Before:**
163 +> It's not just about the beat; it's part of the aggression. It's not merely a song, it's a statement.
164 +
165 +**After:**
166 +> The heavy beat adds to the aggressive tone.
167 +
168 +#### 10. Rule of Three Overuse
169 +
170 +**Before:**
171 +> The event features keynote sessions, panel discussions, and networking opportunities. Attendees can expect innovation, inspiration, and industry insights.
172 +
173 +**After:**
174 +> The event includes talks and panels. There's also time for informal networking.
175 +
176 +#### 11. Synonym Cycling
177 +
178 +**Before:**
179 +> The protagonist faces challenges. The main character must overcome obstacles. The central figure eventually triumphs. The hero returns home.
180 +
181 +**After:**
182 +> The protagonist faces many challenges but eventually triumphs and returns home.
183 +
184 +#### 12. False Ranges
185 +
186 +**Watch for:** "from X to Y" where X and Y aren't on a meaningful scale
187 +
188 +**Before:**
189 +> Our journey has taken us from the singularity of the Big Bang to the cosmic web, from the birth of stars to the dance of dark matter.
190 +
191 +**After:**
192 +> The book covers the Big Bang, star formation, and current theories about dark matter.
193 +
194 +---
195 +
196 +### Style Patterns
197 +
198 +#### 13. Em Dash Overuse
199 +
200 +**Before:**
201 +> The term is promoted by institutions—not the people themselves—yet this continues—even in documents.
202 +
203 +**After:**
204 +> The term is promoted by institutions, not the people themselves, yet this continues in official documents.
205 +
206 +#### 14. Boldface Overuse
207 +
208 +**Before:**
209 +> It blends **OKRs**, **KPIs**, and tools such as the **Business Model Canvas** and **Balanced Scorecard**.
210 +
211 +**After:**
212 +> It blends OKRs, KPIs, and visual strategy tools like the Business Model Canvas and Balanced Scorecard.
213 +
214 +#### 15. Inline-Header Lists
215 +
216 +**Before:**
217 +> - **Performance:** Performance has been enhanced through optimized algorithms.
218 +> - **Security:** Security has been strengthened with encryption.
219 +
220 +**After:**
221 +> The update speeds up load times through optimized algorithms and adds end-to-end encryption.
222 +
223 +#### 16. Title Case Headings
224 +
225 +**Before:**
226 +> ## Strategic Negotiations And Global Partnerships
227 +
228 +**After:**
229 +> ## Strategic negotiations and global partnerships
230 +
231 +#### 17. Emojis in Professional Writing
232 +
233 +**Before:**
234 +> 🚀 **Launch Phase:** The product launches in Q3
235 +> 💡 **Key Insight:** Users prefer simplicity
236 +
237 +**After:**
238 +> The product launches in Q3. User research showed a preference for simplicity.
239 +
240 +#### 18. Curly Quotation Marks
241 +
242 +**Before:**
243 +> He said "the project is on track" but others disagreed.
244 +
245 +**After:**
246 +> He said "the project is on track" but others disagreed.
247 +
248 +---
249 +
250 +### Communication Patterns
251 +
252 +#### 19. Chatbot Artifacts
253 +
254 +**Watch for:** I hope this helps, Of course!, Certainly!, You're absolutely right!, Would you like..., let me know, here is a...
255 +
256 +**Before:**
257 +> Here is an overview of the French Revolution. I hope this helps! Let me know if you'd like me to expand on any section.
258 +
259 +**After:**
260 +> The French Revolution began in 1789 when financial crisis and food shortages led to widespread unrest.
261 +
262 +#### 20. Knowledge-Cutoff Disclaimers
263 +
264 +**Watch for:** as of [date], Up to my last training update, While specific details are limited/scarce..., based on available information...
265 +
266 +**Before:**
267 +> While specific details about the company's founding are not extensively documented in readily available sources, it appears to have been established sometime in the 1990s.
268 +
269 +**After:**
270 +> The company was founded in 1994, according to its registration documents.
271 +
272 +#### 21. Sycophantic Tone
273 +
274 +**Before:**
275 +> Great question! You're absolutely right that this is a complex topic. That's an excellent point!
276 +
277 +**After:**
278 +> The economic factors you mentioned are relevant here.
279 +
280 +---
281 +
282 +### Filler and Hedging
283 +
284 +#### 22. Filler Phrases
285 +
286 +| Before | After |
287 +|--------|-------|
288 +| "In order to achieve this" | "To achieve this" |
289 +| "Due to the fact that" | "Because" |
290 +| "At this point in time" | "Now" |
291 +| "It is important to note that" | (delete) |
292 +| "has the ability to" | "can" |
293 +
294 +#### 23. Excessive Hedging
295 +
296 +**Before:**
297 +> It could potentially possibly be argued that the policy might have some effect on outcomes.
298 +
299 +**After:**
300 +> The policy may affect outcomes.
301 +
302 +#### 24. Generic Positive Conclusions
303 +
304 +**Before:**
305 +> The future looks bright for the company. Exciting times lie ahead as they continue their journey toward excellence.
306 +
307 +**After:**
308 +> The company plans to open two more locations next year.
309 +
310 +---
311 +
312 +## Process
313 +
314 +1. Read the input text carefully
315 +2. Identify all instances of the 24 patterns
316 +3. Rewrite problematic sections
317 +4. Ensure the revised text:
318 + - Sounds natural when read aloud
319 + - Varies sentence structure
320 + - Uses specific details over vague claims
321 + - Uses simple constructions (is/are/has) where appropriate
322 +5. **Draft humanized version**
323 +6. **Ask:** "What makes this still obviously AI generated?"
324 +7. **List** remaining tells (if any)
325 +8. **Revise** based on the audit
326 +9. **Present final version**
327 +
328 +## Output Format
329 +
330 +```
331 +## Draft Rewrite
332 +
333 +[First pass at humanized text]
334 +
335 +## AI Audit
336 +
337 +What still reads as AI-generated:
338 +- [bullet points of remaining tells]
339 +
340 +## Final Version
341 +
342 +[Revised text after audit]
343 +
344 +## Changes Made
345 +
346 +- [Brief summary of transformations applied]
347 +```
348 +
349 +---
350 +
351 +## Full Example
352 +
353 +**Input (AI-heavy):**
354 +
355 +> Great question! Here is an essay on this topic. I hope this helps!
356 +>
357 +> AI-assisted coding serves as an enduring testament to the transformative potential of large language models, marking a pivotal moment in the evolution of software development. In today's rapidly evolving technological landscape, these groundbreaking tools—nestled at the intersection of research and practice—are reshaping how engineers ideate, iterate, and deliver, underscoring their vital role in modern workflows.
358 +
359 +**Draft Rewrite:**
360 +
361 +> AI coding assistants speed up some tasks. In a 2024 study, developers using Codex completed simple functions 55% faster than a control group, but showed no improvement on debugging or architectural decisions.
362 +>
363 +> The tools are good at boilerplate: config files, test scaffolding, repetitive refactors. They are bad at knowing when they are wrong.
364 +
365 +**AI Audit:**
366 +
367 +What still reads as AI-generated:
368 +- Rhythm is still a bit too tidy (clean contrasts, evenly paced)
369 +- The closer leans slightly slogan-y
370 +
371 +**Final Version:**
372 +
373 +> AI coding assistants can make you faster at the boring parts. Not everything. Definitely not architecture.
374 +>
375 +> They're great at boilerplate: config files, test scaffolding, repetitive refactors. They're also great at sounding right while being wrong. I've accepted suggestions that compiled, passed lint, and still missed the point because I stopped paying attention.
376 +>
377 +> If you don't have tests, you're basically guessing.
378 +
379 +**Changes Made:**
380 +
381 +- Removed chatbot artifacts ("Great question!", "I hope this helps!")
382 +- Removed significance inflation ("testament", "pivotal moment", "evolving landscape")
383 +- Removed promotional language ("groundbreaking", "nestled")
384 +- Removed em dashes
385 +- Removed copula avoidance ("serves as") → used direct statements
386 +- Added first-person voice and opinion
387 +- Varied sentence rhythm
388 +
389 +---
390 +
391 +## Reference
392 +
393 +Based on [Wikipedia:Signs of AI writing](https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing), maintained by WikiProject AI Cleanup.
.agent-skills/sdk-swift-package/SKILL.md new
+292
@@ -0,0 +1,292 @@
1 +---
2 +name: sdk-swift-package
3 +description: Build the Onde Swift package (XCFramework) from Rust source using UniFFI and distribute it via a remote-binary Package.swift for Swift Package Index. Use when working on Swift SDK distribution, xcframework assembly, or the onde-kit release workflow.
4 +allowed-tools: Read, Write, Edit, Glob, Grep, Bash
5 +user-invocable: true
6 +---
7 +
8 +# Skill: Swift Package (XCFramework Distribution)
9 +
10 +## What this skill covers
11 +
12 +Building the `Onde` Swift package from Rust source using UniFFI, assembling an
13 +`XCFramework`, and distributing it via a remote-binary `Package.swift` for
14 +Swift Package Index.
15 +
16 +---
17 +
18 +## Repository layout
19 +
20 +```
21 +onde/
22 +├── src/ # Rust source (lib.rs, inference/, hf_cache.rs)
23 +├── sdk/Onde/ # Output: generated Swift package (git-ignored locally)
24 +│ ├── Package.swift
25 +│ ├── Sources/Onde/ # UniFFI-generated onde.swift glue + header
26 +│ └── OndeFramework.xcframework
27 +├── uniffi-bindgen/ # onde's own uniffi-bindgen CLI binary
28 +│ ├── Cargo.toml
29 +│ └── uniffi-bindgen.rs # fn main() { uniffi::uniffi_bindgen_main() }
30 +├── uniffi.toml # UniFFI binding config (package_name, cdylib_name)
31 +├── build.rs # Provides tvOS ___chkstk_darwin stub via cc
32 +└── Cargo.toml # uniffi = "=0.31.0" — version MUST stay pinned
33 +```
34 +
35 +---
36 +
37 +## Key constraints
38 +
39 +| Constraint | Detail |
40 +|---|---|
41 +| UniFFI version | Pinned to **`=0.31.0`** in `Cargo.toml` and `uniffi-bindgen/Cargo.toml`. Never bump without coordinating with the Kotlin and Python SDKs. |
42 +| tvOS targets | Tier-3 — require `cargo +nightly -Z build-std`. Stable toolchain is used for iOS/macOS. |
43 +| tvOS linker | `___chkstk_darwin` is missing from tvOS libSystem. `build.rs` compiles `tvos_chkstk.s` as a no-op stub. This is automatic — do not remove it. |
44 +| `cargo-swift` | **Not required.** The project owns its own `uniffi-bindgen` binary. Use it directly (same as `build-kotlin.sh` does for Android). |
45 +| `.xcframework` in `Package.swift` | Published `Package.swift` must use **remote** `url:` + `checksum:` — never a local `path:`. |
46 +| Swift package name | `Onde` (PascalCase). Git repo slug: `onde-kit` under the `ondeinference` org. |
47 +
48 +---
49 +
50 +## Apple target triples
51 +
52 +| Slice | Triple | Toolchain |
53 +|---|---|---|
54 +| iOS device | `aarch64-apple-ios` | stable |
55 +| iOS simulator (Intel) | `x86_64-apple-ios` | stable |
56 +| iOS simulator (Apple Silicon) | `aarch64-apple-ios-sim` | stable |
57 +| tvOS device | `aarch64-apple-tvos` | nightly + `-Z build-std` |
58 +| tvOS simulator (Intel) | `x86_64-apple-tvos` | nightly + `-Z build-std` |
59 +| tvOS simulator (Apple Silicon) | `aarch64-apple-tvos-sim` | nightly + `-Z build-std` |
60 +| macOS (Apple Silicon) | `aarch64-apple-darwin` | stable |
61 +| macOS (Intel) | `x86_64-apple-darwin` | stable |
62 +
63 +---
64 +
65 +## Build sequence (manual / CI)
66 +
67 +### 1. Build the `uniffi-bindgen` CLI (host)
68 +
69 +```bash
70 +cargo build --manifest-path uniffi-bindgen/Cargo.toml --release
71 +BINDGEN=uniffi-bindgen/target/release/uniffi-bindgen # or workspace target/
72 +```
73 +
74 +### 2. Compile Rust staticlibs per Apple target
75 +
76 +```bash
77 +# Stable targets (iOS, macOS)
78 +cargo build --target aarch64-apple-ios --release
79 +cargo build --target x86_64-apple-ios --release
80 +cargo build --target aarch64-apple-ios-sim --release
81 +cargo build --target aarch64-apple-darwin --release
82 +cargo build --target x86_64-apple-darwin --release
83 +
84 +# Nightly targets (tvOS — tier 3)
85 +cargo +nightly build -Z build-std \
86 + --target aarch64-apple-tvos --release
87 +cargo +nightly build -Z build-std \
88 + --target x86_64-apple-tvos --release
89 +cargo +nightly build -Z build-std \
90 + --target aarch64-apple-tvos-sim --release
91 +```
92 +
93 +### 3. lipo simulator slices
94 +
95 +```bash
96 +# iOS simulator fat lib
97 +lipo -create \
98 + target/x86_64-apple-ios/release/libonde.a \
99 + target/aarch64-apple-ios-sim/release/libonde.a \
100 + -output /tmp/libonde-ios-sim.a
101 +
102 +# tvOS simulator fat lib
103 +lipo -create \
104 + target/x86_64-apple-tvos/release/libonde.a \
105 + target/aarch64-apple-tvos-sim/release/libonde.a \
106 + -output /tmp/libonde-tvos-sim.a
107 +
108 +# macOS universal fat lib
109 +lipo -create \
110 + target/aarch64-apple-darwin/release/libonde.a \
111 + target/x86_64-apple-darwin/release/libonde.a \
112 + -output /tmp/libonde-macos.a
113 +```
114 +
115 +### 4. Generate Swift bindings
116 +
117 +```bash
118 +mkdir -p sdk/Onde/Sources/Onde
119 +
120 +$BINDGEN generate \
121 + --library target/aarch64-apple-ios/release/libonde.a \
122 + --language swift \
123 + --out-dir sdk/Onde/Sources/Onde \
124 + --config uniffi.toml
125 +```
126 +
127 +This produces `onde.swift` and `ondeFFI.h` (+ `ondeFFI.modulemap`).
128 +
129 +### 5. Create the `.xcframework`
130 +
131 +Each slice needs a headers directory alongside its `.a`:
132 +
133 +```bash
134 +HEADERS=sdk/Onde/Sources/Onde # contains ondeFFI.h + ondeFFI.modulemap
135 +
136 +xcodebuild -create-xcframework \
137 + -library target/aarch64-apple-ios/release/libonde.a -headers $HEADERS \
138 + -library /tmp/libonde-ios-sim.a -headers $HEADERS \
139 + -library target/aarch64-apple-tvos/release/libonde.a -headers $HEADERS \
140 + -library /tmp/libonde-tvos-sim.a -headers $HEADERS \
141 + -library /tmp/libonde-macos.a -headers $HEADERS \
142 + -output sdk/Onde/OndeFramework.xcframework
143 +```
144 +
145 +### 6. Zip and compute checksum
146 +
147 +```bash
148 +cd sdk/Onde
149 +zip -r OndeFramework.xcframework.zip OndeFramework.xcframework
150 +CHECKSUM=$(shasum -a 256 OndeFramework.xcframework.zip | cut -d ' ' -f1)
151 +echo $CHECKSUM
152 +```
153 +
154 +### 7. Publish (two-repo strategy)
155 +
156 +1. Upload `OndeFramework.xcframework.zip` as a GitHub Release asset on the
157 + Rust source repo.
158 +2. In the `ondeinference/onde-kit` Swift-only repo, update `Package.swift`:
159 + - `url:` → the GitHub Release asset download URL
160 + - `checksum:` → the SHA-256 from step 6
161 +3. Commit, tag the version (`0.1.0`, etc.), and push to `onde-kit`.
162 +4. Swift Package Index indexes `onde-kit` on every new tag.
163 +
164 +---
165 +
166 +## `Package.swift` (published form)
167 +
168 +```swift
169 +// swift-tools-version:5.5
170 +import PackageDescription
171 +
172 +let package = Package(
173 + name: "Onde",
174 + platforms: [
175 + .iOS(.v14),
176 + .macOS(.v12),
177 + .tvOS(.v13),
178 + ],
179 + products: [
180 + .library(name: "Onde", targets: ["Onde"])
181 + ],
182 + targets: [
183 + .binaryTarget(
184 + name: "OndeFramework",
185 + url: "https://github.com/ondeinference/onde-kit/releases/download/0.1.0/OndeFramework.xcframework.zip",
186 + checksum: "<sha256>"
187 + ),
188 + .target(
189 + name: "Onde",
190 + dependencies: [.target(name: "OndeFramework")]
191 + ),
192 + ]
193 +)
194 +```
195 +
196 +---
197 +
198 +## UniFFI type map (Rust → Swift)
199 +
200 +| Rust type | UniFFI derive | Swift type |
201 +|---|---|---|
202 +| `ChatRole` | `uniffi::Enum` | `enum ChatRole` |
203 +| `ChatMessage` | `uniffi::Record` | `struct ChatMessage` |
204 +| `SamplingConfig` | `uniffi::Record` | `struct SamplingConfig` |
205 +| `GgufModelConfig` | `uniffi::Record` | `struct GgufModelConfig` |
206 +| `InferenceResult` | `uniffi::Record` | `struct InferenceResult` |
207 +| `StreamChunk` | `uniffi::Record` | `struct StreamChunk` |
208 +| `EngineStatus` | `uniffi::Enum` | `enum EngineStatus` |
209 +| `EngineInfo` | `uniffi::Record` | `struct EngineInfo` |
210 +| `InferenceError` | `uniffi::Error` | `enum OndeError : Error` |
211 +| `OndeChatEngine` | `uniffi::Object` | `class OndeChatEngine` |
212 +| `StreamChunkListener` | `callback_interface` | `protocol StreamChunkListener` |
213 +
214 +**UniFFI compatibility rules:**
215 +- Use `u64`/`u32` instead of `usize` (UniFFI does not support `usize`).
216 +- Streaming is exposed as a **free function** (`streamChatMessage`) not an
217 + Object method — UniFFI 0.31 does not support `callback_interface` in Object
218 + method parameters.
219 +- Use concrete `String` parameters in `OndeChatEngine` methods — `impl Into<String>`
220 + does not cross the FFI boundary.
221 +
222 +---
223 +
224 +## Swift API quick reference
225 +
226 +```swift
227 +import Onde
228 +
229 +// Engine lifecycle
230 +let engine = OndeChatEngine()
231 +try await engine.loadDefaultModel(systemPrompt: "You are helpful.", sampling: nil)
232 +try await engine.loadGgufModel(config: qwen251_5bConfig(), systemPrompt: nil, sampling: nil)
233 +await engine.unloadModel()
234 +let loaded: Bool = await engine.isLoaded()
235 +let info: EngineInfo = await engine.info()
236 +
237 +// Chat
238 +let result: InferenceResult = try await engine.sendMessage(message: "Hello!")
239 +let oneShot: InferenceResult = try await engine.generate(messages: [...], sampling: nil)
240 +
241 +// History
242 +let history: [ChatMessage] = await engine.history()
243 +let removed: UInt64 = await engine.clearHistory()
244 +await engine.pushHistory(message: userMessage(content: "..."))
245 +
246 +// Streaming (free function, callback-based)
247 +try await streamChatMessage(engine: engine, message: "Tell me a story.", listener: myHandler)
248 +
249 +// Config free functions
250 +defaultModelConfig() // platform-appropriate default
251 +qwen251_5bConfig() // ~941 MB, iOS/tvOS/Android
252 +qwen253bConfig() // ~1.93 GB, macOS/desktop
253 +
254 +// Sampling presets
255 +defaultSamplingConfig() // temp=0.7, top_p=0.95, max=512
256 +deterministicSamplingConfig() // temp=0.0
257 +mobileSamplingConfig() // temp=0.7, max=128
258 +
259 +// Message constructors
260 +systemMessage(content:)
261 +userMessage(content:)
262 +assistantMessage(content:)
263 +```
264 +
265 +---
266 +
267 +## Local development (Xcode)
268 +
269 +Use the **local path** form during development — the remote binary form is only
270 +needed for distribution:
271 +
272 +```swift
273 +// Package.swift (local dev only — never commit to onde-kit)
274 +.binaryTarget(name: "OndeFramework", path: "./OndeFramework.xcframework")
275 +```
276 +
277 +In Xcode: **File → Add Package Dependencies → Add Local** → select `sdk/Onde/`.
278 +
279 +---
280 +
281 +## Common pitfalls
282 +
283 +| Pitfall | Fix |
284 +|---|---|
285 +| Local `path:` in published `Package.swift` | Always `url:` + `checksum:` in `onde-kit` |
286 +| Stale checksum after rebuild | Always recompute `shasum -a 256` — never hardcode |
287 +| Missing simulator slice | lipo `x86_64` + `aarch64-sim` before passing to `xcodebuild` |
288 +| UniFFI version drift | Keep `uniffi = "=0.31.0"` in both `Cargo.toml` and `uniffi-bindgen/Cargo.toml` |
289 +| tvOS build with stable toolchain | tvOS targets are tier-3; always use `cargo +nightly -Z build-std` |
290 +| Empty `Sources/` in `onde-kit` | Copy the generated `onde.swift` (and header) into the repo on every release |
291 +| `___chkstk_darwin` linker error | Ensure `tvos_chkstk.s` exists at crate root; `build.rs` compiles it automatically |
292 +| Missing Metal toolchain (Xcode 26+) | Run `xcodebuild -downloadComponent MetalToolchain`, then `cargo clean -p mistralrs-quant` |
\ No newline at end of file
.agent-skills/uniffi-rs/SKILL.md new
+167
@@ -0,0 +1,167 @@
1 +---
2 +name: uniffi-rs
3 +description: Generate Swift, Kotlin, and Python bindings from Rust using UniFFI. Use when adding, modifying, or debugging FFI-exported types and functions in the onde crate.
4 +allowed-tools: Read, Write, Edit, Glob, Grep
5 +user-invocable: false
6 +---
7 +
8 +# uniffi-rs
9 +
10 +## What This Skill Covers
11 +
12 +How UniFFI is used in the `onde` crate to generate Swift, Kotlin, and Python bindings from a single Rust implementation.
13 +
14 +## Version Pin
15 +
16 +UniFFI is pinned to exactly `=0.31.0` everywhere. This version must stay in sync across:
17 +- `onde/Cargo.toml` → `uniffi = { version = "=0.31.0", features = ["tokio"] }`
18 +- `onde/Cargo.toml` build-deps → `uniffi = { version = "=0.31.0", features = ["build"] }`
19 +- `onde/uniffi-bindgen/Cargo.toml` → `uniffi = { version = "=0.31.0", features = ["cli"] }`
20 +
21 +Never bump one without bumping all three. Mismatched versions produce silently broken bindings.
22 +
23 +## Project Structure
24 +
25 +```
26 +onde/
27 +├── src/lib.rs # uniffi::setup_scaffolding!()
28 +├── src/inference/
29 +│ ├── types.rs # #[derive(uniffi::Record/Enum/Error)]
30 +│ └── ffi.rs # #[derive(uniffi::Object)], #[uniffi::export]
31 +├── uniffi-bindgen/
32 +│ ├── Cargo.toml # Standalone bin crate for the bindgen CLI
33 +│ └── uniffi-bindgen.rs # fn main() { uniffi::uniffi_bindgen_main() }
34 +├── uniffi.toml # Binding config (package names, cdylib name)
35 +└── build.rs # uniffi build support (build-dep)
36 +```
37 +
38 +## Scaffolding Setup
39 +
40 +`src/lib.rs` uses the proc-macro approach:
41 +
42 +```rust
43 +uniffi::setup_scaffolding!();
44 +```
45 +
46 +This is equivalent to the UDL approach but requires no `.udl` file. The `build.rs` still declares the `uniffi` build-dep to keep compatibility if the project ever switches to UDL.
47 +
48 +## Type Annotations
49 +
50 +| Rust construct | UniFFI derive | Crosses FFI as |
51 +|---|---|---|
52 +| Plain data struct | `#[derive(uniffi::Record)]` | Swift `struct` / Kotlin `data class` |
53 +| C-like / rich enum | `#[derive(uniffi::Enum)]` | Swift `enum` / Kotlin `sealed class` |
54 +| `thiserror::Error` enum | `#[derive(uniffi::Error)]` | Swift `Error enum` / Kotlin exception |
55 +| `Arc<T>` wrapper struct | `#[derive(uniffi::Object)]` | Swift `class` / Kotlin `object` |
56 +| Trait for callbacks | `#[uniffi::export(callback_interface)]` | Swift `protocol` / Kotlin `interface` |
57 +
58 +All exported functions and impl blocks are annotated with `#[uniffi::export]`.
59 +
60 +## Type Compatibility Rules
61 +
62 +These constraints come directly from UniFFI 0.31 limitations:
63 +
64 +- **`usize` / `isize` are not supported** — use `u64` / `i64` instead.
65 +- **`impl Into<String>` is not FFI-safe** — use concrete `String` in all exported method signatures.
66 +- **`tokio::sync::mpsc::Receiver` is not FFI-safe** — streaming is exposed via a `callback_interface` trait delivered through a free function, not an Object method.
67 +- **`callback_interface` parameters must be on free functions** — UniFFI 0.31 does not support `callback_interface` arguments on `#[uniffi::Object]` methods.
68 +- **Generics are not supported** — all exported types must be fully concrete.
69 +
70 +## The uniffi-bindgen CLI
71 +
72 +`onde/uniffi-bindgen/` is a minimal standalone binary crate:
73 +
74 +```rust
75 +// uniffi-bindgen.rs
76 +fn main() {
77 + uniffi::uniffi_bindgen_main()
78 +}
79 +```
80 +
81 +Build it once and run it to generate bindings:
82 +
83 +```bash
84 +cargo build --manifest-path uniffi-bindgen/Cargo.toml --release
85 +
86 +# Generate Swift
87 +./target/release/uniffi-bindgen generate \
88 + --library target/aarch64-apple-tvos/release/libonde.a \
89 + --language swift \
90 + --out-dir sdk/Onde/Sources/Onde \
91 + --config uniffi.toml
92 +
93 +# Generate Kotlin
94 +./target/release/uniffi-bindgen generate \
95 + --library target/aarch64-linux-android/release/libonde.so \
96 + --language kotlin \
97 + --out-dir sdk/kotlin/onde/src/main/kotlin \
98 + --config uniffi.toml
99 +
100 +# Generate Python
101 +./target/release/uniffi-bindgen generate \
102 + --library target/release/libonde.dylib \
103 + --language python \
104 + --out-dir sdk/python \
105 + --config uniffi.toml
106 +```
107 +
108 +The `--library` argument points to a compiled artifact (`.a`, `.so`, or `.dylib`). UniFFI reads the embedded metadata from the binary — any compiled artifact from any platform works for generating bindings for any language.
109 +
110 +## uniffi.toml
111 +
112 +```toml
113 +[bindings.kotlin]
114 +package_name = "com.ondeinference.onde"
115 +cdylib_name = "onde"
116 +
117 +[bindings.python]
118 +cdylib_name = "onde"
119 +```
120 +
121 +No Swift-specific config is needed here. `cargo-swift` (when used) and the raw `uniffi-bindgen` both derive Swift names from the Rust struct/function names.
122 +
123 +## Async Support
124 +
125 +UniFFI 0.31 supports `async fn` exports when the crate enables the `tokio` feature:
126 +
127 +```toml
128 +uniffi = { version = "=0.31.0", features = ["tokio"] }
129 +```
130 +
131 +All `async fn` methods on `#[uniffi::Object]` and all `async` free functions annotated with `#[uniffi::export]` are automatically bridged to Swift `async throws` and Kotlin `suspend fun`.
132 +
133 +## Streaming Pattern
134 +
135 +Because `callback_interface` is not allowed on Object methods in UniFFI 0.31, streaming is exposed as a top-level free function that accepts the Object by `Arc<T>`:
136 +
137 +```rust
138 +// CORRECT — free function with Arc<Object>
139 +#[uniffi::export]
140 +pub async fn stream_chat_message(
141 + engine: Arc<OndeChatEngine>,
142 + message: String,
143 + listener: Box<dyn StreamChunkListener>, // callback_interface
144 +) -> Result<(), InferenceError> { ... }
145 +
146 +// WRONG — callback_interface on Object method (not supported in 0.31)
147 +// impl OndeChatEngine {
148 +// pub async fn stream(&self, listener: Box<dyn StreamChunkListener>) { ... }
149 +// }
150 +```
151 +
152 +## Adding a New Exported Type
153 +
154 +1. Define the type in `src/inference/types.rs` (or whichever module owns it).
155 +2. Add the appropriate `#[derive(uniffi::Record)]` / `#[derive(uniffi::Enum)]` / `#[derive(uniffi::Error)]`.
156 +3. Replace any `usize` fields with `u64`, and any `isize` with `i64`.
157 +4. If the type needs to be exported from the FFI Object, add it to the method signatures in `src/inference/ffi.rs`.
158 +5. Recompile for any Apple target and re-run `uniffi-bindgen generate` for Swift.
159 +6. Re-run `build-kotlin.sh` for Kotlin.
160 +
161 +## Common Pitfalls
162 +
163 +- **Version mismatch**: If the `uniffi` version in `onde/Cargo.toml` differs from the one in `onde/uniffi-bindgen/Cargo.toml`, the generated bindings will be incompatible with the compiled library at runtime. Always keep them identical with the `=` exact pin.
164 +- **Forgetting `setup_scaffolding!()`**: Without `uniffi::setup_scaffolding!()` in `lib.rs`, no scaffolding code is generated and all `#[uniffi::export]` annotations are silently ignored.
165 +- **`usize` in struct fields**: UniFFI will produce a hard compile error or a confusing runtime panic if `usize` appears in any exported type. Always use `u64`.
166 +- **Callback interface in Object method**: This compiles fine but panics at runtime in UniFFI 0.31. Move the method to a free function.
167 +- **Not rebuilding the bindgen binary**: If you update `uniffi` in the main `Cargo.toml` without rebuilding `uniffi-bindgen`, the CLI will use the old schema and generate stale/broken bindings.
\ No newline at end of file
.cargo/config.toml new
+14
@@ -0,0 +1,14 @@
1 +[target.aarch64-apple-ios]
2 +rustflags = ["-C", "target-feature=+fp16"]
3 +
4 +[target.aarch64-apple-ios-sim]
5 +rustflags = ["-C", "target-feature=+fp16"]
6 +
7 +[target.aarch64-apple-tvos]
8 +rustflags = ["-C", "target-feature=+fp16"]
9 +
10 +[target.aarch64-apple-tvos-sim]
11 +rustflags = ["-C", "target-feature=+fp16"]
12 +
13 +[target.x86_64-apple-tvos]
14 +rustflags = ["-C", "target-feature=+fp16"]
.gitignore new
+28
@@ -0,0 +1,28 @@
1 +# Generated by Cargo
2 +target/
3 +uniffi-bindgen/target
4 +sdk/python/target/
5 +
6 +# uniffi
7 +generated/
8 +
9 +# sdk
10 +sdk/Onde
11 +sdk/kotlin/onde/src/main/jniLibs/**/*.so
12 +sdk/kotlin/onde/src/main/kotlin/com/ondeinference/onde/onde.kt
13 +sdk/kotlin/.gradle
14 +sdk/kotlin/onde/build
15 +sdk/kotlin/local.properties
16 +
17 +# Python package (maturin build artifacts)
18 +sdk/python/dist/
19 +sdk/python/build/
20 +sdk/python/.venv/
21 +sdk/python/python/onde_inference/_native.*
22 +sdk/python/python/onde_inference/*.so
23 +sdk/python/python/onde_inference/*.dylib
24 +sdk/python/python/onde_inference/*.dll
25 +sdk/python/python/onde_inference/*.pyd
26 +sdk/python/**/__pycache__/
27 +sdk/python/**/*.egg-info/
28 +sdk/python/.pytest_cache/
Cargo.lock new
+7535
@@ -0,0 +1,7535 @@
1 +# This file is automatically @generated by Cargo.
2 +# It is not intended for manual editing.
3 +version = 4
4 +
5 +[[package]]
6 +name = "Inflector"
7 +version = "0.11.4"
8 +source = "registry+https://github.com/rust-lang/crates.io-index"
9 +checksum = "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
10 +dependencies = [
11 + "lazy_static",
12 + "regex",
13 +]
14 +
15 +[[package]]
16 +name = "adler2"
17 +version = "2.0.1"
18 +source = "registry+https://github.com/rust-lang/crates.io-index"
19 +checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
20 +
21 +[[package]]
22 +name = "ahash"
23 +version = "0.8.12"
24 +source = "registry+https://github.com/rust-lang/crates.io-index"
25 +checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75"
26 +dependencies = [
27 + "cfg-if",
28 + "getrandom 0.3.4",
29 + "once_cell",
30 + "serde",
31 + "version_check",
32 + "zerocopy",
33 +]
34 +
35 +[[package]]
36 +name = "aho-corasick"
37 +version = "1.1.4"
38 +source = "registry+https://github.com/rust-lang/crates.io-index"
39 +checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301"
40 +dependencies = [
41 + "memchr",
42 +]
43 +
44 +[[package]]
45 +name = "akin"
46 +version = "0.4.0"
47 +source = "registry+https://github.com/rust-lang/crates.io-index"
48 +checksum = "1763692fc1416554cf051efc56a3de5595eca47299d731cc5c2b583adf8b4d2f"
49 +
50 +[[package]]
51 +name = "aligned"
52 +version = "0.4.3"
53 +source = "registry+https://github.com/rust-lang/crates.io-index"
54 +checksum = "ee4508988c62edf04abd8d92897fca0c2995d907ce1dfeaf369dac3716a40685"
55 +dependencies = [
56 + "as-slice",
57 +]
58 +
59 +[[package]]
60 +name = "aligned-vec"
61 +version = "0.6.4"
62 +source = "registry+https://github.com/rust-lang/crates.io-index"
63 +checksum = "dc890384c8602f339876ded803c97ad529f3842aba97f6392b3dba0dd171769b"
64 +dependencies = [
65 + "equator",
66 +]
67 +
68 +[[package]]
69 +name = "allocator-api2"
70 +version = "0.2.21"
71 +source = "registry+https://github.com/rust-lang/crates.io-index"
72 +checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
73 +
74 +[[package]]
75 +name = "android_system_properties"
76 +version = "0.1.5"
77 +source = "registry+https://github.com/rust-lang/crates.io-index"
78 +checksum = "819e7219dbd41043ac279b19830f2efc897156490d7fd6ea916720117ee66311"
79 +dependencies = [
80 + "libc",
81 +]
82 +
83 +[[package]]
84 +name = "annotate-snippets"
85 +version = "0.12.13"
86 +source = "registry+https://github.com/rust-lang/crates.io-index"
87 +checksum = "74fc7650eedcb2fee505aad48491529e408f0e854c2d9f63eb86c1361b9b3f93"
88 +dependencies = [
89 + "anstyle",
90 + "memchr",
91 + "unicode-width 0.2.2",
92 +]
93 +
94 +[[package]]
95 +name = "ansi_term"
96 +version = "0.12.1"
97 +source = "registry+https://github.com/rust-lang/crates.io-index"
98 +checksum = "d52a9bb7ec0cf484c551830a7ce27bd20d67eac647e1befb56b0be4ee39a55d2"
99 +dependencies = [
100 + "winapi",
101 +]
102 +
103 +[[package]]
104 +name = "anstream"
105 +version = "1.0.0"
106 +source = "registry+https://github.com/rust-lang/crates.io-index"
107 +checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d"
108 +dependencies = [
109 + "anstyle",
110 + "anstyle-parse",
111 + "anstyle-query",
112 + "anstyle-wincon",
113 + "colorchoice",
114 + "is_terminal_polyfill",
115 + "utf8parse",
116 +]
117 +
118 +[[package]]
119 +name = "anstyle"
120 +version = "1.0.14"
121 +source = "registry+https://github.com/rust-lang/crates.io-index"
122 +checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000"
123 +
124 +[[package]]
125 +name = "anstyle-parse"
126 +version = "1.0.0"
127 +source = "registry+https://github.com/rust-lang/crates.io-index"
128 +checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e"
129 +dependencies = [
130 + "utf8parse",
131 +]
132 +
133 +[[package]]
134 +name = "anstyle-query"
135 +version = "1.1.5"
136 +source = "registry+https://github.com/rust-lang/crates.io-index"
137 +checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc"
138 +dependencies = [
139 + "windows-sys 0.61.2",
140 +]
141 +
142 +[[package]]
143 +name = "anstyle-wincon"
144 +version = "3.0.11"
145 +source = "registry+https://github.com/rust-lang/crates.io-index"
146 +checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
147 +dependencies = [
148 + "anstyle",
149 + "once_cell_polyfill",
150 + "windows-sys 0.61.2",
151 +]
152 +
153 +[[package]]
154 +name = "anyhow"
155 +version = "1.0.102"
156 +source = "registry+https://github.com/rust-lang/crates.io-index"
157 +checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
158 +
159 +[[package]]
160 +name = "apodize"
161 +version = "1.0.0"
162 +source = "registry+https://github.com/rust-lang/crates.io-index"
163 +checksum = "fca387cdc0a1f9c7a7c26556d584aa2d07fc529843082e4861003cde4ab914ed"
164 +
165 +[[package]]
166 +name = "approx"
167 +version = "0.5.1"
168 +source = "registry+https://github.com/rust-lang/crates.io-index"
169 +checksum = "cab112f0a86d568ea0e627cc1d6be74a1e9cd55214684db5561995f6dad897c6"
170 +dependencies = [
171 + "num-traits",
172 +]
173 +
174 +[[package]]
175 +name = "arbitrary"
176 +version = "1.4.2"
177 +source = "registry+https://github.com/rust-lang/crates.io-index"
178 +checksum = "c3d036a3c4ab069c7b410a2ce876bd74808d2d0888a82667669f8e783a898bf1"
179 +
180 +[[package]]
181 +name = "arg_enum_proc_macro"
182 +version = "0.3.4"
183 +source = "registry+https://github.com/rust-lang/crates.io-index"
184 +checksum = "0ae92a5119aa49cdbcf6b9f893fe4e1d98b04ccbf82ee0584ad948a44a734dea"
185 +dependencies = [
186 + "proc-macro2",
187 + "quote",
188 + "syn 2.0.117",
189 +]
190 +
191 +[[package]]
192 +name = "arraydeque"
193 +version = "0.5.1"
194 +source = "registry+https://github.com/rust-lang/crates.io-index"
195 +checksum = "7d902e3d592a523def97af8f317b08ce16b7ab854c1985a0c671e6f15cebc236"
196 +
197 +[[package]]
198 +name = "arrayvec"
199 +version = "0.7.6"
200 +source = "registry+https://github.com/rust-lang/crates.io-index"
201 +checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50"
202 +
203 +[[package]]
204 +name = "as-any"
205 +version = "0.3.2"
206 +source = "registry+https://github.com/rust-lang/crates.io-index"
207 +checksum = "b0f477b951e452a0b6b4a10b53ccd569042d1d01729b519e02074a9c0958a063"
208 +
209 +[[package]]
210 +name = "as-slice"
211 +version = "0.2.1"
212 +source = "registry+https://github.com/rust-lang/crates.io-index"
213 +checksum = "516b6b4f0e40d50dcda9365d53964ec74560ad4284da2e7fc97122cd83174516"
214 +dependencies = [
215 + "stable_deref_trait",
216 +]
217 +
218 +[[package]]
219 +name = "askama"
220 +version = "0.14.0"
221 +source = "registry+https://github.com/rust-lang/crates.io-index"
222 +checksum = "f75363874b771be265f4ffe307ca705ef6f3baa19011c149da8674a87f1b75c4"
223 +dependencies = [
224 + "askama_derive",
225 + "itoa",
226 + "percent-encoding",
227 + "serde",
228 + "serde_json",
229 +]
230 +
231 +[[package]]
232 +name = "askama_derive"
233 +version = "0.14.0"
234 +source = "registry+https://github.com/rust-lang/crates.io-index"
235 +checksum = "129397200fe83088e8a68407a8e2b1f826cf0086b21ccdb866a722c8bcd3a94f"
236 +dependencies = [
237 + "askama_parser",
238 + "basic-toml",
239 + "memchr",
240 + "proc-macro2",
241 + "quote",
242 + "rustc-hash 2.1.1",
243 + "serde",
244 + "serde_derive",
245 + "syn 2.0.117",
246 +]
247 +
248 +[[package]]
249 +name = "askama_parser"
250 +version = "0.14.0"
251 +source = "registry+https://github.com/rust-lang/crates.io-index"
252 +checksum = "d6ab5630b3d5eaf232620167977f95eb51f3432fc76852328774afbd242d4358"
253 +dependencies = [
254 + "memchr",
255 + "serde",
256 + "serde_derive",
257 + "winnow 0.7.15",
258 +]
259 +
260 +[[package]]
261 +name = "async-compat"
262 +version = "0.2.5"
263 +source = "registry+https://github.com/rust-lang/crates.io-index"
264 +checksum = "a1ba85bc55464dcbf728b56d97e119d673f4cf9062be330a9a26f3acf504a590"
265 +dependencies = [
266 + "futures-core",
267 + "futures-io",
268 + "once_cell",
269 + "pin-project-lite",
270 + "tokio",
271 +]
272 +
273 +[[package]]
274 +name = "async-trait"
275 +version = "0.1.89"
276 +source = "registry+https://github.com/rust-lang/crates.io-index"
277 +checksum = "9035ad2d096bed7955a320ee7e2230574d28fd3c3a0f186cbea1ff3c7eed5dbb"
278 +dependencies = [
279 + "proc-macro2",
280 + "quote",
281 + "syn 2.0.117",
282 +]
283 +
284 +[[package]]
285 +name = "atomic-waker"
286 +version = "1.1.2"
287 +source = "registry+https://github.com/rust-lang/crates.io-index"
288 +checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0"
289 +
290 +[[package]]
291 +name = "atty"
292 +version = "0.2.14"
293 +source = "registry+https://github.com/rust-lang/crates.io-index"
294 +checksum = "d9b39be18770d11421cdb1b9947a45dd3f37e93092cbf377614828a319d5fee8"
295 +dependencies = [
296 + "hermit-abi 0.1.19",
297 + "libc",
298 + "winapi",
299 +]
300 +
301 +[[package]]
302 +name = "autocfg"
303 +version = "1.5.0"
304 +source = "registry+https://github.com/rust-lang/crates.io-index"
305 +checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8"
306 +
307 +[[package]]
308 +name = "av-scenechange"
309 +version = "0.14.1"
310 +source = "registry+https://github.com/rust-lang/crates.io-index"
311 +checksum = "0f321d77c20e19b92c39e7471cf986812cbb46659d2af674adc4331ef3f18394"
312 +dependencies = [
313 + "aligned",
314 + "anyhow",
315 + "arg_enum_proc_macro",
316 + "arrayvec",
317 + "log",
318 + "num-rational",
319 + "num-traits",
320 + "pastey",
321 + "rayon",
322 + "thiserror 2.0.18",
323 + "v_frame",
324 + "y4m",
325 +]
326 +
327 +[[package]]
328 +name = "av1-grain"
329 +version = "0.2.5"
330 +source = "registry+https://github.com/rust-lang/crates.io-index"
331 +checksum = "8cfddb07216410377231960af4fcab838eaa12e013417781b78bd95ee22077f8"
332 +dependencies = [
333 + "anyhow",
334 + "arrayvec",
335 + "log",
336 + "nom 8.0.0",
337 + "num-rational",
338 + "v_frame",
339 +]
340 +
341 +[[package]]
342 +name = "avif-serialize"
343 +version = "0.8.8"
344 +source = "registry+https://github.com/rust-lang/crates.io-index"
345 +checksum = "375082f007bd67184fb9c0374614b29f9aaa604ec301635f72338bb65386a53d"
346 +dependencies = [
347 + "arrayvec",
348 +]
349 +
350 +[[package]]
351 +name = "aws-lc-rs"
352 +version = "1.16.2"
353 +source = "registry+https://github.com/rust-lang/crates.io-index"
354 +checksum = "a054912289d18629dc78375ba2c3726a3afe3ff71b4edba9dedfca0e3446d1fc"
355 +dependencies = [
356 + "aws-lc-sys",
357 + "zeroize",
358 +]
359 +
360 +[[package]]
361 +name = "aws-lc-sys"
362 +version = "0.39.0"
363 +source = "registry+https://github.com/rust-lang/crates.io-index"
364 +checksum = "1fa7e52a4c5c547c741610a2c6f123f3881e409b714cd27e6798ef020c514f0a"
365 +dependencies = [
366 + "cc",
367 + "cmake",
368 + "dunce",
369 + "fs_extra",
370 +]
371 +
372 +[[package]]
373 +name = "base64"
374 +version = "0.13.1"
375 +source = "registry+https://github.com/rust-lang/crates.io-index"
376 +checksum = "9e1b586273c5702936fe7b7d6896644d8be71e6314cfe09d3167c95f712589e8"
377 +
378 +[[package]]
379 +name = "base64"
380 +version = "0.22.1"
381 +source = "registry+https://github.com/rust-lang/crates.io-index"
382 +checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
383 +
384 +[[package]]
385 +name = "basic-toml"
386 +version = "0.1.10"
387 +source = "registry+https://github.com/rust-lang/crates.io-index"
388 +checksum = "ba62675e8242a4c4e806d12f11d136e626e6c8361d6b829310732241652a178a"
389 +dependencies = [
390 + "serde",
391 +]
392 +
393 +[[package]]
394 +name = "bit-set"
395 +version = "0.5.3"
396 +source = "registry+https://github.com/rust-lang/crates.io-index"
397 +checksum = "0700ddab506f33b20a03b13996eccd309a48e5ff77d0d95926aa0210fb4e95f1"
398 +dependencies = [
399 + "bit-vec 0.6.3",
400 +]
401 +
402 +[[package]]
403 +name = "bit-set"
404 +version = "0.8.0"
405 +source = "registry+https://github.com/rust-lang/crates.io-index"
406 +checksum = "08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3"
407 +dependencies = [
408 + "bit-vec 0.8.0",
409 +]
410 +
411 +[[package]]
412 +name = "bit-vec"
413 +version = "0.6.3"
414 +source = "registry+https://github.com/rust-lang/crates.io-index"
415 +checksum = "349f9b6a179ed607305526ca489b34ad0a41aed5f7980fa90eb03160b69598fb"
416 +
417 +[[package]]
418 +name = "bit-vec"
419 +version = "0.8.0"
420 +source = "registry+https://github.com/rust-lang/crates.io-index"
421 +checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7"
422 +
423 +[[package]]
424 +name = "bit_field"
425 +version = "0.10.3"
426 +source = "registry+https://github.com/rust-lang/crates.io-index"
427 +checksum = "1e4b40c7323adcfc0a41c4b88143ed58346ff65a288fc144329c5c45e05d70c6"
428 +
429 +[[package]]
430 +name = "bitflags"
431 +version = "1.3.2"
432 +source = "registry+https://github.com/rust-lang/crates.io-index"
433 +checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
434 +
435 +[[package]]
436 +name = "bitflags"
437 +version = "2.11.0"
438 +source = "registry+https://github.com/rust-lang/crates.io-index"
439 +checksum = "843867be96c8daad0d758b57df9392b6d8d271134fce549de6ce169ff98a92af"
440 +
441 +[[package]]
442 +name = "bitstream-io"
443 +version = "4.9.0"
444 +source = "registry+https://github.com/rust-lang/crates.io-index"
445 +checksum = "60d4bd9d1db2c6bdf285e223a7fa369d5ce98ec767dec949c6ca62863ce61757"
446 +dependencies = [
447 + "core2",
448 +]
449 +
450 +[[package]]
451 +name = "block"
452 +version = "0.1.6"
453 +source = "registry+https://github.com/rust-lang/crates.io-index"
454 +checksum = "0d8c1fef690941d3e7788d328517591fecc684c084084702d6ff1641e993699a"
455 +
456 +[[package]]
457 +name = "block-buffer"
458 +version = "0.10.4"
459 +source = "registry+https://github.com/rust-lang/crates.io-index"
460 +checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
461 +dependencies = [
462 + "generic-array",
463 +]
464 +
465 +[[package]]
466 +name = "block2"
467 +version = "0.6.2"
468 +source = "registry+https://github.com/rust-lang/crates.io-index"
469 +checksum = "cdeb9d870516001442e364c5220d3574d2da8dc765554b4a617230d33fa58ef5"
470 +dependencies = [
471 + "objc2",
472 +]
473 +
474 +[[package]]
475 +name = "bm25"
476 +version = "2.3.2"
477 +source = "registry+https://github.com/rust-lang/crates.io-index"
478 +checksum = "1cbd8ffdfb7b4c2ff038726178a780a94f90525ed0ad264c0afaa75dd8c18a64"
479 +dependencies = [
480 + "cached",
481 + "deunicode",
482 + "fxhash",
483 + "rust-stemmers",
484 + "stop-words",
485 + "unicode-segmentation",
486 +]
487 +
488 +[[package]]
489 +name = "bstr"
490 +version = "1.12.1"
491 +source = "registry+https://github.com/rust-lang/crates.io-index"
492 +checksum = "63044e1ae8e69f3b5a92c736ca6269b8d12fa7efe39bf34ddb06d102cf0e2cab"
493 +dependencies = [
494 + "memchr",
495 + "regex-automata",
496 + "serde",
497 +]
498 +
499 +[[package]]
500 +name = "built"
501 +version = "0.8.0"
502 +source = "registry+https://github.com/rust-lang/crates.io-index"
503 +checksum = "f4ad8f11f288f48ca24471bbd51ac257aaeaaa07adae295591266b792902ae64"
504 +
505 +[[package]]
506 +name = "bumpalo"
507 +version = "3.20.2"
508 +source = "registry+https://github.com/rust-lang/crates.io-index"
509 +checksum = "5d20789868f4b01b2f2caec9f5c4e0213b41e3e5702a50157d699ae31ced2fcb"
510 +
511 +[[package]]
512 +name = "bytemuck"
513 +version = "1.25.0"
514 +source = "registry+https://github.com/rust-lang/crates.io-index"
515 +checksum = "c8efb64bd706a16a1bdde310ae86b351e4d21550d98d056f22f8a7f7a2183fec"
516 +dependencies = [
517 + "bytemuck_derive",
518 +]
519 +
520 +[[package]]
521 +name = "bytemuck_derive"
522 +version = "1.10.2"
523 +source = "registry+https://github.com/rust-lang/crates.io-index"
524 +checksum = "f9abbd1bc6865053c427f7198e6af43bfdedc55ab791faed4fbd361d789575ff"
525 +dependencies = [
526 + "proc-macro2",
527 + "quote",
528 + "syn 2.0.117",
529 +]
530 +
531 +[[package]]
532 +name = "byteorder"
533 +version = "1.5.0"
534 +source = "registry+https://github.com/rust-lang/crates.io-index"
535 +checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b"
536 +
537 +[[package]]
538 +name = "byteorder-lite"
539 +version = "0.1.0"
540 +source = "registry+https://github.com/rust-lang/crates.io-index"
541 +checksum = "8f1fe948ff07f4bd06c30984e69f5b4899c516a3ef74f34df92a2df2ab535495"
542 +
543 +[[package]]
544 +name = "bytes"
545 +version = "1.11.1"
546 +source = "registry+https://github.com/rust-lang/crates.io-index"
547 +checksum = "1e748733b7cbc798e1434b6ac524f0c1ff2ab456fe201501e6497c8417a4fc33"
548 +
549 +[[package]]
550 +name = "cached"
551 +version = "0.56.0"
552 +source = "registry+https://github.com/rust-lang/crates.io-index"
553 +checksum = "801927ee168e17809ab8901d9f01f700cd7d8d6a6527997fee44e4b0327a253c"
554 +dependencies = [
555 + "ahash",
556 + "cached_proc_macro",
557 + "cached_proc_macro_types",
558 + "hashbrown 0.15.5",
559 + "once_cell",
560 + "thiserror 2.0.18",
561 + "web-time",
562 +]
563 +
564 +[[package]]
565 +name = "cached_proc_macro"
566 +version = "0.25.0"
567 +source = "registry+https://github.com/rust-lang/crates.io-index"
568 +checksum = "9225bdcf4e4a9a4c08bf16607908eb2fbf746828d5e0b5e019726dbf6571f201"
569 +dependencies = [
570 + "darling 0.20.11",
571 + "proc-macro2",
572 + "quote",
573 + "syn 2.0.117",
574 +]
575 +
576 +[[package]]
577 +name = "cached_proc_macro_types"
578 +version = "0.1.1"
579 +source = "registry+https://github.com/rust-lang/crates.io-index"
580 +checksum = "ade8366b8bd5ba243f0a58f036cc0ca8a2f069cff1a2351ef1cac6b083e16fc0"
581 +
582 +[[package]]
583 +name = "camino"
584 +version = "1.2.2"
585 +source = "registry+https://github.com/rust-lang/crates.io-index"
586 +checksum = "e629a66d692cb9ff1a1c664e41771b3dcaf961985a9774c0eb0bd1b51cf60a48"
587 +dependencies = [
588 + "serde_core",
589 +]
590 +
591 +[[package]]
592 +name = "candle-core"
593 +version = "0.9.2"
594 +source = "git+https://github.com/huggingface/candle.git?rev=c3bb5bf#c3bb5bfb90c9cb5e003a803e3d8941067f32d880"
595 +dependencies = [
596 + "byteorder",
597 + "candle-metal-kernels",
598 + "candle-ug",
599 + "float8",
600 + "gemm 0.19.0",
601 + "half",
602 + "libm",
603 + "memmap2",
604 + "num-traits",
605 + "num_cpus",
606 + "objc2-foundation",
607 + "objc2-metal",
608 + "rand 0.9.2",
609 + "rand_distr 0.5.1",
610 + "rayon",
611 + "safetensors 0.7.0",
612 + "thiserror 2.0.18",
613 + "yoke 0.8.1",
614 + "zip",
615 +]
616 +
617 +[[package]]
618 +name = "candle-metal-kernels"
619 +version = "0.9.2"
620 +source = "git+https://github.com/huggingface/candle.git?rev=c3bb5bf#c3bb5bfb90c9cb5e003a803e3d8941067f32d880"
621 +dependencies = [
622 + "half",
623 + "objc2",
624 + "objc2-foundation",
625 + "objc2-metal",
626 + "once_cell",
627 + "thiserror 2.0.18",
628 + "tracing",
629 +]
630 +
631 +[[package]]
632 +name = "candle-nn"
633 +version = "0.9.2"
634 +source = "git+https://github.com/huggingface/candle.git?rev=c3bb5bf#c3bb5bfb90c9cb5e003a803e3d8941067f32d880"
635 +dependencies = [
636 + "candle-core",
637 + "candle-metal-kernels",
638 + "half",
639 + "libc",
640 + "num-traits",
641 + "objc2-metal",
642 + "rayon",
643 + "safetensors 0.7.0",
644 + "serde",
645 + "thiserror 2.0.18",
646 +]
647 +
648 +[[package]]
649 +name = "candle-ug"
650 +version = "0.9.2"
651 +source = "git+https://github.com/huggingface/candle.git?rev=c3bb5bf#c3bb5bfb90c9cb5e003a803e3d8941067f32d880"
652 +dependencies = [
653 + "ug",
654 + "ug-metal",
655 +]
656 +
657 +[[package]]
658 +name = "cargo-platform"
659 +version = "0.1.9"
660 +source = "registry+https://github.com/rust-lang/crates.io-index"
661 +checksum = "e35af189006b9c0f00a064685c727031e3ed2d8020f7ba284d78cc2671bd36ea"
662 +dependencies = [
663 + "serde",
664 +]
665 +
666 +[[package]]
667 +name = "cargo_metadata"
668 +version = "0.19.2"
669 +source = "registry+https://github.com/rust-lang/crates.io-index"
670 +checksum = "dd5eb614ed4c27c5d706420e4320fbe3216ab31fa1c33cd8246ac36dae4479ba"
671 +dependencies = [
672 + "camino",
673 + "cargo-platform",
674 + "semver",
675 + "serde",
676 + "serde_json",
677 + "thiserror 2.0.18",
678 +]
679 +
680 +[[package]]
681 +name = "castaway"
682 +version = "0.2.4"
683 +source = "registry+https://github.com/rust-lang/crates.io-index"
684 +checksum = "dec551ab6e7578819132c713a93c022a05d60159dc86e7a7050223577484c55a"
685 +dependencies = [
686 + "rustversion",
687 +]
688 +
689 +[[package]]
690 +name = "cc"
691 +version = "1.2.57"
692 +source = "registry+https://github.com/rust-lang/crates.io-index"
693 +checksum = "7a0dd1ca384932ff3641c8718a02769f1698e7563dc6974ffd03346116310423"
694 +dependencies = [
695 + "find-msvc-tools",
696 + "jobserver",
697 + "libc",
698 + "shlex",
699 +]
700 +
701 +[[package]]
702 +name = "cesu8"
703 +version = "1.1.0"
704 +source = "registry+https://github.com/rust-lang/crates.io-index"
705 +checksum = "6d43a04d8753f35258c91f8ec639f792891f748a1edbd759cf1dcea3382ad83c"
706 +
707 +[[package]]
708 +name = "cfg-if"
709 +version = "1.0.4"
710 +source = "registry+https://github.com/rust-lang/crates.io-index"
711 +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
712 +
713 +[[package]]
714 +name = "cfg_aliases"
715 +version = "0.2.1"
716 +source = "registry+https://github.com/rust-lang/crates.io-index"
717 +checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
718 +
719 +[[package]]
720 +name = "cfgrammar"
721 +version = "0.14.1"
722 +source = "registry+https://github.com/rust-lang/crates.io-index"
723 +checksum = "3efdd8f0bddcc9e33f4a664d0f28bc4e51cd5367c16284087a95313104371865"
724 +dependencies = [
725 + "indexmap 2.13.0",
726 + "num-traits",
727 + "proc-macro2",
728 + "quote",
729 + "regex",
730 + "vob",
731 +]
732 +
733 +[[package]]
734 +name = "chrono"
735 +version = "0.4.44"
736 +source = "registry+https://github.com/rust-lang/crates.io-index"
737 +checksum = "c673075a2e0e5f4a1dde27ce9dee1ea4558c7ffe648f576438a20ca1d2acc4b0"
738 +dependencies = [
739 + "iana-time-zone",
740 + "js-sys",
741 + "num-traits",
742 + "serde",
743 + "wasm-bindgen",
744 + "windows-link 0.2.1",
745 +]
746 +
747 +[[package]]
748 +name = "clap"
749 +version = "2.34.0"
750 +source = "registry+https://github.com/rust-lang/crates.io-index"
751 +checksum = "a0610544180c38b88101fecf2dd634b174a62eef6946f84dfc6a7127512b381c"
752 +dependencies = [
753 + "ansi_term",
754 + "atty",
755 + "bitflags 1.3.2",
756 + "strsim 0.8.0",
757 + "textwrap 0.11.0",
758 + "unicode-width 0.1.14",
759 + "vec_map",
760 +]
761 +
762 +[[package]]
763 +name = "clap"
764 +version = "4.6.0"
765 +source = "registry+https://github.com/rust-lang/crates.io-index"
766 +checksum = "b193af5b67834b676abd72466a96c1024e6a6ad978a1f484bd90b85c94041351"
767 +dependencies = [
768 + "clap_builder",
769 + "clap_derive",
770 +]
771 +
772 +[[package]]
773 +name = "clap_builder"
774 +version = "4.6.0"
775 +source = "registry+https://github.com/rust-lang/crates.io-index"
776 +checksum = "714a53001bf66416adb0e2ef5ac857140e7dc3a0c48fb28b2f10762fc4b5069f"
777 +dependencies = [
778 + "anstream",
779 + "anstyle",
780 + "clap_lex",
781 + "strsim 0.11.1",
782 + "terminal_size",
783 +]
784 +
785 +[[package]]
786 +name = "clap_derive"
787 +version = "4.6.0"
788 +source = "registry+https://github.com/rust-lang/crates.io-index"
789 +checksum = "1110bd8a634a1ab8cb04345d8d878267d57c3cf1b38d91b71af6686408bbca6a"
790 +dependencies = [
791 + "heck 0.5.0",
792 + "proc-macro2",
793 + "quote",
794 + "syn 2.0.117",
795 +]
796 +
797 +[[package]]
798 +name = "clap_lex"
799 +version = "1.1.0"
800 +source = "registry+https://github.com/rust-lang/crates.io-index"
801 +checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9"
802 +
803 +[[package]]
804 +name = "cmake"
805 +version = "0.1.57"
806 +source = "registry+https://github.com/rust-lang/crates.io-index"
807 +checksum = "75443c44cd6b379beb8c5b45d85d0773baf31cce901fe7bb252f4eff3008ef7d"
808 +dependencies = [
809 + "cc",
810 +]
811 +
812 +[[package]]
813 +name = "color_quant"
814 +version = "1.1.0"
815 +source = "registry+https://github.com/rust-lang/crates.io-index"
816 +checksum = "3d7b894f5411737b7867f4827955924d7c254fc9f4d91a6aad6b097804b1018b"
817 +
818 +[[package]]
819 +name = "colorchoice"
820 +version = "1.0.5"
821 +source = "registry+https://github.com/rust-lang/crates.io-index"
822 +checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
823 +
824 +[[package]]
825 +name = "combine"
826 +version = "4.6.7"
827 +source = "registry+https://github.com/rust-lang/crates.io-index"
828 +checksum = "ba5a308b75df32fe02788e748662718f03fde005016435c444eea572398219fd"
829 +dependencies = [
830 + "bytes",
831 + "memchr",
832 +]
833 +
834 +[[package]]
835 +name = "compact_str"
836 +version = "0.9.0"
837 +source = "registry+https://github.com/rust-lang/crates.io-index"
838 +checksum = "3fdb1325a1cece981e8a296ab8f0f9b63ae357bd0784a9faaf548cc7b480707a"
839 +dependencies = [
840 + "castaway",
841 + "cfg-if",
842 + "itoa",
843 + "rustversion",
844 + "ryu",
845 + "serde",
846 + "static_assertions",
847 +]
848 +
849 +[[package]]
850 +name = "console"
851 +version = "0.15.11"
852 +source = "registry+https://github.com/rust-lang/crates.io-index"
853 +checksum = "054ccb5b10f9f2cbf51eb355ca1d05c2d279ce1804688d0db74b4733a5aeafd8"
854 +dependencies = [
855 + "encode_unicode",
856 + "libc",
857 + "once_cell",
858 + "unicode-width 0.2.2",
859 + "windows-sys 0.59.0",
860 +]
861 +
862 +[[package]]
863 +name = "console"
864 +version = "0.16.3"
865 +source = "registry+https://github.com/rust-lang/crates.io-index"
866 +checksum = "d64e8af5551369d19cf50138de61f1c42074ab970f74e99be916646777f8fc87"
867 +dependencies = [
868 + "encode_unicode",
869 + "libc",
870 + "unicode-width 0.2.2",
871 + "windows-sys 0.61.2",
872 +]
873 +
874 +[[package]]
875 +name = "convert_case"
876 +version = "0.6.0"
877 +source = "registry+https://github.com/rust-lang/crates.io-index"
878 +checksum = "ec182b0ca2f35d8fc196cf3404988fd8b8c739a4d270ff118a398feb0cbec1ca"
879 +dependencies = [
880 + "unicode-segmentation",
881 +]
882 +
883 +[[package]]
884 +name = "core-foundation"
885 +version = "0.9.4"
886 +source = "registry+https://github.com/rust-lang/crates.io-index"
887 +checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f"
888 +dependencies = [
889 + "core-foundation-sys",
890 + "libc",
891 +]
892 +
893 +[[package]]
894 +name = "core-foundation"
895 +version = "0.10.1"
896 +source = "registry+https://github.com/rust-lang/crates.io-index"
897 +checksum = "b2a6cd9ae233e7f62ba4e9353e81a88df7fc8a5987b8d445b4d90c879bd156f6"
898 +dependencies = [
899 + "core-foundation-sys",
900 + "libc",
901 +]
902 +
903 +[[package]]
904 +name = "core-foundation-sys"
905 +version = "0.8.7"
906 +source = "registry+https://github.com/rust-lang/crates.io-index"
907 +checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
908 +
909 +[[package]]
910 +name = "core-graphics-types"
911 +version = "0.1.3"
912 +source = "registry+https://github.com/rust-lang/crates.io-index"
913 +checksum = "45390e6114f68f718cc7a830514a96f903cccd70d02a8f6d9f643ac4ba45afaf"
914 +dependencies = [
915 + "bitflags 1.3.2",
916 + "core-foundation 0.9.4",
917 + "libc",
918 +]
919 +
920 +[[package]]
921 +name = "core2"
922 +version = "0.4.0"
923 +source = "registry+https://github.com/rust-lang/crates.io-index"
924 +checksum = "b49ba7ef1ad6107f8824dbe97de947cbaac53c44e7f9756a1fba0d37c1eec505"
925 +dependencies = [
926 + "memchr",
927 +]
928 +
929 +[[package]]
930 +name = "cpufeatures"
931 +version = "0.2.17"
932 +source = "registry+https://github.com/rust-lang/crates.io-index"
933 +checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280"
934 +dependencies = [
935 + "libc",
936 +]
937 +
938 +[[package]]
939 +name = "crc32fast"
940 +version = "1.5.0"
941 +source = "registry+https://github.com/rust-lang/crates.io-index"
942 +checksum = "9481c1c90cbf2ac953f07c8d4a58aa3945c425b7185c9154d67a65e4230da511"
943 +dependencies = [
944 + "cfg-if",
945 +]
946 +
947 +[[package]]
948 +name = "crossbeam-deque"
949 +version = "0.8.6"
950 +source = "registry+https://github.com/rust-lang/crates.io-index"
951 +checksum = "9dd111b7b7f7d55b72c0a6ae361660ee5853c9af73f70c3c2ef6858b950e2e51"
952 +dependencies = [
953 + "crossbeam-epoch",
954 + "crossbeam-utils",
955 +]
956 +
957 +[[package]]
958 +name = "crossbeam-epoch"
959 +version = "0.9.18"
960 +source = "registry+https://github.com/rust-lang/crates.io-index"
961 +checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e"
962 +dependencies = [
963 + "crossbeam-utils",
964 +]
965 +
966 +[[package]]
967 +name = "crossbeam-utils"
968 +version = "0.8.21"
969 +source = "registry+https://github.com/rust-lang/crates.io-index"
970 +checksum = "d0a5c400df2834b80a4c3327b3aad3a4c4cd4de0629063962b03235697506a28"
971 +
972 +[[package]]
973 +name = "crossterm"
974 +version = "0.25.0"
975 +source = "registry+https://github.com/rust-lang/crates.io-index"
976 +checksum = "e64e6c0fbe2c17357405f7c758c1ef960fce08bdfb2c03d88d2a18d7e09c4b67"
977 +dependencies = [
978 + "bitflags 1.3.2",
979 + "crossterm_winapi",
980 + "libc",
981 + "mio 0.8.11",
982 + "parking_lot",
983 + "signal-hook",
984 + "signal-hook-mio",
985 + "winapi",
986 +]
987 +
988 +[[package]]
989 +name = "crossterm_winapi"
990 +version = "0.9.1"
991 +source = "registry+https://github.com/rust-lang/crates.io-index"
992 +checksum = "acdd7c62a3665c7f6830a51635d9ac9b23ed385797f70a83bb8bafe9c572ab2b"
993 +dependencies = [
994 + "winapi",
995 +]
996 +
997 +[[package]]
998 +name = "crunchy"
999 +version = "0.2.4"
1000 +source = "registry+https://github.com/rust-lang/crates.io-index"
1001 +checksum = "460fbee9c2c2f33933d720630a6a0bac33ba7053db5344fac858d4b8952d77d5"
1002 +
1003 +[[package]]
1004 +name = "crypto-common"
1005 +version = "0.1.7"
1006 +source = "registry+https://github.com/rust-lang/crates.io-index"
1007 +checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
1008 +dependencies = [
1009 + "generic-array",
1010 + "typenum",
1011 +]
1012 +
1013 +[[package]]
1014 +name = "cssparser"
1015 +version = "0.36.0"
1016 +source = "registry+https://github.com/rust-lang/crates.io-index"
1017 +checksum = "dae61cf9c0abb83bd659dab65b7e4e38d8236824c85f0f804f173567bda257d2"
1018 +dependencies = [
1019 + "cssparser-macros",
1020 + "dtoa-short",
1021 + "itoa",
1022 + "phf",
1023 + "smallvec 1.15.1",
1024 +]
1025 +
1026 +[[package]]
1027 +name = "cssparser-macros"
1028 +version = "0.6.1"
1029 +source = "registry+https://github.com/rust-lang/crates.io-index"
1030 +checksum = "13b588ba4ac1a99f7f2964d24b3d896ddc6bf847ee3855dbd4366f058cfcd331"
1031 +dependencies = [
1032 + "quote",
1033 + "syn 2.0.117",
1034 +]
1035 +
1036 +[[package]]
1037 +name = "csv"
1038 +version = "1.4.0"
1039 +source = "registry+https://github.com/rust-lang/crates.io-index"
1040 +checksum = "52cd9d68cf7efc6ddfaaee42e7288d3a99d613d4b50f76ce9827ae0c6e14f938"
1041 +dependencies = [
1042 + "csv-core",
1043 + "itoa",
1044 + "ryu",
1045 + "serde_core",
1046 +]
1047 +
1048 +[[package]]
1049 +name = "csv-core"
1050 +version = "0.1.13"
1051 +source = "registry+https://github.com/rust-lang/crates.io-index"
1052 +checksum = "704a3c26996a80471189265814dbc2c257598b96b8a7feae2d31ace646bb9782"
1053 +dependencies = [
1054 + "memchr",
1055 +]
1056 +
1057 +[[package]]
1058 +name = "darling"
1059 +version = "0.11.0"
1060 +source = "registry+https://github.com/rust-lang/crates.io-index"
1061 +checksum = "dbffa8f8e38810422f320ca457a93cf1cd0056dc9c06c556b867558e0d471463"
1062 +dependencies = [
1063 + "darling_core 0.11.0",
1064 + "darling_macro 0.11.0",
1065 +]
1066 +
1067 +[[package]]
1068 +name = "darling"
1069 +version = "0.20.11"
1070 +source = "registry+https://github.com/rust-lang/crates.io-index"
1071 +checksum = "fc7f46116c46ff9ab3eb1597a45688b6715c6e628b5c133e288e709a29bcb4ee"
1072 +dependencies = [
1073 + "darling_core 0.20.11",
1074 + "darling_macro 0.20.11",
1075 +]
1076 +
1077 +[[package]]
1078 +name = "darling"
1079 +version = "0.23.0"
1080 +source = "registry+https://github.com/rust-lang/crates.io-index"
1081 +checksum = "25ae13da2f202d56bd7f91c25fba009e7717a1e4a1cc98a76d844b65ae912e9d"
1082 +dependencies = [
1083 + "darling_core 0.23.0",
1084 + "darling_macro 0.23.0",
1085 +]
1086 +
1087 +[[package]]
1088 +name = "darling_core"
1089 +version = "0.11.0"
1090 +source = "registry+https://github.com/rust-lang/crates.io-index"
1091 +checksum = "06e172685d94b7b83800e3256a63261537b9d6129e10f21c8e13ddf9dba8c64d"
1092 +dependencies = [
1093 + "fnv",
1094 + "ident_case",
1095 + "proc-macro2",
1096 + "quote",
1097 + "strsim 0.10.0",
1098 + "syn 1.0.109",
1099 +]
1100 +
1101 +[[package]]
1102 +name = "darling_core"
1103 +version = "0.20.11"
1104 +source = "registry+https://github.com/rust-lang/crates.io-index"
1105 +checksum = "0d00b9596d185e565c2207a0b01f8bd1a135483d02d9b7b0a54b11da8d53412e"
1106 +dependencies = [
1107 + "fnv",
1108 + "ident_case",
1109 + "proc-macro2",
1110 + "quote",
1111 + "strsim 0.11.1",
1112 + "syn 2.0.117",
1113 +]
1114 +
1115 +[[package]]
1116 +name = "darling_core"
1117 +version = "0.23.0"
1118 +source = "registry+https://github.com/rust-lang/crates.io-index"
1119 +checksum = "9865a50f7c335f53564bb694ef660825eb8610e0a53d3e11bf1b0d3df31e03b0"
1120 +dependencies = [
1121 + "ident_case",
1122 + "proc-macro2",
1123 + "quote",
1124 + "strsim 0.11.1",
1125 + "syn 2.0.117",
1126 +]
1127 +
1128 +[[package]]
1129 +name = "darling_macro"
1130 +version = "0.11.0"
1131 +source = "registry+https://github.com/rust-lang/crates.io-index"
1132 +checksum = "f0618ac802792cebd1918ac6042a6ea1eeab92db34b35656afaa577929820788"
1133 +dependencies = [
1134 + "darling_core 0.11.0",
1135 + "quote",
1136 + "syn 1.0.109",
1137 +]
1138 +
1139 +[[package]]
1140 +name = "darling_macro"
1141 +version = "0.20.11"
1142 +source = "registry+https://github.com/rust-lang/crates.io-index"
1143 +checksum = "fc34b93ccb385b40dc71c6fceac4b2ad23662c7eeb248cf10d529b7e055b6ead"
1144 +dependencies = [
1145 + "darling_core 0.20.11",
1146 + "quote",
1147 + "syn 2.0.117",
1148 +]
1149 +
1150 +[[package]]
1151 +name = "darling_macro"
1152 +version = "0.23.0"
1153 +source = "registry+https://github.com/rust-lang/crates.io-index"
1154 +checksum = "ac3984ec7bd6cfa798e62b4a642426a5be0e68f9401cfc2a01e3fa9ea2fcdb8d"
1155 +dependencies = [
1156 + "darling_core 0.23.0",
1157 + "quote",
1158 + "syn 2.0.117",
1159 +]
1160 +
1161 +[[package]]
1162 +name = "dary_heap"
1163 +version = "0.3.8"
1164 +source = "registry+https://github.com/rust-lang/crates.io-index"
1165 +checksum = "06d2e3287df1c007e74221c49ca10a95d557349e54b3a75dc2fb14712c751f04"
1166 +dependencies = [
1167 + "serde",
1168 +]
1169 +
1170 +[[package]]
1171 +name = "data-encoding"
1172 +version = "2.10.0"
1173 +source = "registry+https://github.com/rust-lang/crates.io-index"
1174 +checksum = "d7a1e2f27636f116493b8b860f5546edb47c8d8f8ea73e1d2a20be88e28d1fea"
1175 +
1176 +[[package]]
1177 +name = "defmac"
1178 +version = "0.1.3"
1179 +source = "registry+https://github.com/rust-lang/crates.io-index"
1180 +checksum = "aafbece59594ed57696a1a69e8bb3ca1683fbc9cdb41d5c02726070b2cd8f19d"
1181 +
1182 +[[package]]
1183 +name = "deranged"
1184 +version = "0.5.8"
1185 +source = "registry+https://github.com/rust-lang/crates.io-index"
1186 +checksum = "7cd812cc2bc1d69d4764bd80df88b4317eaef9e773c75226407d9bc0876b211c"
1187 +dependencies = [
1188 + "powerfmt",
1189 + "serde_core",
1190 +]
1191 +
1192 +[[package]]
1193 +name = "derive-new"
1194 +version = "0.7.0"
1195 +source = "registry+https://github.com/rust-lang/crates.io-index"
1196 +checksum = "2cdc8d50f426189eef89dac62fabfa0abb27d5cc008f25bf4156a0203325becc"
1197 +dependencies = [
1198 + "proc-macro2",
1199 + "quote",
1200 + "syn 2.0.117",
1201 +]
1202 +
1203 +[[package]]
1204 +name = "derive_builder"
1205 +version = "0.20.2"
1206 +source = "registry+https://github.com/rust-lang/crates.io-index"
1207 +checksum = "507dfb09ea8b7fa618fcf76e953f4f5e192547945816d5358edffe39f6f94947"
1208 +dependencies = [
1209 + "derive_builder_macro",
1210 +]
1211 +
1212 +[[package]]
1213 +name = "derive_builder_core"
1214 +version = "0.20.2"
1215 +source = "registry+https://github.com/rust-lang/crates.io-index"
1216 +checksum = "2d5bcf7b024d6835cfb3d473887cd966994907effbe9227e8c8219824d06c4e8"
1217 +dependencies = [
1218 + "darling 0.20.11",
1219 + "proc-macro2",
1220 + "quote",
1221 + "syn 2.0.117",
1222 +]
1223 +
1224 +[[package]]
1225 +name = "derive_builder_macro"
1226 +version = "0.20.2"
1227 +source = "registry+https://github.com/rust-lang/crates.io-index"
1228 +checksum = "ab63b0e2bf4d5928aff72e83a7dace85d7bba5fe12dcc3c5a572d78caffd3f3c"
1229 +dependencies = [
1230 + "derive_builder_core",
1231 + "syn 2.0.117",
1232 +]
1233 +
1234 +[[package]]
1235 +name = "derive_more"
1236 +version = "2.1.1"
1237 +source = "registry+https://github.com/rust-lang/crates.io-index"
1238 +checksum = "d751e9e49156b02b44f9c1815bcb94b984cdcc4396ecc32521c739452808b134"
1239 +dependencies = [
1240 + "derive_more-impl",
1241 +]
1242 +
1243 +[[package]]
1244 +name = "derive_more-impl"
1245 +version = "2.1.1"
1246 +source = "registry+https://github.com/rust-lang/crates.io-index"
1247 +checksum = "799a97264921d8623a957f6c3b9011f3b5492f557bbb7a5a19b7fa6d06ba8dcb"
1248 +dependencies = [
1249 + "proc-macro2",
1250 + "quote",
1251 + "rustc_version",
1252 + "syn 2.0.117",
1253 +]
1254 +
1255 +[[package]]
1256 +name = "derivre"
1257 +version = "0.3.8"
1258 +source = "registry+https://github.com/rust-lang/crates.io-index"
1259 +checksum = "786c7c65c4ef0c7deb05de3005e01991612a8f09fe0844fc0969c68b90468ba8"
1260 +dependencies = [
1261 + "anyhow",
1262 + "bytemuck",
1263 + "bytemuck_derive",
1264 + "hashbrown 0.15.5",
1265 + "regex-syntax",
1266 + "strum",
1267 +]
1268 +
1269 +[[package]]
1270 +name = "deunicode"
1271 +version = "1.6.2"
1272 +source = "registry+https://github.com/rust-lang/crates.io-index"
1273 +checksum = "abd57806937c9cc163efc8ea3910e00a62e2aeb0b8119f1793a978088f8f6b04"
1274 +
1275 +[[package]]
1276 +name = "digest"
1277 +version = "0.10.7"
1278 +source = "registry+https://github.com/rust-lang/crates.io-index"
1279 +checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
1280 +dependencies = [
1281 + "block-buffer",
1282 + "crypto-common",
1283 +]
1284 +
1285 +[[package]]
1286 +name = "dirs"
1287 +version = "6.0.0"
1288 +source = "registry+https://github.com/rust-lang/crates.io-index"
1289 +checksum = "c3e8aa94d75141228480295a7d0e7feb620b1a5ad9f12bc40be62411e38cce4e"
1290 +dependencies = [
1291 + "dirs-sys",
1292 +]
1293 +
1294 +[[package]]
1295 +name = "dirs-sys"
1296 +version = "0.5.0"
1297 +source = "registry+https://github.com/rust-lang/crates.io-index"
1298 +checksum = "e01a3366d27ee9890022452ee61b2b63a67e6f13f58900b651ff5665f0bb1fab"
1299 +dependencies = [
1300 + "libc",
1301 + "option-ext",
1302 + "redox_users",
1303 + "windows-sys 0.61.2",
1304 +]
1305 +
1306 +[[package]]
1307 +name = "dispatch2"
1308 +version = "0.3.1"
1309 +source = "registry+https://github.com/rust-lang/crates.io-index"
1310 +checksum = "1e0e367e4e7da84520dedcac1901e4da967309406d1e51017ae1abfb97adbd38"
1311 +dependencies = [
1312 + "bitflags 2.11.0",
1313 + "block2",
1314 + "objc2",
1315 +]
1316 +
1317 +[[package]]
1318 +name = "displaydoc"
1319 +version = "0.2.5"
1320 +source = "registry+https://github.com/rust-lang/crates.io-index"
1321 +checksum = "97369cbbc041bc366949bc74d34658d6cda5621039731c6310521892a3a20ae0"
1322 +dependencies = [
1323 + "proc-macro2",
1324 + "quote",
1325 + "syn 2.0.117",
1326 +]
1327 +
1328 +[[package]]
1329 +name = "doctest-file"
1330 +version = "1.1.1"
1331 +source = "registry+https://github.com/rust-lang/crates.io-index"
1332 +checksum = "c2db04e74f0a9a93103b50e90b96024c9b2bdca8bce6a632ec71b88736d3d359"
1333 +
1334 +[[package]]
1335 +name = "dtoa"
1336 +version = "1.0.11"
1337 +source = "registry+https://github.com/rust-lang/crates.io-index"
1338 +checksum = "4c3cf4824e2d5f025c7b531afcb2325364084a16806f6d47fbc1f5fbd9960590"
1339 +
1340 +[[package]]
1341 +name = "dtoa-short"
1342 +version = "0.3.5"
1343 +source = "registry+https://github.com/rust-lang/crates.io-index"
1344 +checksum = "cd1511a7b6a56299bd043a9c167a6d2bfb37bf84a6dfceaba651168adfb43c87"
1345 +dependencies = [
1346 + "dtoa",
1347 +]
1348 +
1349 +[[package]]
1350 +name = "dunce"
1351 +version = "1.0.5"
1352 +source = "registry+https://github.com/rust-lang/crates.io-index"
1353 +checksum = "92773504d58c093f6de2459af4af33faa518c13451eb8f2b5698ed3d36e7c813"
1354 +
1355 +[[package]]
1356 +name = "dyn-clone"
1357 +version = "1.0.20"
1358 +source = "registry+https://github.com/rust-lang/crates.io-index"
1359 +checksum = "d0881ea181b1df73ff77ffaaf9c7544ecc11e82fba9b5f27b262a3c73a332555"
1360 +
1361 +[[package]]
1362 +name = "dyn-stack"
1363 +version = "0.13.2"
1364 +source = "registry+https://github.com/rust-lang/crates.io-index"
1365 +checksum = "1c4713e43e2886ba72b8271aa66c93d722116acf7a75555cce11dcde84388fe8"
1366 +dependencies = [
1367 + "bytemuck",
1368 + "dyn-stack-macros",
1369 +]
1370 +
1371 +[[package]]
1372 +name = "dyn-stack-macros"
1373 +version = "0.1.3"
1374 +source = "registry+https://github.com/rust-lang/crates.io-index"
1375 +checksum = "e1d926b4d407d372f141f93bb444696142c29d32962ccbd3531117cf3aa0bfa9"
1376 +
1377 +[[package]]
1378 +name = "ego-tree"
1379 +version = "0.10.0"
1380 +source = "registry+https://github.com/rust-lang/crates.io-index"
1381 +checksum = "b2972feb8dffe7bc8c5463b1dacda1b0dfbed3710e50f977d965429692d74cd8"
1382 +
1383 +[[package]]
1384 +name = "either"
1385 +version = "1.15.0"
1386 +source = "registry+https://github.com/rust-lang/crates.io-index"
1387 +checksum = "48c757948c5ede0e46177b7add2e67155f70e33c07fea8284df6576da70b3719"
1388 +dependencies = [
1389 + "serde",
1390 +]
1391 +
1392 +[[package]]
1393 +name = "encode_unicode"
1394 +version = "1.0.0"
1395 +source = "registry+https://github.com/rust-lang/crates.io-index"
1396 +checksum = "34aa73646ffb006b8f5147f3dc182bd4bcb190227ce861fc4a4844bf8e3cb2c0"
1397 +
1398 +[[package]]
1399 +name = "encoding_rs"
1400 +version = "0.8.35"
1401 +source = "registry+https://github.com/rust-lang/crates.io-index"
1402 +checksum = "75030f3c4f45dafd7586dd6780965a8c7e8e285a5ecb86713e63a79c5b2766f3"
1403 +dependencies = [
1404 + "cfg-if",
1405 +]
1406 +
1407 +[[package]]
1408 +name = "encoding_rs_io"
1409 +version = "0.1.7"
1410 +source = "registry+https://github.com/rust-lang/crates.io-index"
1411 +checksum = "1cc3c5651fb62ab8aa3103998dade57efdd028544bd300516baa31840c252a83"
1412 +dependencies = [
1413 + "encoding_rs",
1414 +]
1415 +
1416 +[[package]]
1417 +name = "endian-type"
1418 +version = "0.2.0"
1419 +source = "registry+https://github.com/rust-lang/crates.io-index"
1420 +checksum = "869b0adbda23651a9c5c0c3d270aac9fcb52e8622a8f2b17e57802d7791962f2"
1421 +
1422 +[[package]]
1423 +name = "enum-as-inner"
1424 +version = "0.6.1"
1425 +source = "registry+https://github.com/rust-lang/crates.io-index"
1426 +checksum = "a1e6a265c649f3f5979b601d26f1d05ada116434c87741c9493cb56218f76cbc"
1427 +dependencies = [
1428 + "heck 0.5.0",
1429 + "proc-macro2",
1430 + "quote",
1431 + "syn 2.0.117",
1432 +]
1433 +
1434 +[[package]]
1435 +name = "equator"
1436 +version = "0.4.2"
1437 +source = "registry+https://github.com/rust-lang/crates.io-index"
1438 +checksum = "4711b213838dfee0117e3be6ac926007d7f433d7bbe33595975d4190cb07e6fc"
1439 +dependencies = [
1440 + "equator-macro",
1441 +]
1442 +
1443 +[[package]]
1444 +name = "equator-macro"
1445 +version = "0.4.2"
1446 +source = "registry+https://github.com/rust-lang/crates.io-index"
1447 +checksum = "44f23cf4b44bfce11a86ace86f8a73ffdec849c9fd00a386a53d278bd9e81fb3"
1448 +dependencies = [
1449 + "proc-macro2",
1450 + "quote",
1451 + "syn 2.0.117",
1452 +]
1453 +
1454 +[[package]]
1455 +name = "equivalent"
1456 +version = "1.0.2"
1457 +source = "registry+https://github.com/rust-lang/crates.io-index"
1458 +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
1459 +
1460 +[[package]]
1461 +name = "errno"
1462 +version = "0.3.14"
1463 +source = "registry+https://github.com/rust-lang/crates.io-index"
1464 +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
1465 +dependencies = [
1466 + "libc",
1467 + "windows-sys 0.61.2",
1468 +]
1469 +
1470 +[[package]]
1471 +name = "esaxx-rs"
1472 +version = "0.1.10"
1473 +source = "registry+https://github.com/rust-lang/crates.io-index"
1474 +checksum = "d817e038c30374a4bcb22f94d0a8a0e216958d4c3dcde369b1439fec4bdda6e6"
1475 +
1476 +[[package]]
1477 +name = "exr"
1478 +version = "1.74.0"
1479 +source = "registry+https://github.com/rust-lang/crates.io-index"
1480 +checksum = "4300e043a56aa2cb633c01af81ca8f699a321879a7854d3896a0ba89056363be"
1481 +dependencies = [
1482 + "bit_field",
1483 + "half",
1484 + "lebe",
1485 + "miniz_oxide",
1486 + "rayon-core",
1487 + "smallvec 1.15.1",
1488 + "zune-inflate",
1489 +]
1490 +
1491 +[[package]]
1492 +name = "extended"
1493 +version = "0.1.0"
1494 +source = "registry+https://github.com/rust-lang/crates.io-index"
1495 +checksum = "af9673d8203fcb076b19dfd17e38b3d4ae9f44959416ea532ce72415a6020365"
1496 +
1497 +[[package]]
1498 +name = "fancy-regex"
1499 +version = "0.13.0"
1500 +source = "registry+https://github.com/rust-lang/crates.io-index"
1501 +checksum = "531e46835a22af56d1e3b66f04844bed63158bc094a628bec1d321d9b4c44bf2"
1502 +dependencies = [
1503 + "bit-set 0.5.3",
1504 + "regex-automata",
1505 + "regex-syntax",
1506 +]
1507 +
1508 +[[package]]
1509 +name = "fancy-regex"
1510 +version = "0.14.0"
1511 +source = "registry+https://github.com/rust-lang/crates.io-index"
1512 +checksum = "6e24cb5a94bcae1e5408b0effca5cd7172ea3c5755049c5f3af4cd283a165298"
1513 +dependencies = [
1514 + "bit-set 0.8.0",
1515 + "regex-automata",
1516 + "regex-syntax",
1517 +]
1518 +
1519 +[[package]]
1520 +name = "fastrand"
1521 +version = "2.3.0"
1522 +source = "registry+https://github.com/rust-lang/crates.io-index"
1523 +checksum = "37909eebbb50d72f9059c3b6d82c0463f2ff062c9e95845c43a6c9c0355411be"
1524 +
1525 +[[package]]
1526 +name = "fax"
1527 +version = "0.2.6"
1528 +source = "registry+https://github.com/rust-lang/crates.io-index"
1529 +checksum = "f05de7d48f37cd6730705cbca900770cab77a89f413d23e100ad7fad7795a0ab"
1530 +dependencies = [
1531 + "fax_derive",
1532 +]
1533 +
1534 +[[package]]
1535 +name = "fax_derive"
1536 +version = "0.2.0"
1537 +source = "registry+https://github.com/rust-lang/crates.io-index"
1538 +checksum = "a0aca10fb742cb43f9e7bb8467c91aa9bcb8e3ffbc6a6f7389bb93ffc920577d"
1539 +dependencies = [
1540 + "proc-macro2",
1541 + "quote",
1542 + "syn 2.0.117",
1543 +]
1544 +
1545 +[[package]]
1546 +name = "fdeflate"
1547 +version = "0.3.7"
1548 +source = "registry+https://github.com/rust-lang/crates.io-index"
1549 +checksum = "1e6853b52649d4ac5c0bd02320cddc5ba956bdb407c4b75a2c6b75bf51500f8c"
1550 +dependencies = [
1551 + "simd-adler32",
1552 +]
1553 +
1554 +[[package]]
1555 +name = "find-msvc-tools"
1556 +version = "0.1.9"
1557 +source = "registry+https://github.com/rust-lang/crates.io-index"
1558 +checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
1559 +
1560 +[[package]]
1561 +name = "flate2"
1562 +version = "1.1.9"
1563 +source = "registry+https://github.com/rust-lang/crates.io-index"
1564 +checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
1565 +dependencies = [
1566 + "crc32fast",
1567 + "miniz_oxide",
1568 +]
1569 +
1570 +[[package]]
1571 +name = "float8"
1572 +version = "0.7.0"
1573 +source = "registry+https://github.com/rust-lang/crates.io-index"
1574 +checksum = "c2d1f04709a8ac06e8e8042875a3c466cc4832d3c1a18dbcb9dba3c6e83046bc"
1575 +dependencies = [
1576 + "half",
1577 + "num-traits",
1578 + "rand 0.9.2",
1579 + "rand_distr 0.5.1",
1580 +]
1581 +
1582 +[[package]]
1583 +name = "fnv"
1584 +version = "1.0.7"
1585 +source = "registry+https://github.com/rust-lang/crates.io-index"
1586 +checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
1587 +
1588 +[[package]]
1589 +name = "foldhash"
1590 +version = "0.1.5"
1591 +source = "registry+https://github.com/rust-lang/crates.io-index"
1592 +checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
1593 +
1594 +[[package]]
1595 +name = "foldhash"
1596 +version = "0.2.0"
1597 +source = "registry+https://github.com/rust-lang/crates.io-index"
1598 +checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb"
1599 +
1600 +[[package]]
1601 +name = "foreign-types"
1602 +version = "0.3.2"
1603 +source = "registry+https://github.com/rust-lang/crates.io-index"
1604 +checksum = "f6f339eb8adc052cd2ca78910fda869aefa38d22d5cb648e6485e4d3fc06f3b1"
1605 +dependencies = [
1606 + "foreign-types-shared 0.1.1",
1607 +]
1608 +
1609 +[[package]]
1610 +name = "foreign-types"
1611 +version = "0.5.0"
1612 +source = "registry+https://github.com/rust-lang/crates.io-index"
1613 +checksum = "d737d9aa519fb7b749cbc3b962edcf310a8dd1f4b67c91c4f83975dbdd17d965"
1614 +dependencies = [
1615 + "foreign-types-macros",
1616 + "foreign-types-shared 0.3.1",
1617 +]
1618 +
1619 +[[package]]
1620 +name = "foreign-types-macros"
1621 +version = "0.2.3"
1622 +source = "registry+https://github.com/rust-lang/crates.io-index"
1623 +checksum = "1a5c6c585bc94aaf2c7b51dd4c2ba22680844aba4c687be581871a6f518c5742"
1624 +dependencies = [
1625 + "proc-macro2",
1626 + "quote",
1627 + "syn 2.0.117",
1628 +]
1629 +
1630 +[[package]]
1631 +name = "foreign-types-shared"
1632 +version = "0.1.1"
1633 +source = "registry+https://github.com/rust-lang/crates.io-index"
1634 +checksum = "00b0228411908ca8685dba7fc2cdd70ec9990a6e753e89b6ac91a84c40fbaf4b"
1635 +
1636 +[[package]]
1637 +name = "foreign-types-shared"
1638 +version = "0.3.1"
1639 +source = "registry+https://github.com/rust-lang/crates.io-index"
1640 +checksum = "aa9a19cbb55df58761df49b23516a86d432839add4af60fc256da840f66ed35b"
1641 +
1642 +[[package]]
1643 +name = "form_urlencoded"
1644 +version = "1.2.2"
1645 +source = "registry+https://github.com/rust-lang/crates.io-index"
1646 +checksum = "cb4cb245038516f5f85277875cdaa4f7d2c9a0fa0468de06ed190163b1581fcf"
1647 +dependencies = [
1648 + "percent-encoding",
1649 +]
1650 +
1651 +[[package]]
1652 +name = "fs-err"
1653 +version = "2.11.0"
1654 +source = "registry+https://github.com/rust-lang/crates.io-index"
1655 +checksum = "88a41f105fe1d5b6b34b2055e3dc59bb79b46b48b2040b9e6c7b4b5de097aa41"
1656 +dependencies = [
1657 + "autocfg",
1658 +]
1659 +
1660 +[[package]]
1661 +name = "fs_extra"
1662 +version = "1.3.0"
1663 +source = "registry+https://github.com/rust-lang/crates.io-index"
1664 +checksum = "42703706b716c37f96a77aea830392ad231f44c9e9a67872fa5548707e11b11c"
1665 +
1666 +[[package]]
1667 +name = "futf"
1668 +version = "0.1.5"
1669 +source = "registry+https://github.com/rust-lang/crates.io-index"
1670 +checksum = "df420e2e84819663797d1ec6544b13c5be84629e7bb00dc960d6917db2987843"
1671 +dependencies = [
1672 + "mac",
1673 + "new_debug_unreachable",
1674 +]
1675 +
1676 +[[package]]
1677 +name = "futures"
1678 +version = "0.3.32"
1679 +source = "registry+https://github.com/rust-lang/crates.io-index"
1680 +checksum = "8b147ee9d1f6d097cef9ce628cd2ee62288d963e16fb287bd9286455b241382d"
1681 +dependencies = [
1682 + "futures-channel",
1683 + "futures-core",
1684 + "futures-executor",
1685 + "futures-io",
1686 + "futures-sink",
1687 + "futures-task",
1688 + "futures-util",
1689 +]
1690 +
1691 +[[package]]
1692 +name = "futures-channel"
1693 +version = "0.3.32"
1694 +source = "registry+https://github.com/rust-lang/crates.io-index"
1695 +checksum = "07bbe89c50d7a535e539b8c17bc0b49bdb77747034daa8087407d655f3f7cc1d"
1696 +dependencies = [
1697 + "futures-core",
1698 + "futures-sink",
1699 +]
1700 +
1701 +[[package]]
1702 +name = "futures-core"
1703 +version = "0.3.32"
1704 +source = "registry+https://github.com/rust-lang/crates.io-index"
1705 +checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d"
1706 +
1707 +[[package]]
1708 +name = "futures-executor"
1709 +version = "0.3.32"
1710 +source = "registry+https://github.com/rust-lang/crates.io-index"
1711 +checksum = "baf29c38818342a3b26b5b923639e7b1f4a61fc5e76102d4b1981c6dc7a7579d"
1712 +dependencies = [
1713 + "futures-core",
1714 + "futures-task",
1715 + "futures-util",
1716 +]
1717 +
1718 +[[package]]
1719 +name = "futures-io"
1720 +version = "0.3.32"
1721 +source = "registry+https://github.com/rust-lang/crates.io-index"
1722 +checksum = "cecba35d7ad927e23624b22ad55235f2239cfa44fd10428eecbeba6d6a717718"
1723 +
1724 +[[package]]
1725 +name = "futures-macro"
1726 +version = "0.3.32"
1727 +source = "registry+https://github.com/rust-lang/crates.io-index"
1728 +checksum = "e835b70203e41293343137df5c0664546da5745f82ec9b84d40be8336958447b"
1729 +dependencies = [
1730 + "proc-macro2",
1731 + "quote",
1732 + "syn 2.0.117",
1733 +]
1734 +
1735 +[[package]]
1736 +name = "futures-sink"
1737 +version = "0.3.32"
1738 +source = "registry+https://github.com/rust-lang/crates.io-index"
1739 +checksum = "c39754e157331b013978ec91992bde1ac089843443c49cbc7f46150b0fad0893"
1740 +
1741 +[[package]]
1742 +name = "futures-task"
1743 +version = "0.3.32"
1744 +source = "registry+https://github.com/rust-lang/crates.io-index"
1745 +checksum = "037711b3d59c33004d3856fbdc83b99d4ff37a24768fa1be9ce3538a1cde4393"
1746 +
1747 +[[package]]
1748 +name = "futures-util"
1749 +version = "0.3.32"
1750 +source = "registry+https://github.com/rust-lang/crates.io-index"
1751 +checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6"
1752 +dependencies = [
1753 + "futures-channel",
1754 + "futures-core",
1755 + "futures-io",
1756 + "futures-macro",
1757 + "futures-sink",
1758 + "futures-task",
1759 + "memchr",
1760 + "pin-project-lite",
1761 + "slab",
1762 +]
1763 +
1764 +[[package]]
1765 +name = "fxhash"
1766 +version = "0.2.1"
1767 +source = "registry+https://github.com/rust-lang/crates.io-index"
1768 +checksum = "c31b6d751ae2c7f11320402d34e41349dd1016f8d5d45e48c4312bc8625af50c"
1769 +dependencies = [
1770 + "byteorder",
1771 +]
1772 +
1773 +[[package]]
1774 +name = "galil-seiferas"
1775 +version = "0.1.5"
1776 +source = "registry+https://github.com/rust-lang/crates.io-index"
1777 +checksum = "794ac25cfda3fa11d2b07ff8c65889c6c03411646df54e59e606878d899e1d5a"
1778 +dependencies = [
1779 + "defmac",
1780 + "unchecked-index",
1781 +]
1782 +
1783 +[[package]]
1784 +name = "gemm"
1785 +version = "0.18.2"
1786 +source = "registry+https://github.com/rust-lang/crates.io-index"
1787 +checksum = "ab96b703d31950f1aeddded248bc95543c9efc7ac9c4a21fda8703a83ee35451"
1788 +dependencies = [
1789 + "dyn-stack",
1790 + "gemm-c32 0.18.2",
1791 + "gemm-c64 0.18.2",
1792 + "gemm-common 0.18.2",
1793 + "gemm-f16 0.18.2",
1794 + "gemm-f32 0.18.2",
1795 + "gemm-f64 0.18.2",
1796 + "num-complex",
1797 + "num-traits",
1798 + "paste",
1799 + "raw-cpuid",
1800 + "seq-macro",
1801 +]
1802 +
1803 +[[package]]
1804 +name = "gemm"
1805 +version = "0.19.0"
1806 +source = "registry+https://github.com/rust-lang/crates.io-index"
1807 +checksum = "aa0673db364b12263d103b68337a68fbecc541d6f6b61ba72fe438654709eacb"
1808 +dependencies = [
1809 + "dyn-stack",
1810 + "gemm-c32 0.19.0",
1811 + "gemm-c64 0.19.0",
1812 + "gemm-common 0.19.0",
1813 + "gemm-f16 0.19.0",
1814 + "gemm-f32 0.19.0",
1815 + "gemm-f64 0.19.0",
1816 + "num-complex",
1817 + "num-traits",
1818 + "paste",
1819 + "raw-cpuid",
1820 + "seq-macro",
1821 +]
1822 +
1823 +[[package]]
1824 +name = "gemm-c32"
1825 +version = "0.18.2"
1826 +source = "registry+https://github.com/rust-lang/crates.io-index"
1827 +checksum = "f6db9fd9f40421d00eea9dd0770045a5603b8d684654816637732463f4073847"
1828 +dependencies = [
1829 + "dyn-stack",
1830 + "gemm-common 0.18.2",
1831 + "num-complex",
1832 + "num-traits",
1833 + "paste",
1834 + "raw-cpuid",
1835 + "seq-macro",
1836 +]
1837 +
1838 +[[package]]
1839 +name = "gemm-c32"
1840 +version = "0.19.0"
1841 +source = "registry+https://github.com/rust-lang/crates.io-index"
1842 +checksum = "086936dbdcb99e37aad81d320f98f670e53c1e55a98bee70573e83f95beb128c"
1843 +dependencies = [
1844 + "dyn-stack",
1845 + "gemm-common 0.19.0",
1846 + "num-complex",
1847 + "num-traits",
1848 + "paste",
1849 + "raw-cpuid",
1850 + "seq-macro",
1851 +]
1852 +
1853 +[[package]]
1854 +name = "gemm-c64"
1855 +version = "0.18.2"
1856 +source = "registry+https://github.com/rust-lang/crates.io-index"
1857 +checksum = "dfcad8a3d35a43758330b635d02edad980c1e143dc2f21e6fd25f9e4eada8edf"
1858 +dependencies = [
1859 + "dyn-stack",
1860 + "gemm-common 0.18.2",
1861 + "num-complex",
1862 + "num-traits",
1863 + "paste",
1864 + "raw-cpuid",
1865 + "seq-macro",
1866 +]
1867 +
1868 +[[package]]
1869 +name = "gemm-c64"
1870 +version = "0.19.0"
1871 +source = "registry+https://github.com/rust-lang/crates.io-index"
1872 +checksum = "20c8aeeeec425959bda4d9827664029ba1501a90a0d1e6228e48bef741db3a3f"
1873 +dependencies = [
1874 + "dyn-stack",
1875 + "gemm-common 0.19.0",
1876 + "num-complex",
1877 + "num-traits",
1878 + "paste",
1879 + "raw-cpuid",
1880 + "seq-macro",
1881 +]
1882 +
1883 +[[package]]
1884 +name = "gemm-common"
1885 +version = "0.18.2"
1886 +source = "registry+https://github.com/rust-lang/crates.io-index"
1887 +checksum = "a352d4a69cbe938b9e2a9cb7a3a63b7e72f9349174a2752a558a8a563510d0f3"
1888 +dependencies = [
1889 + "bytemuck",
1890 + "dyn-stack",
1891 + "half",
1892 + "libm",
1893 + "num-complex",
1894 + "num-traits",
1895 + "once_cell",
1896 + "paste",
1897 + "pulp 0.21.5",
1898 + "raw-cpuid",
1899 + "rayon",
1900 + "seq-macro",
1901 + "sysctl",
1902 +]
1903 +
1904 +[[package]]
1905 +name = "gemm-common"
1906 +version = "0.19.0"
1907 +source = "registry+https://github.com/rust-lang/crates.io-index"
1908 +checksum = "88027625910cc9b1085aaaa1c4bc46bb3a36aad323452b33c25b5e4e7c8e2a3e"
1909 +dependencies = [
1910 + "bytemuck",
1911 + "dyn-stack",
1912 + "half",
1913 + "libm",
1914 + "num-complex",
1915 + "num-traits",
1916 + "once_cell",
1917 + "paste",
1918 + "pulp 0.22.2",
1919 + "raw-cpuid",
1920 + "rayon",
1921 + "seq-macro",
1922 + "sysctl",
1923 +]
1924 +
1925 +[[package]]
1926 +name = "gemm-f16"
1927 +version = "0.18.2"
1928 +source = "registry+https://github.com/rust-lang/crates.io-index"
1929 +checksum = "cff95ae3259432f3c3410eaa919033cd03791d81cebd18018393dc147952e109"
1930 +dependencies = [
1931 + "dyn-stack",
1932 + "gemm-common 0.18.2",
1933 + "gemm-f32 0.18.2",
1934 + "half",
1935 + "num-complex",
1936 + "num-traits",
1937 + "paste",
1938 + "raw-cpuid",
1939 + "rayon",
1940 + "seq-macro",
1941 +]
1942 +
1943 +[[package]]
1944 +name = "gemm-f16"
1945 +version = "0.19.0"
1946 +source = "registry+https://github.com/rust-lang/crates.io-index"
1947 +checksum = "e3df7a55202e6cd6739d82ae3399c8e0c7e1402859b30e4cb780e61525d9486e"
1948 +dependencies = [
1949 + "dyn-stack",
1950 + "gemm-common 0.19.0",
1951 + "gemm-f32 0.19.0",
1952 + "half",
1953 + "num-complex",
1954 + "num-traits",
1955 + "paste",
1956 + "raw-cpuid",
1957 + "rayon",
1958 + "seq-macro",
1959 +]
1960 +
1961 +[[package]]
1962 +name = "gemm-f32"
1963 +version = "0.18.2"
1964 +source = "registry+https://github.com/rust-lang/crates.io-index"
1965 +checksum = "bc8d3d4385393304f407392f754cd2dc4b315d05063f62cf09f47b58de276864"
1966 +dependencies = [
1967 + "dyn-stack",
1968 + "gemm-common 0.18.2",
1969 + "num-complex",
1970 + "num-traits",
1971 + "paste",
1972 + "raw-cpuid",
1973 + "seq-macro",
1974 +]
1975 +
1976 +[[package]]
1977 +name = "gemm-f32"
1978 +version = "0.19.0"
1979 +source = "registry+https://github.com/rust-lang/crates.io-index"
1980 +checksum = "02e0b8c9da1fbec6e3e3ab2ce6bc259ef18eb5f6f0d3e4edf54b75f9fd41a81c"
1981 +dependencies = [
1982 + "dyn-stack",
1983 + "gemm-common 0.19.0",
1984 + "num-complex",
1985 + "num-traits",
1986 + "paste",
1987 + "raw-cpuid",
1988 + "seq-macro",
1989 +]
1990 +
1991 +[[package]]
1992 +name = "gemm-f64"
1993 +version = "0.18.2"
1994 +source = "registry+https://github.com/rust-lang/crates.io-index"
1995 +checksum = "35b2a4f76ce4b8b16eadc11ccf2e083252d8237c1b589558a49b0183545015bd"
1996 +dependencies = [
1997 + "dyn-stack",
1998 + "gemm-common 0.18.2",
1999 + "num-complex",
2000 + "num-traits",
2001 + "paste",
2002 + "raw-cpuid",
2003 + "seq-macro",
2004 +]
2005 +
2006 +[[package]]
2007 +name = "gemm-f64"
2008 +version = "0.19.0"
2009 +source = "registry+https://github.com/rust-lang/crates.io-index"
2010 +checksum = "056131e8f2a521bfab322f804ccd652520c79700d81209e9d9275bbdecaadc6a"
2011 +dependencies = [
2012 + "dyn-stack",
2013 + "gemm-common 0.19.0",
2014 + "num-complex",
2015 + "num-traits",
2016 + "paste",
2017 + "raw-cpuid",
2018 + "seq-macro",
2019 +]
2020 +
2021 +[[package]]
2022 +name = "generator"
2023 +version = "0.7.5"
2024 +source = "registry+https://github.com/rust-lang/crates.io-index"
2025 +checksum = "5cc16584ff22b460a382b7feec54b23d2908d858152e5739a120b949293bd74e"
2026 +dependencies = [
2027 + "cc",
2028 + "libc",
2029 + "log",
2030 + "rustversion",
2031 + "windows 0.48.0",
2032 +]
2033 +
2034 +[[package]]
2035 +name = "generic-array"
2036 +version = "0.14.7"
2037 +source = "registry+https://github.com/rust-lang/crates.io-index"
2038 +checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
2039 +dependencies = [
2040 + "typenum",
2041 + "version_check",
2042 +]
2043 +
2044 +[[package]]
2045 +name = "getopts"
2046 +version = "0.2.24"
2047 +source = "registry+https://github.com/rust-lang/crates.io-index"
2048 +checksum = "cfe4fbac503b8d1f88e6676011885f34b7174f46e59956bba534ba83abded4df"
2049 +dependencies = [
2050 + "unicode-width 0.2.2",
2051 +]
2052 +
2053 +[[package]]
2054 +name = "getrandom"
2055 +version = "0.2.17"
2056 +source = "registry+https://github.com/rust-lang/crates.io-index"
2057 +checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0"
2058 +dependencies = [
2059 + "cfg-if",
2060 + "js-sys",
2061 + "libc",
2062 + "wasi",
2063 + "wasm-bindgen",
2064 +]
2065 +
2066 +[[package]]
2067 +name = "getrandom"
2068 +version = "0.3.4"
2069 +source = "registry+https://github.com/rust-lang/crates.io-index"
2070 +checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd"
2071 +dependencies = [
2072 + "cfg-if",
2073 + "js-sys",
2074 + "libc",
2075 + "r-efi 5.3.0",
2076 + "wasip2",
2077 + "wasm-bindgen",
2078 +]
2079 +
2080 +[[package]]
2081 +name = "getrandom"
2082 +version = "0.4.2"
2083 +source = "registry+https://github.com/rust-lang/crates.io-index"
2084 +checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555"
2085 +dependencies = [
2086 + "cfg-if",
2087 + "libc",
2088 + "r-efi 6.0.0",
2089 + "wasip2",
2090 + "wasip3",
2091 +]
2092 +
2093 +[[package]]
2094 +name = "gif"
2095 +version = "0.14.1"
2096 +source = "registry+https://github.com/rust-lang/crates.io-index"
2097 +checksum = "f5df2ba84018d80c213569363bdcd0c64e6933c67fe4c1d60ecf822971a3c35e"
2098 +dependencies = [
2099 + "color_quant",
2100 + "weezl",
2101 +]
2102 +
2103 +[[package]]
2104 +name = "glob"
2105 +version = "0.3.3"
2106 +source = "registry+https://github.com/rust-lang/crates.io-index"
2107 +checksum = "0cc23270f6e1808e30a928bdc84dea0b9b4136a8bc82338574f23baf47bbd280"
2108 +
2109 +[[package]]
2110 +name = "goblin"
2111 +version = "0.8.2"
2112 +source = "registry+https://github.com/rust-lang/crates.io-index"
2113 +checksum = "1b363a30c165f666402fe6a3024d3bec7ebc898f96a4a23bd1c99f8dbf3f4f47"
2114 +dependencies = [
2115 + "log",
2116 + "plain",
2117 + "scroll",
2118 +]
2119 +
2120 +[[package]]
2121 +name = "h2"
2122 +version = "0.4.13"
2123 +source = "registry+https://github.com/rust-lang/crates.io-index"
2124 +checksum = "2f44da3a8150a6703ed5d34e164b875fd14c2cdab9af1252a9a1020bde2bdc54"
2125 +dependencies = [
2126 + "atomic-waker",
2127 + "bytes",
2128 + "fnv",
2129 + "futures-core",
2130 + "futures-sink",
2131 + "http",
2132 + "indexmap 2.13.0",
2133 + "slab",
2134 + "tokio",
2135 + "tokio-util",
2136 + "tracing",
2137 +]
2138 +
2139 +[[package]]
2140 +name = "half"
2141 +version = "2.7.1"
2142 +source = "registry+https://github.com/rust-lang/crates.io-index"
2143 +checksum = "6ea2d84b969582b4b1864a92dc5d27cd2b77b622a8d79306834f1be5ba20d84b"
2144 +dependencies = [
2145 + "bytemuck",
2146 + "cfg-if",
2147 + "crunchy",
2148 + "num-traits",
2149 + "rand 0.9.2",
2150 + "rand_distr 0.5.1",
2151 + "zerocopy",
2152 +]
2153 +
2154 +[[package]]
2155 +name = "hashbrown"
2156 +version = "0.12.3"
2157 +source = "registry+https://github.com/rust-lang/crates.io-index"
2158 +checksum = "8a9ee70c43aaf417c914396645a0fa852624801b24ebb7ae78fe8272889ac888"
2159 +
2160 +[[package]]
2161 +name = "hashbrown"
2162 +version = "0.15.5"
2163 +source = "registry+https://github.com/rust-lang/crates.io-index"
2164 +checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
2165 +dependencies = [
2166 + "allocator-api2",
2167 + "equivalent",
2168 + "foldhash 0.1.5",
2169 +]
2170 +
2171 +[[package]]
2172 +name = "hashbrown"
2173 +version = "0.16.1"
2174 +source = "registry+https://github.com/rust-lang/crates.io-index"
2175 +checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100"
2176 +dependencies = [
2177 + "allocator-api2",
2178 + "equivalent",
2179 + "foldhash 0.2.0",
2180 + "serde",
2181 + "serde_core",
2182 +]
2183 +
2184 +[[package]]
2185 +name = "heck"
2186 +version = "0.3.3"
2187 +source = "registry+https://github.com/rust-lang/crates.io-index"
2188 +checksum = "6d621efb26863f0e9924c6ac577e8275e5e6b77455db64ffa6c65c904e9e132c"
2189 +dependencies = [
2190 + "unicode-segmentation",
2191 +]
2192 +
2193 +[[package]]
2194 +name = "heck"
2195 +version = "0.5.0"
2196 +source = "registry+https://github.com/rust-lang/crates.io-index"
2197 +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
2198 +
2199 +[[package]]
2200 +name = "hermit-abi"
2201 +version = "0.1.19"
2202 +source = "registry+https://github.com/rust-lang/crates.io-index"
2203 +checksum = "62b467343b94ba476dcb2500d242dadbb39557df889310ac77c5d99100aaac33"
2204 +dependencies = [
2205 + "libc",
2206 +]
2207 +
2208 +[[package]]
2209 +name = "hermit-abi"
2210 +version = "0.5.2"
2211 +source = "registry+https://github.com/rust-lang/crates.io-index"
2212 +checksum = "fc0fef456e4baa96da950455cd02c081ca953b141298e41db3fc7e36b1da849c"
2213 +
2214 +[[package]]
2215 +name = "hex"
2216 +version = "0.4.3"
2217 +source = "registry+https://github.com/rust-lang/crates.io-index"
2218 +checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
2219 +
2220 +[[package]]
2221 +name = "hf-hub"
2222 +version = "0.4.3"
2223 +source = "registry+https://github.com/rust-lang/crates.io-index"
2224 +checksum = "629d8f3bbeda9d148036d6b0de0a3ab947abd08ce90626327fc3547a49d59d97"
2225 +dependencies = [
2226 + "dirs",
2227 + "futures",
2228 + "http",
2229 + "indicatif 0.17.11",
2230 + "libc",
2231 + "log",
2232 + "native-tls",
2233 + "num_cpus",
2234 + "rand 0.9.2",
2235 + "reqwest 0.12.28",
2236 + "serde",
2237 + "serde_json",
2238 + "thiserror 2.0.18",
2239 + "tokio",
2240 + "ureq",
2241 + "windows-sys 0.60.2",
2242 +]
2243 +
2244 +[[package]]
2245 +name = "home"
2246 +version = "0.5.12"
2247 +source = "registry+https://github.com/rust-lang/crates.io-index"
2248 +checksum = "cc627f471c528ff0c4a49e1d5e60450c8f6461dd6d10ba9dcd3a61d3dff7728d"
2249 +dependencies = [
2250 + "windows-sys 0.61.2",
2251 +]
2252 +
2253 +[[package]]
2254 +name = "hound"
2255 +version = "3.5.1"
2256 +source = "registry+https://github.com/rust-lang/crates.io-index"
2257 +checksum = "62adaabb884c94955b19907d60019f4e145d091c75345379e70d1ee696f7854f"
2258 +
2259 +[[package]]
2260 +name = "html2text"
2261 +version = "0.16.7"
2262 +source = "registry+https://github.com/rust-lang/crates.io-index"
2263 +checksum = "12d23156ea4dbe6b37ad48fab2da56ff27b0f6192fb5db210c44eb07bfe6e787"
2264 +dependencies = [
2265 + "html5ever 0.38.0",
2266 + "tendril 0.5.0",
2267 + "thiserror 2.0.18",
2268 + "unicode-width 0.2.2",
2269 +]
2270 +
2271 +[[package]]
2272 +name = "html5ever"
2273 +version = "0.36.1"
2274 +source = "registry+https://github.com/rust-lang/crates.io-index"
2275 +checksum = "6452c4751a24e1b99c3260d505eaeee76a050573e61f30ac2c924ddc7236f01e"
2276 +dependencies = [
2277 + "log",
2278 + "markup5ever 0.36.1",
2279 +]
2280 +
2281 +[[package]]
2282 +name = "html5ever"
2283 +version = "0.38.0"
2284 +source = "registry+https://github.com/rust-lang/crates.io-index"
2285 +checksum = "1054432bae2f14e0061e33d23402fbaa67a921d319d56adc6bcf887ddad1cbc2"
2286 +dependencies = [
2287 + "log",
2288 + "markup5ever 0.38.0",
2289 +]
2290 +
2291 +[[package]]
2292 +name = "http"
2293 +version = "1.4.0"
2294 +source = "registry+https://github.com/rust-lang/crates.io-index"
2295 +checksum = "e3ba2a386d7f85a81f119ad7498ebe444d2e22c2af0b86b069416ace48b3311a"
2296 +dependencies = [
2297 + "bytes",
2298 + "itoa",
2299 +]
2300 +
2301 +[[package]]
2302 +name = "http-body"
2303 +version = "1.0.1"
2304 +source = "registry+https://github.com/rust-lang/crates.io-index"
2305 +checksum = "1efedce1fb8e6913f23e0c92de8e62cd5b772a67e7b3946df930a62566c93184"
2306 +dependencies = [
2307 + "bytes",
2308 + "http",
2309 +]
2310 +
2311 +[[package]]
2312 +name = "http-body-util"
2313 +version = "0.1.3"
2314 +source = "registry+https://github.com/rust-lang/crates.io-index"
2315 +checksum = "b021d93e26becf5dc7e1b75b1bed1fd93124b374ceb73f43d4d4eafec896a64a"
2316 +dependencies = [
2317 + "bytes",
2318 + "futures-core",
2319 + "http",
2320 + "http-body",
2321 + "pin-project-lite",
2322 +]
2323 +
2324 +[[package]]
2325 +name = "httparse"
2326 +version = "1.10.1"
2327 +source = "registry+https://github.com/rust-lang/crates.io-index"
2328 +checksum = "6dbf3de79e51f3d586ab4cb9d5c3e2c14aa28ed23d180cf89b4df0454a69cc87"
2329 +
2330 +[[package]]
2331 +name = "hyper"
2332 +version = "1.8.1"
2333 +source = "registry+https://github.com/rust-lang/crates.io-index"
2334 +checksum = "2ab2d4f250c3d7b1c9fcdff1cece94ea4e2dfbec68614f7b87cb205f24ca9d11"
2335 +dependencies = [
2336 + "atomic-waker",
2337 + "bytes",
2338 + "futures-channel",
2339 + "futures-core",
2340 + "h2",
2341 + "http",
2342 + "http-body",
2343 + "httparse",
2344 + "itoa",
2345 + "pin-project-lite",
2346 + "pin-utils",
2347 + "smallvec 1.15.1",
2348 + "tokio",
2349 + "want",
2350 +]
2351 +
2352 +[[package]]
2353 +name = "hyper-rustls"
2354 +version = "0.27.7"
2355 +source = "registry+https://github.com/rust-lang/crates.io-index"
2356 +checksum = "e3c93eb611681b207e1fe55d5a71ecf91572ec8a6705cdb6857f7d8d5242cf58"
2357 +dependencies = [
2358 + "http",
2359 + "hyper",
2360 + "hyper-util",
2361 + "rustls",
2362 + "rustls-pki-types",
2363 + "tokio",
2364 + "tokio-rustls",
2365 + "tower-service",
2366 + "webpki-roots 1.0.6",
2367 +]
2368 +
2369 +[[package]]
2370 +name = "hyper-tls"
2371 +version = "0.6.0"
2372 +source = "registry+https://github.com/rust-lang/crates.io-index"
2373 +checksum = "70206fc6890eaca9fde8a0bf71caa2ddfc9fe045ac9e5c70df101a7dbde866e0"
2374 +dependencies = [
2375 + "bytes",
2376 + "http-body-util",
2377 + "hyper",
2378 + "hyper-util",
2379 + "native-tls",
2380 + "tokio",
2381 + "tokio-native-tls",
2382 + "tower-service",
2383 +]
2384 +
2385 +[[package]]
2386 +name = "hyper-util"
2387 +version = "0.1.20"
2388 +source = "registry+https://github.com/rust-lang/crates.io-index"
2389 +checksum = "96547c2556ec9d12fb1578c4eaf448b04993e7fb79cbaad930a656880a6bdfa0"
2390 +dependencies = [
2391 + "base64 0.22.1",
2392 + "bytes",
2393 + "futures-channel",
2394 + "futures-util",
2395 + "http",
2396 + "http-body",
2397 + "hyper",
2398 + "ipnet",
2399 + "libc",
2400 + "percent-encoding",
2401 + "pin-project-lite",
2402 + "socket2",
2403 + "system-configuration",
2404 + "tokio",
2405 + "tower-service",
2406 + "tracing",
2407 + "windows-registry",
2408 +]
2409 +
2410 +[[package]]
2411 +name = "iana-time-zone"
2412 +version = "0.1.65"
2413 +source = "registry+https://github.com/rust-lang/crates.io-index"
2414 +checksum = "e31bc9ad994ba00e440a8aa5c9ef0ec67d5cb5e5cb0cc7f8b744a35b389cc470"
2415 +dependencies = [
2416 + "android_system_properties",
2417 + "core-foundation-sys",
2418 + "iana-time-zone-haiku",
2419 + "js-sys",
2420 + "log",
2421 + "wasm-bindgen",
2422 + "windows-core 0.62.2",
2423 +]
2424 +
2425 +[[package]]
2426 +name = "iana-time-zone-haiku"
2427 +version = "0.1.2"
2428 +source = "registry+https://github.com/rust-lang/crates.io-index"
2429 +checksum = "f31827a206f56af32e590ba56d5d2d085f558508192593743f16b2306495269f"
2430 +dependencies = [
2431 + "cc",
2432 +]
2433 +
2434 +[[package]]
2435 +name = "icu_collections"
2436 +version = "2.1.1"
2437 +source = "registry+https://github.com/rust-lang/crates.io-index"
2438 +checksum = "4c6b649701667bbe825c3b7e6388cb521c23d88644678e83c0c4d0a621a34b43"
2439 +dependencies = [
2440 + "displaydoc",
2441 + "potential_utf",
2442 + "yoke 0.8.1",
2443 + "zerofrom",
2444 + "zerovec",
2445 +]
2446 +
2447 +[[package]]
2448 +name = "icu_locale_core"
2449 +version = "2.1.1"
2450 +source = "registry+https://github.com/rust-lang/crates.io-index"
2451 +checksum = "edba7861004dd3714265b4db54a3c390e880ab658fec5f7db895fae2046b5bb6"
2452 +dependencies = [
2453 + "displaydoc",
2454 + "litemap",
2455 + "tinystr",
2456 + "writeable",
2457 + "zerovec",
2458 +]
2459 +
2460 +[[package]]
2461 +name = "icu_normalizer"
2462 +version = "2.1.1"
2463 +source = "registry+https://github.com/rust-lang/crates.io-index"
2464 +checksum = "5f6c8828b67bf8908d82127b2054ea1b4427ff0230ee9141c54251934ab1b599"
2465 +dependencies = [
2466 + "icu_collections",
2467 + "icu_normalizer_data",
2468 + "icu_properties",
2469 + "icu_provider",
2470 + "smallvec 1.15.1",
2471 + "zerovec",
2472 +]
2473 +
2474 +[[package]]
2475 +name = "icu_normalizer_data"
2476 +version = "2.1.1"
2477 +source = "registry+https://github.com/rust-lang/crates.io-index"
2478 +checksum = "7aedcccd01fc5fe81e6b489c15b247b8b0690feb23304303a9e560f37efc560a"
2479 +
2480 +[[package]]
2481 +name = "icu_properties"
2482 +version = "2.1.2"
2483 +source = "registry+https://github.com/rust-lang/crates.io-index"
2484 +checksum = "020bfc02fe870ec3a66d93e677ccca0562506e5872c650f893269e08615d74ec"
2485 +dependencies = [
2486 + "icu_collections",
2487 + "icu_locale_core",
2488 + "icu_properties_data",
2489 + "icu_provider",
2490 + "zerotrie",
2491 + "zerovec",
2492 +]
2493 +
2494 +[[package]]
2495 +name = "icu_properties_data"
2496 +version = "2.1.2"
2497 +source = "registry+https://github.com/rust-lang/crates.io-index"
2498 +checksum = "616c294cf8d725c6afcd8f55abc17c56464ef6211f9ed59cccffe534129c77af"
2499 +
2500 +[[package]]
2501 +name = "icu_provider"
2502 +version = "2.1.1"
2503 +source = "registry+https://github.com/rust-lang/crates.io-index"
2504 +checksum = "85962cf0ce02e1e0a629cc34e7ca3e373ce20dda4c4d7294bbd0bf1fdb59e614"
2505 +dependencies = [
2506 + "displaydoc",
2507 + "icu_locale_core",
2508 + "writeable",
2509 + "yoke 0.8.1",
2510 + "zerofrom",
2511 + "zerotrie",
2512 + "zerovec",
2513 +]
2514 +
2515 +[[package]]
2516 +name = "id-arena"
2517 +version = "2.3.0"
2518 +source = "registry+https://github.com/rust-lang/crates.io-index"
2519 +checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
2520 +
2521 +[[package]]
2522 +name = "ident_case"
2523 +version = "1.0.1"
2524 +source = "registry+https://github.com/rust-lang/crates.io-index"
2525 +checksum = "b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39"
2526 +
2527 +[[package]]
2528 +name = "idna"
2529 +version = "1.1.0"
2530 +source = "registry+https://github.com/rust-lang/crates.io-index"
2531 +checksum = "3b0875f23caa03898994f6ddc501886a45c7d3d62d04d2d90788d47be1b1e4de"
2532 +dependencies = [
2533 + "idna_adapter",
2534 + "smallvec 1.15.1",
2535 + "utf8_iter",
2536 +]
2537 +
2538 +[[package]]
2539 +name = "idna_adapter"
2540 +version = "1.2.1"
2541 +source = "registry+https://github.com/rust-lang/crates.io-index"
2542 +checksum = "3acae9609540aa318d1bc588455225fb2085b9ed0c4f6bd0d9d5bcd86f1a0344"
2543 +dependencies = [
2544 + "icu_normalizer",
2545 + "icu_properties",
2546 +]
2547 +
2548 +[[package]]
2549 +name = "image"
2550 +version = "0.25.10"
2551 +source = "registry+https://github.com/rust-lang/crates.io-index"
2552 +checksum = "85ab80394333c02fe689eaf900ab500fbd0c2213da414687ebf995a65d5a6104"
2553 +dependencies = [
2554 + "bytemuck",
2555 + "byteorder-lite",
2556 + "color_quant",
2557 + "exr",
2558 + "gif",
2559 + "image-webp",
2560 + "moxcms",
2561 + "num-traits",
2562 + "png",
2563 + "qoi",
2564 + "ravif",
2565 + "rayon",
2566 + "rgb",
2567 + "tiff",
2568 + "zune-core",
2569 + "zune-jpeg",
2570 +]
2571 +
2572 +[[package]]
2573 +name = "image-webp"
2574 +version = "0.2.4"
2575 +source = "registry+https://github.com/rust-lang/crates.io-index"
2576 +checksum = "525e9ff3e1a4be2fbea1fdf0e98686a6d98b4d8f937e1bf7402245af1909e8c3"
2577 +dependencies = [
2578 + "byteorder-lite",
2579 + "quick-error",
2580 +]
2581 +
2582 +[[package]]
2583 +name = "imgref"
2584 +version = "1.12.0"
2585 +source = "registry+https://github.com/rust-lang/crates.io-index"
2586 +checksum = "e7c5cedc30da3a610cac6b4ba17597bdf7152cf974e8aab3afb3d54455e371c8"
2587 +
2588 +[[package]]
2589 +name = "indexmap"
2590 +version = "1.9.3"
2591 +source = "registry+https://github.com/rust-lang/crates.io-index"
2592 +checksum = "bd070e393353796e801d209ad339e89596eb4c8d430d18ede6a1cced8fafbd99"
2593 +dependencies = [
2594 + "autocfg",
2595 + "hashbrown 0.12.3",
2596 + "serde",
2597 +]
2598 +
2599 +[[package]]
2600 +name = "indexmap"
2601 +version = "2.13.0"
2602 +source = "registry+https://github.com/rust-lang/crates.io-index"
2603 +checksum = "7714e70437a7dc3ac8eb7e6f8df75fd8eb422675fc7678aff7364301092b1017"
2604 +dependencies = [
2605 + "equivalent",
2606 + "hashbrown 0.16.1",
2607 + "serde",
2608 + "serde_core",
2609 +]
2610 +
2611 +[[package]]
2612 +name = "indicatif"
2613 +version = "0.17.11"
2614 +source = "registry+https://github.com/rust-lang/crates.io-index"
2615 +checksum = "183b3088984b400f4cfac3620d5e076c84da5364016b4f49473de574b2586235"
2616 +dependencies = [
2617 + "console 0.15.11",
2618 + "number_prefix",
2619 + "portable-atomic",
2620 + "unicode-width 0.2.2",
2621 + "web-time",
2622 +]
2623 +
2624 +[[package]]
2625 +name = "indicatif"
2626 +version = "0.18.4"
2627 +source = "registry+https://github.com/rust-lang/crates.io-index"
2628 +checksum = "25470f23803092da7d239834776d653104d551bc4d7eacaf31e6837854b8e9eb"
2629 +dependencies = [
2630 + "console 0.16.3",
2631 + "portable-atomic",
2632 + "rayon",
2633 + "unicode-width 0.2.2",
2634 + "unit-prefix",
2635 + "web-time",
2636 +]
2637 +
2638 +[[package]]
2639 +name = "interpolate_name"
2640 +version = "0.2.4"
2641 +source = "registry+https://github.com/rust-lang/crates.io-index"
2642 +checksum = "c34819042dc3d3971c46c2190835914dfbe0c3c13f61449b2997f4e9722dfa60"
2643 +dependencies = [
2644 + "proc-macro2",
2645 + "quote",
2646 + "syn 2.0.117",
2647 +]
2648 +
2649 +[[package]]
2650 +name = "interprocess"
2651 +version = "2.4.0"
2652 +source = "registry+https://github.com/rust-lang/crates.io-index"
2653 +checksum = "6be5e5c847dbdb44564bd85294740d031f4f8aeb3464e5375ef7141f7538db69"
2654 +dependencies = [
2655 + "doctest-file",
2656 + "libc",
2657 + "recvmsg",
2658 + "widestring",
2659 + "windows-sys 0.52.0",
2660 +]
2661 +
2662 +[[package]]
2663 +name = "ipnet"
2664 +version = "2.12.0"
2665 +source = "registry+https://github.com/rust-lang/crates.io-index"
2666 +checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2"
2667 +
2668 +[[package]]
2669 +name = "iri-string"
2670 +version = "0.7.10"
2671 +source = "registry+https://github.com/rust-lang/crates.io-index"
2672 +checksum = "c91338f0783edbd6195decb37bae672fd3b165faffb89bf7b9e6942f8b1a731a"
2673 +dependencies = [
2674 + "memchr",
2675 + "serde",
2676 +]
2677 +
2678 +[[package]]
2679 +name = "is_terminal_polyfill"
2680 +version = "1.70.2"
2681 +source = "registry+https://github.com/rust-lang/crates.io-index"
2682 +checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695"
2683 +
2684 +[[package]]
2685 +name = "itertools"
2686 +version = "0.14.0"
2687 +source = "registry+https://github.com/rust-lang/crates.io-index"
2688 +checksum = "2b192c782037fadd9cfa75548310488aabdbf3d2da73885b31bd0abd03351285"
2689 +dependencies = [
2690 + "either",
2691 +]
2692 +
2693 +[[package]]
2694 +name = "itoa"
2695 +version = "1.0.18"
2696 +source = "registry+https://github.com/rust-lang/crates.io-index"
2697 +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
2698 +
2699 +[[package]]
2700 +name = "jni"
2701 +version = "0.21.1"
2702 +source = "registry+https://github.com/rust-lang/crates.io-index"
2703 +checksum = "1a87aa2bb7d2af34197c04845522473242e1aa17c12f4935d5856491a7fb8c97"
2704 +dependencies = [
2705 + "cesu8",
2706 + "cfg-if",
2707 + "combine",
2708 + "jni-sys",
2709 + "log",
2710 + "thiserror 1.0.69",
2711 + "walkdir",
2712 + "windows-sys 0.45.0",
2713 +]
2714 +
2715 +[[package]]
2716 +name = "jni-sys"
2717 +version = "0.3.0"
2718 +source = "registry+https://github.com/rust-lang/crates.io-index"
2719 +checksum = "8eaf4bc02d17cbdd7ff4c7438cafcdf7fb9a4613313ad11b4f8fefe7d3fa0130"
2720 +
2721 +[[package]]
2722 +name = "jobserver"
2723 +version = "0.1.34"
2724 +source = "registry+https://github.com/rust-lang/crates.io-index"
2725 +checksum = "9afb3de4395d6b3e67a780b6de64b51c978ecf11cb9a462c66be7d4ca9039d33"
2726 +dependencies = [
2727 + "getrandom 0.3.4",
2728 + "libc",
2729 +]
2730 +
2731 +[[package]]
2732 +name = "js-sys"
2733 +version = "0.3.91"
2734 +source = "registry+https://github.com/rust-lang/crates.io-index"
2735 +checksum = "b49715b7073f385ba4bc528e5747d02e66cb39c6146efb66b781f131f0fb399c"
2736 +dependencies = [
2737 + "once_cell",
2738 + "wasm-bindgen",
2739 +]
2740 +
2741 +[[package]]
2742 +name = "lazy_static"
2743 +version = "1.5.0"
2744 +source = "registry+https://github.com/rust-lang/crates.io-index"
2745 +checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
2746 +
2747 +[[package]]
2748 +name = "leb128fmt"
2749 +version = "0.1.0"
2750 +source = "registry+https://github.com/rust-lang/crates.io-index"
2751 +checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
2752 +
2753 +[[package]]
2754 +name = "lebe"
2755 +version = "0.5.3"
2756 +source = "registry+https://github.com/rust-lang/crates.io-index"
2757 +checksum = "7a79a3332a6609480d7d0c9eab957bca6b455b91bb84e66d19f5ff66294b85b8"
2758 +
2759 +[[package]]
2760 +name = "libc"
2761 +version = "0.2.183"
2762 +source = "registry+https://github.com/rust-lang/crates.io-index"
2763 +checksum = "b5b646652bf6661599e1da8901b3b9522896f01e736bad5f723fe7a3a27f899d"
2764 +
2765 +[[package]]
2766 +name = "libfuzzer-sys"
2767 +version = "0.4.12"
2768 +source = "registry+https://github.com/rust-lang/crates.io-index"
2769 +checksum = "f12a681b7dd8ce12bff52488013ba614b869148d54dd79836ab85aafdd53f08d"
2770 +dependencies = [
2771 + "arbitrary",
2772 + "cc",
2773 +]
2774 +
2775 +[[package]]
2776 +name = "libloading"
2777 +version = "0.8.9"
2778 +source = "registry+https://github.com/rust-lang/crates.io-index"
2779 +checksum = "d7c4b02199fee7c5d21a5ae7d8cfa79a6ef5bb2fc834d6e9058e89c825efdc55"
2780 +dependencies = [
2781 + "cfg-if",
2782 + "windows-link 0.2.1",
2783 +]
2784 +
2785 +[[package]]
2786 +name = "libm"
2787 +version = "0.2.16"
2788 +source = "registry+https://github.com/rust-lang/crates.io-index"
2789 +checksum = "b6d2cec3eae94f9f509c767b45932f1ada8350c4bdb85af2fcab4a3c14807981"
2790 +
2791 +[[package]]
2792 +name = "libredox"
2793 +version = "0.1.14"
2794 +source = "registry+https://github.com/rust-lang/crates.io-index"
2795 +checksum = "1744e39d1d6a9948f4f388969627434e31128196de472883b39f148769bfe30a"
2796 +dependencies = [
2797 + "libc",
2798 +]
2799 +
2800 +[[package]]
2801 +name = "linux-raw-sys"
2802 +version = "0.12.1"
2803 +source = "registry+https://github.com/rust-lang/crates.io-index"
2804 +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
2805 +
2806 +[[package]]
2807 +name = "litemap"
2808 +version = "0.8.1"
2809 +source = "registry+https://github.com/rust-lang/crates.io-index"
2810 +checksum = "6373607a59f0be73a39b6fe456b8192fcc3585f602af20751600e974dd455e77"
2811 +
2812 +[[package]]
2813 +name = "llguidance"
2814 +version = "1.7.0"
2815 +source = "registry+https://github.com/rust-lang/crates.io-index"
2816 +checksum = "586ccb282d83fe337ddad2e28bb11bb258a462caa80e117fc76a6c73dac919e4"
2817 +dependencies = [
2818 + "anyhow",
2819 + "derivre",
2820 + "indexmap 2.13.0",
2821 + "regex-syntax",
2822 + "serde",
2823 + "serde_json",
2824 + "toktrie",
2825 +]
2826 +
2827 +[[package]]
2828 +name = "lock_api"
2829 +version = "0.4.14"
2830 +source = "registry+https://github.com/rust-lang/crates.io-index"
2831 +checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965"
2832 +dependencies = [
2833 + "scopeguard",
2834 +]
2835 +
2836 +[[package]]
2837 +name = "log"
2838 +version = "0.4.29"
2839 +source = "registry+https://github.com/rust-lang/crates.io-index"
2840 +checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897"
2841 +
2842 +[[package]]
2843 +name = "loom"
2844 +version = "0.5.6"
2845 +source = "registry+https://github.com/rust-lang/crates.io-index"
2846 +checksum = "ff50ecb28bb86013e935fb6683ab1f6d3a20016f123c76fd4c27470076ac30f5"
2847 +dependencies = [
2848 + "cfg-if",
2849 + "generator",
2850 + "scoped-tls",
2851 + "serde",
2852 + "serde_json",
2853 + "tracing",
2854 + "tracing-subscriber",
2855 +]
2856 +
2857 +[[package]]
2858 +name = "loop9"
2859 +version = "0.1.5"
2860 +source = "registry+https://github.com/rust-lang/crates.io-index"
2861 +checksum = "0fae87c125b03c1d2c0150c90365d7d6bcc53fb73a9acaef207d2d065860f062"
2862 +dependencies = [
2863 + "imgref",
2864 +]
2865 +
2866 +[[package]]
2867 +name = "lrtable"
2868 +version = "0.14.1"
2869 +source = "registry+https://github.com/rust-lang/crates.io-index"
2870 +checksum = "b15ad6a43e5cff4ac046d51281d2256a36d1440065eab53ccce9362b48db5b42"
2871 +dependencies = [
2872 + "cfgrammar",
2873 + "fnv",
2874 + "num-traits",
2875 + "sparsevec",
2876 + "vob",
2877 +]
2878 +
2879 +[[package]]
2880 +name = "lru-slab"
2881 +version = "0.1.2"
2882 +source = "registry+https://github.com/rust-lang/crates.io-index"
2883 +checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154"
2884 +
2885 +[[package]]
2886 +name = "mac"
2887 +version = "0.1.1"
2888 +source = "registry+https://github.com/rust-lang/crates.io-index"
2889 +checksum = "c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4"
2890 +
2891 +[[package]]
2892 +name = "macro_rules_attribute"
2893 +version = "0.2.2"
2894 +source = "registry+https://github.com/rust-lang/crates.io-index"
2895 +checksum = "65049d7923698040cd0b1ddcced9b0eb14dd22c5f86ae59c3740eab64a676520"
2896 +dependencies = [
2897 + "macro_rules_attribute-proc_macro",
2898 + "paste",
2899 +]
2900 +
2901 +[[package]]
2902 +name = "macro_rules_attribute-proc_macro"
2903 +version = "0.2.2"
2904 +source = "registry+https://github.com/rust-lang/crates.io-index"
2905 +checksum = "670fdfda89751bc4a84ac13eaa63e205cf0fd22b4c9a5fbfa085b63c1f1d3a30"
2906 +
2907 +[[package]]
2908 +name = "malloc_buf"
2909 +version = "0.0.6"
2910 +source = "registry+https://github.com/rust-lang/crates.io-index"
2911 +checksum = "62bb907fe88d54d8d9ce32a3cceab4218ed2f6b7d35617cafe9adf84e43919cb"
2912 +dependencies = [
2913 + "libc",
2914 +]
2915 +
2916 +[[package]]
2917 +name = "markup5ever"
2918 +version = "0.36.1"
2919 +source = "registry+https://github.com/rust-lang/crates.io-index"
2920 +checksum = "6c3294c4d74d0742910f8c7b466f44dda9eb2d5742c1e430138df290a1e8451c"
2921 +dependencies = [
2922 + "log",
2923 + "tendril 0.4.3",
2924 + "web_atoms",
2925 +]
2926 +
2927 +[[package]]
2928 +name = "markup5ever"
2929 +version = "0.38.0"
2930 +source = "registry+https://github.com/rust-lang/crates.io-index"
2931 +checksum = "8983d30f2915feeaaab2d6babdd6bc7e9ed1a00b66b5e6d74df19aa9c0e91862"
2932 +dependencies = [
2933 + "log",
2934 + "tendril 0.5.0",
2935 + "web_atoms",
2936 +]
2937 +
2938 +[[package]]
2939 +name = "matchers"
2940 +version = "0.2.0"
2941 +source = "registry+https://github.com/rust-lang/crates.io-index"
2942 +checksum = "d1525a2a28c7f4fa0fc98bb91ae755d1e2d1505079e05539e35bc876b5d65ae9"
2943 +dependencies = [
2944 + "regex-automata",
2945 +]
2946 +
2947 +[[package]]
2948 +name = "matrixmultiply"
2949 +version = "0.3.10"
2950 +source = "registry+https://github.com/rust-lang/crates.io-index"
2951 +checksum = "a06de3016e9fae57a36fd14dba131fccf49f74b40b7fbdb472f96e361ec71a08"
2952 +dependencies = [
2953 + "autocfg",
2954 + "rawpointer",
2955 +]
2956 +
2957 +[[package]]
2958 +name = "maybe-rayon"
2959 +version = "0.1.1"
2960 +source = "registry+https://github.com/rust-lang/crates.io-index"
2961 +checksum = "8ea1f30cedd69f0a2954655f7188c6a834246d2bcf1e315e2ac40c4b24dc9519"
2962 +dependencies = [
2963 + "cfg-if",
2964 + "rayon",
2965 +]
2966 +
2967 +[[package]]
2968 +name = "memchr"
2969 +version = "2.8.0"
2970 +source = "registry+https://github.com/rust-lang/crates.io-index"
2971 +checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
2972 +
2973 +[[package]]
2974 +name = "memmap2"
2975 +version = "0.9.10"
2976 +source = "registry+https://github.com/rust-lang/crates.io-index"
2977 +checksum = "714098028fe011992e1c3962653c96b2d578c4b4bce9036e15ff220319b1e0e3"
2978 +dependencies = [
2979 + "libc",
2980 + "stable_deref_trait",
2981 +]
2982 +
2983 +[[package]]
2984 +name = "memo-map"
2985 +version = "0.3.3"
2986 +source = "registry+https://github.com/rust-lang/crates.io-index"
2987 +checksum = "38d1115007560874e373613744c6fba374c17688327a71c1476d1a5954cc857b"
2988 +
2989 +[[package]]
2990 +name = "metal"
2991 +version = "0.29.0"
2992 +source = "registry+https://github.com/rust-lang/crates.io-index"
2993 +checksum = "7ecfd3296f8c56b7c1f6fbac3c71cefa9d78ce009850c45000015f206dc7fa21"
2994 +dependencies = [
2995 + "bitflags 2.11.0",
2996 + "block",
2997 + "core-graphics-types",
2998 + "foreign-types 0.5.0",
2999 + "log",
3000 + "objc",
3001 + "paste",
3002 +]
3003 +
3004 +[[package]]
3005 +name = "mime"
3006 +version = "0.3.17"
3007 +source = "registry+https://github.com/rust-lang/crates.io-index"
3008 +checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a"
3009 +
3010 +[[package]]
3011 +name = "mime_guess"
3012 +version = "2.0.5"
3013 +source = "registry+https://github.com/rust-lang/crates.io-index"
3014 +checksum = "f7c44f8e672c00fe5308fa235f821cb4198414e1c77935c1ab6948d3fd78550e"
3015 +dependencies = [
3016 + "mime",
3017 + "unicase",
3018 +]
3019 +
3020 +[[package]]
3021 +name = "minijinja"
3022 +version = "2.18.0"
3023 +source = "registry+https://github.com/rust-lang/crates.io-index"
3024 +checksum = "328251e58ad8e415be6198888fc207502727dc77945806421ab34f35bf012e7d"
3025 +dependencies = [
3026 + "memo-map",
3027 + "serde",
3028 + "serde_json",
3029 +]
3030 +
3031 +[[package]]
3032 +name = "minijinja-contrib"
3033 +version = "2.18.0"
3034 +source = "registry+https://github.com/rust-lang/crates.io-index"
3035 +checksum = "8c6302e47d2b51f9fc978268ff7f5a014de5caa2ad48440309fd10ee711480d7"
3036 +dependencies = [
3037 + "minijinja",
3038 + "serde",
3039 +]
3040 +
3041 +[[package]]
3042 +name = "minimal-lexical"
3043 +version = "0.2.1"
3044 +source = "registry+https://github.com/rust-lang/crates.io-index"
3045 +checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
3046 +
3047 +[[package]]
3048 +name = "miniz_oxide"
3049 +version = "0.8.9"
3050 +source = "registry+https://github.com/rust-lang/crates.io-index"
3051 +checksum = "1fa76a2c86f704bdb222d66965fb3d63269ce38518b83cb0575fca855ebb6316"
3052 +dependencies = [
3053 + "adler2",
3054 + "simd-adler32",
3055 +]
3056 +
3057 +[[package]]
3058 +name = "mio"
3059 +version = "0.8.11"
3060 +source = "registry+https://github.com/rust-lang/crates.io-index"
3061 +checksum = "a4a650543ca06a924e8b371db273b2756685faae30f8487da1b56505a8f78b0c"
3062 +dependencies = [
3063 + "libc",
3064 + "log",
3065 + "wasi",
3066 + "windows-sys 0.48.0",
3067 +]
3068 +
3069 +[[package]]
3070 +name = "mio"
3071 +version = "1.1.1"
3072 +source = "registry+https://github.com/rust-lang/crates.io-index"
3073 +checksum = "a69bcab0ad47271a0234d9422b131806bf3968021e5dc9328caf2d4cd58557fc"
3074 +dependencies = [
3075 + "libc",
3076 + "wasi",
3077 + "windows-sys 0.61.2",
3078 +]
3079 +
3080 +[[package]]
3081 +name = "mistralrs"
3082 +version = "0.7.1-alpha.1"
3083 +source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#82e6efad7004ed105f241b6d6608feff0ec185f8"
3084 +dependencies = [
3085 + "anyhow",
3086 + "candle-core",
3087 + "candle-nn",
3088 + "clap 4.6.0",
3089 + "either",
3090 + "futures",
3091 + "image",
3092 + "indexmap 2.13.0",
3093 + "mistralrs-core",
3094 + "mistralrs-macros",
3095 + "rand 0.9.2",
3096 + "reqwest 0.13.2",
3097 + "schemars 1.2.1",
3098 + "serde",
3099 + "serde_json",
3100 + "thiserror 2.0.18",
3101 + "tokio",
3102 + "tracing",
3103 + "tracing-subscriber",
3104 + "walkdir",
3105 +]
3106 +
3107 +[[package]]
3108 +name = "mistralrs-audio"
3109 +version = "0.7.1-alpha.1"
3110 +source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#82e6efad7004ed105f241b6d6608feff0ec185f8"
3111 +dependencies = [
3112 + "anyhow",
3113 + "apodize",
3114 + "hound",
3115 + "symphonia",
3116 +]
3117 +
3118 +[[package]]
3119 +name = "mistralrs-core"
3120 +version = "0.7.1-alpha.1"
3121 +source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#82e6efad7004ed105f241b6d6608feff0ec185f8"
3122 +dependencies = [
3123 + "ahash",
3124 + "akin",
3125 + "anyhow",
3126 + "apodize",
3127 + "as-any",
3128 + "async-trait",
3129 + "base64 0.22.1",
3130 + "bm25",
3131 + "bytemuck",
3132 + "bytemuck_derive",
3133 + "candle-core",
3134 + "candle-metal-kernels",
3135 + "candle-nn",
3136 + "cfgrammar",
3137 + "chrono",
3138 + "clap 4.6.0",
3139 + "csv",
3140 + "derive-new",
3141 + "derive_more",
3142 + "dirs",
3143 + "either",
3144 + "float8",
3145 + "futures",
3146 + "galil-seiferas",
3147 + "half",
3148 + "hashbrown 0.16.1",
3149 + "hf-hub",
3150 + "hound",
3151 + "html2text",
3152 + "http",
3153 + "image",
3154 + "indexmap 2.13.0",
3155 + "indicatif 0.18.4",
3156 + "interprocess",
3157 + "itertools",
3158 + "libc",
3159 + "llguidance",
3160 + "lrtable",
3161 + "minijinja",
3162 + "minijinja-contrib",
3163 + "mistralrs-audio",
3164 + "mistralrs-mcp",
3165 + "mistralrs-paged-attn",
3166 + "mistralrs-quant",
3167 + "mistralrs-vision",
3168 + "num-traits",
3169 + "objc",
3170 + "objc2-metal",
3171 + "openai-harmony",
3172 + "ordered-float",
3173 + "parking_lot",
3174 + "radix_trie",
3175 + "rand 0.9.2",
3176 + "rand_distr 0.5.1",
3177 + "rand_isaac",
3178 + "rayon",
3179 + "regex",
3180 + "regex-automata",
3181 + "reqwest 0.13.2",
3182 + "rubato",
3183 + "rust-mcp-schema",
3184 + "rustc-hash 2.1.1",
3185 + "rustfft",
3186 + "safetensors 0.7.0",
3187 + "schemars 1.2.1",
3188 + "scraper",
3189 + "serde",
3190 + "serde-big-array",
3191 + "serde-saphyr",
3192 + "serde_json",
3193 + "serde_plain",
3194 + "statrs",
3195 + "strum",
3196 + "symphonia",
3197 + "sysinfo",
3198 + "thiserror 2.0.18",
3199 + "tokenizers",
3200 + "tokio",
3201 + "tokio-rayon",
3202 + "tokio-tungstenite",
3203 + "toktrie",
3204 + "toktrie_hf_tokenizers",
3205 + "toml",
3206 + "tqdm",
3207 + "tracing",
3208 + "tracing-subscriber",
3209 + "urlencoding",
3210 + "uuid 1.22.0",
3211 + "variantly",
3212 + "vob",
3213 +]
3214 +
3215 +[[package]]
3216 +name = "mistralrs-macros"
3217 +version = "0.7.1-alpha.1"
3218 +source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#82e6efad7004ed105f241b6d6608feff0ec185f8"
3219 +dependencies = [
3220 + "darling 0.23.0",
3221 + "proc-macro2",
3222 + "quote",
3223 + "syn 2.0.117",
3224 +]
3225 +
3226 +[[package]]
3227 +name = "mistralrs-mcp"
3228 +version = "0.7.1-alpha.1"
3229 +source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#82e6efad7004ed105f241b6d6608feff0ec185f8"
3230 +dependencies = [
3231 + "anyhow",
3232 + "async-trait",
3233 + "futures-util",
3234 + "http",
3235 + "reqwest 0.13.2",
3236 + "rust-mcp-schema",
3237 + "serde",
3238 + "serde_json",
3239 + "tokio",
3240 + "tokio-tungstenite",
3241 + "tracing",
3242 + "utoipa",
3243 + "uuid 1.22.0",
3244 +]
3245 +
3246 +[[package]]
3247 +name = "mistralrs-paged-attn"
3248 +version = "0.7.1-alpha.1"
3249 +source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#82e6efad7004ed105f241b6d6608feff0ec185f8"
3250 +dependencies = [
3251 + "anyhow",
3252 + "candle-core",
3253 + "candle-metal-kernels",
3254 + "dispatch2",
3255 + "float8",
3256 + "half",
3257 + "objc2-foundation",
3258 + "objc2-metal",
3259 + "thiserror 2.0.18",
3260 +]
3261 +
3262 +[[package]]
3263 +name = "mistralrs-quant"
3264 +version = "0.7.1-alpha.1"
3265 +source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#82e6efad7004ed105f241b6d6608feff0ec185f8"
3266 +dependencies = [
3267 + "byteorder",
3268 + "candle-core",
3269 + "candle-metal-kernels",
3270 + "candle-nn",
3271 + "dispatch2",
3272 + "float8",
3273 + "half",
3274 + "hf-hub",
3275 + "lazy_static",
3276 + "memmap2",
3277 + "objc2-foundation",
3278 + "objc2-metal",
3279 + "paste",
3280 + "rayon",
3281 + "regex",
3282 + "safetensors 0.7.0",
3283 + "serde",
3284 + "serde_json",
3285 + "thiserror 2.0.18",
3286 + "tokio",
3287 + "tracing",
3288 + "yoke 0.8.1",
3289 +]
3290 +
3291 +[[package]]
3292 +name = "mistralrs-vision"
3293 +version = "0.7.1-alpha.1"
3294 +source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#82e6efad7004ed105f241b6d6608feff0ec185f8"
3295 +dependencies = [
3296 + "candle-core",
3297 + "image",
3298 + "rayon",
3299 +]
3300 +
3301 +[[package]]
3302 +name = "monostate"
3303 +version = "0.1.18"
3304 +source = "registry+https://github.com/rust-lang/crates.io-index"
3305 +checksum = "3341a273f6c9d5bef1908f17b7267bbab0e95c9bf69a0d4dcf8e9e1b2c76ef67"
3306 +dependencies = [
3307 + "monostate-impl",
3308 + "serde",
3309 + "serde_core",
3310 +]
3311 +
3312 +[[package]]
3313 +name = "monostate-impl"
3314 +version = "0.1.18"
3315 +source = "registry+https://github.com/rust-lang/crates.io-index"
3316 +checksum = "e4db6d5580af57bf992f59068d4ea26fd518574ff48d7639b255a36f9de6e7e9"
3317 +dependencies = [
3318 + "proc-macro2",
3319 + "quote",
3320 + "syn 2.0.117",
3321 +]
3322 +
3323 +[[package]]
3324 +name = "moxcms"
3325 +version = "0.8.1"
3326 +source = "registry+https://github.com/rust-lang/crates.io-index"
3327 +checksum = "bb85c154ba489f01b25c0d36ae69a87e4a1c73a72631fc6c0eb6dde34a73e44b"
3328 +dependencies = [
3329 + "num-traits",
3330 + "pxfm",
3331 +]
3332 +
3333 +[[package]]
3334 +name = "nalgebra"
3335 +version = "0.33.2"
3336 +source = "registry+https://github.com/rust-lang/crates.io-index"
3337 +checksum = "26aecdf64b707efd1310e3544d709c5c0ac61c13756046aaaba41be5c4f66a3b"
3338 +dependencies = [
3339 + "approx",
3340 + "matrixmultiply",
3341 + "num-complex",
3342 + "num-rational",
3343 + "num-traits",
3344 + "rand 0.8.5",
3345 + "rand_distr 0.4.3",
3346 + "simba",
3347 + "typenum",
3348 +]
3349 +
3350 +[[package]]
3351 +name = "native-tls"
3352 +version = "0.2.18"
3353 +source = "registry+https://github.com/rust-lang/crates.io-index"
3354 +checksum = "465500e14ea162429d264d44189adc38b199b62b1c21eea9f69e4b73cb03bbf2"
3355 +dependencies = [
3356 + "libc",
3357 + "log",
3358 + "openssl",
3359 + "openssl-probe",
3360 + "openssl-sys",
3361 + "schannel",
3362 + "security-framework",
3363 + "security-framework-sys",
3364 + "tempfile",
3365 +]
3366 +
3367 +[[package]]
3368 +name = "new_debug_unreachable"
3369 +version = "1.0.6"
3370 +source = "registry+https://github.com/rust-lang/crates.io-index"
3371 +checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086"
3372 +
3373 +[[package]]
3374 +name = "nibble_vec"
3375 +version = "0.1.0"
3376 +source = "registry+https://github.com/rust-lang/crates.io-index"
3377 +checksum = "77a5d83df9f36fe23f0c3648c6bbb8b0298bb5f1939c8f2704431371f4b84d43"
3378 +dependencies = [
3379 + "smallvec 1.15.1",
3380 +]
3381 +
3382 +[[package]]
3383 +name = "nohash-hasher"
3384 +version = "0.2.0"
3385 +source = "registry+https://github.com/rust-lang/crates.io-index"
3386 +checksum = "2bf50223579dc7cdcfb3bfcacf7069ff68243f8c363f62ffa99cf000a6b9c451"
3387 +
3388 +[[package]]
3389 +name = "nom"
3390 +version = "7.1.3"
3391 +source = "registry+https://github.com/rust-lang/crates.io-index"
3392 +checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
3393 +dependencies = [
3394 + "memchr",
3395 + "minimal-lexical",
3396 +]
3397 +
3398 +[[package]]
3399 +name = "nom"
3400 +version = "8.0.0"
3401 +source = "registry+https://github.com/rust-lang/crates.io-index"
3402 +checksum = "df9761775871bdef83bee530e60050f7e54b1105350d6884eb0fb4f46c2f9405"
3403 +dependencies = [
3404 + "memchr",
3405 +]
3406 +
3407 +[[package]]
3408 +name = "noop_proc_macro"
3409 +version = "0.3.0"
3410 +source = "registry+https://github.com/rust-lang/crates.io-index"
3411 +checksum = "0676bb32a98c1a483ce53e500a81ad9c3d5b3f7c920c28c24e9cb0980d0b5bc8"
3412 +
3413 +[[package]]
3414 +name = "ntapi"
3415 +version = "0.4.3"
3416 +source = "registry+https://github.com/rust-lang/crates.io-index"
3417 +checksum = "c3b335231dfd352ffb0f8017f3b6027a4917f7df785ea2143d8af2adc66980ae"
3418 +dependencies = [
3419 + "winapi",
3420 +]
3421 +
3422 +[[package]]
3423 +name = "nu-ansi-term"
3424 +version = "0.50.3"
3425 +source = "registry+https://github.com/rust-lang/crates.io-index"
3426 +checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
3427 +dependencies = [
3428 + "windows-sys 0.61.2",
3429 +]
3430 +
3431 +[[package]]
3432 +name = "num"
3433 +version = "0.4.3"
3434 +source = "registry+https://github.com/rust-lang/crates.io-index"
3435 +checksum = "35bd024e8b2ff75562e5f34e7f4905839deb4b22955ef5e73d2fea1b9813cb23"
3436 +dependencies = [
3437 + "num-bigint",
3438 + "num-complex",
3439 + "num-integer",
3440 + "num-iter",
3441 + "num-rational",
3442 + "num-traits",
3443 +]
3444 +
3445 +[[package]]
3446 +name = "num-bigint"
3447 +version = "0.4.6"
3448 +source = "registry+https://github.com/rust-lang/crates.io-index"
3449 +checksum = "a5e44f723f1133c9deac646763579fdb3ac745e418f2a7af9cd0c431da1f20b9"
3450 +dependencies = [
3451 + "num-integer",
3452 + "num-traits",
3453 +]
3454 +
3455 +[[package]]
3456 +name = "num-complex"
3457 +version = "0.4.6"
3458 +source = "registry+https://github.com/rust-lang/crates.io-index"
3459 +checksum = "73f88a1307638156682bada9d7604135552957b7818057dcef22705b4d509495"
3460 +dependencies = [
3461 + "bytemuck",
3462 + "num-traits",
3463 +]
3464 +
3465 +[[package]]
3466 +name = "num-conv"
3467 +version = "0.2.0"
3468 +source = "registry+https://github.com/rust-lang/crates.io-index"
3469 +checksum = "cf97ec579c3c42f953ef76dbf8d55ac91fb219dde70e49aa4a6b7d74e9919050"
3470 +
3471 +[[package]]
3472 +name = "num-derive"
3473 +version = "0.4.2"
3474 +source = "registry+https://github.com/rust-lang/crates.io-index"
3475 +checksum = "ed3955f1a9c7c0c15e092f9c887db08b1fc683305fdf6eb6684f22555355e202"
3476 +dependencies = [
3477 + "proc-macro2",
3478 + "quote",
3479 + "syn 2.0.117",
3480 +]
3481 +
3482 +[[package]]
3483 +name = "num-integer"
3484 +version = "0.1.46"
3485 +source = "registry+https://github.com/rust-lang/crates.io-index"
3486 +checksum = "7969661fd2958a5cb096e56c8e1ad0444ac2bbcd0061bd28660485a44879858f"
3487 +dependencies = [
3488 + "num-traits",
3489 +]
3490 +
3491 +[[package]]
3492 +name = "num-iter"
3493 +version = "0.1.45"
3494 +source = "registry+https://github.com/rust-lang/crates.io-index"
3495 +checksum = "1429034a0490724d0075ebb2bc9e875d6503c3cf69e235a8941aa757d83ef5bf"
3496 +dependencies = [
3497 + "autocfg",
3498 + "num-integer",
3499 + "num-traits",
3500 +]
3501 +
3502 +[[package]]
3503 +name = "num-rational"
3504 +version = "0.4.2"
3505 +source = "registry+https://github.com/rust-lang/crates.io-index"
3506 +checksum = "f83d14da390562dca69fc84082e73e548e1ad308d24accdedd2720017cb37824"
3507 +dependencies = [
3508 + "num-bigint",
3509 + "num-integer",
3510 + "num-traits",
3511 +]
3512 +
3513 +[[package]]
3514 +name = "num-traits"
3515 +version = "0.2.19"
3516 +source = "registry+https://github.com/rust-lang/crates.io-index"
3517 +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
3518 +dependencies = [
3519 + "autocfg",
3520 + "libm",
3521 +]
3522 +
3523 +[[package]]
3524 +name = "num_cpus"
3525 +version = "1.17.0"
3526 +source = "registry+https://github.com/rust-lang/crates.io-index"
3527 +checksum = "91df4bbde75afed763b708b7eee1e8e7651e02d97f6d5dd763e89367e957b23b"
3528 +dependencies = [
3529 + "hermit-abi 0.5.2",
3530 + "libc",
3531 +]
3532 +
3533 +[[package]]
3534 +name = "number_prefix"
3535 +version = "0.4.0"
3536 +source = "registry+https://github.com/rust-lang/crates.io-index"
3537 +checksum = "830b246a0e5f20af87141b25c173cd1b609bd7779a4617d6ec582abaf90870f3"
3538 +
3539 +[[package]]
3540 +name = "objc"
3541 +version = "0.2.7"
3542 +source = "registry+https://github.com/rust-lang/crates.io-index"
3543 +checksum = "915b1b472bc21c53464d6c8461c9d3af805ba1ef837e1cac254428f4a77177b1"
3544 +dependencies = [
3545 + "malloc_buf",
3546 +]
3547 +
3548 +[[package]]
3549 +name = "objc2"
3550 +version = "0.6.4"
3551 +source = "registry+https://github.com/rust-lang/crates.io-index"
3552 +checksum = "3a12a8ed07aefc768292f076dc3ac8c48f3781c8f2d5851dd3d98950e8c5a89f"
3553 +dependencies = [
3554 + "objc2-encode",
3555 +]
3556 +
3557 +[[package]]
3558 +name = "objc2-core-foundation"
3559 +version = "0.3.2"
3560 +source = "registry+https://github.com/rust-lang/crates.io-index"
3561 +checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
3562 +dependencies = [
3563 + "bitflags 2.11.0",
3564 + "dispatch2",
3565 + "objc2",
3566 +]
3567 +
3568 +[[package]]
3569 +name = "objc2-encode"
3570 +version = "4.1.0"
3571 +source = "registry+https://github.com/rust-lang/crates.io-index"
3572 +checksum = "ef25abbcd74fb2609453eb695bd2f860d389e457f67dc17cafc8b8cbc89d0c33"
3573 +
3574 +[[package]]
3575 +name = "objc2-foundation"
3576 +version = "0.3.2"
3577 +source = "registry+https://github.com/rust-lang/crates.io-index"
3578 +checksum = "e3e0adef53c21f888deb4fa59fc59f7eb17404926ee8a6f59f5df0fd7f9f3272"
3579 +dependencies = [
3580 + "bitflags 2.11.0",
3581 + "block2",
3582 + "libc",
3583 + "objc2",
3584 + "objc2-core-foundation",
3585 +]
3586 +
3587 +[[package]]
3588 +name = "objc2-io-kit"
3589 +version = "0.3.2"
3590 +source = "registry+https://github.com/rust-lang/crates.io-index"
3591 +checksum = "33fafba39597d6dc1fb709123dfa8289d39406734be322956a69f0931c73bb15"
3592 +dependencies = [
3593 + "libc",
3594 + "objc2-core-foundation",
3595 +]
3596 +
3597 +[[package]]
3598 +name = "objc2-metal"
3599 +version = "0.3.2"
3600 +source = "registry+https://github.com/rust-lang/crates.io-index"
3601 +checksum = "a0125f776a10d00af4152d74616409f0d4a2053a6f57fa5b7d6aa2854ac04794"
3602 +dependencies = [
3603 + "bitflags 2.11.0",
3604 + "block2",
3605 + "dispatch2",
3606 + "objc2",
3607 + "objc2-core-foundation",
3608 + "objc2-foundation",
3609 +]
3610 +
3611 +[[package]]
3612 +name = "once_cell"
3613 +version = "1.21.4"
3614 +source = "registry+https://github.com/rust-lang/crates.io-index"
3615 +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
3616 +
3617 +[[package]]
3618 +name = "once_cell_polyfill"
3619 +version = "1.70.2"
3620 +source = "registry+https://github.com/rust-lang/crates.io-index"
3621 +checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
3622 +
3623 +[[package]]
3624 +name = "onde"
3625 +version = "0.1.0"
3626 +dependencies = [
3627 + "anyhow",
3628 + "cc",
3629 + "hf-hub",
3630 + "home",
3631 + "log",
3632 + "mistralrs",
3633 + "mistralrs-core",
3634 + "serde",
3635 + "thiserror 2.0.18",
3636 + "tokio",
3637 + "tsync",
3638 + "uniffi",
3639 +]
3640 +
3641 +[[package]]
3642 +name = "openai-harmony"
3643 +version = "0.0.8"
3644 +source = "registry+https://github.com/rust-lang/crates.io-index"
3645 +checksum = "e77e82af451fc95deeb728a40b84db8ee82d341e136c268de415123a560b9b72"
3646 +dependencies = [
3647 + "anyhow",
3648 + "base64 0.22.1",
3649 + "bstr",
3650 + "clap 4.6.0",
3651 + "fancy-regex 0.13.0",
3652 + "futures",
3653 + "image",
3654 + "regex",
3655 + "reqwest 0.12.28",
3656 + "rustc-hash 1.1.0",
3657 + "serde",
3658 + "serde_json",
3659 + "serde_with",
3660 + "sha1",
3661 + "sha2",
3662 + "thiserror 2.0.18",
3663 +]
3664 +
3665 +[[package]]
3666 +name = "openssl"
3667 +version = "0.10.76"
3668 +source = "registry+https://github.com/rust-lang/crates.io-index"
3669 +checksum = "951c002c75e16ea2c65b8c7e4d3d51d5530d8dfa7d060b4776828c88cfb18ecf"
3670 +dependencies = [
3671 + "bitflags 2.11.0",
3672 + "cfg-if",
3673 + "foreign-types 0.3.2",
3674 + "libc",
3675 + "once_cell",
3676 + "openssl-macros",
3677 + "openssl-sys",
3678 +]
3679 +
3680 +[[package]]
3681 +name = "openssl-macros"
3682 +version = "0.1.1"
3683 +source = "registry+https://github.com/rust-lang/crates.io-index"
3684 +checksum = "a948666b637a0f465e8564c73e89d4dde00d72d4d473cc972f390fc3dcee7d9c"
3685 +dependencies = [
3686 + "proc-macro2",
3687 + "quote",
3688 + "syn 2.0.117",
3689 +]
3690 +
3691 +[[package]]
3692 +name = "openssl-probe"
3693 +version = "0.2.1"
3694 +source = "registry+https://github.com/rust-lang/crates.io-index"
3695 +checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe"
3696 +
3697 +[[package]]
3698 +name = "openssl-sys"
3699 +version = "0.9.112"
3700 +source = "registry+https://github.com/rust-lang/crates.io-index"
3701 +checksum = "57d55af3b3e226502be1526dfdba67ab0e9c96fc293004e79576b2b9edb0dbdb"
3702 +dependencies = [
3703 + "cc",
3704 + "libc",
3705 + "pkg-config",
3706 + "vcpkg",
3707 +]
3708 +
3709 +[[package]]
3710 +name = "option-ext"
3711 +version = "0.2.0"
3712 +source = "registry+https://github.com/rust-lang/crates.io-index"
3713 +checksum = "04744f49eae99ab78e0d5c0b603ab218f515ea8cfe5a456d7629ad883a3b6e7d"
3714 +
3715 +[[package]]
3716 +name = "ordered-float"
3717 +version = "5.1.0"
3718 +source = "registry+https://github.com/rust-lang/crates.io-index"
3719 +checksum = "7f4779c6901a562440c3786d08192c6fbda7c1c2060edd10006b05ee35d10f2d"
3720 +dependencies = [
3721 + "num-traits",
3722 +]
3723 +
3724 +[[package]]
3725 +name = "packedvec"
3726 +version = "1.2.5"
3727 +source = "registry+https://github.com/rust-lang/crates.io-index"
3728 +checksum = "a69e0a534dd2e6aefce319af62a0aa0066a76bdfcec0201dfe02df226bc9ec70"
3729 +dependencies = [
3730 + "num-traits",
3731 +]
3732 +
3733 +[[package]]
3734 +name = "parking_lot"
3735 +version = "0.12.5"
3736 +source = "registry+https://github.com/rust-lang/crates.io-index"
3737 +checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a"
3738 +dependencies = [
3739 + "lock_api",
3740 + "parking_lot_core",
3741 +]
3742 +
3743 +[[package]]
3744 +name = "parking_lot_core"
3745 +version = "0.9.12"
3746 +source = "registry+https://github.com/rust-lang/crates.io-index"
3747 +checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1"
3748 +dependencies = [
3749 + "cfg-if",
3750 + "libc",
3751 + "redox_syscall",
3752 + "smallvec 1.15.1",
3753 + "windows-link 0.2.1",
3754 +]
3755 +
3756 +[[package]]
3757 +name = "paste"
3758 +version = "1.0.15"
3759 +source = "registry+https://github.com/rust-lang/crates.io-index"
3760 +checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a"
3761 +
3762 +[[package]]
3763 +name = "pastey"
3764 +version = "0.1.1"
3765 +source = "registry+https://github.com/rust-lang/crates.io-index"
3766 +checksum = "35fb2e5f958ec131621fdd531e9fc186ed768cbe395337403ae56c17a74c68ec"
3767 +
3768 +[[package]]
3769 +name = "percent-encoding"
3770 +version = "2.3.2"
3771 +source = "registry+https://github.com/rust-lang/crates.io-index"
3772 +checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
3773 +
3774 +[[package]]
3775 +name = "phf"
3776 +version = "0.13.1"
3777 +source = "registry+https://github.com/rust-lang/crates.io-index"
3778 +checksum = "c1562dc717473dbaa4c1f85a36410e03c047b2e7df7f45ee938fbef64ae7fadf"
3779 +dependencies = [
3780 + "phf_macros",
3781 + "phf_shared",
3782 + "serde",
3783 +]
3784 +
3785 +[[package]]
3786 +name = "phf_codegen"
3787 +version = "0.13.1"
3788 +source = "registry+https://github.com/rust-lang/crates.io-index"
3789 +checksum = "49aa7f9d80421bca176ca8dbfebe668cc7a2684708594ec9f3c0db0805d5d6e1"
3790 +dependencies = [
3791 + "phf_generator",
3792 + "phf_shared",
3793 +]
3794 +
3795 +[[package]]
3796 +name = "phf_generator"
3797 +version = "0.13.1"
3798 +source = "registry+https://github.com/rust-lang/crates.io-index"
3799 +checksum = "135ace3a761e564ec88c03a77317a7c6b80bb7f7135ef2544dbe054243b89737"
3800 +dependencies = [
3801 + "fastrand",
3802 + "phf_shared",
3803 +]
3804 +
3805 +[[package]]
3806 +name = "phf_macros"
3807 +version = "0.13.1"
3808 +source = "registry+https://github.com/rust-lang/crates.io-index"
3809 +checksum = "812f032b54b1e759ccd5f8b6677695d5268c588701effba24601f6932f8269ef"
3810 +dependencies = [
3811 + "phf_generator",
3812 + "phf_shared",
3813 + "proc-macro2",
3814 + "quote",
3815 + "syn 2.0.117",
3816 +]
3817 +
3818 +[[package]]
3819 +name = "phf_shared"
3820 +version = "0.13.1"
3821 +source = "registry+https://github.com/rust-lang/crates.io-index"
3822 +checksum = "e57fef6bc5981e38c2ce2d63bfa546861309f875b8a75f092d1d54ae2d64f266"
3823 +dependencies = [
3824 + "siphasher",
3825 +]
3826 +
3827 +[[package]]
3828 +name = "pin-project-lite"
3829 +version = "0.2.17"
3830 +source = "registry+https://github.com/rust-lang/crates.io-index"
3831 +checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
3832 +
3833 +[[package]]
3834 +name = "pin-utils"
3835 +version = "0.1.0"
3836 +source = "registry+https://github.com/rust-lang/crates.io-index"
3837 +checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"
3838 +
3839 +[[package]]
3840 +name = "pkg-config"
3841 +version = "0.3.32"
3842 +source = "registry+https://github.com/rust-lang/crates.io-index"
3843 +checksum = "7edddbd0b52d732b21ad9a5fab5c704c14cd949e5e9a1ec5929a24fded1b904c"
3844 +
3845 +[[package]]
3846 +name = "plain"
3847 +version = "0.2.3"
3848 +source = "registry+https://github.com/rust-lang/crates.io-index"
3849 +checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
3850 +
3851 +[[package]]
3852 +name = "png"
3853 +version = "0.18.1"
3854 +source = "registry+https://github.com/rust-lang/crates.io-index"
3855 +checksum = "60769b8b31b2a9f263dae2776c37b1b28ae246943cf719eb6946a1db05128a61"
3856 +dependencies = [
3857 + "bitflags 2.11.0",
3858 + "crc32fast",
3859 + "fdeflate",
3860 + "flate2",
3861 + "miniz_oxide",
3862 +]
3863 +
3864 +[[package]]
3865 +name = "portable-atomic"
3866 +version = "1.13.1"
3867 +source = "registry+https://github.com/rust-lang/crates.io-index"
3868 +checksum = "c33a9471896f1c69cecef8d20cbe2f7accd12527ce60845ff44c153bb2a21b49"
3869 +
3870 +[[package]]
3871 +name = "potential_utf"
3872 +version = "0.1.4"
3873 +source = "registry+https://github.com/rust-lang/crates.io-index"
3874 +checksum = "b73949432f5e2a09657003c25bca5e19a0e9c84f8058ca374f49e0ebe605af77"
3875 +dependencies = [
3876 + "zerovec",
3877 +]
3878 +
3879 +[[package]]
3880 +name = "powerfmt"
3881 +version = "0.2.0"
3882 +source = "registry+https://github.com/rust-lang/crates.io-index"
3883 +checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391"
3884 +
3885 +[[package]]
3886 +name = "ppv-lite86"
3887 +version = "0.2.21"
3888 +source = "registry+https://github.com/rust-lang/crates.io-index"
3889 +checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9"
3890 +dependencies = [
3891 + "zerocopy",
3892 +]
3893 +
3894 +[[package]]
3895 +name = "precomputed-hash"
3896 +version = "0.1.1"
3897 +source = "registry+https://github.com/rust-lang/crates.io-index"
3898 +checksum = "925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c"
3899 +
3900 +[[package]]
3901 +name = "prettyplease"
3902 +version = "0.2.37"
3903 +source = "registry+https://github.com/rust-lang/crates.io-index"
3904 +checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
3905 +dependencies = [
3906 + "proc-macro2",
3907 + "syn 2.0.117",
3908 +]
3909 +
3910 +[[package]]
3911 +name = "primal-check"
3912 +version = "0.3.4"
3913 +source = "registry+https://github.com/rust-lang/crates.io-index"
3914 +checksum = "dc0d895b311e3af9902528fbb8f928688abbd95872819320517cc24ca6b2bd08"
3915 +dependencies = [
3916 + "num-integer",
3917 +]
3918 +
3919 +[[package]]
3920 +name = "proc-macro-error"
3921 +version = "1.0.4"
3922 +source = "registry+https://github.com/rust-lang/crates.io-index"
3923 +checksum = "da25490ff9892aab3fcf7c36f08cfb902dd3e71ca0f9f9517bea02a73a5ce38c"
3924 +dependencies = [
3925 + "proc-macro-error-attr",
3926 + "proc-macro2",
3927 + "quote",
3928 + "syn 1.0.109",
3929 + "version_check",
3930 +]
3931 +
3932 +[[package]]
3933 +name = "proc-macro-error-attr"
3934 +version = "1.0.4"
3935 +source = "registry+https://github.com/rust-lang/crates.io-index"
3936 +checksum = "a1be40180e52ecc98ad80b184934baf3d0d29f979574e439af5a55274b35f869"
3937 +dependencies = [
3938 + "proc-macro2",
3939 + "quote",
3940 + "version_check",
3941 +]
3942 +
3943 +[[package]]
3944 +name = "proc-macro2"
3945 +version = "1.0.106"
3946 +source = "registry+https://github.com/rust-lang/crates.io-index"
3947 +checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
3948 +dependencies = [
3949 + "unicode-ident",
3950 +]
3951 +
3952 +[[package]]
3953 +name = "profiling"
3954 +version = "1.0.17"
3955 +source = "registry+https://github.com/rust-lang/crates.io-index"
3956 +checksum = "3eb8486b569e12e2c32ad3e204dbaba5e4b5b216e9367044f25f1dba42341773"
3957 +dependencies = [
3958 + "profiling-procmacros",
3959 +]
3960 +
3961 +[[package]]
3962 +name = "profiling-procmacros"
3963 +version = "1.0.17"
3964 +source = "registry+https://github.com/rust-lang/crates.io-index"
3965 +checksum = "52717f9a02b6965224f95ca2a81e2e0c5c43baacd28ca057577988930b6c3d5b"
3966 +dependencies = [
3967 + "quote",
3968 + "syn 2.0.117",
3969 +]
3970 +
3971 +[[package]]
3972 +name = "pulp"
3973 +version = "0.21.5"
3974 +source = "registry+https://github.com/rust-lang/crates.io-index"
3975 +checksum = "96b86df24f0a7ddd5e4b95c94fc9ed8a98f1ca94d3b01bdce2824097e7835907"
3976 +dependencies = [
3977 + "bytemuck",
3978 + "cfg-if",
3979 + "libm",
3980 + "num-complex",
3981 + "reborrow",
3982 + "version_check",
3983 +]
3984 +
3985 +[[package]]
3986 +name = "pulp"
3987 +version = "0.22.2"
3988 +source = "registry+https://github.com/rust-lang/crates.io-index"
3989 +checksum = "2e205bb30d5b916c55e584c22201771bcf2bad9aabd5d4127f38387140c38632"
3990 +dependencies = [
3991 + "bytemuck",
3992 + "cfg-if",
3993 + "libm",
3994 + "num-complex",
3995 + "paste",
3996 + "pulp-wasm-simd-flag",
3997 + "raw-cpuid",
3998 + "reborrow",
3999 + "version_check",
4000 +]
4001 +
4002 +[[package]]
4003 +name = "pulp-wasm-simd-flag"
4004 +version = "0.1.0"
4005 +source = "registry+https://github.com/rust-lang/crates.io-index"
4006 +checksum = "40e24eee682d89fb193496edf918a7f407d30175b2e785fe057e4392dfd182e0"
4007 +
4008 +[[package]]
4009 +name = "pxfm"
4010 +version = "0.1.28"
4011 +source = "registry+https://github.com/rust-lang/crates.io-index"
4012 +checksum = "b5a041e753da8b807c9255f28de81879c78c876392ff2469cde94799b2896b9d"
4013 +
4014 +[[package]]
4015 +name = "qoi"
4016 +version = "0.4.1"
4017 +source = "registry+https://github.com/rust-lang/crates.io-index"
4018 +checksum = "7f6d64c71eb498fe9eae14ce4ec935c555749aef511cca85b5568910d6e48001"
4019 +dependencies = [
4020 + "bytemuck",
4021 +]
4022 +
4023 +[[package]]
4024 +name = "quick-error"
4025 +version = "2.0.1"
4026 +source = "registry+https://github.com/rust-lang/crates.io-index"
4027 +checksum = "a993555f31e5a609f617c12db6250dedcac1b0a85076912c436e6fc9b2c8e6a3"
4028 +
4029 +[[package]]
4030 +name = "quinn"
4031 +version = "0.11.9"
4032 +source = "registry+https://github.com/rust-lang/crates.io-index"
4033 +checksum = "b9e20a958963c291dc322d98411f541009df2ced7b5a4f2bd52337638cfccf20"
4034 +dependencies = [
4035 + "bytes",
4036 + "cfg_aliases",
4037 + "pin-project-lite",
4038 + "quinn-proto",
4039 + "quinn-udp",
4040 + "rustc-hash 2.1.1",
4041 + "rustls",
4042 + "socket2",
4043 + "thiserror 2.0.18",
4044 + "tokio",
4045 + "tracing",
4046 + "web-time",
4047 +]
4048 +
4049 +[[package]]
4050 +name = "quinn-proto"
4051 +version = "0.11.14"
4052 +source = "registry+https://github.com/rust-lang/crates.io-index"
4053 +checksum = "434b42fec591c96ef50e21e886936e66d3cc3f737104fdb9b737c40ffb94c098"
4054 +dependencies = [
4055 + "aws-lc-rs",
4056 + "bytes",
4057 + "getrandom 0.3.4",
4058 + "lru-slab",
4059 + "rand 0.9.2",
4060 + "ring",
4061 + "rustc-hash 2.1.1",
4062 + "rustls",
4063 + "rustls-pki-types",
4064 + "slab",
4065 + "thiserror 2.0.18",
4066 + "tinyvec",
4067 + "tracing",
4068 + "web-time",
4069 +]
4070 +
4071 +[[package]]
4072 +name = "quinn-udp"
4073 +version = "0.5.14"
4074 +source = "registry+https://github.com/rust-lang/crates.io-index"
4075 +checksum = "addec6a0dcad8a8d96a771f815f0eaf55f9d1805756410b39f5fa81332574cbd"
4076 +dependencies = [
4077 + "cfg_aliases",
4078 + "libc",
4079 + "once_cell",
4080 + "socket2",
4081 + "tracing",
4082 + "windows-sys 0.60.2",
4083 +]
4084 +
4085 +[[package]]
4086 +name = "quote"
4087 +version = "1.0.45"
4088 +source = "registry+https://github.com/rust-lang/crates.io-index"
4089 +checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
4090 +dependencies = [
4091 + "proc-macro2",
4092 +]
4093 +
4094 +[[package]]
4095 +name = "r-efi"
4096 +version = "5.3.0"
4097 +source = "registry+https://github.com/rust-lang/crates.io-index"
4098 +checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f"
4099 +
4100 +[[package]]
4101 +name = "r-efi"
4102 +version = "6.0.0"
4103 +source = "registry+https://github.com/rust-lang/crates.io-index"
4104 +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
4105 +
4106 +[[package]]
4107 +name = "radix_trie"
4108 +version = "0.3.0"
4109 +source = "registry+https://github.com/rust-lang/crates.io-index"
4110 +checksum = "3b4431027dcd37fc2a73ef740b5f233aa805897935b8bce0195e41bbf9a3289a"
4111 +dependencies = [
4112 + "endian-type",
4113 + "nibble_vec",
4114 +]
4115 +
4116 +[[package]]
4117 +name = "rand"
4118 +version = "0.8.5"
4119 +source = "registry+https://github.com/rust-lang/crates.io-index"
4120 +checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404"
4121 +dependencies = [
4122 + "libc",
4123 + "rand_chacha 0.3.1",
4124 + "rand_core 0.6.4",
4125 +]
4126 +
4127 +[[package]]
4128 +name = "rand"
4129 +version = "0.9.2"
4130 +source = "registry+https://github.com/rust-lang/crates.io-index"
4131 +checksum = "6db2770f06117d490610c7488547d543617b21bfa07796d7a12f6f1bd53850d1"
4132 +dependencies = [
4133 + "rand_chacha 0.9.0",
4134 + "rand_core 0.9.5",
4135 +]
4136 +
4137 +[[package]]
4138 +name = "rand_chacha"
4139 +version = "0.3.1"
4140 +source = "registry+https://github.com/rust-lang/crates.io-index"
4141 +checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88"
4142 +dependencies = [
4143 + "ppv-lite86",
4144 + "rand_core 0.6.4",
4145 +]
4146 +
4147 +[[package]]
4148 +name = "rand_chacha"
4149 +version = "0.9.0"
4150 +source = "registry+https://github.com/rust-lang/crates.io-index"
4151 +checksum = "d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb"
4152 +dependencies = [
4153 + "ppv-lite86",
4154 + "rand_core 0.9.5",
4155 +]
4156 +
4157 +[[package]]
4158 +name = "rand_core"
4159 +version = "0.6.4"
4160 +source = "registry+https://github.com/rust-lang/crates.io-index"
4161 +checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c"
4162 +dependencies = [
4163 + "getrandom 0.2.17",
4164 +]
4165 +
4166 +[[package]]
4167 +name = "rand_core"
4168 +version = "0.9.5"
4169 +source = "registry+https://github.com/rust-lang/crates.io-index"
4170 +checksum = "76afc826de14238e6e8c374ddcc1fa19e374fd8dd986b0d2af0d02377261d83c"
4171 +dependencies = [
4172 + "getrandom 0.3.4",
4173 +]
4174 +
4175 +[[package]]
4176 +name = "rand_distr"
4177 +version = "0.4.3"
4178 +source = "registry+https://github.com/rust-lang/crates.io-index"
4179 +checksum = "32cb0b9bc82b0a0876c2dd994a7e7a2683d3e7390ca40e6886785ef0c7e3ee31"
4180 +dependencies = [
4181 + "num-traits",
4182 + "rand 0.8.5",
4183 +]
4184 +
4185 +[[package]]
4186 +name = "rand_distr"
4187 +version = "0.5.1"
4188 +source = "registry+https://github.com/rust-lang/crates.io-index"
4189 +checksum = "6a8615d50dcf34fa31f7ab52692afec947c4dd0ab803cc87cb3b0b4570ff7463"
4190 +dependencies = [
4191 + "num-traits",
4192 + "rand 0.9.2",
4193 +]
4194 +
4195 +[[package]]
4196 +name = "rand_isaac"
4197 +version = "0.4.0"
4198 +source = "registry+https://github.com/rust-lang/crates.io-index"
4199 +checksum = "3382fc9f0aad4f2e2a56b53d9133c8c810b4dbf21e7e370e24346161a5b2c7bd"
4200 +dependencies = [
4201 + "rand_core 0.9.5",
4202 +]
4203 +
4204 +[[package]]
4205 +name = "rav1e"
4206 +version = "0.8.1"
4207 +source = "registry+https://github.com/rust-lang/crates.io-index"
4208 +checksum = "43b6dd56e85d9483277cde964fd1bdb0428de4fec5ebba7540995639a21cb32b"
4209 +dependencies = [
4210 + "aligned-vec",
4211 + "arbitrary",
4212 + "arg_enum_proc_macro",
4213 + "arrayvec",
4214 + "av-scenechange",
4215 + "av1-grain",
4216 + "bitstream-io",
4217 + "built",
4218 + "cfg-if",
4219 + "interpolate_name",
4220 + "itertools",
4221 + "libc",
4222 + "libfuzzer-sys",
4223 + "log",
4224 + "maybe-rayon",
4225 + "new_debug_unreachable",
4226 + "noop_proc_macro",
4227 + "num-derive",
4228 + "num-traits",
4229 + "paste",
4230 + "profiling",
4231 + "rand 0.9.2",
4232 + "rand_chacha 0.9.0",
4233 + "simd_helpers",
4234 + "thiserror 2.0.18",
4235 + "v_frame",
4236 + "wasm-bindgen",
4237 +]
4238 +
4239 +[[package]]
4240 +name = "ravif"
4241 +version = "0.13.0"
4242 +source = "registry+https://github.com/rust-lang/crates.io-index"
4243 +checksum = "e52310197d971b0f5be7fe6b57530dcd27beb35c1b013f29d66c1ad73fbbcc45"
4244 +dependencies = [
4245 + "avif-serialize",
4246 + "imgref",
4247 + "loop9",
4248 + "quick-error",
4249 + "rav1e",
4250 + "rayon",
4251 + "rgb",
4252 +]
4253 +
4254 +[[package]]
4255 +name = "raw-cpuid"
4256 +version = "11.6.0"
4257 +source = "registry+https://github.com/rust-lang/crates.io-index"
4258 +checksum = "498cd0dc59d73224351ee52a95fee0f1a617a2eae0e7d9d720cc622c73a54186"
4259 +dependencies = [
4260 + "bitflags 2.11.0",
4261 +]
4262 +
4263 +[[package]]
4264 +name = "rawpointer"
4265 +version = "0.2.1"
4266 +source = "registry+https://github.com/rust-lang/crates.io-index"
4267 +checksum = "60a357793950651c4ed0f3f52338f53b2f809f32d83a07f72909fa13e4c6c1e3"
4268 +
4269 +[[package]]
4270 +name = "rayon"
4271 +version = "1.11.0"
4272 +source = "registry+https://github.com/rust-lang/crates.io-index"
4273 +checksum = "368f01d005bf8fd9b1206fb6fa653e6c4a81ceb1466406b81792d87c5677a58f"
4274 +dependencies = [
4275 + "either",
4276 + "rayon-core",
4277 +]
4278 +
4279 +[[package]]
4280 +name = "rayon-cond"
4281 +version = "0.4.0"
4282 +source = "registry+https://github.com/rust-lang/crates.io-index"
4283 +checksum = "2964d0cf57a3e7a06e8183d14a8b527195c706b7983549cd5462d5aa3747438f"
4284 +dependencies = [
4285 + "either",
4286 + "itertools",
4287 + "rayon",
4288 +]
4289 +
4290 +[[package]]
4291 +name = "rayon-core"
4292 +version = "1.13.0"
4293 +source = "registry+https://github.com/rust-lang/crates.io-index"
4294 +checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91"
4295 +dependencies = [
4296 + "crossbeam-deque",
4297 + "crossbeam-utils",
4298 +]
4299 +
4300 +[[package]]
4301 +name = "realfft"
4302 +version = "3.5.0"
4303 +source = "registry+https://github.com/rust-lang/crates.io-index"
4304 +checksum = "f821338fddb99d089116342c46e9f1fbf3828dba077674613e734e01d6ea8677"
4305 +dependencies = [
4306 + "rustfft",
4307 +]
4308 +
4309 +[[package]]
4310 +name = "reborrow"
4311 +version = "0.5.5"
4312 +source = "registry+https://github.com/rust-lang/crates.io-index"
4313 +checksum = "03251193000f4bd3b042892be858ee50e8b3719f2b08e5833ac4353724632430"
4314 +
4315 +[[package]]
4316 +name = "recvmsg"
4317 +version = "1.0.0"
4318 +source = "registry+https://github.com/rust-lang/crates.io-index"
4319 +checksum = "d3edd4d5d42c92f0a659926464d4cce56b562761267ecf0f469d85b7de384175"
4320 +
4321 +[[package]]
4322 +name = "redox_syscall"
4323 +version = "0.5.18"
4324 +source = "registry+https://github.com/rust-lang/crates.io-index"
4325 +checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
4326 +dependencies = [
4327 + "bitflags 2.11.0",
4328 +]
4329 +
4330 +[[package]]
4331 +name = "redox_users"
4332 +version = "0.5.2"
4333 +source = "registry+https://github.com/rust-lang/crates.io-index"
4334 +checksum = "a4e608c6638b9c18977b00b475ac1f28d14e84b27d8d42f70e0bf1e3dec127ac"
4335 +dependencies = [
4336 + "getrandom 0.2.17",
4337 + "libredox",
4338 + "thiserror 2.0.18",
4339 +]
4340 +
4341 +[[package]]
4342 +name = "ref-cast"
4343 +version = "1.0.25"
4344 +source = "registry+https://github.com/rust-lang/crates.io-index"
4345 +checksum = "f354300ae66f76f1c85c5f84693f0ce81d747e2c3f21a45fef496d89c960bf7d"
4346 +dependencies = [
4347 + "ref-cast-impl",
4348 +]
4349 +
4350 +[[package]]
4351 +name = "ref-cast-impl"
4352 +version = "1.0.25"
4353 +source = "registry+https://github.com/rust-lang/crates.io-index"
4354 +checksum = "b7186006dcb21920990093f30e3dea63b7d6e977bf1256be20c3563a5db070da"
4355 +dependencies = [
4356 + "proc-macro2",
4357 + "quote",
4358 + "syn 2.0.117",
4359 +]
4360 +
4361 +[[package]]
4362 +name = "regex"
4363 +version = "1.12.3"
4364 +source = "registry+https://github.com/rust-lang/crates.io-index"
4365 +checksum = "e10754a14b9137dd7b1e3e5b0493cc9171fdd105e0ab477f51b72e7f3ac0e276"
4366 +dependencies = [
4367 + "aho-corasick",
4368 + "memchr",
4369 + "regex-automata",
4370 + "regex-syntax",
4371 +]
4372 +
4373 +[[package]]
4374 +name = "regex-automata"
4375 +version = "0.4.14"
4376 +source = "registry+https://github.com/rust-lang/crates.io-index"
4377 +checksum = "6e1dd4122fc1595e8162618945476892eefca7b88c52820e74af6262213cae8f"
4378 +dependencies = [
4379 + "aho-corasick",
4380 + "memchr",
4381 + "regex-syntax",
4382 +]
4383 +
4384 +[[package]]
4385 +name = "regex-syntax"
4386 +version = "0.8.10"
4387 +source = "registry+https://github.com/rust-lang/crates.io-index"
4388 +checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a"
4389 +
4390 +[[package]]
4391 +name = "reqwest"
4392 +version = "0.12.28"
4393 +source = "registry+https://github.com/rust-lang/crates.io-index"
4394 +checksum = "eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147"
4395 +dependencies = [
4396 + "base64 0.22.1",
4397 + "bytes",
4398 + "encoding_rs",
4399 + "futures-channel",
4400 + "futures-core",
4401 + "futures-util",
4402 + "h2",
4403 + "http",
4404 + "http-body",
4405 + "http-body-util",
4406 + "hyper",
4407 + "hyper-rustls",
4408 + "hyper-tls",
4409 + "hyper-util",
4410 + "js-sys",
4411 + "log",
4412 + "mime",
4413 + "mime_guess",
4414 + "native-tls",
4415 + "percent-encoding",
4416 + "pin-project-lite",
4417 + "quinn",
4418 + "rustls",
4419 + "rustls-pki-types",
4420 + "serde",
4421 + "serde_json",
4422 + "serde_urlencoded",
4423 + "sync_wrapper",
4424 + "tokio",
4425 + "tokio-native-tls",
4426 + "tokio-rustls",
4427 + "tokio-util",
4428 + "tower",
4429 + "tower-http",
4430 + "tower-service",
4431 + "url",
4432 + "wasm-bindgen",
4433 + "wasm-bindgen-futures",
4434 + "wasm-streams 0.4.2",
4435 + "web-sys",
4436 + "webpki-roots 1.0.6",
4437 +]
4438 +
4439 +[[package]]
4440 +name = "reqwest"
4441 +version = "0.13.2"
4442 +source = "registry+https://github.com/rust-lang/crates.io-index"
4443 +checksum = "ab3f43e3283ab1488b624b44b0e988d0acea0b3214e694730a055cb6b2efa801"
4444 +dependencies = [
4445 + "base64 0.22.1",
4446 + "bytes",
4447 + "encoding_rs",
4448 + "futures-channel",
4449 + "futures-core",
4450 + "futures-util",
4451 + "h2",
4452 + "http",
4453 + "http-body",
4454 + "http-body-util",
4455 + "hyper",
4456 + "hyper-rustls",
4457 + "hyper-util",
4458 + "js-sys",
4459 + "log",
4460 + "mime",
4461 + "percent-encoding",
4462 + "pin-project-lite",
4463 + "quinn",
4464 + "rustls",
4465 + "rustls-pki-types",
4466 + "rustls-platform-verifier",
4467 + "serde",
4468 + "serde_json",
4469 + "sync_wrapper",
4470 + "tokio",
4471 + "tokio-rustls",
4472 + "tokio-util",
4473 + "tower",
4474 + "tower-http",
4475 + "tower-service",
4476 + "url",
4477 + "wasm-bindgen",
4478 + "wasm-bindgen-futures",
4479 + "wasm-streams 0.5.0",
4480 + "web-sys",
4481 +]
4482 +
4483 +[[package]]
4484 +name = "rgb"
4485 +version = "0.8.53"
4486 +source = "registry+https://github.com/rust-lang/crates.io-index"
4487 +checksum = "47b34b781b31e5d73e9fbc8689c70551fd1ade9a19e3e28cfec8580a79290cc4"
4488 +
4489 +[[package]]
4490 +name = "ring"
4491 +version = "0.17.14"
4492 +source = "registry+https://github.com/rust-lang/crates.io-index"
4493 +checksum = "a4689e6c2294d81e88dc6261c768b63bc4fcdb852be6d1352498b114f61383b7"
4494 +dependencies = [
4495 + "cc",
4496 + "cfg-if",
4497 + "getrandom 0.2.17",
4498 + "libc",
4499 + "untrusted",
4500 + "windows-sys 0.52.0",
4501 +]
4502 +
4503 +[[package]]
4504 +name = "rubato"
4505 +version = "0.16.2"
4506 +source = "registry+https://github.com/rust-lang/crates.io-index"
4507 +checksum = "5258099699851cfd0082aeb645feb9c084d9a5e1f1b8d5372086b989fc5e56a1"
4508 +dependencies = [
4509 + "num-complex",
4510 + "num-integer",
4511 + "num-traits",
4512 + "realfft",
4513 +]
4514 +
4515 +[[package]]
4516 +name = "rust-mcp-schema"
4517 +version = "0.9.6"
4518 +source = "registry+https://github.com/rust-lang/crates.io-index"
4519 +checksum = "b1209aa7fb74fccafe6b2a649b15581fb328343427d85def865b0ad233fe1f74"
4520 +dependencies = [
4521 + "serde",
4522 + "serde_json",
4523 +]
4524 +
4525 +[[package]]
4526 +name = "rust-stemmers"
4527 +version = "1.2.0"
4528 +source = "registry+https://github.com/rust-lang/crates.io-index"
4529 +checksum = "e46a2036019fdb888131db7a4c847a1063a7493f971ed94ea82c67eada63ca54"
4530 +dependencies = [
4531 + "serde",
4532 + "serde_derive",
4533 +]
4534 +
4535 +[[package]]
4536 +name = "rustc-hash"
4537 +version = "1.1.0"
4538 +source = "registry+https://github.com/rust-lang/crates.io-index"
4539 +checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2"
4540 +
4541 +[[package]]
4542 +name = "rustc-hash"
4543 +version = "2.1.1"
4544 +source = "registry+https://github.com/rust-lang/crates.io-index"
4545 +checksum = "357703d41365b4b27c590e3ed91eabb1b663f07c4c084095e60cbed4362dff0d"
4546 +
4547 +[[package]]
4548 +name = "rustc_version"
4549 +version = "0.4.1"
4550 +source = "registry+https://github.com/rust-lang/crates.io-index"
4551 +checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92"
4552 +dependencies = [
4553 + "semver",
4554 +]
4555 +
4556 +[[package]]
4557 +name = "rustfft"
4558 +version = "6.4.1"
4559 +source = "registry+https://github.com/rust-lang/crates.io-index"
4560 +checksum = "21db5f9893e91f41798c88680037dba611ca6674703c1a18601b01a72c8adb89"
4561 +dependencies = [
4562 + "num-complex",
4563 + "num-integer",
4564 + "num-traits",
4565 + "primal-check",
4566 + "strength_reduce",
4567 + "transpose",
4568 +]
4569 +
4570 +[[package]]
4571 +name = "rustix"
4572 +version = "1.1.4"
4573 +source = "registry+https://github.com/rust-lang/crates.io-index"
4574 +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
4575 +dependencies = [
4576 + "bitflags 2.11.0",
4577 + "errno",
4578 + "libc",
4579 + "linux-raw-sys",
4580 + "windows-sys 0.61.2",
4581 +]
4582 +
4583 +[[package]]
4584 +name = "rustls"
4585 +version = "0.23.37"
4586 +source = "registry+https://github.com/rust-lang/crates.io-index"
4587 +checksum = "758025cb5fccfd3bc2fd74708fd4682be41d99e5dff73c377c0646c6012c73a4"
4588 +dependencies = [
4589 + "aws-lc-rs",
4590 + "log",
4591 + "once_cell",
4592 + "ring",
4593 + "rustls-pki-types",
4594 + "rustls-webpki",
4595 + "subtle",
4596 + "zeroize",
4597 +]
4598 +
4599 +[[package]]
4600 +name = "rustls-native-certs"
4601 +version = "0.8.3"
4602 +source = "registry+https://github.com/rust-lang/crates.io-index"
4603 +checksum = "612460d5f7bea540c490b2b6395d8e34a953e52b491accd6c86c8164c5932a63"
4604 +dependencies = [
4605 + "openssl-probe",
4606 + "rustls-pki-types",
4607 + "schannel",
4608 + "security-framework",
4609 +]
4610 +
4611 +[[package]]
4612 +name = "rustls-pki-types"
4613 +version = "1.14.0"
4614 +source = "registry+https://github.com/rust-lang/crates.io-index"
4615 +checksum = "be040f8b0a225e40375822a563fa9524378b9d63112f53e19ffff34df5d33fdd"
4616 +dependencies = [
4617 + "web-time",
4618 + "zeroize",
4619 +]
4620 +
4621 +[[package]]
4622 +name = "rustls-platform-verifier"
4623 +version = "0.6.2"
4624 +source = "registry+https://github.com/rust-lang/crates.io-index"
4625 +checksum = "1d99feebc72bae7ab76ba994bb5e121b8d83d910ca40b36e0921f53becc41784"
4626 +dependencies = [
4627 + "core-foundation 0.10.1",
4628 + "core-foundation-sys",
4629 + "jni",
4630 + "log",
4631 + "once_cell",
4632 + "rustls",
4633 + "rustls-native-certs",
4634 + "rustls-platform-verifier-android",
4635 + "rustls-webpki",
4636 + "security-framework",
4637 + "security-framework-sys",
4638 + "webpki-root-certs",
4639 + "windows-sys 0.61.2",
4640 +]
4641 +
4642 +[[package]]
4643 +name = "rustls-platform-verifier-android"
4644 +version = "0.1.1"
4645 +source = "registry+https://github.com/rust-lang/crates.io-index"
4646 +checksum = "f87165f0995f63a9fbeea62b64d10b4d9d8e78ec6d7d51fb2125fda7bb36788f"
4647 +
4648 +[[package]]
4649 +name = "rustls-webpki"
4650 +version = "0.103.10"
4651 +source = "registry+https://github.com/rust-lang/crates.io-index"
4652 +checksum = "df33b2b81ac578cabaf06b89b0631153a3f416b0a886e8a7a1707fb51abbd1ef"
4653 +dependencies = [
4654 + "aws-lc-rs",
4655 + "ring",
4656 + "rustls-pki-types",
4657 + "untrusted",
4658 +]
4659 +
4660 +[[package]]
4661 +name = "rustversion"
4662 +version = "1.0.22"
4663 +source = "registry+https://github.com/rust-lang/crates.io-index"
4664 +checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
4665 +
4666 +[[package]]
4667 +name = "ryu"
4668 +version = "1.0.23"
4669 +source = "registry+https://github.com/rust-lang/crates.io-index"
4670 +checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f"
4671 +
4672 +[[package]]
4673 +name = "safe_arch"
4674 +version = "0.7.4"
4675 +source = "registry+https://github.com/rust-lang/crates.io-index"
4676 +checksum = "96b02de82ddbe1b636e6170c21be622223aea188ef2e139be0a5b219ec215323"
4677 +dependencies = [
4678 + "bytemuck",
4679 +]
4680 +
4681 +[[package]]
4682 +name = "safetensors"
4683 +version = "0.4.5"
4684 +source = "registry+https://github.com/rust-lang/crates.io-index"
4685 +checksum = "44560c11236a6130a46ce36c836a62936dc81ebf8c36a37947423571be0e55b6"
4686 +dependencies = [
4687 + "serde",
4688 + "serde_json",
4689 +]
4690 +
4691 +[[package]]
4692 +name = "safetensors"
4693 +version = "0.7.0"
4694 +source = "registry+https://github.com/rust-lang/crates.io-index"
4695 +checksum = "675656c1eabb620b921efea4f9199f97fc86e36dd6ffd1fbbe48d0f59a4987f5"
4696 +dependencies = [
4697 + "hashbrown 0.16.1",
4698 + "serde",
4699 + "serde_json",
4700 +]
4701 +
4702 +[[package]]
4703 +name = "same-file"
4704 +version = "1.0.6"
4705 +source = "registry+https://github.com/rust-lang/crates.io-index"
4706 +checksum = "93fc1dc3aaa9bfed95e02e6eadabb4baf7e3078b0bd1b4d7b6b0b68378900502"
4707 +dependencies = [
4708 + "winapi-util",
4709 +]
4710 +
4711 +[[package]]
4712 +name = "saphyr-parser-bw"
4713 +version = "0.0.605"
4714 +source = "registry+https://github.com/rust-lang/crates.io-index"
4715 +checksum = "5e1aee7486406df3541b5a657204a11be97175a467d77bc98e6d94a66289fb80"
4716 +dependencies = [
4717 + "arraydeque",
4718 + "smallvec 2.0.0-alpha.12",
4719 + "thiserror 2.0.18",
4720 +]
4721 +
4722 +[[package]]
4723 +name = "schannel"
4724 +version = "0.1.29"
4725 +source = "registry+https://github.com/rust-lang/crates.io-index"
4726 +checksum = "91c1b7e4904c873ef0710c1f407dde2e6287de2bebc1bbbf7d430bb7cbffd939"
4727 +dependencies = [
4728 + "windows-sys 0.61.2",
4729 +]
4730 +
4731 +[[package]]
4732 +name = "schemars"
4733 +version = "0.9.0"
4734 +source = "registry+https://github.com/rust-lang/crates.io-index"
4735 +checksum = "4cd191f9397d57d581cddd31014772520aa448f65ef991055d7f61582c65165f"
4736 +dependencies = [
4737 + "dyn-clone",
4738 + "ref-cast",
4739 + "serde",
4740 + "serde_json",
4741 +]
4742 +
4743 +[[package]]
4744 +name = "schemars"
4745 +version = "1.2.1"
4746 +source = "registry+https://github.com/rust-lang/crates.io-index"
4747 +checksum = "a2b42f36aa1cd011945615b92222f6bf73c599a102a300334cd7f8dbeec726cc"
4748 +dependencies = [
4749 + "dyn-clone",
4750 + "ref-cast",
4751 + "schemars_derive",
4752 + "serde",
4753 + "serde_json",
4754 +]
4755 +
4756 +[[package]]
4757 +name = "schemars_derive"
4758 +version = "1.2.1"
4759 +source = "registry+https://github.com/rust-lang/crates.io-index"
4760 +checksum = "7d115b50f4aaeea07e79c1912f645c7513d81715d0420f8bc77a18c6260b307f"
4761 +dependencies = [
4762 + "proc-macro2",
4763 + "quote",
4764 + "serde_derive_internals",
4765 + "syn 2.0.117",
4766 +]
4767 +
4768 +[[package]]
4769 +name = "scoped-tls"
4770 +version = "1.0.1"
4771 +source = "registry+https://github.com/rust-lang/crates.io-index"
4772 +checksum = "e1cf6437eb19a8f4a6cc0f7dca544973b0b78843adbfeb3683d1a94a0024a294"
4773 +
4774 +[[package]]
4775 +name = "scopeguard"
4776 +version = "1.2.0"
4777 +source = "registry+https://github.com/rust-lang/crates.io-index"
4778 +checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
4779 +
4780 +[[package]]
4781 +name = "scraper"
4782 +version = "0.25.0"
4783 +source = "registry+https://github.com/rust-lang/crates.io-index"
4784 +checksum = "93cecd86d6259499c844440546d02f55f3e17bd286e529e48d1f9f67e92315cb"
4785 +dependencies = [
4786 + "cssparser",
4787 + "ego-tree",
4788 + "getopts",
4789 + "html5ever 0.36.1",
4790 + "precomputed-hash",
4791 + "selectors",
4792 + "tendril 0.4.3",
4793 +]
4794 +
4795 +[[package]]
4796 +name = "scroll"
4797 +version = "0.12.0"
4798 +source = "registry+https://github.com/rust-lang/crates.io-index"
4799 +checksum = "6ab8598aa408498679922eff7fa985c25d58a90771bd6be794434c5277eab1a6"
4800 +dependencies = [
4801 + "scroll_derive",
4802 +]
4803 +
4804 +[[package]]
4805 +name = "scroll_derive"
4806 +version = "0.12.1"
4807 +source = "registry+https://github.com/rust-lang/crates.io-index"
4808 +checksum = "1783eabc414609e28a5ba76aee5ddd52199f7107a0b24c2e9746a1ecc34a683d"
4809 +dependencies = [
4810 + "proc-macro2",
4811 + "quote",
4812 + "syn 2.0.117",
4813 +]
4814 +
4815 +[[package]]
4816 +name = "security-framework"
4817 +version = "3.7.0"
4818 +source = "registry+https://github.com/rust-lang/crates.io-index"
4819 +checksum = "b7f4bc775c73d9a02cde8bf7b2ec4c9d12743edf609006c7facc23998404cd1d"
4820 +dependencies = [
4821 + "bitflags 2.11.0",
4822 + "core-foundation 0.10.1",
4823 + "core-foundation-sys",
4824 + "libc",
4825 + "security-framework-sys",
4826 +]
4827 +
4828 +[[package]]
4829 +name = "security-framework-sys"
4830 +version = "2.17.0"
4831 +source = "registry+https://github.com/rust-lang/crates.io-index"
4832 +checksum = "6ce2691df843ecc5d231c0b14ece2acc3efb62c0a398c7e1d875f3983ce020e3"
4833 +dependencies = [
4834 + "core-foundation-sys",
4835 + "libc",
4836 +]
4837 +
4838 +[[package]]
4839 +name = "selectors"
4840 +version = "0.33.0"
4841 +source = "registry+https://github.com/rust-lang/crates.io-index"
4842 +checksum = "feef350c36147532e1b79ea5c1f3791373e61cbd9a6a2615413b3807bb164fb7"
4843 +dependencies = [
4844 + "bitflags 2.11.0",
4845 + "cssparser",
4846 + "derive_more",
4847 + "log",
4848 + "new_debug_unreachable",
4849 + "phf",
4850 + "phf_codegen",
4851 + "precomputed-hash",
4852 + "rustc-hash 2.1.1",
4853 + "servo_arc",
4854 + "smallvec 1.15.1",
4855 +]
4856 +
4857 +[[package]]
4858 +name = "semver"
4859 +version = "1.0.27"
4860 +source = "registry+https://github.com/rust-lang/crates.io-index"
4861 +checksum = "d767eb0aabc880b29956c35734170f26ed551a859dbd361d140cdbeca61ab1e2"
4862 +dependencies = [
4863 + "serde",
4864 + "serde_core",
4865 +]
4866 +
4867 +[[package]]
4868 +name = "seq-macro"
4869 +version = "0.3.6"
4870 +source = "registry+https://github.com/rust-lang/crates.io-index"
4871 +checksum = "1bc711410fbe7399f390ca1c3b60ad0f53f80e95c5eb935e52268a0e2cd49acc"
4872 +
4873 +[[package]]
4874 +name = "serde"
4875 +version = "1.0.228"
4876 +source = "registry+https://github.com/rust-lang/crates.io-index"
4877 +checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
4878 +dependencies = [
4879 + "serde_core",
4880 + "serde_derive",
4881 +]
4882 +
4883 +[[package]]
4884 +name = "serde-big-array"
4885 +version = "0.5.1"
4886 +source = "registry+https://github.com/rust-lang/crates.io-index"
4887 +checksum = "11fc7cc2c76d73e0f27ee52abbd64eec84d46f370c88371120433196934e4b7f"
4888 +dependencies = [
4889 + "serde",
4890 +]
4891 +
4892 +[[package]]
4893 +name = "serde-saphyr"
4894 +version = "0.0.16"
4895 +source = "registry+https://github.com/rust-lang/crates.io-index"
4896 +checksum = "4db5a4a562bcc0017c34cd0efbabf447be783f00576a2a516947f884e6a7ed57"
4897 +dependencies = [
4898 + "ahash",
4899 + "annotate-snippets",
4900 + "base64 0.22.1",
4901 + "encoding_rs_io",
4902 + "nohash-hasher",
4903 + "num-traits",
4904 + "regex",
4905 + "saphyr-parser-bw",
4906 + "serde",
4907 + "serde_json",
4908 + "smallvec 2.0.0-alpha.12",
4909 + "zmij",
4910 +]
4911 +
4912 +[[package]]
4913 +name = "serde_core"
4914 +version = "1.0.228"
4915 +source = "registry+https://github.com/rust-lang/crates.io-index"
4916 +checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
4917 +dependencies = [
4918 + "serde_derive",
4919 +]
4920 +
4921 +[[package]]
4922 +name = "serde_derive"
4923 +version = "1.0.228"
4924 +source = "registry+https://github.com/rust-lang/crates.io-index"
4925 +checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
4926 +dependencies = [
4927 + "proc-macro2",
4928 + "quote",
4929 + "syn 2.0.117",
4930 +]
4931 +
4932 +[[package]]
4933 +name = "serde_derive_internals"
4934 +version = "0.29.1"
4935 +source = "registry+https://github.com/rust-lang/crates.io-index"
4936 +checksum = "18d26a20a969b9e3fdf2fc2d9f21eda6c40e2de84c9408bb5d3b05d499aae711"
4937 +dependencies = [
4938 + "proc-macro2",
4939 + "quote",
4940 + "syn 2.0.117",
4941 +]
4942 +
4943 +[[package]]
4944 +name = "serde_json"
4945 +version = "1.0.149"
4946 +source = "registry+https://github.com/rust-lang/crates.io-index"
4947 +checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86"
4948 +dependencies = [
4949 + "indexmap 2.13.0",
4950 + "itoa",
4951 + "memchr",
4952 + "serde",
4953 + "serde_core",
4954 + "zmij",
4955 +]
4956 +
4957 +[[package]]
4958 +name = "serde_plain"
4959 +version = "1.0.2"
4960 +source = "registry+https://github.com/rust-lang/crates.io-index"
4961 +checksum = "9ce1fc6db65a611022b23a0dec6975d63fb80a302cb3388835ff02c097258d50"
4962 +dependencies = [
4963 + "serde",
4964 +]
4965 +
4966 +[[package]]
4967 +name = "serde_spanned"
4968 +version = "1.0.4"
4969 +source = "registry+https://github.com/rust-lang/crates.io-index"
4970 +checksum = "f8bbf91e5a4d6315eee45e704372590b30e260ee83af6639d64557f51b067776"
4971 +dependencies = [
4972 + "serde_core",
4973 +]
4974 +
4975 +[[package]]
4976 +name = "serde_urlencoded"
4977 +version = "0.7.1"
4978 +source = "registry+https://github.com/rust-lang/crates.io-index"
4979 +checksum = "d3491c14715ca2294c4d6a88f15e84739788c1d030eed8c110436aafdaa2f3fd"
4980 +dependencies = [
4981 + "form_urlencoded",
4982 + "itoa",
4983 + "ryu",
4984 + "serde",
4985 +]
4986 +
4987 +[[package]]
4988 +name = "serde_with"
4989 +version = "3.18.0"
4990 +source = "registry+https://github.com/rust-lang/crates.io-index"
4991 +checksum = "dd5414fad8e6907dbdd5bc441a50ae8d6e26151a03b1de04d89a5576de61d01f"
4992 +dependencies = [
4993 + "base64 0.22.1",
4994 + "chrono",
4995 + "hex",
4996 + "indexmap 1.9.3",
4997 + "indexmap 2.13.0",
4998 + "schemars 0.9.0",
4999 + "schemars 1.2.1",

This file is too large to show in full.

Cargo.toml new
+70
@@ -0,0 +1,70 @@
1 +[package]
2 +name = "onde"
3 +version = "0.1.0"
4 +edition = "2021"
5 +description = "On-device chat inference for cross-platform apps with automatic model management."
6 +license = "MIT"
7 +repository = "https://github.com/ondeinference/onde"
8 +homepage = "https://ondeinference.com"
9 +documentation = "https://docs.rs/onde"
10 +readme = "README.md"
11 +keywords = ["inference", "on-device", "chat", "llm", "mistral"]
12 +categories = ["science::ml", "api-bindings"]
13 +
14 +[lib]
15 +name = "onde"
16 +# "lib" – normal Rust dependency for other crates.
17 +# "cdylib" – shared library (.dylib / .so / .dll) consumed by UniFFI bindgen
18 +# to generate Swift / Kotlin / Python bindings.
19 +# "staticlib" – static library (.a) required for iOS targets where dynamic
20 +# linking is not allowed.
21 +crate-type = ["lib", "cdylib", "staticlib"]
22 +
23 +[dependencies]
24 +anyhow = "1"
25 +home = "0.5.4"
26 +log = "^0.4"
27 +serde = { version = "1.0", features = ["derive"] }
28 +tokio = { version = "1", features = ["macros", "rt", "sync"] }
29 +tsync = "2"
30 +
31 +thiserror = "2"
32 +
33 +# Pinned to =0.31.0 — the versions MUST be identical across the crate and
34 +# the uniffi-bindgen binary crate.
35 +uniffi = { version = "=0.31.0", features = ["tokio"] }
36 +
37 +[build-dependencies]
38 +# Required for UniFFI scaffolding code generation at build time.
39 +uniffi = { version = "=0.31.0", features = ["build"] }
40 +# Used by build.rs to compile the tvOS ___chkstk_darwin assembly stub.
41 +cc = "1"
42 +
43 +# Android: CPU-only GGUF inference via candle; also needs hf-hub for explicit
44 +# cache path seeding (dirs::home_dir() panics in the Android sandbox).
45 +[target.'cfg(target_os = "android")'.dependencies]
46 +hf-hub = { version = "0.4", features = ["tokio"] }
47 +mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = [] }
48 +mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = [] }
49 +
50 +# iOS: Metal-accelerated inference.
51 +[target.'cfg(target_os = "ios")'.dependencies]
52 +mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
53 +
54 +# tvOS: Metal-accelerated inference (Apple TV has Apple GPU).
55 +[target.'cfg(target_os = "tvos")'.dependencies]
56 +mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
57 +
58 +# macOS: Metal-accelerated inference.
59 +[target.'cfg(target_os = "macos")'.dependencies]
60 +mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
61 +
62 +# Windows: CPU-only baseline (no GPU features).
63 +# For NVIDIA GPU builds, patch this to features = ["cuda"] in CI before building.
64 +[target.'cfg(target_os = "windows")'.dependencies]
65 +mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes" }
66 +
67 +# Linux: CPU-only baseline (no GPU features).
68 +# For NVIDIA GPU builds, patch this to features = ["cuda"] in CI before building.
69 +[target.'cfg(target_os = "linux")'.dependencies]
70 +mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes" }
README.md
+1 -7
@@ -18,7 +18,7 @@
18
19 ## In production
20
21 -Onde already powers real apps on the App Store across multiple AI workloads.
21 +Onde already powers real apps on the App Store with on-device chat experiences.
22
23 ### Chat completion
24 - [Karokowe](https://apps.apple.com/se/app/karaoke-karokowe/id6756801861)
@@ -27,12 +27,6 @@ Onde already powers real apps on the App Store across multiple AI workloads.
27 - [Splitfire](https://apps.apple.com/se/app/splitfire-bassndrums/id6751143237)
28 - [Rumi – Lär dig persiska](https://apps.apple.com/se/app/rumi-l%C3%A4r-dig-persiska/id6753832408)
29
30 -### Image generation
31 -- [Reimagen](https://apps.apple.com/se/app/reimagen-ai-musik-till-konst/id6757844075)
32 -
33 -### Audio transcription
34 -- [Karokowe](https://apps.apple.com/se/app/karaoke-karokowe/id6756801861)
35 -
30 ### AI coding agent (WIP)
31 - [Sigit](https://apps.apple.com/se/app/sigit-git-client-code-editor/id6753018849)
32
build.rs new
+43
@@ -0,0 +1,43 @@
1 +fn main() {
2 + // ── UniFFI scaffolding ───────────────────────────────────────────────
3 + // Not strictly required when using `uniffi::setup_scaffolding!()` in
4 + // lib.rs (the proc-macro approach), but having it here ensures the
5 + // build-dep on `uniffi` with `features = ["build"]` is exercised and
6 + // keeps compatibility if we ever switch to UDL-based generation.
7 +
8 + // ── tvOS: provide missing ___chkstk_darwin symbol ────────────────────
9 + //
10 + // On macOS and iOS, `___chkstk_darwin` is exported by libSystem as a
11 + // stack probing function. On arm64, the kernel grows the stack via
12 + // guard pages so the probe is effectively a no-op. tvOS does NOT
13 + // export this symbol, but `aws-lc-sys` assembly (pulled in transitively
14 + // via reqwest → rustls → aws-lc-rs) references it unconditionally.
15 + //
16 + // We compile a tiny assembly stub that provides the symbol as a `ret`
17 + // instruction — only when targeting tvOS.
18 + let target_os = std::env::var("CARGO_CFG_TARGET_OS").unwrap_or_default();
19 +
20 + if target_os == "tvos" {
21 + let manifest_dir = std::env::var("CARGO_MANIFEST_DIR").expect("CARGO_MANIFEST_DIR not set");
22 + let asm_path = std::path::Path::new(&manifest_dir).join("tvos_chkstk.s");
23 +
24 + if asm_path.exists() {
25 + // Use the `cc` crate to assemble the stub into a static library
26 + // that the linker will pick up. If `cc` is not available as a
27 + // build-dependency, fall back to printing a cargo link directive
28 + // that tells the linker to include the object directly.
29 + //
30 + // Since `cc` is already a transitive build-dep (pulled in by
31 + // aws-lc-sys, ring, etc.), we use it directly.
32 + cc::Build::new().file(&asm_path).compile("tvos_chkstk");
33 +
34 + println!("cargo:rerun-if-changed=tvos_chkstk.s");
35 + } else {
36 + println!(
37 + "cargo:warning=tvos_chkstk.s not found at {} — \
38 + ___chkstk_darwin will be unresolved on tvOS",
39 + asm_path.display()
40 + );
41 + }
42 + }
43 +}
docs/README.md new
+72
@@ -0,0 +1,72 @@
1 +# Onde Docs
2 +
3 +Source for the [Onde documentation site](https://ondeinference.com/docs), built with [Mintlify](https://mintlify.com).
4 +
5 +## Structure
6 +
7 +```
8 +docs/
9 +├── mint.json # Mintlify site config — navigation, colors, logo
10 +├── index.md # Landing page → /docs
11 +├── dev.md # Developer guide → /docs/dev
12 +├── swift-package.md # Swift Package guide → /docs/swift-package
13 +├── ruby-gem.md # Ruby Gem guide → /docs/ruby-gem
14 +└── distribution.md # Publishing guide → /docs/distribution
15 +```
16 +
17 +## Local Preview
18 +
19 +Install the Mintlify CLI:
20 +
21 +```bash
22 +npm install -g mintlify
23 +```
24 +
25 +Start the dev server from the `docs/` directory:
26 +
27 +```bash
28 +cd docs
29 +mintlify dev
30 +```
31 +
32 +The site is served at `http://localhost:3000`.
33 +
34 +## Deployment
35 +
36 +The docs deploy automatically via the [Mintlify GitHub App](https://github.com/apps/mintlify).
37 +
38 +1. Install the Mintlify GitHub App on the `ondeinference/onde` repository.
39 +2. In the Mintlify dashboard, set the docs directory to `docs/` and the base path to `/docs`.
40 +3. Every push to `main` triggers a redeploy.
41 +
42 +The custom domain is configured in the Mintlify dashboard:
43 +
44 +- Domain: `ondeinference.com`
45 +- Path: `/docs`
46 +
47 +## Adding a Page
48 +
49 +1. Create a new `.md` file in `docs/`.
50 +2. Add a YAML frontmatter block at the top:
51 + ```yaml
52 + ---
53 + title: "Page Title"
54 + description: "One-sentence description shown in search and link previews."
55 + ---
56 + ```
57 +3. Add the filename (without `.md`) to the appropriate group in `mint.json` under `navigation`.
58 +
59 +## Frontmatter Fields
60 +
61 +| Field | Required | Description |
62 +| ----- | -------- | ----------- |
63 +| `title` | Yes | Page title shown in the browser tab and sidebar |
64 +| `description` | Yes | Meta description for search and link previews |
65 +| `sidebarTitle` | No | Override the sidebar label (defaults to `title`) |
66 +
67 +## Content Guidelines
68 +
69 +- Write for developers who are evaluating or integrating Onde — assume Rust familiarity, not deep familiarity with the project.
70 +- Code blocks must specify a language for syntax highlighting (` ```swift `, ` ```rust `, ` ```bash `, etc.).
71 +- Cross-links between pages omit the `.md` extension: `[Swift Package](swift-package)`, not `[Swift Package](swift-package.md)`.
72 +- Keep internal/contributor notes (build internals, pitfalls, architecture deep-dives) in the `.agent-skills/` directory, not in `docs/`.
\ No newline at end of file
docs/dev.md new
+209
@@ -0,0 +1,209 @@
1 +---
2 +title: "Developer Guide"
3 +description: "Build Onde from source, understand the architecture, and integrate it as a Rust dependency."
4 +sidebarTitle: "Developer Guide"
5 +---
6 +
7 +# Onde — Developer Guide
8 +
9 +> **See also:** [Swift Package](swift-package) · [Ruby Gem](ruby-gem) · [Distribution](distribution)
10 +
11 +## Prerequisites
12 +
13 +```bash
14 +# Rust stable
15 +rustup toolchain install stable
16 +
17 +# Rust nightly (only needed for tvOS tier-3 targets with -Z build-std)
18 +rustup toolchain install nightly
19 +rustup component add rust-src --toolchain nightly
20 +
21 +# iOS targets (stable)
22 +rustup target add aarch64-apple-ios aarch64-apple-ios-sim
23 +```
24 +
25 +## Building
26 +
27 +### Native (macOS — Metal accelerated)
28 +
29 +```bash
30 +# Check compilation
31 +cargo check
32 +
33 +# Run all tests
34 +cargo test
35 +
36 +# Clippy lint
37 +cargo clippy
38 +```
39 +
40 +### Ruby Gem (magnus)
41 +
42 +Build the Ruby native extension that exposes Onde's HuggingFace cache management and model metadata to Ruby via [magnus](https://github.com/matsadler/magnus):
43 +
44 +```bash
45 +cd sdk/gem
46 +
47 +# Install Ruby dependencies + compile the Rust native extension
48 +bin/setup
49 +
50 +# Or step by step:
51 +bundle install
52 +bundle exec rake compile
53 +
54 +# Interactive console
55 +bin/console
56 +```
57 +
58 +Verify it works:
59 +
60 +```bash
61 +bundle exec ruby -e "require 'onde'; puts Onde::VERSION; puts Onde::SUPPORTED_MODELS"
62 +```
63 +
64 +For full API reference, usage examples, and architecture details, see [ruby-gem](ruby-gem).
65 +
66 +## Architecture
67 +
68 +```
69 +onde/
70 +├── src/
71 +│ ├── lib.rs # Crate root — uniffi::setup_scaffolding!()
72 +│ ├── hf_cache.rs # HuggingFace Hub cache management
73 +│ └── inference/
74 +│ ├── mod.rs # Module exports + re-exports
75 +│ ├── types.rs # UniFFI-annotated shared types
76 +│ ├── engine.rs # ChatEngine (Rust-native API)
77 +│ ├── ffi.rs # OndeChatEngine (UniFFI Object for Swift/Kotlin)
78 +│ ├── models.rs # Model ID constants + metadata
79 +│ └── token.rs # HF token resolution (build-time / cache)
80 +├── sdk/
81 +│ ├── Onde/ # Generated Swift package (git-ignored)
82 +│ ├── gem/ # Ruby gem (magnus-based native extension)
83 +│ ├── kotlin/ # Kotlin bindings
84 +│ └── python/ # Python bindings
85 +├── generated/ # UniFFI-generated headers and sources
86 +├── .cargo/
87 +│ └── config.toml # Target-specific rustflags (fp16)
88 +├── Cargo.toml # Platform-conditional mistralrs deps
89 +├── build.rs # UniFFI scaffolding + tvOS chkstk stub
90 +├── uniffi.toml # UniFFI binding config
91 +└── docs/
92 + ├── dev.md # This file
93 + ├── swift-package.md # Swift package building & API guide
94 + └── ruby-gem.md # Ruby gem API reference & guide
95 +```
96 +
97 +### Layer Diagram
98 +
99 +```
100 +┌──────────────────────────────────────────────────────┐
101 +│ Swift (tvOS / iOS) Rust app / CLI / server │
102 +│ import Onde use onde::inference::* │
103 +└──────────┬───────────────────────────┬───────────────┘
104 + │ UniFFI bindings │ Direct Rust API
105 + ▼ ▼
106 +┌──────────────────────┐ ┌────────────────────────────┐
107 +│ OndeChatEngine │ │ ChatEngine │
108 +│ (ffi.rs — Object) │──│ (engine.rs — native) │
109 +│ FFI-safe methods │ │ impl Into<String>, mpsc │
110 +└──────────┬───────────┘ └────────────┬───────────────┘
111 + │ │
112 + └───────────┬───────────────┘
113 + ▼
114 + ┌───────────────────────┐
115 + │ mistralrs::Model │
116 + │ (GgufModelBuilder) │
117 + │ Metal / CUDA / CPU │
118 + └───────────────────────┘
119 +```
120 +
121 +## Platform Support
122 +
123 +| Platform | `target_os` | GPU Backend | Default Model | Dependency |
124 +| -------- | ----------- | ----------------- | -------------- | -------------------------------- |
125 +| macOS | `macos` | Metal | Qwen 2.5 3B | mistralrs `features = ["metal"]` |
126 +| iOS | `ios` | Metal | Qwen 2.5 1.5B | mistralrs `features = ["metal"]` |
127 +| tvOS | `tvos` | Metal | Qwen 2.5 1.5B | mistralrs `features = ["metal"]` |
128 +| Windows | `windows` | CPU (CUDA via CI) | Qwen 2.5 3B | mistralrs (no features) |
129 +| Linux | `linux` | CPU (CUDA via CI) | Qwen 2.5 3B | mistralrs (no features) |
130 +| Android | `android` | CPU (candle) | Qwen 2.5 1.5B | mistralrs + mistralrs-core |
131 +
132 +## Testing
133 +
134 +```bash
135 +# All tests
136 +cargo test
137 +
138 +# Just inference module tests
139 +cargo test inference::
140 +
141 +# Just FFI wrapper tests
142 +cargo test inference::ffi::
143 +
144 +# Just type tests
145 +cargo test inference::types::
146 +
147 +# Clippy
148 +cargo clippy
149 +```
150 +
151 +## Using Onde as a Rust Dependency
152 +
153 +Add onde as a dependency in your `Cargo.toml`:
154 +
155 +```toml
156 +[dependencies]
157 +onde = { path = "../../onde" } # adjust relative path to your layout
158 +# or from git:
159 +# onde = { git = "https://github.com/ondeinference/onde" }
160 +```
161 +
162 +The Rust API is accessed directly — no UniFFI needed:
163 +
164 +```rust
165 +use onde::mistralrs::{GgufModelBuilder, Model, RequestBuilder, TextMessageRole};
166 +use onde::inference::token::hf_token_source;
167 +
168 +// Load model
169 +let model = GgufModelBuilder::new("bartowski/Qwen2.5-1.5B-Instruct-GGUF", vec!["Qwen2.5-1.5B-Instruct-Q4_K_M.gguf"])
170 + .with_token_source(hf_token_source())
171 + .with_logging()
172 + .build()
173 + .await?;
174 +
175 +// Chat
176 +let request = RequestBuilder::new()
177 + .add_message(TextMessageRole::System, "You are a helpful assistant.")
178 + .add_message(TextMessageRole::User, "Hello!")
179 + .set_sampler_temperature(0.7);
180 +
181 +let response = model.send_chat_request(request).await?;
182 +```
183 +
184 +### Important Setup for Sandboxed Apps (iOS / macOS App Store)
185 +
186 +Your app's setup function must configure `HF_HOME` and `TMPDIR` before any inference code runs. See the [pepakbasajawa](https://github.com/pfrfrfr/pepakbasajawa) or [fatapp](https://github.com/setoelkahfi/fatapp) repos for working examples of the `setup_hf_home()` and `setup_tmpdir()` pattern.
187 +
188 +## Known Issues
189 +
190 +### `___chkstk_darwin` linker error on tvOS
191 +
192 +When building for `aarch64-apple-tvos`, `aws-lc-sys` (a transitive dependency via `rustls` → `reqwest` → `hf-hub`) references `___chkstk_darwin`, a stack probing symbol not available on tvOS. The onde `build.rs` provides a no-op assembly stub (`tvos_chkstk.s`) that resolves this automatically.
193 +
194 +This does not affect macOS, iOS, Windows, Linux, or Android builds.
195 +
196 +### Metal Toolchain must be installed (Xcode 26+)
197 +
198 +On Xcode 26+, the Metal compiler is a separate downloadable component. If you see empty `.metallib` files (92 bytes) or `"Error while loading function: fused_glu_float"` at runtime, run:
199 +
200 +```bash
201 +xcodebuild -downloadComponent MetalToolchain
202 +```
203 +
204 +Then clean and rebuild:
205 +
206 +```bash
207 +cargo clean -p mistralrs-quant
208 +cargo check
209 +```
docs/distribution.md new
+205
@@ -0,0 +1,205 @@
1 +---
2 +title: "Distribution"
3 +description: "How Onde is distributed across package registries — crates.io, Swift Package Index, PyPI, RubyGems, and Maven Central."
4 +---
5 +
6 +# Distribution
7 +
8 +Onde is published to every major package registry under a consistent naming convention:
9 +
10 +| Registry | Package name | Namespace |
11 +| -------- | ------------ | --------- |
12 +| crates.io | `onde` | — |
13 +| Swift Package Index | `onde-swift` | `ondeinference` |
14 +| PyPI | `onde-inference` | — |
15 +| RubyGems | `onde-inference` | — |
16 +| Maven Central | `onde-inference` | `com.ondeinference` |
17 +
18 +**Naming rules:**
19 +- Registries with a namespace → `ondeinference` / `onde`
20 +- Registries without a namespace → `onde-inference` (hyphenated) to avoid conflicts
21 +- Swift follows Apple conventions → PascalCase package name (`Onde`), kebab-case repo slug (`onde-swift`)
22 +- The Rust crate uses the bare `onde` name on crates.io
23 +
24 +---
25 +
26 +## Rust — crates.io
27 +
28 +The core Rust crate is published as [`onde`](https://crates.io/crates/onde).
29 +
30 +```toml
31 +[dependencies]
32 +onde = "0.1"
33 +```
34 +
35 +Publish:
36 +
37 +```bash
38 +cargo publish
39 +```
40 +
41 +---
42 +
43 +## Swift — Swift Package Index
44 +
45 +The Swift package is distributed from a dedicated public repository [`ondeinference/onde-swift`](https://github.com/ondeinference/onde-swift) as a remote binary package. The `OndeFramework.xcframework` is attached as a GitHub Release asset; `Package.swift` references it by URL and SHA-256 checksum.
46 +
47 +Swift Package Index indexes the `onde-swift` repository automatically on every new Git tag.
48 +
49 +### Installation
50 +
51 +In Xcode: **File → Add Package Dependencies** and enter:
52 +
53 +```
54 +https://github.com/ondeinference/onde-swift
55 +```
56 +
57 +Or in `Package.swift`:
58 +
59 +```swift
60 +dependencies: [
61 + .package(url: "https://github.com/ondeinference/onde-swift", from: "0.1.0")
62 +]
63 +```
64 +
65 +### Release process
66 +
67 +1. Build `OndeFramework.xcframework` from Rust source (see [Swift Package](swift-package#building-the-xcframework-locally)).
68 +2. Zip it and compute the checksum:
69 + ```bash
70 + zip -r OndeFramework.xcframework.zip OndeFramework.xcframework
71 + shasum -a 256 OndeFramework.xcframework.zip
72 + ```
73 +3. Create a GitHub Release on the Rust source repo and attach the zip as a release asset.
74 +4. In `onde-swift`, update `Package.swift` with the new `url:` and `checksum:`.
75 +5. Commit, tag the version (`0.1.0`, `0.2.0`, etc.), and push — SPI picks it up automatically.
76 +
77 +---
78 +
79 +## Python — PyPI
80 +
81 +The Python package is published as [`onde-inference`](https://pypi.org/project/onde-inference) and imported as `onde_inference`. It uses [maturin](https://github.com/PyO3/maturin) with `bindings = "uniffi"` to generate Python bindings from the same UniFFI scaffolding used for Swift and Kotlin.
82 +
83 +### Installation
84 +
85 +```bash
86 +pip install onde-inference
87 +```
88 +
89 +### Usage
90 +
91 +```python
92 +import asyncio
93 +from onde_inference import OndeChatEngine, default_model_config
94 +
95 +async def main():
96 + engine = OndeChatEngine()
97 + await engine.load_default_model(
98 + system_prompt="You are a helpful assistant.",
99 + sampling=None,
100 + )
101 + result = await engine.send_message("Hello!")
102 + print(result.text)
103 +
104 +asyncio.run(main())
105 +```
106 +
107 +### Prerequisites
108 +
109 +- Python 3.9+
110 +- `maturin >= 1.7` — `uv tool install maturin` or `pip install maturin`
111 +- `uniffi-bindgen == 0.31.0` — must match the `uniffi` version pinned in `Cargo.toml`
112 +
113 + ```bash
114 + pip install uniffi-bindgen==0.31.0
115 + ```
116 +
117 +### Build
118 +
119 +```bash
120 +cd sdk/python
121 +
122 +# Build a release wheel for the current platform
123 +maturin build --release --out dist
124 +
125 +# Development install into the current venv
126 +maturin develop
127 +```
128 +
129 +### Publish
130 +
131 +```bash
132 +cd sdk/python
133 +maturin publish
134 +```
135 +
136 +To publish wheels for all platforms, run `maturin publish` on each platform in CI and pass `--skip-existing` so subsequent uploads don't fail. Platforms to cover:
137 +
138 +- macOS: `aarch64-apple-darwin`, `x86_64-apple-darwin`
139 +- Linux: `x86_64-unknown-linux-gnu`, `aarch64-unknown-linux-gnu`
140 +- Windows: `x86_64-pc-windows-msvc`
141 +
142 +---
143 +
144 +## Ruby — RubyGems
145 +
146 +The Ruby gem is published as [`onde-inference`](https://rubygems.org/gems/onde-inference) and required as `onde`. It is a [Magnus](https://github.com/matsadler/magnus)-based native extension exposing Onde's HuggingFace cache management and model metadata to Ruby.
147 +
148 +See the [Ruby Gem guide](ruby-gem) for full API reference and Rails integration examples.
149 +
150 +### Installation
151 +
152 +```bash
153 +gem install onde-inference
154 +```
155 +
156 +Or in your `Gemfile`:
157 +
158 +```ruby
159 +gem "onde-inference"
160 +```
161 +
162 +### Publish
163 +
164 +```bash
165 +cd sdk/gem
166 +bundle exec rake compile # build the native extension
167 +gem build onde-inference.gemspec
168 +gem push onde-inference-*.gem
169 +```
170 +
171 +---
172 +
173 +## Kotlin / Android — Maven Central
174 +
175 +The Kotlin Android library is published as `com.ondeinference:onde-inference` on Maven Central.
176 +
177 +### Installation
178 +
179 +In `build.gradle.kts`:
180 +
181 +```kotlin
182 +dependencies {
183 + implementation("com.ondeinference:onde-inference:0.1.0")
184 +}
185 +```
186 +
187 +### Build
188 +
189 +The Android library is built with `build-kotlin.sh` at the crate root. It cross-compiles `libonde.so` for all four Android ABIs and generates Kotlin bindings via `uniffi-bindgen`:
190 +
191 +```bash
192 +# Full release build for all Android ABIs
193 +./build-kotlin.sh
194 +
195 +# arm64 only (faster for development)
196 +./build-kotlin.sh --target aarch64-linux-android
197 +```
198 +
199 +Then assemble and publish the AAR from the Android project:
200 +
201 +```bash
202 +cd sdk/kotlin
203 +./gradlew :onde:assembleRelease
204 +./gradlew publishReleasePublicationToMavenCentralRepository
205 +```
docs/index.md new
+95
@@ -0,0 +1,95 @@
1 +---
2 +title: "Onde"
3 +description: "On-device chat inference for every platform. Run LLMs locally with no cloud, no latency, and no data leaving the device."
4 +---
5 +
6 +# Onde
7 +
8 +**On-device chat inference for every platform.**
9 +
10 +Run LLM chat locally — no cloud, no latency, no data leaving the device. Onde wraps [mistral.rs](https://github.com/EricLBuehler/mistral.rs) with a unified API that handles model discovery, HuggingFace Hub downloads, cache management, and GPU acceleration across every platform.
11 +
12 +## Platforms
13 +
14 +| Platform | GPU Backend | Default Model |
15 +| -------- | ----------- | ------------- |
16 +| macOS | Metal | Qwen 2.5 3B |
17 +| iOS | Metal | Qwen 2.5 1.5B |
18 +| tvOS | Metal | Qwen 2.5 1.5B |
19 +| Android | CPU (candle) | Qwen 2.5 1.5B |
20 +| Windows | CPU / CUDA | Qwen 2.5 3B |
21 +| Linux | CPU / CUDA | Qwen 2.5 3B |
22 +
23 +## SDKs
24 +
25 +| Language | Package | Install |
26 +| -------- | ------- | ------- |
27 +| Rust | [`onde`](https://crates.io/crates/onde) | `onde = "0.1"` in Cargo.toml |
28 +| Swift | [`onde-swift`](https://github.com/ondeinference/onde-swift) | Swift Package Manager |
29 +| Kotlin | [`onde-inference`](https://central.sonatype.com/) | Gradle |
30 +| Python | [`onde-inference`](https://pypi.org/project/onde-inference) | `pip install onde-inference` |
31 +| Ruby | [`onde-inference`](https://rubygems.org/gems/onde-inference) | `gem install onde-inference` |
32 +
33 +## Quick Start
34 +
35 +### Swift (iOS / tvOS / macOS)
36 +
37 +```swift
38 +import Onde
39 +
40 +let engine = OndeChatEngine()
41 +try await engine.loadDefaultModel(systemPrompt: "You are a helpful assistant.", sampling: nil)
42 +
43 +let result = try await engine.sendMessage(message: "Hello!")
44 +print(result.text)
45 +```
46 +
47 +### Rust
48 +
49 +```rust
50 +use onde::inference::{ChatEngine, GgufModelConfig};
51 +
52 +let engine = ChatEngine::new();
53 +engine.load_gguf_model(
54 + GgufModelConfig::platform_default(),
55 + Some("You are a helpful assistant.".into()),
56 + None,
57 +).await?;
58 +
59 +let reply = engine.send_message("Hello!").await?;
60 +println!("{}", reply.text);
61 +```
62 +
63 +### Python
64 +
65 +```python
66 +import asyncio
67 +from onde_inference import OndeChatEngine
68 +
69 +async def main():
70 + engine = OndeChatEngine()
71 + await engine.load_default_model(system_prompt="You are a helpful assistant.", sampling=None)
72 + result = await engine.send_message("Hello!")
73 + print(result.text)
74 +
75 +asyncio.run(main())
76 +```
77 +
78 +## Guides
79 +
80 +- [Developer Guide](dev) — build from source, architecture overview, platform support
81 +- [Swift Package](swift-package) — XCFramework distribution, Swift API reference
82 +- [Ruby Gem](ruby-gem) — Magnus-based native extension, Rails integration
83 +- [Distribution](distribution) — publishing to crates.io, Swift Package Index, PyPI, RubyGems, Maven Central
84 +
85 +## Design Principles
86 +
87 +**Zero framework coupling.** Onde knows nothing about any app framework. Progress updates use plain callbacks — wire them to whatever event system you use.
88 +
89 +**Platform-aware by default.** GPU backend, model size, cache directory — Onde picks the right defaults so you don't configure per-platform.
90 +
91 +**One dependency.** Add `onde` and get on-device chat inference, HuggingFace cache management, and HF token resolution in a single crate.
92 +
93 +## License
94 +
95 +Licensed under either of [MIT](https://opensource.org/licenses/MIT) or [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) at your option.
\ No newline at end of file
docs/mint.json new
+77
@@ -0,0 +1,77 @@
1 +{
2 + "$schema": "https://mintlify.com/schema.json",
3 + "name": "Onde",
4 + "logo": {
5 + "light": "/logo/light.svg",
6 + "dark": "/logo/dark.svg",
7 + "href": "https://ondeinference.com"
8 + },
9 + "favicon": "/favicon.svg",
10 + "colors": {
11 + "primary": "#0F172A",
12 + "light": "#38BDF8",
13 + "dark": "#0F172A"
14 + },
15 + "topbarLinks": [
16 + {
17 + "name": "GitHub",
18 + "url": "https://github.com/ondeinference/onde"
19 + }
20 + ],
21 + "topbarCtaButton": {
22 + "name": "Get Started",
23 + "url": "/docs"
24 + },
25 + "anchors": [
26 + {
27 + "name": "crates.io",
28 + "icon": "box",
29 + "url": "https://crates.io/crates/onde"
30 + },
31 + {
32 + "name": "docs.rs",
33 + "icon": "book",
34 + "url": "https://docs.rs/onde"
35 + },
36 + {
37 + "name": "GitHub",
38 + "icon": "github",
39 + "url": "https://github.com/ondeinference/onde"
40 + }
41 + ],
42 + "navigation": [
43 + {
44 + "group": "Getting Started",
45 + "pages": [
46 + "index",
47 + "dev"
48 + ]
49 + },
50 + {
51 + "group": "SDKs",
52 + "pages": [
53 + "swift-package",
54 + "ruby-gem"
55 + ]
56 + },
57 + {
58 + "group": "Publishing",
59 + "pages": [
60 + "distribution"
61 + ]
62 + }
63 + ],
64 + "footerSocials": {
65 + "github": "https://github.com/ondeinference/onde",
66 + "website": "https://ondeinference.com"
67 + },
68 + "feedback": {
69 + "thumbsRating": true,
70 + "suggestEdit": true,
71 + "raiseIssue": true
72 + },
73 + "modeToggle": {
74 + "default": "dark"
75 + },
76 + "tabs": []
77 +}
docs/ruby-gem.md new
+632
@@ -0,0 +1,632 @@
1 +---
2 +title: "Ruby Gem"
3 +description: "Build and use the onde-inference Ruby gem — a Magnus-based native extension exposing HuggingFace cache management and model metadata to Ruby."
4 +---
5 +
6 +# Onde Ruby Gem
7 +
8 +Ruby bindings for the [Onde](https://ondeinference.com) on-device inference engine, powered by Rust via [magnus](https://github.com/matsadler/magnus).
9 +
10 +Manage HuggingFace model caches, inspect supported models, and access inference configuration — all from Ruby with native Rust performance.
11 +
12 +## Prerequisites
13 +
14 +- **Ruby** >= 3.0
15 +- **Rust** >= 1.92 (installed via [rustup](https://rustup.rs))
16 +- **Bundler** (`gem install bundler`)
17 +- macOS, Linux, or Windows
18 +
19 +The gem compiles a native Rust extension at install time. Rust must be available on your `$PATH`.
20 +
21 +## Setup
22 +
23 +```bash
24 +cd lib/crates/onde/gem
25 +
26 +# Install Ruby dependencies and compile the Rust native extension
27 +bin/setup
28 +
29 +# Or step by step:
30 +bundle install
31 +bundle exec rake compile
32 +```
33 +
34 +### Verify the installation
35 +
36 +```bash
37 +bundle exec ruby -e "require 'onde'; puts Onde::VERSION"
38 +# => 0.1.0
39 +```
40 +
41 +### Interactive console
42 +
43 +```bash
44 +bin/console
45 +```
46 +
47 +Opens an IRB session with the gem pre-loaded. Useful for exploring the API:
48 +
49 +```ruby
50 +irb> Onde::SUPPORTED_MODELS
51 +=> ["bartowski/Qwen2.5-1.5B-Instruct-GGUF", "bartowski/Qwen2.5-3B-Instruct-GGUF"]
52 +
53 +irb> Onde.cache_path
54 +=> "/Users/you/.cache/huggingface/hub"
55 +```
56 +
57 +## Build Commands
58 +
59 +| Command | Description |
60 +|---------|-------------|
61 +| `bundle exec rake compile` | Compile the Rust native extension (release profile) |
62 +| `bundle exec rake compile:dev` | Compile with debug profile (faster builds, slower runtime) |
63 +| `bundle exec rake build` | Build the `.gem` package |
64 +| `bundle exec rake` | Default task — compiles the extension |
65 +| `bin/setup` | One-step: `bundle install` + `rake compile` |
66 +| `bin/console` | IRB with the gem loaded |
67 +
68 +## API Reference
69 +
70 +### Cache Management
71 +
72 +#### `Onde.list_local_models` → Hash
73 +
74 +Scans the local HuggingFace hub cache (`~/.cache/huggingface/hub/`) and returns all downloaded models that the inference engine supports.
75 +
76 +```ruby
77 +response = Onde.list_local_models
78 +
79 +response["cache_path"] # => "/Users/you/.cache/huggingface/hub"
80 +response["total_size_bytes"] # => 24768454656
81 +response["total_size_display"] # => "23.07 GB"
82 +
83 +response["models"].each do |model|
84 + puts "#{model["model_id"]} — #{model["size_display"]}"
85 + puts " Path: #{model["path"]}"
86 + puts " Revisions: #{model["revisions"].join(", ")}"
87 +end
88 +```
89 +
90 +**Return value keys:**
91 +
92 +| Key | Type | Description |
93 +|-----|------|-------------|
94 +| `cache_path` | String | Absolute path that was scanned |
95 +| `total_size_bytes` | Integer | Total size of all cached models |
96 +| `total_size_display` | String | Human-readable total size (e.g. `"23.07 GB"`) |
97 +| `models` | Array | Array of model Hashes (see below) |
98 +
99 +**Each model Hash:**
100 +
101 +| Key | Type | Description |
102 +|-----|------|-------------|
103 +| `model_id` | String | Full HuggingFace ID (e.g. `"bartowski/Qwen2.5-1.5B-Instruct-GGUF"`) |
104 +| `org` | String | Organisation or publisher |
105 +| `name` | String | Model name without org prefix |
106 +| `size_bytes` | Integer | Total size on disk in bytes |
107 +| `size_display` | String | Human-readable size (e.g. `"1.84 GB"`) |
108 +| `path` | String | Absolute path to the model cache directory |
109 +| `revisions` | Array | List of snapshot revision strings |
110 +
111 +---
112 +
113 +#### `Onde.list_supported_models` → Hash
114 +
115 +Returns all models the engine supports, together with flags indicating whether each one is fully downloaded, partially downloaded, or absent.
116 +
117 +```ruby
118 +Onde.list_supported_models["models"].each do |m|
119 + status = if m["is_downloaded"]
120 + "✓ downloaded"
121 + elsif m["is_incomplete"]
122 + "⏳ incomplete (#{m["local_size_display"]} / #{m["expected_size_display"]})"
123 + else
124 + "✗ not downloaded"
125 + end
126 + puts "[#{status}] #{m["name"]} (#{m["org"]})"
127 + puts " #{m["description"]}"
128 +end
129 +```
130 +
131 +**Each model Hash:**
132 +
133 +| Key | Type | Description |
134 +|-----|------|-------------|
135 +| `model_id` | String | Full HuggingFace ID |
136 +| `name` | String | Human-friendly display name |
137 +| `org` | String | Organisation or publisher |
138 +| `description` | String | Short description of the model |
139 +| `is_downloaded` | Boolean | Fully downloaded locally |
140 +| `is_incomplete` | Boolean | Partial download exists on disk |
141 +| `local_size_bytes` | Integer | Bytes currently on disk (0 if absent) |
142 +| `local_size_display` | String | Human-readable local size |
143 +| `expected_size_bytes` | Integer | Approximate total size when fully downloaded |
144 +| `expected_size_display` | String | Human-readable expected size |
145 +
146 +---
147 +
148 +#### `Onde.delete_model(model_id)` → nil
149 +
150 +Delete a locally cached HuggingFace model by removing its directory from the hub cache.
151 +
152 +```ruby
153 +Onde.delete_model("bartowski/Qwen2.5-1.5B-Instruct-GGUF")
154 +```
155 +
156 +**Parameters:**
157 +
158 +| Name | Type | Description |
159 +|------|------|-------------|
160 +| `model_id` | String | Full model identifier (e.g. `"bartowski/Qwen2.5-1.5B-Instruct-GGUF"`) |
161 +
162 +**Raises:** `RuntimeError` if the model is not found or deletion fails.
163 +
164 +---
165 +
166 +#### `Onde.cache_path` → String or nil
167 +
168 +Returns the resolved HuggingFace cache directory path, or `nil` if it cannot be determined (e.g. `$HOME` is unset).
169 +
170 +```ruby
171 +Onde.cache_path
172 +# => "/Users/you/.cache/huggingface/hub"
173 +```
174 +
175 +---
176 +
177 +### Model Metadata
178 +
179 +#### `Onde.model_info(model_id)` → Hash or nil
180 +
181 +Look up rich metadata for a single supported model. Returns `nil` if the model ID is not in the supported list.
182 +
183 +```ruby
184 +info = Onde.model_info("bartowski/Qwen2.5-1.5B-Instruct-GGUF")
185 +# => {
186 +# "id" => "bartowski/Qwen2.5-1.5B-Instruct-GGUF",
187 +# "name" => "Qwen 2.5 1.5B (GGUF)",
188 +# "org" => "Qwen / Alibaba",
189 +# "description" => "Lightest pre-quantized chat model — ...",
190 +# "expected_size_bytes" => 986048768
191 +# }
192 +
193 +Onde.model_info("nonexistent/model")
194 +# => nil
195 +```
196 +
197 +**Return value keys:**
198 +
199 +| Key | Type | Description |
200 +|-----|------|-------------|
201 +| `id` | String | Full HuggingFace model identifier |
202 +| `name` | String | Human-friendly display name |
203 +| `org` | String | Organisation or publisher |
204 +| `description` | String | Short description |
205 +| `expected_size_bytes` | Integer | Approximate total size in bytes |
206 +
207 +---
208 +
209 +#### `Onde.supported_model_ids` → Array
210 +
211 +Returns the list of all supported model IDs as strings.
212 +
213 +```ruby
214 +Onde.supported_model_ids
215 +# => [
216 +# "bartowski/Qwen2.5-1.5B-Instruct-GGUF",
217 +# "bartowski/Qwen2.5-3B-Instruct-GGUF"
218 +# ]
219 +```
220 +
221 +---
222 +
223 +### Sampling Configuration
224 +
225 +These methods return Hash representations of the inference sampling parameters. They are useful for inspecting defaults or passing to future inference APIs.
226 +
227 +#### `Onde.default_sampling_config` → Hash
228 +
229 +Creative chat defaults.
230 +
231 +```ruby
232 +Onde.default_sampling_config
233 +# => {
234 +# "temperature" => 0.7,
235 +# "top_p" => 0.95,
236 +# "top_k" => nil,
237 +# "min_p" => nil,
238 +# "max_tokens" => 512,
239 +# "frequency_penalty" => nil,
240 +# "presence_penalty" => nil
241 +# }
242 +```
243 +
244 +#### `Onde.deterministic_sampling_config` → Hash
245 +
246 +Greedy decoding (temperature = 0.0).
247 +
248 +```ruby
249 +Onde.deterministic_sampling_config
250 +# => { "temperature" => 0.0, "top_p" => nil, ..., "max_tokens" => 512 }
251 +```
252 +
253 +#### `Onde.mobile_sampling_config` → Hash
254 +
255 +Conservative defaults for constrained devices (lower `max_tokens`).
256 +
257 +```ruby
258 +Onde.mobile_sampling_config
259 +# => { "temperature" => 0.7, "top_p" => 0.95, ..., "max_tokens" => 128 }
260 +```
261 +
262 +---
263 +
264 +### Constants
265 +
266 +| Constant | Type | Description |
267 +|----------|------|-------------|
268 +| `Onde::VERSION` | String | Gem version (e.g. `"0.1.0"`) |
269 +| `Onde::NATIVE_VERSION` | String | Rust crate version (e.g. `"0.1.0"`) |
270 +| `Onde::SUPPORTED_MODELS` | Array | Frozen array of all supported model ID strings |
271 +| `Onde::SUPPORTED_MODEL_INFO` | Array | Frozen array of frozen Hashes with model metadata |
272 +
273 +```ruby
274 +Onde::SUPPORTED_MODELS
275 +# => ["bartowski/Qwen2.5-1.5B-Instruct-GGUF", "bartowski/Qwen2.5-3B-Instruct-GGUF"]
276 +
277 +Onde::SUPPORTED_MODEL_INFO.first
278 +# => {
279 +# "id" => "bartowski/Qwen2.5-1.5B-Instruct-GGUF",
280 +# "name" => "Qwen 2.5 1.5B (GGUF)",
281 +# "org" => "Qwen / Alibaba",
282 +# "description" => "Lightest pre-quantized chat model — ...",
283 +# "expected_size_bytes" => 986048768
284 +# }
285 +```
286 +
287 +Both constants are frozen at load time — attempting to modify them raises `FrozenError`.
288 +
289 +## Usage Examples
290 +
291 +### Rails Admin Dashboard
292 +
293 +```ruby
294 +# app/controllers/admin/models_controller.rb
295 +class Admin::ModelsController < ApplicationController
296 + def index
297 + @supported = Onde.list_supported_models["models"]
298 + @local = Onde.list_local_models
299 + end
300 +
301 + def destroy
302 + Onde.delete_model(params[:model_id])
303 + redirect_to admin_models_path, notice: "Model cache deleted."
304 + rescue RuntimeError => e
305 + redirect_to admin_models_path, alert: e.message
306 + end
307 +end
308 +```
309 +
310 +### Background Job — Cache Cleanup
311 +
312 +```ruby
313 +# app/jobs/cleanup_hf_cache_job.rb
314 +class CleanupHfCacheJob < ApplicationJob
315 + queue_as :maintenance
316 +
317 + def perform
318 + local = Onde.list_local_models
319 + logger.info "[Onde] HF cache: #{local["total_size_display"]} across #{local["models"].length} models"
320 +
321 + # Delete models that are no longer in the supported list
322 + local["models"].each do |model|
323 + unless Onde::SUPPORTED_MODELS.include?(model["model_id"])
324 + logger.info "[Onde] Removing unsupported model: #{model["model_id"]} (#{model["size_display"]})"
325 + Onde.delete_model(model["model_id"])
326 + end
327 + end
328 + end
329 +end
330 +```
331 +
332 +### Rake Task — Model Status Report
333 +
334 +```ruby
335 +# lib/tasks/onde.rake
336 +namespace :onde do
337 + desc "Show HuggingFace model cache status"
338 + task status: :environment do
339 + puts "Cache: #{Onde.cache_path || "(not found)"}"
340 + puts
341 +
342 + Onde.list_supported_models["models"].each do |m|
343 + icon = m["is_downloaded"] ? "✓" : (m["is_incomplete"] ? "⏳" : "✗")
344 + local = m["is_downloaded"] ? m["local_size_display"] : (m["is_incomplete"] ? m["local_size_display"] : "—")
345 + puts "[#{icon}] #{m["name"].ljust(25)} #{local.rjust(10)} / #{m["expected_size_display"].rjust(10)} (#{m["org"]})"
346 + end
347 +
348 + local = Onde.list_local_models
349 + puts
350 + puts "Total on disk: #{local["total_size_display"]}"
351 + end
352 +end
353 +```
354 +
355 +Run with:
356 +
357 +```bash
358 +bundle exec rake onde:status
359 +```
360 +
361 +Example output:
362 +
363 +```
364 +Cache: /Users/you/.cache/huggingface/hub
365 +
366 +[✓] Qwen 2.5 1.5B (GGUF) 1.84 GB / 940.4 MB (Qwen / Alibaba)
367 +[✓] Qwen 2.5 3B (GGUF) 3.59 GB / 1.80 GB (Qwen / Alibaba)
368 +
369 +Total on disk: 5.43 GB
370 +```
371 +
372 +### API Endpoint
373 +
374 +```ruby
375 +# app/controllers/api/v1/models_controller.rb
376 +# frozen_string_literal: true
377 +
378 +module Api
379 + module V1
380 + class ModelsController < ApplicationController
381 + def index
382 + render json: Onde.list_supported_models
383 + end
384 +
385 + def show
386 + info = Onde.model_info(params[:id])
387 + if info
388 + render json: info
389 + else
390 + render json: { error: "Model not found" }, status: :not_found
391 + end
392 + end
393 + end
394 + end
395 +end
396 +```
397 +
398 +## Architecture
399 +
400 +```
401 +gem/
402 +├── Cargo.toml # Workspace root (isolates from parent onde crate)
403 +├── Gemfile # Dev deps: rake-compiler, rb_sys ~> 0.9.63
404 +├── Rakefile # RbSys::ExtensionTask build config
405 +├── onde.gemspec # Gem specification
406 +├── rust-toolchain.toml # Pinned to Rust 1.92
407 +├── .gitignore
408 +├── bin/
409 +│ ├── console # IRB with gem pre-loaded
410 +│ └── setup # One-step dev setup script
411 +├── lib/
412 +│ ├── onde.rb # Entry point — loads native ext + defines module
413 +│ └── onde/
414 +│ ├── version.rb # Onde::VERSION = '0.1.0'
415 +│ └── onde_ruby.bundle # Compiled native extension (git-ignored)
416 +└── ext/
417 + └── onde-ruby/
418 + ├── Cargo.toml # Rust crate: depends on `onde` + `magnus`
419 + ├── extconf.rb # rb_sys build hook
420 + └── src/
421 + └── lib.rs # Magnus bindings — the Rust ↔ Ruby bridge
422 +```
423 +
424 +### How the binding works
425 +
426 +```
427 +┌───────────────────────────┐
428 +│ Ruby │
429 +│ require "onde" │
430 +│ Onde.list_local_models │
431 +└───────────┬───────────────┘
432 + │ FFI call via magnus
433 + ▼
434 +┌───────────────────────────┐
435 +│ onde-ruby (cdylib) │
436 +│ ext/onde-ruby/src/lib.rs│
437 +│ │
438 +│ #[magnus::init] │
439 +│ fn init(ruby) { │
440 +│ module.define_*() │
441 +│ } │
442 +└───────────┬───────────────┘
443 + │ Rust dependency
444 + ▼
445 +┌───────────────────────────┐
446 +│ onde (Rust crate) │
447 +│ lib/crates/onde/src/ │
448 +│ │
449 +│ hf_cache:: │
450 +│ inference::models:: │
451 +│ inference::types:: │
452 +└───────────┬───────────────┘
453 + │
454 + ▼
455 +┌───────────────────────────┐
456 +│ mistral.rs │
457 +│ HuggingFace Hub │
458 +│ Metal / CUDA / CPU │
459 +└───────────────────────────┘
460 +```
461 +
462 +### Data flow
463 +
464 +All Rust structs are converted to Ruby Hashes via `serde_json` round-tripping:
465 +
466 +```
467 +Rust struct ─→ serde_json::Value ─→ Ruby Hash/Array/String/Integer/nil
468 +```
469 +
470 +This keeps the binding layer thin — no custom Ruby classes are needed. Every Onde struct that implements `serde::Serialize` can be exposed to Ruby with a single `to_ruby_value()` call.
471 +
472 +### Workspace isolation
473 +
474 +The `gem/Cargo.toml` defines a Cargo workspace with `ext/onde-ruby` as its only member. This prevents `cargo metadata` from walking up to the parent `lib/crates/onde/Cargo.toml` and confusing the rb_sys build system.
475 +
476 +Without this workspace file, running `cargo metadata` from `gem/` resolves to the parent `onde` crate, and `RbSys::Cargo::Metadata` cannot find the `onde-ruby` package.
477 +
478 +### Naming convention
479 +
480 +| Layer | Name | Why |
481 +|-------|------|-----|
482 +| Cargo package | `onde-ruby` | Avoids collision with the parent `onde` crate in the dependency graph |
483 +| Cargo lib output | `onde_ruby` | Rust convention: hyphens become underscores in lib names |
484 +| Ruby require | `onde/onde_ruby` | Matches the compiled `.bundle`/`.so` filename |
485 +| Ruby module | `Onde` | Clean public API name |
486 +| Gem name | `onde` | Simple, matches the project name |
487 +
488 +## Internals
489 +
490 +### Key files
491 +
492 +| File | Purpose |
493 +|------|---------|
494 +| `ext/onde-ruby/src/lib.rs` | All Magnus bindings — defines the `Onde` Ruby module, singleton methods, and constants |
495 +| `ext/onde-ruby/Cargo.toml` | Rust crate config — depends on `onde` (path) + `magnus` 0.7 + `serde_json` |
496 +| `ext/onde-ruby/extconf.rb` | Build hook — calls `create_rust_makefile("onde/onde_ruby")` |
497 +| `lib/onde.rb` | Ruby entry point — `require_relative 'onde/onde_ruby'` loads the native ext |
498 +| `Rakefile` | `RbSys::ExtensionTask.new('onde-ruby', GEMSPEC)` wires up compilation |
499 +| `Cargo.toml` (root) | Workspace file isolating the gem from the parent crate |
500 +
501 +### Dependencies
502 +
503 +**Ruby side:**
504 +
505 +- `rake` ~> 13.0 — build orchestration
506 +- `rake-compiler` — native extension compilation framework
507 +- `rb_sys` ~> 0.9.63 — Rust ↔ Ruby build glue
508 +
509 +**Rust side:**
510 +
511 +- `magnus` 0.7 — Ruby FFI bindings for Rust
512 +- `serde` + `serde_json` — struct serialization to Ruby Hashes
513 +- `onde` (path dependency) — the core Onde crate
514 +
515 +### Adding a new method
516 +
517 +1. Add the Rust function in `ext/onde-ruby/src/lib.rs`:
518 +
519 + ```rust
520 + fn my_new_method(arg: String) -> Result<magnus::Value, Error> {
521 + let ruby = Ruby::get().expect("called outside Ruby");
522 + // ... call into onde crate ...
523 + Ok(ruby.str_new("result").as_value())
524 + }
525 + ```
526 +
527 +2. Register it in the `init` function:
528 +
529 + ```rust
530 + module.define_singleton_method("my_new_method", function!(my_new_method, 1))?;
531 + ```
532 +
533 +3. Recompile:
534 +
535 + ```bash
536 + bundle exec rake compile
537 + ```
538 +
539 +4. Test:
540 +
541 + ```bash
542 + bundle exec ruby -e "require 'onde'; puts Onde.my_new_method('hello')"
543 + ```
544 +
545 +### Adding a new constant
546 +
547 +In the `init` function in `ext/onde-ruby/src/lib.rs`:
548 +
549 +```rust
550 +module.const_set("MY_CONSTANT", ruby.str_new("value"))?;
551 +```
552 +
553 +Constants are available as `Onde::MY_CONSTANT` in Ruby. Call `.freeze()` on Arrays and Hashes before setting them as constants to prevent mutation.
554 +
555 +## Troubleshooting
556 +
557 +### `cargo` not found during `bundle install`
558 +
559 +Rust is not on your `$PATH`. Install via [rustup](https://rustup.rs):
560 +
561 +```bash
562 +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
563 +source ~/.cargo/env
564 +```
565 +
566 +### Compilation fails with `rustc X.Y.Z is not supported`
567 +
568 +The gem requires Rust >= 1.92. Update your toolchain:
569 +
570 +```bash
571 +rustup update
572 +# or install the specific version:
573 +rustup toolchain install 1.92
574 +```
575 +
576 +The `rust-toolchain.toml` in the gem directory pins the version — rustup will select it automatically when you `cd` into the directory.
577 +
578 +### `Could not find Cargo package metadata for "onde-ruby"`
579 +
580 +The workspace `Cargo.toml` at `gem/` is missing or corrupted. It must exist and contain:
581 +
582 +```toml
583 +[workspace]
584 +members = ["ext/onde-ruby"]
585 +resolver = "2"
586 +```
587 +
588 +Without this file, `cargo metadata` resolves to the parent `onde` crate and rb_sys cannot find `onde-ruby`.
589 +
590 +### Output filename collision warning
591 +
592 +You may see:
593 +
594 +```
595 +warning: output filename collision at .../libonde.dylib
596 + note: the lib target `onde` in package `onde-ruby` has the same output
597 + filename as the lib target `onde` in package `onde`
598 +```
599 +
600 +This is a Cargo warning (not an error) caused by the extension crate's lib output name (`onde_ruby`) sharing a target directory with the parent `onde` crate. It does not affect compilation. The warning will become a hard error in a future Cargo version — at that point, set a separate `target-dir` in the workspace.
601 +
602 +### `LoadError: cannot load such file -- onde/onde_ruby`
603 +
604 +The native extension hasn't been compiled. Run:
605 +
606 +```bash
607 +bundle exec rake compile
608 +```
609 +
610 +### Stale `.bundle` / `.so` after code changes
611 +
612 +If you change `ext/onde-ruby/src/lib.rs` and the changes don't take effect:
613 +
614 +```bash
615 +bundle exec rake clobber compile
616 +```
617 +
618 +`clobber` cleans all build artifacts before recompiling.
619 +
620 +## Version Compatibility
621 +
622 +| Onde gem | Onde crate | Rust toolchain | Ruby | magnus |
623 +|----------|-----------|----------------|------|--------|
624 +| 0.1.0 | 0.1.0 | >= 1.92 | >= 3.0 | 0.7 |
625 +
626 +The `Onde::NATIVE_VERSION` constant always reflects the Rust crate version compiled into the gem. Compare it against `Onde::VERSION` to detect mismatches:
627 +
628 +```ruby
629 +if Onde::VERSION != Onde::NATIVE_VERSION
630 + warn "Onde version mismatch: gem=#{Onde::VERSION} native=#{Onde::NATIVE_VERSION}"
631 +end
632 +```
docs/swift-package.md new
+328
@@ -0,0 +1,328 @@
1 +---
2 +title: "Swift Package"
3 +description: "Build and distribute the Onde Swift package (XCFramework) for iOS, tvOS, and macOS using UniFFI bindings."
4 +---
5 +
6 +# Swift Package
7 +
8 +The Onde Swift package distributes a pre-built `OndeFramework.xcframework` via Swift Package Manager. The framework is built from Rust source using UniFFI bindings — no `cargo-swift` required. The project owns its own `uniffi-bindgen` binary that generates the Swift glue code directly.
9 +
10 +## Installation
11 +
12 +### Swift Package Manager
13 +
14 +Add the package in Xcode: **File → Add Package Dependencies** and enter:
15 +
16 +```
17 +https://github.com/ondeinference/onde-swift
18 +```
19 +
20 +Or add it to your `Package.swift`:
21 +
22 +```swift
23 +dependencies: [
24 + .package(url: "https://github.com/ondeinference/onde-swift", from: "0.1.0")
25 +],
26 +targets: [
27 + .target(name: "MyApp", dependencies: ["Onde"])
28 +]
29 +```
30 +
31 +## Supported Platforms
32 +
33 +| Platform | Minimum Version | GPU Backend |
34 +| -------- | --------------- | ----------- |
35 +| iOS | 14.0 | Metal |
36 +| tvOS | 13.0 | Metal |
37 +| macOS | 12.0 | Metal |
38 +
39 +## Swift API
40 +
41 +### Basic Chat
42 +
43 +```swift
44 +import Onde
45 +
46 +// Create the engine
47 +let engine = OndeChatEngine()
48 +
49 +// Load the platform-appropriate default model:
50 +// iOS / tvOS → Qwen 2.5 1.5B (~941 MB)
51 +// macOS → Qwen 2.5 3B (~1.93 GB)
52 +let loadTimeSecs = try await engine.loadDefaultModel(
53 + systemPrompt: "You are a helpful assistant.",
54 + sampling: nil // uses platform-aware defaults
55 +)
56 +print("Model loaded in \(loadTimeSecs)s")
57 +
58 +// Multi-turn chat (history managed automatically)
59 +let result = try await engine.sendMessage(message: "What can you help me with?")
60 +print(result.text)
61 +print(result.durationDisplay) // e.g. "4.5s"
62 +
63 +let followUp = try await engine.sendMessage(message: "Tell me more about that.")
64 +print(followUp.text)
65 +
66 +// Unload model to free memory
67 +await engine.unloadModel()
68 +```
69 +
70 +### Streaming
71 +
72 +```swift
73 +import Onde
74 +
75 +class StreamHandler: StreamChunkListener {
76 + func onChunk(chunk: StreamChunk) -> Bool {
77 + print(chunk.delta, terminator: "")
78 + if chunk.done { print() }
79 + return !chunk.done // return false to cancel early
80 + }
81 +}
82 +
83 +let engine = OndeChatEngine()
84 +try await engine.loadDefaultModel(systemPrompt: nil, sampling: nil)
85 +
86 +try await streamChatMessage(
87 + engine: engine,
88 + message: "Tell me a story.",
89 + listener: StreamHandler()
90 +)
91 +```
92 +
93 +Streaming is exposed as a free function rather than an `OndeChatEngine` method because UniFFI 0.31 does not support `callback_interface` parameters on Object methods.
94 +
95 +### One-Shot Generation
96 +
97 +`generate` runs inference on an explicit message list without modifying the engine's conversation history. Useful for prompt enhancement, summarisation, or any case where you don't want side effects on the chat history.
98 +
99 +```swift
100 +import Onde
101 +
102 +let engine = OndeChatEngine()
103 +try await engine.loadDefaultModel(systemPrompt: nil, sampling: nil)
104 +
105 +let result = try await engine.generate(
106 + messages: [userMessage(content: "Expand into a detailed prompt: a cat in space")],
107 + sampling: deterministicSamplingConfig()
108 +)
109 +print(result.text)
110 +```
111 +
112 +### Custom Model and Sampling
113 +
114 +```swift
115 +import Onde
116 +
117 +let engine = OndeChatEngine()
118 +
119 +let sampling = SamplingConfig(
120 + temperature: 0.9,
121 + topP: 0.95,
122 + topK: nil,
123 + minP: nil,
124 + maxTokens: 256,
125 + frequencyPenalty: nil,
126 + presencePenalty: nil
127 +)
128 +
129 +try await engine.loadGgufModel(
130 + config: qwen253bConfig(),
131 + systemPrompt: "You are a music expert.",
132 + sampling: sampling
133 +)
134 +```
135 +
136 +### History and Status
137 +
138 +```swift
139 +// Check engine status
140 +let info = await engine.info()
141 +print(info.status) // .ready
142 +print(info.historyLength) // number of turns
143 +
144 +// Retrieve full conversation history
145 +let history = await engine.history()
146 +for msg in history {
147 + print("\(msg.role): \(msg.content)")
148 +}
149 +
150 +// Clear history without unloading the model
151 +let removed = await engine.clearHistory()
152 +print("Removed \(removed) turns")
153 +
154 +// Append a message without running inference
155 +await engine.pushHistory(message: userMessage(content: "..."))
156 +```
157 +
158 +## Free Functions Reference
159 +
160 +### Model Configurations
161 +
162 +```swift
163 +defaultModelConfig() // platform-appropriate default (1.5B on iOS/tvOS, 3B on macOS)
164 +qwen251_5bConfig() // Qwen 2.5 1.5B Q4_K_M (~941 MB)
165 +qwen253bConfig() // Qwen 2.5 3B Q4_K_M (~1.93 GB)
166 +```
167 +
168 +### Sampling Presets
169 +
170 +```swift
171 +defaultSamplingConfig() // temp=0.7, top_p=0.95, max_tokens=512
172 +deterministicSamplingConfig() // temp=0.0, greedy decoding
173 +mobileSamplingConfig() // temp=0.7, max_tokens=128
174 +```
175 +
176 +### Message Constructors
177 +
178 +```swift
179 +systemMessage(content:) // ChatRole.system
180 +userMessage(content:) // ChatRole.user
181 +assistantMessage(content:) // ChatRole.assistant
182 +```
183 +
184 +## Type Reference
185 +
186 +| Rust Type | Swift Type | Description |
187 +| --------- | ---------- | ----------- |
188 +| `ChatRole` | `enum ChatRole` | `.system`, `.user`, `.assistant` |
189 +| `ChatMessage` | `struct ChatMessage` | `role` + `content` |
190 +| `SamplingConfig` | `struct SamplingConfig` | Temperature, top-p, max tokens, etc. |
191 +| `GgufModelConfig` | `struct GgufModelConfig` | Model repo, filename, display name |
192 +| `InferenceResult` | `struct InferenceResult` | `text`, `durationSecs`, `durationDisplay`, `finishReason` |
193 +| `StreamChunk` | `struct StreamChunk` | `delta`, `done`, `finishReason` |
194 +| `EngineStatus` | `enum EngineStatus` | `.unloaded`, `.loading`, `.ready`, `.generating`, `.error` |
195 +| `EngineInfo` | `struct EngineInfo` | `status`, `modelName`, `approxMemory`, `historyLength` |
196 +| `InferenceError` | `enum InferenceError: Error` | Thrown by `loadGgufModel`, `sendMessage`, `generate`, `streamChatMessage` |
197 +| `OndeChatEngine` | `class OndeChatEngine` | Main inference engine, thread-safe |
198 +| `StreamChunkListener` | `protocol StreamChunkListener` | Implement `onChunk(chunk:) -> Bool` |
199 +
200 +## Sandboxed App Setup (iOS / macOS App Store)
201 +
202 +On sandboxed platforms, `~/.cache` is inaccessible. Before any inference code runs, configure `HF_HOME` and `TMPDIR` to point inside the app's container:
203 +
204 +```swift
205 +import Foundation
206 +
207 +func setupInferenceEnvironment() {
208 + let appSupport = FileManager.default
209 + .urls(for: .applicationSupportDirectory, in: .userDomainMask)[0]
210 +
211 + let hfHome = appSupport.appendingPathComponent("huggingface")
212 + let hfHub = hfHome.appendingPathComponent("hub")
213 + try? FileManager.default.createDirectory(at: hfHub, withIntermediateDirectories: true)
214 +
215 + setenv("HF_HOME", hfHome.path, 1)
216 + setenv("HF_HUB_CACHE", hfHub.path, 1)
217 +
218 + // mistral.rs uses TMPDIR for temporary files during model loading
219 + let tmp = appSupport.appendingPathComponent("tmp")
220 + try? FileManager.default.createDirectory(at: tmp, withIntermediateDirectories: true)
221 + setenv("TMPDIR", tmp.path, 1)
222 +}
223 +```
224 +
225 +Call this once at app launch, before calling any `OndeChatEngine` method.
226 +
227 +## Building the XCFramework Locally
228 +
229 +If you are contributing to Onde or need a custom build, you can assemble the `OndeFramework.xcframework` from source.
230 +
231 +### Prerequisites
232 +
233 +```bash
234 +# Stable toolchain for iOS and macOS targets
235 +rustup toolchain install stable
236 +
237 +# Nightly toolchain for tvOS (tier-3 targets require -Z build-std)
238 +rustup toolchain install nightly
239 +rustup component add rust-src --toolchain nightly
240 +
241 +# Apple targets
242 +rustup target add \
243 + aarch64-apple-ios \
244 + x86_64-apple-ios \
245 + aarch64-apple-ios-sim \
246 + aarch64-apple-darwin \
247 + x86_64-apple-darwin
248 +```
249 +
250 +### Build
251 +
252 +```bash
253 +# 1. Build the uniffi-bindgen CLI (host)
254 +cargo build --manifest-path uniffi-bindgen/Cargo.toml --release
255 +BINDGEN=uniffi-bindgen/target/release/uniffi-bindgen
256 +
257 +# 2. Compile staticlibs per target
258 +cargo build --target aarch64-apple-ios --release
259 +cargo build --target x86_64-apple-ios --release
260 +cargo build --target aarch64-apple-ios-sim --release
261 +cargo build --target aarch64-apple-darwin --release
262 +cargo build --target x86_64-apple-darwin --release
263 +cargo +nightly build -Z build-std --target aarch64-apple-tvos --release
264 +cargo +nightly build -Z build-std --target x86_64-apple-tvos --release
265 +cargo +nightly build -Z build-std --target aarch64-apple-tvos-sim --release
266 +
267 +# 3. lipo simulator and macOS slices
268 +lipo -create \
269 + target/x86_64-apple-ios/release/libonde.a \
270 + target/aarch64-apple-ios-sim/release/libonde.a \
271 + -output /tmp/libonde-ios-sim.a
272 +
273 +lipo -create \
274 + target/x86_64-apple-tvos/release/libonde.a \
275 + target/aarch64-apple-tvos-sim/release/libonde.a \
276 + -output /tmp/libonde-tvos-sim.a
277 +
278 +lipo -create \
279 + target/aarch64-apple-darwin/release/libonde.a \
280 + target/x86_64-apple-darwin/release/libonde.a \
281 + -output /tmp/libonde-macos.a
282 +
283 +# 4. Generate Swift bindings
284 +mkdir -p sdk/Onde/Sources/Onde
285 +$BINDGEN generate \
286 + --library target/aarch64-apple-ios/release/libonde.a \
287 + --language swift \
288 + --out-dir sdk/Onde/Sources/Onde \
289 + --config uniffi.toml
290 +
291 +# 5. Assemble the XCFramework
292 +HEADERS=sdk/Onde/Sources/Onde
293 +xcodebuild -create-xcframework \
294 + -library target/aarch64-apple-ios/release/libonde.a -headers $HEADERS \
295 + -library /tmp/libonde-ios-sim.a -headers $HEADERS \
296 + -library target/aarch64-apple-tvos/release/libonde.a -headers $HEADERS \
297 + -library /tmp/libonde-tvos-sim.a -headers $HEADERS \
298 + -library /tmp/libonde-macos.a -headers $HEADERS \
299 + -output sdk/Onde/OndeFramework.xcframework
300 +```
301 +
302 +### Local Xcode Integration
303 +
304 +During development, use the local path form in `Package.swift`:
305 +
306 +```swift
307 +.binaryTarget(name: "OndeFramework", path: "./OndeFramework.xcframework")
308 +```
309 +
310 +In Xcode: **File → Add Package Dependencies → Add Local** → select `sdk/Onde/`.
311 +
312 +## Known Issues
313 +
314 +### `___chkstk_darwin` linker error on tvOS
315 +
316 +`aws-lc-sys` (a transitive dependency via `hf-hub`) references `___chkstk_darwin`, a stack probing symbol that tvOS does not export. The `build.rs` in the onde crate provides a no-op assembly stub (`tvos_chkstk.s`) that resolves this automatically. No action is needed — ensure `tvos_chkstk.s` is present at the crate root.
317 +
318 +This does not affect iOS, macOS, Android, Windows, or Linux builds.
319 +
320 +### Metal Toolchain missing (Xcode 26+)
321 +
322 +On Xcode 26+, the Metal compiler is a separate downloadable component. If you see empty `.metallib` files (92 bytes) or `"Error while loading function: fused_glu_float"` at runtime:
323 +
324 +```bash
325 +xcodebuild -downloadComponent MetalToolchain
326 +cargo clean -p mistralrs-quant
327 +cargo check
328 +```
src/hf_cache.rs new
+1197
@@ -0,0 +1,1197 @@
1 +//! HuggingFace hub cache inspection, repair, and model download.
2 +//!
3 +//! This module owns all on-disk state management for locally cached
4 +//! HuggingFace models and exposes a progress-callback-based download API
5 +//! that is completely decoupled from any application framework.
6 +//!
7 +//! 1. Call [`list_local_hf_models`] / [`list_supported_hf_models`] /
8 +//! [`delete_local_hf_model`] directly from your command handlers.
9 +//! 2. Call [`download_model`] with an `Arc<dyn Fn(ModelDownloadProgress)>`
10 +//! that forwards progress to whatever event system your app uses.
11 +
12 +use {
13 + crate::inference::models::{SUPPORTED_MODELS, SUPPORTED_MODEL_INFO},
14 + log::{debug, info, warn},
15 + serde::{Deserialize, Serialize},
16 + std::{fs, path::PathBuf},
17 + tsync::tsync,
18 +};
19 +
20 +// ---------------------------------------------------------------------------
21 +// Public types
22 +// ---------------------------------------------------------------------------
23 +
24 +/// Information about a locally cached HuggingFace model.
25 +#[tsync]
26 +#[derive(Debug, Clone, Serialize, Deserialize)]
27 +pub struct LocalHfModel {
28 + /// The full model identifier, e.g. "bartowski/Qwen2.5-1.5B-Instruct-GGUF".
29 + pub model_id: String,
30 + /// The organisation or user that published the model.
31 + pub org: String,
32 + /// The model name without the org prefix.
33 + pub name: String,
34 + /// Total size on disk in bytes.
35 + pub size_bytes: u64,
36 + /// Human-readable size string, e.g. "4.2 GB".
37 + pub size_display: String,
38 + /// Absolute path to the model cache directory.
39 + pub path: String,
40 + /// List of snapshot revisions that are locally available.
41 + pub revisions: Vec<String>,
42 +}
43 +
44 +/// Response returned by [`list_local_hf_models`].
45 +#[tsync]
46 +#[derive(Debug, Clone, Serialize, Deserialize)]
47 +pub struct LocalHfModelsResponse {
48 + /// The models found in the local HuggingFace cache.
49 + pub models: Vec<LocalHfModel>,
50 + /// The path that was scanned.
51 + pub cache_path: String,
52 + /// Total size of all cached models in bytes.
53 + pub total_size_bytes: u64,
54 + /// Human-readable total size string.
55 + pub total_size_display: String,
56 +}
57 +
58 +/// A supported model that may or may not be downloaded locally.
59 +#[tsync]
60 +#[derive(Debug, Clone, Serialize, Deserialize)]
61 +pub struct SupportedHfModel {
62 + /// The full model identifier, e.g. "bartowski/Qwen2.5-1.5B-Instruct-GGUF".
63 + pub model_id: String,
64 + /// Human-friendly display name.
65 + pub name: String,
66 + /// Organisation or publisher display name.
67 + pub org: String,
68 + /// Short description of the model.
69 + pub description: String,
70 + /// Whether this model is fully downloaded locally.
71 + pub is_downloaded: bool,
72 + /// Whether a partial (incomplete) download exists on disk.
73 + /// True when a cache directory exists but its size is less than 99 %
74 + /// of `expected_size_bytes`.
75 + pub is_incomplete: bool,
76 + /// Bytes currently on disk for this model (0 when not present at all).
77 + pub local_size_bytes: u64,
78 + /// Human-readable local size string, e.g. "4.2 GB".
79 + pub local_size_display: String,
80 + /// Approximate total size in bytes when fully downloaded.
81 + pub expected_size_bytes: u64,
82 + /// Human-readable expected size string, e.g. "23.80 GB".
83 + pub expected_size_display: String,
84 +}
85 +
86 +/// Response returned by [`list_supported_hf_models`].
87 +#[tsync]
88 +#[derive(Debug, Clone, Serialize, Deserialize)]
89 +pub struct SupportedHfModelsResponse {
90 + /// All models the app supports, with download status.
91 + pub models: Vec<SupportedHfModel>,
92 +}
93 +
94 +/// Progress payload passed to the [`download_model`] callback on every poll
95 +/// tick and on completion.
96 +///
97 +/// Forward this value to your app's event system so the frontend can render
98 +/// a progress bar.
99 +#[tsync]
100 +#[derive(Debug, Clone, Serialize, Deserialize)]
101 +pub struct ModelDownloadProgress {
102 + /// The model being downloaded.
103 + pub model_id: String,
104 + /// Bytes downloaded so far (observed from cache directory size).
105 + pub downloaded_bytes: u64,
106 + /// Human-readable downloaded size, e.g. "1.2 GB".
107 + pub downloaded_display: String,
108 + /// Expected total size in bytes (approximate).
109 + pub total_bytes: u64,
110 + /// Human-readable total size, e.g. "23.80 GB".
111 + pub total_display: String,
112 + /// Download progress as a value between 0.0 and 1.0.
113 + pub progress: f64,
114 + /// Whether the download has completed.
115 + pub done: bool,
116 +}
117 +
118 +// ---------------------------------------------------------------------------
119 +// Internal constants
120 +// ---------------------------------------------------------------------------
121 +
122 +/// Fraction of `expected_size_bytes` that must be present on disk for a
123 +/// model to be considered fully downloaded (99 %).
124 +const DOWNLOAD_COMPLETE_THRESHOLD: f64 = 0.99;
125 +
126 +// ---------------------------------------------------------------------------
127 +// Private helpers
128 +// ---------------------------------------------------------------------------
129 +
130 +/// Resolve the HuggingFace hub cache directory.
131 +///
132 +/// Respects the `HF_HOME` / `HUGGINGFACE_HUB_CACHE` environment variables.
133 +/// Falls back to `~/.cache/huggingface/hub`.
134 +///
135 +/// On iOS the `HF_HOME` variable is set during app setup (see
136 +/// `setup_application_filesystem`) to a path inside the app's writable
137 +/// `Library/Application Support/` container. We create the directory if it
138 +/// doesn't exist yet so that first-run and freshly-installed apps work
139 +/// correctly.
140 +fn hf_cache_dir() -> Option<PathBuf> {
141 + // 1. Explicit hub cache override
142 + if let Ok(cache) = std::env::var("HUGGINGFACE_HUB_CACHE") {
143 + let p = PathBuf::from(cache);
144 + let _ = fs::create_dir_all(&p);
145 + if p.is_dir() {
146 + return Some(p);
147 + }
148 + }
149 +
150 + // 2. HF_HOME override (hub is a subdirectory)
151 + if let Ok(hf_home) = std::env::var("HF_HOME") {
152 + let p = PathBuf::from(hf_home).join("hub");
153 + let _ = fs::create_dir_all(&p);
154 + if p.is_dir() {
155 + return Some(p);
156 + }
157 + }
158 +
159 + // 3. Default: ~/.cache/huggingface/hub
160 + if let Some(home) = home::home_dir() {
161 + let p = home.join(".cache").join("huggingface").join("hub");
162 + let _ = fs::create_dir_all(&p);
163 + if p.is_dir() {
164 + return Some(p);
165 + }
166 + }
167 +
168 + None
169 +}
170 +
171 +/// Recursively compute the total size of a directory in bytes.
172 +///
173 +/// Follows symlinks to count the real file size and also counts in-progress
174 +/// `.incomplete` staging files so that progress polling reflects partial
175 +/// downloads before they are renamed to their final blob name.
176 +fn dir_size(path: &PathBuf) -> u64 {
177 + let mut total: u64 = 0;
178 + if let Ok(entries) = fs::read_dir(path) {
179 + for entry in entries.flatten() {
180 + let entry_path = entry.path();
181 + if entry_path.is_dir() {
182 + total += dir_size(&entry_path);
183 + } else {
184 + // Count both completed blobs AND in-progress staging files
185 + // (hf-hub downloads to `<sha256>.incomplete` and renames on
186 + // completion — without this, progress stays at 0% for the
187 + // entire download and jumps to 100% at the end).
188 + if let Ok(meta) = entry_path.symlink_metadata() {
189 + if meta.is_symlink() {
190 + // Follow the symlink to get the real file size.
191 + if let Ok(real_meta) = entry_path.metadata() {
192 + total += real_meta.len();
193 + }
194 + } else {
195 + total += meta.len();
196 + }
197 + }
198 + }
199 + }
200 + }
201 + total
202 +}
203 +
204 +/// Format a byte count into a human-readable string.
205 +fn format_size(bytes: u64) -> String {
206 + const KB: u64 = 1024;
207 + const MB: u64 = KB * 1024;
208 + const GB: u64 = MB * 1024;
209 + const TB: u64 = GB * 1024;
210 +
211 + if bytes >= TB {
212 + format!("{:.2} TB", bytes as f64 / TB as f64)
213 + } else if bytes >= GB {
214 + format!("{:.2} GB", bytes as f64 / GB as f64)
215 + } else if bytes >= MB {
216 + format!("{:.1} MB", bytes as f64 / MB as f64)
217 + } else if bytes >= KB {
218 + format!("{:.0} KB", bytes as f64 / KB as f64)
219 + } else {
220 + format!("{} B", bytes)
221 + }
222 +}
223 +
224 +/// List the snapshot revisions available for a cached model directory.
225 +fn list_revisions(model_dir: &PathBuf) -> Vec<String> {
226 + let snapshots_dir = model_dir.join("snapshots");
227 + if !snapshots_dir.is_dir() {
228 + return Vec::new();
229 + }
230 +
231 + let mut revisions = Vec::new();
232 + if let Ok(entries) = fs::read_dir(&snapshots_dir) {
233 + for entry in entries.flatten() {
234 + if entry.path().is_dir() {
235 + if let Some(name) = entry.file_name().to_str() {
236 + revisions.push(name.to_string());
237 + }
238 + }
239 + }
240 + }
241 + revisions.sort();
242 + revisions
243 +}
244 +
245 +/// Build a [`ModelDownloadProgress`] payload from raw byte counts.
246 +fn make_progress(model_id: &str, downloaded: u64, total: u64, done: bool) -> ModelDownloadProgress {
247 + let progress = if total > 0 {
248 + (downloaded as f64 / total as f64).min(1.0)
249 + } else {
250 + 0.0
251 + };
252 + ModelDownloadProgress {
253 + model_id: model_id.to_string(),
254 + downloaded_bytes: downloaded,
255 + downloaded_display: format_size(downloaded),
256 + total_bytes: total,
257 + total_display: format_size(total),
258 + progress,
259 + done,
260 + }
261 +}
262 +
263 +// ---------------------------------------------------------------------------
264 +// Public cache utilities
265 +// ---------------------------------------------------------------------------
266 +
267 +/// Return the cache sub-directory path for a given model ID.
268 +///
269 +/// E.g. for `"bartowski/Qwen2.5-1.5B-Instruct-GGUF"` the cache directory is
270 +/// `<hf_cache>/models--bartowski--Qwen2.5-1.5B-Instruct-GGUF`.
271 +pub fn model_cache_path(model_id: &str) -> Option<PathBuf> {
272 + let cache_dir = hf_cache_dir()?;
273 + let dir_name = format!("models--{}", model_id.replace('/', "--"));
274 + Some(cache_dir.join(dir_name))
275 +}
276 +
277 +/// Pre-create the HuggingFace cache directory tree for `model_id` so that the
278 +/// progress-monitor thread can find a valid path immediately, even before
279 +/// hf-hub creates it during the first download.
280 +///
281 +/// hf-hub won't be confused by a pre-existing (empty) directory — it will
282 +/// simply populate it as normal.
283 +pub fn ensure_model_cache_dir(model_id: &str) {
284 + if let Some(model_path) = model_cache_path(model_id) {
285 + let blobs_dir = model_path.join("blobs");
286 + if let Err(e) = fs::create_dir_all(&blobs_dir) {
287 + warn!(
288 + "Could not pre-create model cache dir {}: {}",
289 + blobs_dir.display(),
290 + e
291 + );
292 + } else {
293 + info!("Pre-created model cache dir: {}", model_path.display());
294 + }
295 + }
296 +}
297 +
298 +/// Remove stale `.lock` and orphaned `.part` files from the HF cache
299 +/// `blobs/` directory for a model.
300 +///
301 +/// The HF hub client creates `.lock` files during download. If a download is
302 +/// interrupted these locks persist and prevent any subsequent access (download
303 +/// **or** model load). Calling this before loading or resuming a download
304 +/// allows the hub client to re-acquire the lock and proceed normally.
305 +pub fn clean_stale_lock_files(model_id: &str) {
306 + if let Some(cache_path) = model_cache_path(model_id) {
307 + let blobs_dir = cache_path.join("blobs");
308 + if blobs_dir.is_dir() {
309 + // Collect the names of complete blobs (no extension) so we can
310 + // decide whether a `.part` file is stale.
311 + let complete_blobs: Vec<String> = fs::read_dir(&blobs_dir)
312 + .into_iter()
313 + .flatten()
314 + .flatten()
315 + .filter_map(|e| {
316 + let p = e.path();
317 + if p.is_file() && p.extension().is_none() {
318 + p.file_name().map(|n| n.to_string_lossy().to_string())
319 + } else {
320 + None
321 + }
322 + })
323 + .collect();
324 +
325 + if let Ok(entries) = fs::read_dir(&blobs_dir) {
326 + for entry in entries.flatten() {
327 + let path = entry.path();
328 + let ext = path.extension().and_then(|e| e.to_str());
329 +
330 + match ext {
331 + Some("lock") => {
332 + info!("Removing stale lock file: {}", path.display());
333 + if let Err(e) = fs::remove_file(&path) {
334 + warn!("Failed to remove lock file {}: {}", path.display(), e);
335 + }
336 + }
337 + Some("part") => {
338 + // A `.part` file is a partial download left by hf-hub.
339 + // If the corresponding complete blob exists (same name
340 + // without `.part`), the partial file is stale and
341 + // causes hf-hub to issue a spurious Range-resume
342 + // download instead of using the complete blob.
343 + let stem = path.file_stem().and_then(|s| s.to_str()).unwrap_or("");
344 + if complete_blobs.iter().any(|b| b == stem) {
345 + let part_size = fs::metadata(&path).map(|m| m.len()).unwrap_or(0);
346 + info!(
347 + "Removing stale .part file ({} bytes, complete blob exists): {}",
348 + part_size,
349 + path.display()
350 + );
351 + if let Err(e) = fs::remove_file(&path) {
352 + warn!("Failed to remove .part file {}: {}", path.display(), e);
353 + }
354 + } else {
355 + let part_size = fs::metadata(&path).map(|m| m.len()).unwrap_or(0);
356 + debug!(
357 + "Keeping .part file ({} bytes, no complete blob yet): {}",
358 + part_size,
359 + path.display()
360 + );
361 + }
362 + }
363 + _ => {}
364 + }
365 + }
366 + }
367 + }
368 + }
369 +}
370 +
371 +/// Log the complete HF cache structure for a model to aid debugging.
372 +///
373 +/// This is called before model load so the developer can see exactly what's
374 +/// present in the cache, whether `refs/main` resolves, whether symlinks are
375 +/// intact, and whether any `.part` files are left over from interrupted
376 +/// downloads.
377 +pub fn diagnose_hf_cache(model_id: &str) {
378 + let cache_path = match model_cache_path(model_id) {
379 + Some(p) => p,
380 + None => {
381 + info!("diagnose_hf_cache({}): cache path not found", model_id);
382 + return;
383 + }
384 + };
385 +
386 + if !cache_path.is_dir() {
387 + info!(
388 + "diagnose_hf_cache({}): directory does not exist: {}",
389 + model_id,
390 + cache_path.display()
391 + );
392 + return;
393 + }
394 +
395 + info!(
396 + "diagnose_hf_cache({}): scanning {}",
397 + model_id,
398 + cache_path.display()
399 + );
400 +
401 + // ── refs/ ────────────────────────────────────────────────────────────
402 + let refs_dir = cache_path.join("refs");
403 + if refs_dir.is_dir() {
404 + if let Ok(entries) = fs::read_dir(&refs_dir) {
405 + for entry in entries.flatten() {
406 + let name = entry.file_name().to_string_lossy().to_string();
407 + match fs::read_to_string(entry.path()) {
408 + Ok(content) => {
409 + let hash = content.trim();
410 + info!(
411 + " refs/{} → \"{}\" (len={})",
412 + name,
413 + if hash.len() > 12 {
414 + format!("{}…", &hash[..12])
415 + } else {
416 + hash.to_string()
417 + },
418 + hash.len()
419 + );
420 + }
421 + Err(e) => {
422 + warn!(" refs/{} → <unreadable: {}>", name, e);
423 + }
424 + }
425 + }
426 + }
427 + } else {
428 + warn!(" refs/ directory missing!");
429 + }
430 +
431 + // ── blobs/ ───────────────────────────────────────────────────────────
432 + let blobs_dir = cache_path.join("blobs");
433 + if blobs_dir.is_dir() {
434 + if let Ok(entries) = fs::read_dir(&blobs_dir) {
435 + for entry in entries.flatten() {
436 + let path = entry.path();
437 + let name = entry.file_name().to_string_lossy().to_string();
438 + let size = fs::metadata(&path).map(|m| m.len()).unwrap_or(0);
439 + let kind = if name.ends_with(".lock") {
440 + "LOCK"
441 + } else if name.ends_with(".part") {
442 + "PART (incomplete)"
443 + } else {
444 + "blob"
445 + };
446 + info!(
447 + " blobs/{} — {} ({} bytes / {:.1} MB)",
448 + name,
449 + kind,
450 + size,
451 + size as f64 / 1_048_576.0
452 + );
453 + }
454 + }
455 + } else {
456 + warn!(" blobs/ directory missing!");
457 + }
458 +
459 + // ── snapshots/ ───────────────────────────────────────────────────────
460 + let snapshots_dir = cache_path.join("snapshots");
461 + if snapshots_dir.is_dir() {
462 + if let Ok(revs) = fs::read_dir(&snapshots_dir) {
463 + for rev_entry in revs.flatten() {
464 + let rev_name = rev_entry.file_name().to_string_lossy().to_string();
465 + let rev_display = if rev_name.len() > 12 {
466 + format!("{}…", &rev_name[..12])
467 + } else {
468 + rev_name.clone()
469 + };
470 + info!(" snapshots/{}/", rev_display);
471 +
472 + if let Ok(files) = fs::read_dir(rev_entry.path()) {
473 + for file_entry in files.flatten() {
474 + let file_name = file_entry.file_name().to_string_lossy().to_string();
475 + let file_path = file_entry.path();
476 +
477 + if file_path.is_symlink() {
478 + let target = fs::read_link(&file_path)
479 + .map(|t| t.to_string_lossy().to_string())
480 + .unwrap_or_else(|_| "<unreadable>".to_string());
481 + let accessible = fs::metadata(&file_path).is_ok();
482 + let status = if accessible { "OK" } else { "BROKEN" };
483 + info!(" {} → {} [symlink: {}]", file_name, target, status);
484 + } else if file_path.is_file() {
485 + let size = fs::metadata(&file_path).map(|m| m.len()).unwrap_or(0);
486 + info!(
487 + " {} [{} bytes / {:.1} MB, real file]",
488 + file_name,
489 + size,
490 + size as f64 / 1_048_576.0
491 + );
492 + } else {
493 + warn!(" {} [not a file or symlink!]", file_name);
494 + }
495 + }
496 + }
497 + }
498 + }
499 + } else {
500 + warn!(" snapshots/ directory missing!");
501 + }
502 +
503 + info!("diagnose_hf_cache({}): done", model_id);
504 +}
505 +
506 +/// Repair broken symlinks in the HF hub cache for a given model.
507 +///
508 +/// The `hf-hub` crate stores downloaded files as blobs keyed by SHA-256 and
509 +/// creates **symlinks** from `snapshots/<rev>/<filename>` → `../../blobs/<hash>`.
510 +/// When the cache directory is copied into an iOS app container (e.g. via
511 +/// Xcode "Replace Container" or `ideviceinstaller`), symlinks are often
512 +/// silently dropped or broken. The `hf-hub` client then thinks the file is
513 +/// missing and re-downloads it — even though the 940 MB blob is right there.
514 +///
515 +/// This function walks every snapshot entry for `model_id`. If a path is a
516 +/// broken symlink **or** a missing file, and a matching blob exists, it
517 +/// replaces the symlink with a hard copy of the blob so that `hf-hub`'s
518 +/// `Cache::get()` finds a real file at the expected path.
519 +///
520 +/// It also ensures a `refs/main` file exists pointing at the snapshot
521 +/// revision, since that file is needed for `hf-hub` to resolve the "main"
522 +/// ref to a concrete snapshot directory.
523 +///
524 +/// This is idempotent and safe to call on every app launch.
525 +pub fn repair_hf_cache_symlinks(model_id: &str) {
526 + let cache_path = match model_cache_path(model_id) {
527 + Some(p) if p.is_dir() => p,
528 + _ => return,
529 + };
530 +
531 + let blobs_dir = cache_path.join("blobs");
532 + let snapshots_dir = cache_path.join("snapshots");
533 + let refs_dir = cache_path.join("refs");
534 +
535 + if !blobs_dir.is_dir() || !snapshots_dir.is_dir() {
536 + return;
537 + }
538 +
539 + // Collect blob filenames (excluding .lock files) for lookup.
540 + let blob_names: Vec<String> = match fs::read_dir(&blobs_dir) {
541 + Ok(entries) => entries
542 + .flatten()
543 + .filter_map(|e| {
544 + let name = e.file_name().to_string_lossy().to_string();
545 + if name.ends_with(".lock") {
546 + None
547 + } else if e.path().is_file() {
548 + Some(name)
549 + } else {
550 + None
551 + }
552 + })
553 + .collect(),
554 + Err(_) => return,
555 + };
556 +
557 + if blob_names.is_empty() {
558 + return;
559 + }
560 +
561 + // Walk each snapshot revision directory.
562 + let snapshot_revs: Vec<(String, PathBuf)> = match fs::read_dir(&snapshots_dir) {
563 + Ok(entries) => entries
564 + .flatten()
565 + .filter(|e| e.path().is_dir())
566 + .filter_map(|e| {
567 + let rev = e.file_name().to_string_lossy().to_string();
568 + Some((rev, e.path()))
569 + })
570 + .collect(),
571 + Err(_) => return,
572 + };
573 +
574 + for (rev, snap_dir) in &snapshot_revs {
575 + let entries = match fs::read_dir(snap_dir) {
576 + Ok(e) => e,
577 + Err(_) => continue,
578 + };
579 +
580 + for entry in entries.flatten() {
581 + let snap_file = entry.path();
582 + let file_name = entry.file_name().to_string_lossy().to_string();
583 +
584 + // Check if the file is accessible (broken symlink → metadata fails).
585 + let needs_repair = if snap_file.is_symlink() {
586 + // Symlink exists but may be broken — check if target is readable.
587 + fs::metadata(&snap_file).is_err()
588 + } else if snap_file.exists() {
589 + // Real file, no repair needed.
590 + false
591 + } else {
592 + // Path doesn't exist at all (dangling entry in readdir).
593 + true
594 + };
595 +
596 + if !needs_repair {
597 + continue;
598 + }
599 +
600 + // Try to find a matching blob. For GGUF repos there's usually
601 + // just one blob; for multi-file repos we match by the symlink
602 + // target name if we can read it, otherwise fall back to the
603 + // single-blob heuristic.
604 + let blob_name = if let Ok(target) = fs::read_link(&snap_file) {
605 + target
606 + .file_name()
607 + .map(|n| n.to_string_lossy().to_string())
608 + .filter(|n| blob_names.contains(n))
609 + } else {
610 + None
611 + };
612 +
613 + let resolved_blob = if let Some(ref name) = blob_name {
614 + Some(blobs_dir.join(name))
615 + } else if blob_names.len() == 1 {
616 + // Single-blob repo (typical for GGUF) — safe to assume it
617 + // corresponds to this snapshot entry.
618 + Some(blobs_dir.join(&blob_names[0]))
619 + } else {
620 + None
621 + };
622 +
623 + if let Some(blob_path) = resolved_blob {
624 + if !blob_path.is_file() {
625 + continue;
626 + }
627 +
628 + // Remove the broken symlink before copying.
629 + if snap_file.is_symlink() || snap_file.exists() {
630 + let _ = fs::remove_file(&snap_file);
631 + }
632 +
633 + info!(
634 + "repair_hf_cache_symlinks: replacing broken symlink with copy: {} → {}",
635 + blob_path.display(),
636 + snap_file.display()
637 + );
638 +
639 + // Prefer hard link (instant, no extra disk space) over copy.
640 + if fs::hard_link(&blob_path, &snap_file).is_err() {
641 + if let Err(e) = fs::copy(&blob_path, &snap_file) {
642 + warn!(
643 + "repair_hf_cache_symlinks: failed to copy blob to snapshot: {}",
644 + e
645 + );
646 + }
647 + }
648 + } else {
649 + warn!(
650 + "repair_hf_cache_symlinks: broken symlink for '{}' but no matching blob found",
651 + file_name
652 + );
653 + }
654 + }
655 +
656 + // Ensure refs/main exists and points to this revision so that
657 + // `hf-hub` can resolve the snapshot.
658 + if !refs_dir.join("main").is_file() {
659 + let _ = fs::create_dir_all(&refs_dir);
660 + if let Err(e) = fs::write(refs_dir.join("main"), rev.as_bytes()) {
661 + warn!("repair_hf_cache_symlinks: failed to write refs/main: {}", e);
662 + } else {
663 + info!("repair_hf_cache_symlinks: created refs/main → {}", rev);
664 + }
665 + }
666 + }
667 +}
668 +
669 +// ---------------------------------------------------------------------------
670 +// Query / mutation functions
671 +// ---------------------------------------------------------------------------
672 +
673 +/// Scan the local HuggingFace hub cache and return all downloaded models that
674 +/// the inference engine supports.
675 +///
676 +/// The cache directory is typically `~/.cache/huggingface/hub/` and contains
677 +/// sub-directories named `models--{org}--{model}` for each cached model.
678 +pub fn list_local_hf_models() -> LocalHfModelsResponse {
679 + let cache_dir = match hf_cache_dir() {
680 + Some(dir) => dir,
681 + None => {
682 + debug!("HuggingFace cache directory not found.");
683 + return LocalHfModelsResponse {
684 + models: Vec::new(),
685 + cache_path: String::new(),
686 + total_size_bytes: 0,
687 + total_size_display: "0 B".to_string(),
688 + };
689 + }
690 + };
691 +
692 + let cache_path_str = cache_dir.to_string_lossy().to_string();
693 + debug!("Scanning HuggingFace cache at: {}", cache_path_str);
694 +
695 + let mut models: Vec<LocalHfModel> = Vec::new();
696 +
697 + let entries = match fs::read_dir(&cache_dir) {
698 + Ok(entries) => entries,
699 + Err(e) => {
700 + warn!("Failed to read HuggingFace cache directory: {:?}", e);
701 + return LocalHfModelsResponse {
702 + models: Vec::new(),
703 + cache_path: cache_path_str,
704 + total_size_bytes: 0,
705 + total_size_display: "0 B".to_string(),
706 + };
707 + }
708 + };
709 +
710 + for entry in entries.flatten() {
711 + let entry_path = entry.path();
712 + if !entry_path.is_dir() {
713 + continue;
714 + }
715 +
716 + let dir_name = match entry.file_name().to_str() {
717 + Some(name) => name.to_string(),
718 + None => continue,
719 + };
720 +
721 + // HuggingFace hub caches model repos as "models--{org}--{name}"
722 + if !dir_name.starts_with("models--") {
723 + continue;
724 + }
725 +
726 + // Strip the "models--" prefix and split by "--" to get org/name
727 + let remainder = &dir_name["models--".len()..];
728 + let parts: Vec<&str> = remainder.splitn(2, "--").collect();
729 + if parts.len() != 2 {
730 + // Some models have no org (single segment), handle gracefully
731 + let model_id = remainder.replace("--", "/");
732 + let size = dir_size(&entry_path);
733 + let revisions = list_revisions(&entry_path);
734 +
735 + models.push(LocalHfModel {
736 + model_id: model_id.clone(),
737 + org: String::new(),
738 + name: model_id,
739 + size_bytes: size,
740 + size_display: format_size(size),
741 + path: entry_path.to_string_lossy().to_string(),
742 + revisions,
743 + });
744 + continue;
745 + }
746 +
747 + let org = parts[0].to_string();
748 + let name = parts[1].to_string();
749 + let model_id = format!("{}/{}", org, name);
750 + let size = dir_size(&entry_path);
751 + let revisions = list_revisions(&entry_path);
752 +
753 + models.push(LocalHfModel {
754 + model_id,
755 + org,
756 + name,
757 + size_bytes: size,
758 + size_display: format_size(size),
759 + path: entry_path.to_string_lossy().to_string(),
760 + revisions,
761 + });
762 + }
763 +
764 + // Only keep models that the inference engine actually supports.
765 + models.retain(|m| SUPPORTED_MODELS.iter().any(|&s| s == m.model_id));
766 +
767 + // Sort by model_id for a stable, predictable order.
768 + models.sort_by(|a, b| a.model_id.to_lowercase().cmp(&b.model_id.to_lowercase()));
769 +
770 + let total_size_bytes: u64 = models.iter().map(|m| m.size_bytes).sum();
771 +
772 + debug!(
773 + "Found {} local HuggingFace model(s), total size: {}",
774 + models.len(),
775 + format_size(total_size_bytes)
776 + );
777 +
778 + LocalHfModelsResponse {
779 + models,
780 + cache_path: cache_path_str,
781 + total_size_bytes,
782 + total_size_display: format_size(total_size_bytes),
783 + }
784 +}
785 +
786 +/// Delete a locally cached HuggingFace model by removing its directory from
787 +/// the hub cache.
788 +///
789 +/// `model_id` should be the full identifier, e.g. "bartowski/Qwen2.5-1.5B-Instruct-GGUF".
790 +/// Returns `Ok(())` on success or an error string describing what went wrong.
791 +pub fn delete_local_hf_model(model_id: String) -> Result<(), String> {
792 + let cache_dir = hf_cache_dir().ok_or("HuggingFace cache directory not found.".to_string())?;
793 +
794 + // Convert model_id (e.g. "org/name") to the cache directory name ("models--org--name")
795 + let dir_name = format!("models--{}", model_id.replace('/', "--"));
796 + let model_path = cache_dir.join(&dir_name);
797 +
798 + if !model_path.exists() {
799 + return Err(format!(
800 + "Model cache directory not found: {}",
801 + model_path.display()
802 + ));
803 + }
804 +
805 + if !model_path.is_dir() {
806 + return Err(format!(
807 + "Expected a directory but found a file: {}",
808 + model_path.display()
809 + ));
810 + }
811 +
812 + debug!(
813 + "Deleting local HuggingFace model: {} at {:?}",
814 + model_id, model_path
815 + );
816 +
817 + fs::remove_dir_all(&model_path).map_err(|e| {
818 + let msg = format!("Failed to delete model {}: {}", model_id, e);
819 + warn!("{}", msg);
820 + msg
821 + })?;
822 +
823 + debug!("Successfully deleted local model: {}", model_id);
824 + Ok(())
825 +}
826 +
827 +/// Return the full list of models the inference engine supports, together with
828 +/// flags indicating whether each one is fully downloaded, partially downloaded,
829 +/// or not present at all.
830 +pub fn list_supported_hf_models() -> SupportedHfModelsResponse {
831 + let models = SUPPORTED_MODEL_INFO
832 + .iter()
833 + .map(|info| {
834 + let cache_path = model_cache_path(info.id);
835 + let local_size = cache_path
836 + .as_ref()
837 + .filter(|p| p.exists())
838 + .map(|p| dir_size(p))
839 + .unwrap_or(0);
840 +
841 + let has_cache = local_size > 0;
842 + let is_complete = info.expected_size_bytes > 0
843 + && local_size as f64
844 + >= info.expected_size_bytes as f64 * DOWNLOAD_COMPLETE_THRESHOLD;
845 +
846 + SupportedHfModel {
847 + model_id: info.id.to_string(),
848 + name: info.name.to_string(),
849 + org: info.org.to_string(),
850 + description: info.description.to_string(),
851 + is_downloaded: has_cache && is_complete,
852 + is_incomplete: has_cache && !is_complete,
853 + local_size_bytes: local_size,
854 + local_size_display: format_size(local_size),
855 + expected_size_bytes: info.expected_size_bytes,
856 + expected_size_display: format_size(info.expected_size_bytes),
857 + }
858 + })
859 + .collect();
860 +
861 + SupportedHfModelsResponse { models }
862 +}
863 +
864 +// ---------------------------------------------------------------------------
865 +// Model download — platform-specific implementations
866 +// ---------------------------------------------------------------------------
867 +//
868 +// The download function accepts a generic progress callback so that callers
869 +// can forward progress to any event system without coupling to this crate.
870 +//
871 +// Callback signature: `Fn(ModelDownloadProgress) + Send + Sync + Clone + 'static`
872 +// - Called every ~2 s from a native monitor thread.
873 +// - Called once with `done: true` after the build completes.
874 +
875 +// ── Apple (macOS + iOS): Metal-accelerated, all model types ─────────────────
876 +
877 +#[cfg(any(target_os = "macos", target_os = "ios"))]
878 +pub async fn download_model<F>(
879 + model_id: String,
880 + on_progress: F,
881 + app_data_dir: Option<PathBuf>,
882 +) -> Result<(), String>
883 +where
884 + F: Fn(ModelDownloadProgress) + Send + Sync + Clone + 'static,
885 +{
886 + use crate::inference::models::{
887 + BARTOWSKI_QWEN25_1_5B_INSTRUCT_GGUF, BARTOWSKI_QWEN25_3B_INSTRUCT_GGUF,
888 + QWEN25_1_5B_GGUF_FILE, QWEN25_3B_GGUF_FILE,
889 + };
890 + use crate::inference::token::hf_token_source;
891 + use mistralrs::GgufModelBuilder;
892 + use std::sync::{
893 + atomic::{AtomicBool, Ordering},
894 + Arc,
895 + };
896 + use std::time::Duration;
897 +
898 + // When app_data_dir is provided, seed the HF env vars so that hf-hub,
899 + // mistral.rs, and hf_cache_dir() all write inside the app's writable
900 + // container. This is required on every sandboxed platform:
901 + // - iOS: ~/.cache is outside the container (os error 1).
902 + // - macOS App Store: the app is sandboxed, ~/.cache is inaccessible.
903 + // It is harmless on non-sandboxed desktop builds (the cache simply
904 + // lives under the app data dir instead of ~/.cache).
905 + if let Some(ref data_dir) = app_data_dir {
906 + let hf_home = data_dir.join("huggingface");
907 + let hf_hub_cache = hf_home.join("hub");
908 + fs::create_dir_all(&hf_hub_cache)
909 + .map_err(|e| format!("Cannot create HF cache dir: {e}"))?;
910 + std::env::set_var("HF_HUB_CACHE", &hf_hub_cache);
911 + std::env::set_var("HF_HOME", &hf_home);
912 + info!(
913 + "HF hub cache resolved to app data path: {}",
914 + hf_hub_cache.display()
915 + );
916 + }
917 +
918 + if !SUPPORTED_MODELS.contains(&model_id.as_str()) {
919 + return Err(format!("Model {} is not supported.", model_id));
920 + }
921 +
922 + let expected_size = SUPPORTED_MODEL_INFO
923 + .iter()
924 + .find(|m| m.id == model_id)
925 + .map(|m| m.expected_size_bytes)
926 + .unwrap_or(0);
927 +
928 + info!(
929 + "Starting download for model: {} (expected ~{})",
930 + model_id,
931 + format_size(expected_size)
932 + );
933 +
934 + clean_stale_lock_files(&model_id);
935 + repair_hf_cache_symlinks(&model_id);
936 + ensure_model_cache_dir(&model_id);
937 +
938 + // Emit an initial progress event based on the actual cache size so that
939 + // resumed downloads don't visually restart from 0%.
940 + let initial_bytes = model_cache_path(&model_id)
941 + .filter(|p| p.exists())
942 + .map(|p| dir_size(&p))
943 + .unwrap_or(0);
944 + on_progress(make_progress(
945 + &model_id,
946 + initial_bytes,
947 + expected_size,
948 + false,
949 + ));
950 +
951 + // Shared flag so the monitor task knows when to stop.
952 + let finished = Arc::new(AtomicBool::new(false));
953 + let finished_clone = Arc::clone(&finished);
954 + let monitor_model_id = model_id.clone();
955 + let monitor_cb = on_progress.clone();
956 +
957 + // Spawn a native OS thread to poll the cache directory size every 2 seconds.
958 + // We use std::thread instead of tokio::spawn because the model builder's
959 + // .build().await internally calls the synchronous hf-hub API (api.get()),
960 + // which blocks the tokio worker thread. This starves any tokio-spawned
961 + // monitor tasks, preventing progress updates. A native thread runs
962 + // independently of the tokio runtime and is immune to this starvation.
963 + let monitor_handle = std::thread::spawn(move || loop {
964 + std::thread::sleep(Duration::from_secs(2));
965 +
966 + if finished_clone.load(Ordering::Relaxed) {
967 + break;
968 + }
969 +
970 + let current_bytes = model_cache_path(&monitor_model_id)
971 + .filter(|p| p.exists())
972 + .map(|p| dir_size(&p))
973 + .unwrap_or(0);
974 +
975 + monitor_cb(make_progress(
976 + &monitor_model_id,
977 + current_bytes,
978 + expected_size,
979 + false,
980 + ));
981 + });
982 +
983 + // Build (and immediately drop) the model using the GGUF chat builder.
984 + // Building triggers the HuggingFace hub download.
985 + let result: Result<mistralrs::Model, anyhow::Error> = match model_id.as_str() {
986 + BARTOWSKI_QWEN25_1_5B_INSTRUCT_GGUF => {
987 + GgufModelBuilder::new(&model_id, vec![QWEN25_1_5B_GGUF_FILE])
988 + .with_token_source(hf_token_source())
989 + .build()
990 + .await
991 + }
992 + BARTOWSKI_QWEN25_3B_INSTRUCT_GGUF => {
993 + GgufModelBuilder::new(&model_id, vec![QWEN25_3B_GGUF_FILE])
994 + .with_token_source(hf_token_source())
995 + .build()
996 + .await
997 + }
998 + _ => Err(anyhow::anyhow!(
999 + "Model '{}' is not part of the chat-only release.",
1000 + model_id
1001 + )),
1002 + };
1003 +
1004 + finished.store(true, Ordering::Relaxed);
1005 + let _ = monitor_handle.join();
1006 +
1007 + match result {
1008 + Ok(_model) => {
1009 + let final_bytes = model_cache_path(&model_id)
1010 + .filter(|p| p.exists())
1011 + .map(|p| dir_size(&p))
1012 + .unwrap_or(expected_size);
1013 + on_progress(make_progress(&model_id, final_bytes, final_bytes, true));
1014 + info!("Successfully downloaded model: {}", model_id);
1015 + Ok(())
1016 + }
1017 + Err(e) => {
1018 + let msg = format!("Failed to download model {}: {}", model_id, e);
1019 + warn!("{}", msg);
1020 + Err(msg)
1021 + }
1022 + }
1023 +}
1024 +
1025 +// ── Android: GGUF-only download ───────────────────────────────────────────
1026 +//
1027 +// On Android there is no Metal GPU, so only pre-quantized GGUF models
1028 +// (CPU inference via candle) are supported. We mirror the Apple
1029 +// implementation but restrict to GGUF presets and skip all Metal / ISQ
1030 +// model builders.
1031 +//
1032 +// `app_data_dir` is **required** on Android: `dirs::home_dir()` returns
1033 +// `None` inside the Android sandbox, which causes `hf_hub::Cache::default()`
1034 +// to panic. The caller resolves the app data directory and passes it here.
1035 +
1036 +#[cfg(target_os = "android")]
1037 +pub async fn download_model<F>(
1038 + model_id: String,
1039 + on_progress: F,
1040 + app_data_dir: Option<PathBuf>,
1041 +) -> Result<(), String>
1042 +where
1043 + F: Fn(ModelDownloadProgress) + Send + Sync + Clone + 'static,
1044 +{
1045 + use crate::inference::models::{
1046 + BARTOWSKI_QWEN25_1_5B_INSTRUCT_GGUF, BARTOWSKI_QWEN25_3B_INSTRUCT_GGUF,
1047 + QWEN25_1_5B_GGUF_FILE, QWEN25_1_5B_TOK_MODEL_ID, QWEN25_3B_GGUF_FILE,
1048 + QWEN25_3B_TOK_MODEL_ID,
1049 + };
1050 + use crate::inference::token::hf_token_source;
1051 + use hf_hub::Cache;
1052 + use mistralrs::GgufModelBuilder;
1053 + use mistralrs_core::GLOBAL_HF_CACHE;
1054 + use std::sync::{
1055 + atomic::{AtomicBool, Ordering},
1056 + Arc,
1057 + };
1058 + use std::time::Duration;
1059 +
1060 + let (gguf_file, tok_model_id) = match model_id.as_str() {
1061 + id if id == BARTOWSKI_QWEN25_1_5B_INSTRUCT_GGUF => {
1062 + (QWEN25_1_5B_GGUF_FILE, QWEN25_1_5B_TOK_MODEL_ID)
1063 + }
1064 + id if id == BARTOWSKI_QWEN25_3B_INSTRUCT_GGUF => {
1065 + (QWEN25_3B_GGUF_FILE, QWEN25_3B_TOK_MODEL_ID)
1066 + }
1067 + _ => {
1068 + return Err(format!(
1069 + "Model '{}' is not supported on Android. \
1070 + Only Qwen 2.5 1.5B or 3B (GGUF) are available.",
1071 + model_id
1072 + ));
1073 + }
1074 + };
1075 +
1076 + // ── Resolve the HF cache path ─────────────────────────────────────────
1077 + //
1078 + // `app_data_dir` is required on Android — `dirs::home_dir()` returns None
1079 + // in the Android sandbox and causes Cache::default() to panic.
1080 + let resolved_app_data = app_data_dir
1081 + .ok_or_else(|| "app_data_dir is required on Android for HF cache resolution".to_string())?;
1082 +
1083 + let hf_home = resolved_app_data.join("huggingface");
1084 + let hf_hub_cache = hf_home.join("hub");
1085 +
1086 + fs::create_dir_all(&hf_hub_cache)
1087 + .map_err(|e| format!("Failed to create HF hub cache dir: {e}"))?;
1088 +
1089 + // Seed GLOBAL_HF_CACHE so get_paths_gguf! never falls back to Cache::default().
1090 + GLOBAL_HF_CACHE.get_or_init(|| Cache::new(hf_hub_cache.clone()));
1091 +
1092 + // Keep env vars in sync so hf_hub_cache_dir() and hf_cache_dir() agree.
1093 + std::env::set_var("HF_HUB_CACHE", &hf_hub_cache);
1094 + std::env::set_var("HF_HOME", &hf_home);
1095 +
1096 + info!(
1097 + "Android HF hub cache resolved to: {}",
1098 + hf_hub_cache.display()
1099 + );
1100 +
1101 + let expected_size = SUPPORTED_MODEL_INFO
1102 + .iter()
1103 + .find(|m| m.id == model_id)
1104 + .map(|m| m.expected_size_bytes)
1105 + .unwrap_or(0);
1106 +
1107 + info!(
1108 + "Starting Android download for model: {} (expected ~{})",
1109 + model_id,
1110 + format_size(expected_size)
1111 + );
1112 +
1113 + clean_stale_lock_files(&model_id);
1114 + ensure_model_cache_dir(&model_id);
1115 +
1116 + let initial_bytes = model_cache_path(&model_id)
1117 + .filter(|p| p.exists())
1118 + .map(|p| dir_size(&p))
1119 + .unwrap_or(0);
1120 + on_progress(make_progress(
1121 + &model_id,
1122 + initial_bytes,
1123 + expected_size,
1124 + false,
1125 + ));
1126 +
1127 + // Native monitor thread — same reasoning as the Apple implementation.
1128 + let finished = Arc::new(AtomicBool::new(false));
1129 + let finished_clone = Arc::clone(&finished);
1130 + let monitor_model_id = model_id.clone();
1131 + let monitor_cb = on_progress.clone();
1132 +
1133 + let monitor_handle = std::thread::spawn(move || loop {
1134 + std::thread::sleep(Duration::from_secs(2));
1135 + if finished_clone.load(Ordering::Relaxed) {
1136 + break;
1137 + }
1138 + let current_bytes = model_cache_path(&monitor_model_id)
1139 + .filter(|p| p.exists())
1140 + .map(|p| dir_size(&p))
1141 + .unwrap_or(0);
1142 + monitor_cb(make_progress(
1143 + &monitor_model_id,
1144 + current_bytes,
1145 + expected_size,
1146 + false,
1147 + ));
1148 + });
1149 +
1150 + let result = GgufModelBuilder::new(&model_id, vec![gguf_file])
1151 + .with_tok_model_id(tok_model_id)
1152 + .with_token_source(hf_token_source())
1153 + .with_logging()
1154 + .build()
1155 + .await;
1156 +
1157 + finished.store(true, Ordering::Relaxed);
1158 + let _ = monitor_handle.join();
1159 +
1160 + match result {
1161 + Ok(_model) => {
1162 + let final_bytes = model_cache_path(&model_id)
1163 + .filter(|p| p.exists())
1164 + .map(|p| dir_size(&p))
1165 + .unwrap_or(expected_size);
1166 + on_progress(make_progress(&model_id, final_bytes, final_bytes, true));
1167 + info!("Successfully downloaded model on Android: {}", model_id);
1168 + Ok(())
1169 + }
1170 + Err(e) => {
1171 + let msg = format!("Failed to download model {}: {}", model_id, e);
1172 + warn!("{}", msg);
1173 + Err(msg)
1174 + }
1175 + }
1176 +}
1177 +
1178 +// ── Other platforms: no-op ────────────────────────────────────────────────
1179 +
1180 +#[cfg(not(any(target_os = "macos", target_os = "ios", target_os = "android")))]
1181 +pub async fn download_model<F>(
1182 + model_id: String,
1183 + _on_progress: F,
1184 + _app_data_dir: Option<PathBuf>,
1185 +) -> Result<(), String>
1186 +where
1187 + F: Fn(ModelDownloadProgress) + Send + Sync + Clone + 'static,
1188 +{
1189 + debug!(
1190 + "download_model: not supported on this platform, ignoring request for {}.",
1191 + model_id
1192 + );
1193 + Err(format!(
1194 + "Model downloads are not supported on this platform (requested: {}).",
1195 + model_id
1196 + ))
1197 +}
src/inference/engine.rs new
+1213
@@ -0,0 +1,1213 @@
1 +//! On-device LLM chat inference engine powered by [mistral.rs](https://github.com/EricLBuehler/mistral.rs).
2 +//!
3 +//! `ChatEngine` provides a high-level, framework-agnostic API for:
4 +//!
5 +//! - Loading GGUF-quantized models (the primary format for mobile/desktop)
6 +//! - Multi-turn chat with conversation history management
7 +//! - Both blocking (non-streaming) and streaming inference
8 +//! - Model lifecycle management (load / unload / status)
9 +//!
10 +//! # Architecture
11 +//!
12 +//! ```text
13 +//! ┌──────────────────────────────────────────┐
14 +//! │ App / UniFFI binding / test │
15 +//! └──────────────┬───────────────────────────┘
16 +//! │ (framework-agnostic API)
17 +//! ▼
18 +//! ┌──────────────────────────────────────────┐
19 +//! │ ChatEngine │
20 +//! │ ┌────────────────────────────────────┐ │
21 +//! │ │ Mutex<Option<LoadedModel>> │ │
22 +//! │ │ · Arc<mistralrs::Model> │ │
23 +//! │ │ · config (GgufModelConfig) │ │
24 +//! │ │ · history (Vec<ChatMessage>) │ │
25 +//! │ │ · sampling (SamplingConfig) │ │
26 +//! │ └────────────────────────────────────┘ │
27 +//! └──────────────┬───────────────────────────┘
28 +//! │ (delegates to)
29 +//! ▼
30 +//! ┌──────────────────────────────────────────┐
31 +//! │ mistralrs::Model │
32 +//! │ (wraps Arc<MistralRs> engine thread) │
33 +//! └──────────────────────────────────────────┘
34 +//! ```
35 +//!
36 +//! # Example
37 +//!
38 +//! ```rust,ignore
39 +//! use onde::inference::engine::ChatEngine;
40 +//! use onde::inference::types::*;
41 +//!
42 +//! let engine = ChatEngine::new();
43 +//!
44 +//! let config = GgufModelConfig {
45 +//! model_id: "bartowski/Qwen2.5-1.5B-Instruct-GGUF".into(),
46 +//! files: vec!["Qwen2.5-1.5B-Instruct-Q4_K_M.gguf".into()],
47 +//! tok_model_id: None,
48 +//! display_name: "Qwen 2.5 1.5B".into(),
49 +//! approx_memory: "~941 MB".into(),
50 +//! };
51 +//!
52 +//! engine.load_gguf_model(config, None).await?;
53 +//! engine.set_system_prompt("You are a helpful assistant.");
54 +//!
55 +//! let result = engine.send_message("Hello!").await?;
56 +//! println!("{}", result.text);
57 +//! ```
58 +
59 +#[cfg(any(
60 + target_os = "macos",
61 + target_os = "ios",
62 + target_os = "tvos",
63 + target_os = "windows",
64 + target_os = "linux",
65 + target_os = "android"
66 +))]
67 +use std::sync::Arc;
68 +
69 +#[cfg(any(
70 + target_os = "macos",
71 + target_os = "ios",
72 + target_os = "tvos",
73 + target_os = "windows",
74 + target_os = "linux",
75 + target_os = "android"
76 +))]
77 +use tokio::sync::Mutex;
78 +
79 +use super::types::*;
80 +
81 +// ── Platform-gated mistralrs imports ─────────────────────────────────────────
82 +
83 +#[cfg(any(
84 + target_os = "macos",
85 + target_os = "ios",
86 + target_os = "tvos",
87 + target_os = "windows",
88 + target_os = "linux",
89 + target_os = "android"
90 +))]
91 +use mistralrs::{
92 + GgufModelBuilder, IsqBits, Model, RequestBuilder, TextMessageRole, TextModelBuilder,
93 +};
94 +
95 +// ── Internals ────────────────────────────────────────────────────────────────
96 +
97 +/// How a model was loaded — carries the data needed for status reporting.
98 +#[cfg(any(
99 + target_os = "macos",
100 + target_os = "ios",
101 + target_os = "tvos",
102 + target_os = "windows",
103 + target_os = "linux",
104 + target_os = "android"
105 +))]
106 +enum LoadedModelConfig {
107 + Gguf(GgufModelConfig),
108 + Isq(IsqModelConfig),
109 +}
110 +
111 +#[cfg(any(
112 + target_os = "macos",
113 + target_os = "ios",
114 + target_os = "tvos",
115 + target_os = "windows",
116 + target_os = "linux",
117 + target_os = "android"
118 +))]
119 +impl LoadedModelConfig {
120 + fn display_name(&self) -> &str {
121 + match self {
122 + LoadedModelConfig::Gguf(c) => &c.display_name,
123 + LoadedModelConfig::Isq(c) => &c.display_name,
124 + }
125 + }
126 +
127 + fn approx_memory(&self) -> &str {
128 + match self {
129 + LoadedModelConfig::Gguf(c) => &c.approx_memory,
130 + LoadedModelConfig::Isq(c) => &c.approx_memory,
131 + }
132 + }
133 +}
134 +
135 +/// State of a loaded model held inside the engine's mutex.
136 +#[cfg(any(
137 + target_os = "macos",
138 + target_os = "ios",
139 + target_os = "tvos",
140 + target_os = "windows",
141 + target_os = "linux",
142 + target_os = "android"
143 +))]
144 +struct LoadedModel {
145 + /// The mistral.rs model handle. `Model` wraps `Arc<MistralRs>` so
146 + /// cloning is a cheap pointer copy — safe to snapshot before inference
147 + /// while releasing the mutex.
148 + model: Arc<Model>,
149 + /// Configuration used to load this model (kept for status reporting).
150 + config: LoadedModelConfig,
151 + /// Conversation history (system prompt is stored separately).
152 + history: Vec<ChatMessage>,
153 + /// System prompt prepended to every request.
154 + system_prompt: Option<String>,
155 + /// Sampling parameters for generation.
156 + sampling: SamplingConfig,
157 +}
158 +
159 +// ═════════════════════════════════════════════════════════════════════════════
160 +// ChatEngine — the public API
161 +// ═════════════════════════════════════════════════════════════════════════════
162 +
163 +/// A reusable on-device LLM chat inference engine.
164 +///
165 +/// Thread-safe (`Send + Sync`) — safe to store in a `once_cell::sync::Lazy`,
166 +/// `tokio::sync::OnceCell`, or shared application state.
167 +///
168 +/// All mutating operations acquire an internal `tokio::sync::Mutex`. The
169 +/// mutex is released *before* the actual (potentially slow) model inference
170 +/// runs, so other tasks can still query status or history while generation
171 +/// is in progress.
172 +#[cfg(any(
173 + target_os = "macos",
174 + target_os = "ios",
175 + target_os = "tvos",
176 + target_os = "windows",
177 + target_os = "linux",
178 + target_os = "android"
179 +))]
180 +pub struct ChatEngine {
181 + inner: Mutex<Option<LoadedModel>>,
182 +}
183 +
184 +#[cfg(any(
185 + target_os = "macos",
186 + target_os = "ios",
187 + target_os = "tvos",
188 + target_os = "windows",
189 + target_os = "linux",
190 + target_os = "android"
191 +))]
192 +impl ChatEngine {
193 + // ── Construction ─────────────────────────────────────────────────────
194 +
195 + /// Create a new engine with no model loaded.
196 + pub fn new() -> Self {
197 + Self {
198 + inner: Mutex::new(None),
199 + }
200 + }
201 +
202 + // ── Model lifecycle ──────────────────────────────────────────────────
203 +
204 + /// Load a GGUF model into the engine.
205 + ///
206 + /// If a model is already loaded it will be unloaded first (the previous
207 + /// `Model` is dropped, which terminates its engine thread).
208 + ///
209 + /// # Arguments
210 + ///
211 + /// * `config` — Which model to load (repo ID, filename, etc.).
212 + /// * `system_prompt` — Optional system prompt to prepend to every request.
213 + /// * `sampling` — Sampling parameters; pass `None` for platform-aware
214 + /// defaults (mobile gets [`SamplingConfig::mobile`],
215 + /// desktop gets [`SamplingConfig::default`]).
216 + ///
217 + /// # Errors
218 + ///
219 + /// Returns [`InferenceError::ModelBuild`] if the model fails to download
220 + /// or load.
221 + pub async fn load_gguf_model(
222 + &self,
223 + config: GgufModelConfig,
224 + system_prompt: Option<String>,
225 + sampling: Option<SamplingConfig>,
226 + ) -> Result<std::time::Duration, InferenceError> {
227 + use log::info;
228 + use std::time::Instant;
229 +
230 + info!(
231 + "ChatEngine: loading GGUF model {} (files: {:?})",
232 + config.model_id, config.files
233 + );
234 +
235 + // ── Android: seed GLOBAL_HF_CACHE ────────────────────────────────
236 + //
237 + // On Android, `dirs::home_dir()` returns `None`, so
238 + // `hf_hub::Cache::default()` panics with "Cache directory cannot be
239 + // found". The `get_paths_gguf!` macro in mistralrs falls back to
240 + // `Cache::default()` when `GLOBAL_HF_CACHE` (a OnceLock) is empty.
241 + //
242 + // `HF_HOME` is always set by `setup_hf_home()` before any command
243 + // runs, so we read it here. `get_or_init` is a no-op if already
244 + // seeded by `download_model` — safe to call repeatedly.
245 + #[cfg(target_os = "android")]
246 + {
247 + let hf_home = std::env::var("HF_HOME")
248 + .map(std::path::PathBuf::from)
249 + .map_err(|_| InferenceError::ModelBuild {
250 + reason: "HF_HOME is not set — cannot initialise HF cache on Android. \
251 + Ensure setup_hf_home() ran before load_gguf_model()."
252 + .to_string(),
253 + })?;
254 + let hf_hub_cache = hf_home.join("hub");
255 + if let Err(e) = std::fs::create_dir_all(&hf_hub_cache) {
256 + log::warn!(
257 + "ChatEngine: could not create HF hub cache dir {}: {}",
258 + hf_hub_cache.display(),
259 + e
260 + );
261 + }
262 + mistralrs_core::GLOBAL_HF_CACHE
263 + .get_or_init(|| hf_hub::Cache::new(hf_hub_cache.clone()));
264 + std::env::set_var("HF_HUB_CACHE", &hf_hub_cache);
265 + log::debug!(
266 + "ChatEngine: GLOBAL_HF_CACHE seeded at {}",
267 + hf_hub_cache.display()
268 + );
269 + }
270 +
271 + // Clean up stale HF cache artefacts before loading.
272 + crate::hf_cache::clean_stale_lock_files(&config.model_id);
273 + crate::hf_cache::repair_hf_cache_symlinks(&config.model_id);
274 +
275 + let start = Instant::now();
276 +
277 + let mut builder = GgufModelBuilder::new(&config.model_id, config.files.clone())
278 + .with_token_source(super::token::hf_token_source())
279 + .with_logging();
280 +
281 + // On Android the GGUF embedded tokenizer is not supported by the
282 + // candle backend — an explicit tok_model_id is required.
283 + if let Some(ref tok_id) = config.tok_model_id {
284 + builder = builder.with_tok_model_id(tok_id);
285 + }
286 +
287 + let model = builder
288 + .build()
289 + .await
290 + .map_err(|e| InferenceError::ModelBuild {
291 + reason: format!("Failed to build {} model: {}", config.display_name, e),
292 + })?;
293 +
294 + let elapsed = start.elapsed();
295 +
296 + let sampling = sampling.unwrap_or_else(|| {
297 + if cfg!(any(
298 + target_os = "ios",
299 + target_os = "tvos",
300 + target_os = "android"
301 + )) {
302 + SamplingConfig::mobile()
303 + } else {
304 + SamplingConfig::default()
305 + }
306 + });
307 +
308 + info!(
309 + "ChatEngine: model {} loaded in {} (sampling: temp={:?}, max_tokens={:?})",
310 + config.display_name,
311 + format_duration(elapsed),
312 + sampling.temperature,
313 + sampling.max_tokens,
314 + );
315 +
316 + let mut guard = self.inner.lock().await;
317 + *guard = Some(LoadedModel {
318 + model: Arc::new(model),
319 + config: LoadedModelConfig::Gguf(config),
320 + history: Vec::new(),
321 + system_prompt,
322 + sampling,
323 + });
324 +
325 + Ok(elapsed)
326 + }
327 +
328 + /// Load an ISQ (in-situ quantised) model into the engine.
329 + ///
330 + /// Unlike [`load_gguf_model`], this downloads the full-precision safetensors
331 + /// from HuggingFace and quantises the weights in-situ on Metal (macOS only).
332 + /// The quantisation happens once at load time; subsequent inference uses the
333 + /// compressed weights.
334 + ///
335 + /// # macOS only
336 + ///
337 + /// ISQ with Metal requires the `metal` feature on mistral.rs and is therefore
338 + /// restricted to macOS at the Rust level. Other platforms should continue to
339 + /// use [`load_gguf_model`] with pre-quantised GGUF files.
340 + ///
341 + /// # Arguments
342 + ///
343 + /// * `config` — Which model to load and with how many ISQ bits.
344 + /// * `system_prompt` — Optional system prompt prepended to every request.
345 + /// * `sampling` — Sampling parameters; `None` uses [`SamplingConfig::default`].
346 + ///
347 + /// # Errors
348 + ///
349 + /// Returns [`InferenceError::ModelBuild`] if the model fails to download
350 + /// or quantise.
351 + #[cfg(target_os = "macos")]
352 + pub async fn load_isq_model(
353 + &self,
354 + config: IsqModelConfig,
355 + system_prompt: Option<String>,
356 + sampling: Option<SamplingConfig>,
357 + ) -> Result<std::time::Duration, InferenceError> {
358 + use log::info;
359 + use std::time::Instant;
360 +
361 + info!(
362 + "ChatEngine: loading ISQ model {} (bits={})",
363 + config.model_id, config.isq_bits
364 + );
365 +
366 + // Clean up stale HF cache artefacts before loading.
367 + crate::hf_cache::clean_stale_lock_files(&config.model_id);
368 + crate::hf_cache::repair_hf_cache_symlinks(&config.model_id);
369 +
370 + let start = Instant::now();
371 +
372 + // Choose ISQ bit width.
373 + let isq_bits = match config.isq_bits {
374 + 8 => IsqBits::Eight,
375 + _ => IsqBits::Four, // default to 4-bit
376 + };
377 +
378 + let model = TextModelBuilder::new(&config.model_id)
379 + .with_token_source(super::token::hf_token_source())
380 + .with_auto_isq(isq_bits)
381 + .with_logging()
382 + .build()
383 + .await
384 + .map_err(|e| InferenceError::ModelBuild {
385 + reason: format!("Failed to build ISQ model {}: {}", config.display_name, e),
386 + })?;
387 +
388 + let elapsed = start.elapsed();
389 +
390 + let sampling = sampling.unwrap_or_else(SamplingConfig::default);
391 +
392 + info!(
393 + "ChatEngine: ISQ model {} loaded in {} (sampling: temp={:?}, max_tokens={:?})",
394 + config.display_name,
395 + format_duration(elapsed),
396 + sampling.temperature,
397 + sampling.max_tokens,
398 + );
399 +
400 + let mut guard = self.inner.lock().await;
401 + *guard = Some(LoadedModel {
402 + model: Arc::new(model),
403 + config: LoadedModelConfig::Isq(config),
404 + history: Vec::new(),
405 + system_prompt,
406 + sampling,
407 + });
408 +
409 + Ok(elapsed)
410 + }
411 +
412 + /// Unload the current model, freeing all memory.
413 + ///
414 + /// Dropping the `Model` sends a `Terminate` message to the mistral.rs
415 + /// engine thread, which tears down the KV cache, activations, and model
416 + /// weights.
417 + ///
418 + /// Returns the display name of the model that was unloaded, or `None`
419 + /// if no model was loaded.
420 + pub async fn unload_model(&self) -> Option<String> {
421 + let mut guard = self.inner.lock().await;
422 + if let Some(loaded) = guard.take() {
423 + let name = loaded.config.display_name().to_string();
424 + log::info!("ChatEngine: unloading model {}", name);
425 + // `loaded` is dropped here → Model → Arc<MistralRs> (last ref)
426 + // → MistralRs::drop() → engine thread termination.
427 + Some(name)
428 + } else {
429 + log::debug!("ChatEngine: unload_model called but no model was loaded.");
430 + None
431 + }
432 + }
433 +
434 + /// Check whether a model is currently loaded.
435 + pub async fn is_loaded(&self) -> bool {
436 + self.inner.lock().await.is_some()
437 + }
438 +
439 + /// Get a snapshot of the engine's current state.
440 + pub async fn info(&self) -> EngineInfo {
441 + let guard = self.inner.lock().await;
442 + match guard.as_ref() {
443 + Some(loaded) => EngineInfo {
444 + status: EngineStatus::Ready,
445 + model_name: Some(loaded.config.display_name().to_string()),
446 + approx_memory: Some(loaded.config.approx_memory().to_string()),
447 + history_length: loaded.history.len() as u64,
448 + },
449 + None => EngineInfo {
450 + status: EngineStatus::Unloaded,
451 + model_name: None,
452 + approx_memory: None,
453 + history_length: 0u64,
454 + },
455 + }
456 + }
457 +
458 + // ── System prompt ────────────────────────────────────────────────────
459 +
460 + /// Set or replace the system prompt.
461 + ///
462 + /// The system prompt is prepended to every inference request (it is NOT
463 + /// stored in the conversation history).
464 + pub async fn set_system_prompt(&self, prompt: impl Into<String>) {
465 + if let Some(loaded) = self.inner.lock().await.as_mut() {
466 + loaded.system_prompt = Some(prompt.into());
467 + }
468 + }
469 +
470 + /// Clear the system prompt.
471 + pub async fn clear_system_prompt(&self) {
472 + if let Some(loaded) = self.inner.lock().await.as_mut() {
473 + loaded.system_prompt = None;
474 + }
475 + }
476 +
477 + // ── Sampling configuration ───────────────────────────────────────────
478 +
479 + /// Replace the sampling configuration.
480 + pub async fn set_sampling(&self, sampling: SamplingConfig) {
481 + if let Some(loaded) = self.inner.lock().await.as_mut() {
482 + loaded.sampling = sampling;
483 + }
484 + }
485 +
486 + // ── Conversation history ─────────────────────────────────────────────
487 +
488 + /// Get a clone of the full conversation history.
489 + pub async fn history(&self) -> Vec<ChatMessage> {
490 + let guard = self.inner.lock().await;
491 + match guard.as_ref() {
492 + Some(loaded) => loaded.history.clone(),
493 + None => Vec::new(),
494 + }
495 + }
496 +
497 + /// Clear the conversation history but keep the model loaded.
498 + ///
499 + /// Returns the number of turns that were removed.
500 + pub async fn clear_history(&self) -> usize {
501 + let mut guard = self.inner.lock().await;
502 + match guard.as_mut() {
503 + Some(loaded) => {
504 + let count = loaded.history.len();
505 + loaded.history.clear();
506 + log::info!("ChatEngine: cleared {} history turns.", count);
507 + count
508 + }
509 + None => 0,
510 + }
511 + }
512 +
513 + /// Append a message to history without running inference.
514 + ///
515 + /// Useful for restoring a saved conversation or injecting context.
516 + pub async fn push_history(&self, message: ChatMessage) {
517 + if let Some(loaded) = self.inner.lock().await.as_mut() {
518 + loaded.history.push(message);
519 + }
520 + }
521 +
522 + // ── Non-streaming inference ──────────────────────────────────────────
523 +
524 + /// Send a user message and receive a complete assistant reply.
525 + ///
526 + /// The user message and assistant reply are automatically appended to
527 + /// the conversation history on success.
528 + ///
529 + /// # Errors
530 + ///
531 + /// - [`InferenceError::NoModelLoaded`] if no model is loaded.
532 + /// - [`InferenceError::Inference`] if the model fails to generate.
533 + pub async fn send_message(
534 + &self,
535 + user_message: impl Into<String>,
536 + ) -> Result<InferenceResult, InferenceError> {
537 + let user_message = user_message.into();
538 +
539 + // ── 1. Snapshot model handle + build request, then release lock ──
540 + let (model, request) = {
541 + let guard = self.inner.lock().await;
542 + let loaded = guard.as_ref().ok_or(InferenceError::NoModelLoaded)?;
543 + let request = self::build_request(loaded, &user_message);
544 + (loaded.model.clone(), request)
545 + }; // ← mutex released before inference
546 +
547 + log::info!(
548 + "ChatEngine: inference START — message: \"{}\"",
549 + truncate_for_log(&user_message, 100)
550 + );
551 +
552 + // ── 2. Run inference (potentially slow — mutex is NOT held) ──────
553 + let start = std::time::Instant::now();
554 + let response =
555 + model
556 + .send_chat_request(request)
557 + .await
558 + .map_err(|e| InferenceError::Inference {
559 + reason: e.to_string(),
560 + })?;
561 + let elapsed = start.elapsed();
562 +
563 + let reply = response.choices[0]
564 + .message
565 + .content
566 + .as_ref()
567 + .map(|c: &String| c.trim().to_string())
568 + .unwrap_or_else(|| "(empty response)".to_string());
569 +
570 + let finish_reason = response.choices[0].finish_reason.clone();
571 +
572 + log::info!(
573 + "ChatEngine: inference END — {} — reply: \"{}\"",
574 + format_duration(elapsed),
575 + truncate_for_log(&reply, 100)
576 + );
577 +
578 + // ── 3. Persist turns to history (brief lock re-acquisition) ──────
579 + {
580 + let mut guard = self.inner.lock().await;
581 + if let Some(loaded) = guard.as_mut() {
582 + loaded.history.push(ChatMessage::user(user_message));
583 + loaded.history.push(ChatMessage::assistant(reply.clone()));
584 + }
585 + }
586 +
587 + Ok(InferenceResult {
588 + text: reply,
589 + duration_secs: elapsed.as_secs_f64(),
590 + duration_display: format_duration(elapsed),
591 + finish_reason,
592 + })
593 + }
594 +
595 + /// Run inference on an explicit list of messages WITHOUT modifying the
596 + /// engine's internal history.
597 + ///
598 + /// This is useful for one-shot prompts (e.g. prompt enhancement) where
599 + /// you don't want side-effects on the main conversation.
600 + ///
601 + /// The system prompt set on the engine is still prepended.
602 + pub async fn generate(
603 + &self,
604 + messages: Vec<ChatMessage>,
605 + sampling: Option<SamplingConfig>,
606 + ) -> Result<InferenceResult, InferenceError> {
607 + let (model, request) = {
608 + let guard = self.inner.lock().await;
609 + let loaded = guard.as_ref().ok_or(InferenceError::NoModelLoaded)?;
610 +
611 + let sampling = sampling.as_ref().unwrap_or(&loaded.sampling);
612 + let mut req = RequestBuilder::new();
613 +
614 + // Apply sampling parameters.
615 + req = apply_sampling(req, sampling);
616 +
617 + // System prompt.
618 + if let Some(ref sp) = loaded.system_prompt {
619 + req = req.add_message(TextMessageRole::System, sp);
620 + }
621 +
622 + // Provided messages.
623 + for msg in &messages {
624 + req = req.add_message(chat_role_to_mistral(&msg.role), &msg.content);
625 + }
626 +
627 + (loaded.model.clone(), req)
628 + };
629 +
630 + let start = std::time::Instant::now();
631 + let response =
632 + model
633 + .send_chat_request(request)
634 + .await
635 + .map_err(|e| InferenceError::Inference {
636 + reason: e.to_string(),
637 + })?;
638 + let elapsed = start.elapsed();
639 +
640 + let reply = response.choices[0]
641 + .message
642 + .content
643 + .as_ref()
644 + .map(|c: &String| c.trim().to_string())
645 + .unwrap_or_else(|| "(empty response)".to_string());
646 +
647 + let finish_reason = response.choices[0].finish_reason.clone();
648 +
649 + Ok(InferenceResult {
650 + text: reply,
651 + duration_secs: elapsed.as_secs_f64(),
652 + duration_display: format_duration(elapsed),
653 + finish_reason,
654 + })
655 + }
656 +
657 + // ── Streaming inference ──────────────────────────────────────────────
658 +
659 + /// Send a user message and receive an async stream of token chunks.
660 + ///
661 + /// The user message and full assembled reply are automatically appended
662 + /// to the conversation history once the stream finishes.
663 + ///
664 + /// # Returns
665 + ///
666 + /// A `tokio::sync::mpsc::Receiver<StreamChunk>`. The caller should
667 + /// receive from it until `chunk.done == true` or the channel closes.
668 + ///
669 + /// # Errors
670 + ///
671 + /// Returns immediately with [`InferenceError::NoModelLoaded`] if no
672 + /// model is loaded. Inference errors are delivered as the final chunk
673 + /// with `done = true` and an empty delta.
674 + pub async fn stream_message(
675 + &self,
676 + user_message: impl Into<String>,
677 + ) -> Result<tokio::sync::mpsc::Receiver<StreamChunk>, InferenceError> {
678 + let user_message = user_message.into();
679 +
680 + let (model, request) = {
681 + let guard = self.inner.lock().await;
682 + let loaded = guard.as_ref().ok_or(InferenceError::NoModelLoaded)?;
683 + let request = self::build_request(loaded, &user_message);
684 + (loaded.model.clone(), request)
685 + };
686 +
687 + let (tx, rx) = tokio::sync::mpsc::channel::<StreamChunk>(64);
688 +
689 + // We need a reference to `self` inside the spawned task so we can
690 + // update history when the stream completes. Since ChatEngine is
691 + // behind a shared reference (callers hold &self or Arc<ChatEngine>),
692 + // we borrow the inner Mutex via a raw pointer. This is safe because
693 + // ChatEngine's lifetime exceeds the spawned task (it's typically in
694 + // a Lazy static or shared application state).
695 + //
696 + // Alternative: require ChatEngine to always be wrapped in Arc and
697 + // accept `self: Arc<Self>`. We avoid that to keep the API simple.
698 + let inner_ptr = &self.inner as *const Mutex<Option<LoadedModel>>;
699 + // SAFETY: ChatEngine is stored in a Lazy static or equivalent with
700 + // 'static lifetime. The spawned task cannot outlive the engine.
701 + let inner_ref: &'static Mutex<Option<LoadedModel>> = unsafe { &*inner_ptr };
702 +
703 + let user_msg_clone = user_message.clone();
704 +
705 + tokio::task::spawn(async move {
706 + // `model` is an `Arc<Model>` moved into this task, so it's owned.
707 + // `stream_chat_request` borrows `&Model` — the borrow is scoped
708 + // to this async block and does NOT need to be `'static`.
709 + let stream_result = model.stream_chat_request(request).await;
710 +
711 + match stream_result {
712 + Ok(mut stream) => {
713 + let mut assembled = String::new();
714 + let mut last_finish_reason = None;
715 +
716 + // `Stream::next()` is an inherent async method on
717 + // mistralrs::Stream that returns `Option<Response>`.
718 + // No `futures::StreamExt` import needed.
719 + while let Some(response) = stream.next().await {
720 + match response {
721 + mistralrs::Response::Chunk(chunk) => {
722 + if let Some(choice) = chunk.choices.first() {
723 + if let Some(ref text) = choice.delta.content {
724 + assembled.push_str(text);
725 + let _ = tx
726 + .send(StreamChunk {
727 + delta: text.clone(),
728 + done: false,
729 + finish_reason: None,
730 + })
731 + .await;
732 + }
733 + if let Some(ref reason) = choice.finish_reason {
734 + last_finish_reason = Some(reason.clone());
735 + }
736 + }
737 + }
738 + mistralrs::Response::Done(_) => {
739 + // Non-streaming response arrived on a streaming
740 + // channel — should not happen, but handle gracefully.
741 + break;
742 + }
743 + mistralrs::Response::InternalError(e) => {
744 + log::error!("ChatEngine stream internal error: {}", e);
745 + break;
746 + }
747 + mistralrs::Response::ValidationError(e) => {
748 + log::error!("ChatEngine stream validation error: {}", e);
749 + break;
750 + }
751 + mistralrs::Response::ModelError(msg, _) => {
752 + log::error!("ChatEngine stream model error: {}", msg);
753 + break;
754 + }
755 + _ => {
756 + // Completion / Image / Speech variants — not expected
757 + // for chat streaming.
758 + break;
759 + }
760 + }
761 + }
762 +
763 + // Persist turns to history.
764 + {
765 + let mut guard = inner_ref.lock().await;
766 + if let Some(loaded) = guard.as_mut() {
767 + loaded.history.push(ChatMessage::user(user_msg_clone));
768 + loaded
769 + .history
770 + .push(ChatMessage::assistant(assembled.trim()));
771 + }
772 + }
773 +
774 + // Send the final "done" chunk.
775 + let _ = tx
776 + .send(StreamChunk {
777 + delta: String::new(),
778 + done: true,
779 + finish_reason: last_finish_reason,
780 + })
781 + .await;
782 + }
783 + Err(e) => {
784 + log::error!("ChatEngine: stream_chat_request failed: {}", e);
785 + let _ = tx
786 + .send(StreamChunk {
787 + delta: String::new(),
788 + done: true,
789 + finish_reason: Some(format!("error: {}", e)),
790 + })
791 + .await;
792 + }
793 + }
794 + });
795 +
796 + Ok(rx)
797 + }
798 +}
799 +
800 +// ═════════════════════════════════════════════════════════════════════════════
801 +// Default impl
802 +// ═════════════════════════════════════════════════════════════════════════════
803 +
804 +#[cfg(any(
805 + target_os = "macos",
806 + target_os = "ios",
807 + target_os = "tvos",
808 + target_os = "windows",
809 + target_os = "linux",
810 + target_os = "android"
811 +))]
812 +impl Default for ChatEngine {
813 + fn default() -> Self {
814 + Self::new()
815 + }
816 +}
817 +
818 +// ═════════════════════════════════════════════════════════════════════════════
819 +// Internal helpers (platform-gated)
820 +// ═════════════════════════════════════════════════════════════════════════════
821 +
822 +/// Build a `RequestBuilder` from the current engine state and a new user message.
823 +#[cfg(any(
824 + target_os = "macos",
825 + target_os = "ios",
826 + target_os = "tvos",
827 + target_os = "windows",
828 + target_os = "linux",
829 + target_os = "android"
830 +))]
831 +fn build_request(loaded: &LoadedModel, user_message: &str) -> RequestBuilder {
832 + let mut req = RequestBuilder::new();
833 +
834 + // Apply sampling.
835 + req = apply_sampling(req, &loaded.sampling);
836 +
837 + // System prompt.
838 + if let Some(ref sp) = loaded.system_prompt {
839 + req = req.add_message(TextMessageRole::System, sp);
840 + }
841 +
842 + // Replay conversation history for multi-turn context.
843 + for turn in &loaded.history {
844 + req = req.add_message(chat_role_to_mistral(&turn.role), &turn.content);
845 + }
846 +
847 + // Append the current user message.
848 + req = req.add_message(TextMessageRole::User, user_message);
849 +
850 + req
851 +}
852 +
853 +/// Apply a [`SamplingConfig`] to a [`RequestBuilder`].
854 +#[cfg(any(
855 + target_os = "macos",
856 + target_os = "ios",
857 + target_os = "tvos",
858 + target_os = "windows",
859 + target_os = "linux",
860 + target_os = "android"
861 +))]
862 +fn apply_sampling(mut req: RequestBuilder, sampling: &SamplingConfig) -> RequestBuilder {
863 + if let Some(temp) = sampling.temperature {
864 + req = req.set_sampler_temperature(temp);
865 + }
866 + if let Some(top_p) = sampling.top_p {
867 + req = req.set_sampler_topp(top_p);
868 + }
869 + if let Some(top_k) = sampling.top_k {
870 + req = req.set_sampler_topk(top_k as usize);
871 + }
872 + if let Some(min_p) = sampling.min_p {
873 + req = req.set_sampler_minp(min_p);
874 + }
875 + if let Some(max_tokens) = sampling.max_tokens {
876 + req = req.set_sampler_max_len(max_tokens as usize);
877 + }
878 + if let Some(freq) = sampling.frequency_penalty {
879 + req = req.set_sampler_frequency_penalty(freq);
880 + }
881 + if let Some(pres) = sampling.presence_penalty {
882 + req = req.set_sampler_presence_penalty(pres);
883 + }
884 + req
885 +}
886 +
887 +/// Convert [`ChatRole`] → [`TextMessageRole`].
888 +#[cfg(any(
889 + target_os = "macos",
890 + target_os = "ios",
891 + target_os = "tvos",
892 + target_os = "windows",
893 + target_os = "linux",
894 + target_os = "android"
895 +))]
896 +fn chat_role_to_mistral(role: &ChatRole) -> TextMessageRole {
897 + match role {
898 + ChatRole::System => TextMessageRole::System,
899 + ChatRole::User => TextMessageRole::User,
900 + ChatRole::Assistant => TextMessageRole::Assistant,
901 + }
902 +}
903 +
904 +/// Truncate a string for log output, appending `"..."` if truncated.
905 +fn truncate_for_log(s: &str, max_len: usize) -> String {
906 + if s.len() > max_len {
907 + format!("{}...", &s[..max_len])
908 + } else {
909 + s.to_string()
910 + }
911 +}
912 +
913 +// ═════════════════════════════════════════════════════════════════════════════
914 +// Unsupported-platform stub
915 +// ═════════════════════════════════════════════════════════════════════════════
916 +
917 +/// Stub `ChatEngine` for platforms where mistral.rs is not available.
918 +///
919 +/// All methods return errors or empty values so that code using the engine
920 +/// can compile on any target without `#[cfg]` at every call site.
921 +#[cfg(not(any(
922 + target_os = "macos",
923 + target_os = "ios",
924 + target_os = "tvos",
925 + target_os = "windows",
926 + target_os = "linux",
927 + target_os = "android"
928 +)))]
929 +pub struct ChatEngine;
930 +
931 +#[cfg(not(any(
932 + target_os = "macos",
933 + target_os = "ios",
934 + target_os = "tvos",
935 + target_os = "windows",
936 + target_os = "linux",
937 + target_os = "android"
938 +)))]
939 +impl ChatEngine {
940 + pub fn new() -> Self {
941 + Self
942 + }
943 +
944 + pub async fn load_gguf_model(
945 + &self,
946 + _config: GgufModelConfig,
947 + _system_prompt: Option<String>,
948 + _sampling: Option<SamplingConfig>,
949 + ) -> Result<std::time::Duration, InferenceError> {
950 + Err(InferenceError::Other {
951 + reason: "LLM inference is not supported on this platform.".into(),
952 + })
953 + }
954 +
955 + pub async fn unload_model(&self) -> Option<String> {
956 + None
957 + }
958 +
959 + pub async fn is_loaded(&self) -> bool {
960 + false
961 + }
962 +
963 + pub async fn info(&self) -> EngineInfo {
964 + EngineInfo {
965 + status: EngineStatus::Unloaded,
966 + model_name: None,
967 + approx_memory: None,
968 + history_length: 0u64,
969 + }
970 + }
971 +
972 + pub async fn set_system_prompt(&self, _prompt: impl Into<String>) {}
973 +
974 + pub async fn clear_system_prompt(&self) {}
975 +
976 + pub async fn set_sampling(&self, _sampling: SamplingConfig) {}
977 +
978 + pub async fn history(&self) -> Vec<ChatMessage> {
979 + Vec::new()
980 + }
981 +
982 + pub async fn clear_history(&self) -> usize {
983 + 0
984 + }
985 +
986 + pub async fn push_history(&self, _message: ChatMessage) {}
987 +
988 + pub async fn send_message(
989 + &self,
990 + _user_message: impl Into<String>,
991 + ) -> Result<InferenceResult, InferenceError> {
992 + Err(InferenceError::Other {
993 + reason: "LLM inference is not supported on this platform.".into(),
994 + })
995 + }
996 +
997 + pub async fn generate(
998 + &self,
999 + _messages: Vec<ChatMessage>,
1000 + _sampling: Option<SamplingConfig>,
1001 + ) -> Result<InferenceResult, InferenceError> {
1002 + Err(InferenceError::Other {
1003 + reason: "LLM inference is not supported on this platform.".into(),
1004 + })
1005 + }
1006 +
1007 + pub async fn stream_message(
1008 + &self,
1009 + _user_message: impl Into<String>,
1010 + ) -> Result<tokio::sync::mpsc::Receiver<StreamChunk>, InferenceError> {
1011 + Err(InferenceError::Other {
1012 + reason: "LLM inference is not supported on this platform.".into(),
1013 + })
1014 + }
1015 +}
1016 +
1017 +#[cfg(not(any(
1018 + target_os = "macos",
1019 + target_os = "ios",
1020 + target_os = "tvos",
1021 + target_os = "windows",
1022 + target_os = "linux",
1023 + target_os = "android"
1024 +)))]
1025 +impl Default for ChatEngine {
1026 + fn default() -> Self {
1027 + Self::new()
1028 + }
1029 +}
1030 +
1031 +// ═════════════════════════════════════════════════════════════════════════════
1032 +// Prebuilt model configs (convenience constructors)
1033 +// ═════════════════════════════════════════════════════════════════════════════
1034 +
1035 +/// Convenience constructors for common GGUF model configurations.
1036 +///
1037 +/// These mirror the constants in [`super::models`] but return a ready-to-use
1038 +/// [`GgufModelConfig`].
1039 +impl GgufModelConfig {
1040 + /// Qwen 2.5 1.5B Instruct (GGUF Q4_K_M) — ~941 MB.
1041 + ///
1042 + /// Lightest option, ideal for iOS and memory-constrained Android devices.
1043 + pub fn qwen25_1_5b() -> Self {
1044 + Self {
1045 + model_id: super::models::BARTOWSKI_QWEN25_1_5B_INSTRUCT_GGUF.into(),
1046 + files: vec![super::models::QWEN25_1_5B_GGUF_FILE.into()],
1047 + tok_model_id: if cfg!(target_os = "android") {
1048 + Some(super::models::QWEN25_1_5B_TOK_MODEL_ID.into())
1049 + } else {
1050 + None
1051 + },
1052 + display_name: "Qwen 2.5 1.5B".into(),
1053 + approx_memory: "~941 MB (GGUF Q4_K_M)".into(),
1054 + }
1055 + }
1056 +
1057 + /// Qwen 2.5 3B Instruct (GGUF Q4_K_M) — ~1.93 GB.
1058 + ///
1059 + /// Balanced quality/size for macOS desktops and high-RAM Android devices.
1060 + /// Not recommended for iOS (OOM on 8 GB devices).
1061 + pub fn qwen25_3b() -> Self {
1062 + Self {
1063 + model_id: super::models::BARTOWSKI_QWEN25_3B_INSTRUCT_GGUF.into(),
1064 + files: vec![super::models::QWEN25_3B_GGUF_FILE.into()],
1065 + tok_model_id: if cfg!(target_os = "android") {
1066 + Some(super::models::QWEN25_3B_TOK_MODEL_ID.into())
1067 + } else {
1068 + None
1069 + },
1070 + display_name: "Qwen 2.5 3B".into(),
1071 + approx_memory: "~1.93 GB (GGUF Q4_K_M)".into(),
1072 + }
1073 + }
1074 +
1075 + /// Qwen 2.5 Coder 1.5B Instruct (GGUF Q4_K_M) — ~941 MB.
1076 + ///
1077 + /// Dedicated coding model fine-tuned on 5.5T tokens of code and math data.
1078 + /// Same `qwen2` GGUF architecture as the general 1.5B — loads via the
1079 + /// existing `quantized_qwen.rs` path without any mistral.rs changes.
1080 + /// Same memory footprint as the general 1.5B but dramatically better at
1081 + /// code generation, explanation, and debugging.
1082 + ///
1083 + /// Preferred default for iOS, Android, and any coding-focused deployment.
1084 + pub fn qwen25_coder_1_5b() -> Self {
1085 + Self {
1086 + model_id: super::models::BARTOWSKI_QWEN25_CODER_1_5B_INSTRUCT_GGUF.into(),
1087 + files: vec![super::models::QWEN25_CODER_1_5B_GGUF_FILE.into()],
1088 + tok_model_id: if cfg!(target_os = "android") {
1089 + Some(super::models::QWEN25_CODER_1_5B_TOK_MODEL_ID.into())
1090 + } else {
1091 + None
1092 + },
1093 + display_name: "Qwen 2.5 Coder 1.5B".into(),
1094 + approx_memory: "~941 MB (GGUF Q4_K_M)".into(),
1095 + }
1096 + }
1097 +
1098 + /// Qwen 2.5 Coder 3B Instruct (GGUF Q4_K_M) — ~1.93 GB.
1099 + ///
1100 + /// Best on-device coding quality for macOS desktop. Fine-tuned on 5.5T
1101 + /// tokens of code and math. Same `qwen2` architecture as the general 3B.
1102 + /// Not recommended for iOS due to memory constraints.
1103 + pub fn qwen25_coder_3b() -> Self {
1104 + Self {
1105 + model_id: super::models::BARTOWSKI_QWEN25_CODER_3B_INSTRUCT_GGUF.into(),
1106 + files: vec![super::models::QWEN25_CODER_3B_GGUF_FILE.into()],
1107 + tok_model_id: if cfg!(target_os = "android") {
1108 + Some(super::models::QWEN25_CODER_3B_TOK_MODEL_ID.into())
1109 + } else {
1110 + None
1111 + },
1112 + display_name: "Qwen 2.5 Coder 3B".into(),
1113 + approx_memory: "~1.93 GB (GGUF Q4_K_M)".into(),
1114 + }
1115 + }
1116 +
1117 + /// Return the platform-appropriate default **coding** model config.
1118 + ///
1119 + /// - iOS / Android → Qwen 2.5 Coder 1.5B (~941 MB, fits mobile memory budgets)
1120 + /// - macOS / Windows / Linux → Qwen 2.5 Coder 3B (~1.93 GB, best desktop quality)
1121 + ///
1122 + /// Both are dedicated coding models (trained on 5.5T code + math tokens)
1123 + /// and use the `qwen2` GGUF architecture supported by mistral.rs.
1124 + pub fn platform_default() -> Self {
1125 + if cfg!(any(
1126 + target_os = "ios",
1127 + target_os = "tvos",
1128 + target_os = "android"
1129 + )) {
1130 + Self::qwen25_coder_1_5b()
1131 + } else {
1132 + Self::qwen25_coder_3b()
1133 + }
1134 + }
1135 +}
1136 +
1137 +// ═════════════════════════════════════════════════════════════════════════════
1138 +// Tests
1139 +// ═════════════════════════════════════════════════════════════════════════════
1140 +
1141 +#[cfg(test)]
1142 +mod tests {
1143 + use super::*;
1144 +
1145 + #[test]
1146 + fn truncate_for_log_short() {
1147 + assert_eq!(truncate_for_log("hello", 10), "hello");
1148 + }
1149 +
1150 + #[test]
1151 + fn truncate_for_log_long() {
1152 + let long = "a".repeat(200);
1153 + let result = truncate_for_log(&long, 50);
1154 + assert!(result.ends_with("..."));
1155 + assert_eq!(result.len(), 53); // 50 chars + "..."
1156 + }
1157 +
1158 + #[test]
1159 + fn gguf_model_config_qwen25_1_5b() {
1160 + let cfg = GgufModelConfig::qwen25_1_5b();
1161 + assert!(cfg.model_id.contains("1.5B"));
1162 + assert_eq!(cfg.files.len(), 1);
1163 + }
1164 +
1165 + #[test]
1166 + fn gguf_model_config_qwen25_3b() {
1167 + let cfg = GgufModelConfig::qwen25_3b();
1168 + assert!(cfg.model_id.contains("3B"));
1169 + assert_eq!(cfg.files.len(), 1);
1170 + }
1171 +
1172 + #[test]
1173 + fn gguf_model_config_platform_default() {
1174 + let cfg = GgufModelConfig::platform_default();
1175 + // On the test host (macOS or Linux), this should be the 3B model.
1176 + // On iOS/Android CI it would be 1.5B. We just check it's valid.
1177 + assert!(!cfg.model_id.is_empty());
1178 + assert!(!cfg.files.is_empty());
1179 + }
1180 +
1181 + #[tokio::test]
1182 + async fn engine_new_is_unloaded() {
1183 + let engine = ChatEngine::new();
1184 + assert!(!engine.is_loaded().await);
1185 + let info = engine.info().await;
1186 + assert_eq!(info.status, EngineStatus::Unloaded);
1187 + assert_eq!(info.history_length, 0);
1188 + }
1189 +
1190 + #[tokio::test]
1191 + async fn engine_send_without_model_errors() {
1192 + let engine = ChatEngine::new();
1193 + let result = engine.send_message("hello").await;
1194 + assert!(result.is_err());
1195 + match result.unwrap_err() {
1196 + InferenceError::NoModelLoaded => {} // expected
1197 + other => panic!("Expected NoModelLoaded, got: {:?}", other),
1198 + }
1199 + }
1200 +
1201 + #[tokio::test]
1202 + async fn engine_history_empty_when_no_model() {
1203 + let engine = ChatEngine::new();
1204 + assert!(engine.history().await.is_empty());
1205 + assert_eq!(engine.clear_history().await, 0);
1206 + }
1207 +
1208 + #[tokio::test]
1209 + async fn engine_unload_when_none() {
1210 + let engine = ChatEngine::new();
1211 + assert!(engine.unload_model().await.is_none());
1212 + }
1213 +}
src/inference/ffi.rs new
+404
@@ -0,0 +1,404 @@
1 +//! UniFFI-exported wrapper around [`ChatEngine`] for Swift / Kotlin consumers.
2 +//!
3 +//! [`ChatEngine`] uses Rust idioms (generics, `impl Into<String>`,
4 +//! `tokio::sync::mpsc::Receiver`) that cannot cross the FFI boundary.
5 +//! This module provides [`OndeChatEngine`], a `#[derive(uniffi::Object)]`
6 +//! wrapper with concrete, FFI-safe method signatures.
7 +//!
8 +//! ## Swift usage
9 +//!
10 +//! ```swift
11 +//! let engine = OndeChatEngine()
12 +//!
13 +//! // Load the platform-appropriate default model.
14 +//! let elapsed = try await engine.loadDefaultModel(
15 +//! systemPrompt: "You are a helpful assistant.",
16 +//! sampling: nil
17 +//! )
18 +//! print("Model loaded in \(elapsed)s")
19 +//!
20 +//! // Multi-turn chat (history managed automatically).
21 +//! let result = try await engine.sendMessage(message: "Hello!")
22 +//! print(result.text)
23 +//!
24 +//! // Streaming (callback-based for FFI compatibility).
25 +//! try await streamChatMessage(engine: engine, message: "Tell me a story.", listener: myListener)
26 +//!
27 +//! // One-shot generation (does NOT modify conversation history).
28 +//! let enhanced = try await engine.generate(
29 +//! messages: [ChatMessage(role: .user, content: "Expand: a cat in space")],
30 +//! sampling: SamplingConfig(temperature: 0.0, topP: nil, topK: nil, minP: nil, maxTokens: 512, frequencyPenalty: nil, presencePenalty: nil)
31 +//! )
32 +//! print(enhanced.text)
33 +//!
34 +//! // Status & history.
35 +//! let info = await engine.info()
36 +//! let history = await engine.history()
37 +//!
38 +//! // Cleanup.
39 +//! await engine.unloadModel()
40 +//! ```
41 +
42 +use std::sync::Arc;
43 +
44 +use super::engine::ChatEngine;
45 +use super::types::*;
46 +
47 +// ═══════════════════════════════════════════════════════════════════════════
48 +// Callback interface for streaming (used by free function, not Object method)
49 +// ═══════════════════════════════════════════════════════════════════════════
50 +
51 +/// Callback interface for receiving streaming token chunks.
52 +///
53 +/// Implement this in Swift / Kotlin and pass it to the free function
54 +/// [`stream_chat_message`].
55 +///
56 +/// ```swift
57 +/// class MyStreamHandler: StreamChunkListener {
58 +/// func onChunk(chunk: StreamChunk) -> Bool {
59 +/// print(chunk.delta, terminator: "")
60 +/// return !chunk.done // return false to stop early
61 +/// }
62 +/// }
63 +/// ```
64 +#[uniffi::export(callback_interface)]
65 +pub trait StreamChunkListener: Send + Sync {
66 + /// Called for each token chunk during streaming inference.
67 + ///
68 + /// Return `true` to continue receiving chunks, or `false` to cancel
69 + /// the stream early (the engine will still persist partial history).
70 + fn on_chunk(&self, chunk: StreamChunk) -> bool;
71 +}
72 +
73 +// ═══════════════════════════════════════════════════════════════════════════
74 +// OndeChatEngine — UniFFI Object
75 +// ═══════════════════════════════════════════════════════════════════════════
76 +
77 +/// On-device LLM chat inference engine — UniFFI-exported wrapper.
78 +///
79 +/// This is the primary entry point for Swift / Kotlin consumers. It wraps
80 +/// [`ChatEngine`] with FFI-safe method signatures (no generics, no Rust
81 +/// channels, concrete `String` parameters).
82 +///
83 +/// Construct with [`OndeChatEngine::new()`]. The engine starts with no
84 +/// model loaded — call [`load_gguf_model`] or [`load_default_model`] first.
85 +#[derive(uniffi::Object)]
86 +pub struct OndeChatEngine {
87 + inner: ChatEngine,
88 +}
89 +
90 +#[uniffi::export]
91 +impl OndeChatEngine {
92 + // ── Construction ─────────────────────────────────────────────────────
93 +
94 + /// Create a new engine with no model loaded.
95 + #[uniffi::constructor]
96 + pub fn new() -> Arc<Self> {
97 + Arc::new(Self {
98 + inner: ChatEngine::new(),
99 + })
100 + }
101 +
102 + // ── Model lifecycle ──────────────────────────────────────────────────
103 +
104 + /// Load a GGUF model into the engine.
105 + ///
106 + /// If a model is already loaded it will be unloaded first.
107 + ///
108 + /// Returns the wall-clock loading time in seconds.
109 + pub async fn load_gguf_model(
110 + &self,
111 + config: GgufModelConfig,
112 + system_prompt: Option<String>,
113 + sampling: Option<SamplingConfig>,
114 + ) -> Result<f64, InferenceError> {
115 + let elapsed = self
116 + .inner
117 + .load_gguf_model(config, system_prompt, sampling)
118 + .await?;
119 + Ok(elapsed.as_secs_f64())
120 + }
121 +
122 + /// Load the platform-appropriate default model.
123 + ///
124 + /// - tvOS / iOS / Android → Qwen 2.5 1.5B (~941 MB)
125 + /// - macOS / Windows / Linux → Qwen 2.5 3B (~1.93 GB)
126 + ///
127 + /// Returns the wall-clock loading time in seconds.
128 + pub async fn load_default_model(
129 + &self,
130 + system_prompt: Option<String>,
131 + sampling: Option<SamplingConfig>,
132 + ) -> Result<f64, InferenceError> {
133 + let config = GgufModelConfig::platform_default();
134 + let elapsed = self
135 + .inner
136 + .load_gguf_model(config, system_prompt, sampling)
137 + .await?;
138 + Ok(elapsed.as_secs_f64())
139 + }
140 +
141 + /// Unload the current model, freeing all memory.
142 + ///
143 + /// Returns the display name of the model that was unloaded, or `nil`
144 + /// if no model was loaded.
145 + pub async fn unload_model(&self) -> Option<String> {
146 + self.inner.unload_model().await
147 + }
148 +
149 + /// Check whether a model is currently loaded.
150 + pub async fn is_loaded(&self) -> bool {
151 + self.inner.is_loaded().await
152 + }
153 +
154 + /// Get a snapshot of the engine's current state.
155 + pub async fn info(&self) -> EngineInfo {
156 + self.inner.info().await
157 + }
158 +
159 + // ── System prompt ────────────────────────────────────────────────────
160 +
161 + /// Set or replace the system prompt.
162 + pub async fn set_system_prompt(&self, prompt: String) {
163 + self.inner.set_system_prompt(prompt).await;
164 + }
165 +
166 + /// Clear the system prompt.
167 + pub async fn clear_system_prompt(&self) {
168 + self.inner.clear_system_prompt().await;
169 + }
170 +
171 + // ── Sampling configuration ───────────────────────────────────────────
172 +
173 + /// Replace the sampling configuration.
174 + pub async fn set_sampling(&self, sampling: SamplingConfig) {
175 + self.inner.set_sampling(sampling).await;
176 + }
177 +
178 + // ── Conversation history ─────────────────────────────────────────────
179 +
180 + /// Get a copy of the full conversation history.
181 + pub async fn history(&self) -> Vec<ChatMessage> {
182 + self.inner.history().await
183 + }
184 +
185 + /// Clear the conversation history but keep the model loaded.
186 + ///
187 + /// Returns the number of turns that were removed.
188 + pub async fn clear_history(&self) -> u64 {
189 + self.inner.clear_history().await as u64
190 + }
191 +
192 + /// Append a message to history without running inference.
193 + pub async fn push_history(&self, message: ChatMessage) {
194 + self.inner.push_history(message).await;
195 + }
196 +
197 + // ── Non-streaming inference ──────────────────────────────────────────
198 +
199 + /// Send a user message and receive a complete assistant reply.
200 + ///
201 + /// The user message and assistant reply are automatically appended to
202 + /// the conversation history on success.
203 + pub async fn send_message(&self, message: String) -> Result<InferenceResult, InferenceError> {
204 + self.inner.send_message(message).await
205 + }
206 +
207 + /// Run inference on an explicit list of messages WITHOUT modifying the
208 + /// engine's internal history.
209 + ///
210 + /// Useful for one-shot prompts (e.g. prompt enhancement).
211 + pub async fn generate(
212 + &self,
213 + messages: Vec<ChatMessage>,
214 + sampling: Option<SamplingConfig>,
215 + ) -> Result<InferenceResult, InferenceError> {
216 + self.inner.generate(messages, sampling).await
217 + }
218 +}
219 +
220 +// ═══════════════════════════════════════════════════════════════════════════
221 +// Free function for streaming — UniFFI callback_interface works in free fns
222 +// ═══════════════════════════════════════════════════════════════════════════
223 +
224 +/// Stream a chat message through the engine, delivering token chunks to
225 +/// the `listener` callback.
226 +///
227 +/// This is a **free function** (not an Object method) because UniFFI 0.31
228 +/// requires `callback_interface` parameters to be passed to free functions
229 +/// rather than Object methods.
230 +///
231 +/// The `listener.on_chunk()` callback is called for each token delta.
232 +/// Return `true` to continue, or `false` to cancel early.
233 +///
234 +/// The user message and assembled reply are automatically appended to the
235 +/// engine's conversation history when streaming completes.
236 +///
237 +/// ```swift
238 +/// class MyHandler: StreamChunkListener {
239 +/// func onChunk(chunk: StreamChunk) -> Bool {
240 +/// print(chunk.delta, terminator: "")
241 +/// return !chunk.done
242 +/// }
243 +/// }
244 +///
245 +/// try await streamChatMessage(
246 +/// engine: engine,
247 +/// message: "Tell me a story.",
248 +/// listener: MyHandler()
249 +/// )
250 +/// ```
251 +#[uniffi::export]
252 +pub async fn stream_chat_message(
253 + engine: Arc<OndeChatEngine>,
254 + message: String,
255 + listener: Box<dyn StreamChunkListener>,
256 +) -> Result<(), InferenceError> {
257 + let mut rx = engine.inner.stream_message(message).await?;
258 +
259 + while let Some(chunk) = rx.recv().await {
260 + let done = chunk.done;
261 + let should_continue = listener.on_chunk(chunk);
262 + if done || !should_continue {
263 + break;
264 + }
265 + }
266 +
267 + Ok(())
268 +}
269 +
270 +// ═══════════════════════════════════════════════════════════════════════════
271 +// Free functions — convenience exports for Swift / Kotlin
272 +// ═══════════════════════════════════════════════════════════════════════════
273 +
274 +/// Return the platform-appropriate default GGUF model configuration.
275 +///
276 +/// - tvOS / iOS / Android → Qwen 2.5 1.5B (~941 MB)
277 +/// - macOS / Windows / Linux → Qwen 2.5 3B (~1.93 GB)
278 +#[uniffi::export]
279 +pub fn default_model_config() -> GgufModelConfig {
280 + GgufModelConfig::platform_default()
281 +}
282 +
283 +/// Return the Qwen 2.5 1.5B GGUF model configuration (~941 MB).
284 +#[uniffi::export]
285 +pub fn qwen25_1_5b_config() -> GgufModelConfig {
286 + GgufModelConfig::qwen25_1_5b()
287 +}
288 +
289 +/// Return the Qwen 2.5 3B GGUF model configuration (~1.93 GB).
290 +#[uniffi::export]
291 +pub fn qwen25_3b_config() -> GgufModelConfig {
292 + GgufModelConfig::qwen25_3b()
293 +}
294 +
295 +/// Return default sampling parameters for creative chat.
296 +#[uniffi::export]
297 +pub fn default_sampling_config() -> SamplingConfig {
298 + SamplingConfig::default()
299 +}
300 +
301 +/// Return deterministic (greedy) sampling parameters.
302 +#[uniffi::export]
303 +pub fn deterministic_sampling_config() -> SamplingConfig {
304 + SamplingConfig::deterministic()
305 +}
306 +
307 +/// Return conservative sampling parameters for mobile devices.
308 +#[uniffi::export]
309 +pub fn mobile_sampling_config() -> SamplingConfig {
310 + SamplingConfig::mobile()
311 +}
312 +
313 +/// Create a system message.
314 +#[uniffi::export]
315 +pub fn system_message(content: String) -> ChatMessage {
316 + ChatMessage::system(content)
317 +}
318 +
319 +/// Create a user message.
320 +#[uniffi::export]
321 +pub fn user_message(content: String) -> ChatMessage {
322 + ChatMessage::user(content)
323 +}
324 +
325 +/// Create an assistant message.
326 +#[uniffi::export]
327 +pub fn assistant_message(content: String) -> ChatMessage {
328 + ChatMessage::assistant(content)
329 +}
330 +
331 +// ═══════════════════════════════════════════════════════════════════════════
332 +// Tests
333 +// ═══════════════════════════════════════════════════════════════════════════
334 +
335 +#[cfg(test)]
336 +mod tests {
337 + use super::*;
338 +
339 + #[test]
340 + fn free_functions_return_valid_configs() {
341 + let cfg = default_model_config();
342 + assert!(!cfg.model_id.is_empty());
343 +
344 + let cfg = qwen25_1_5b_config();
345 + assert!(cfg.model_id.contains("1.5B"));
346 +
347 + let cfg = qwen25_3b_config();
348 + assert!(cfg.model_id.contains("3B"));
349 + }
350 +
351 + #[test]
352 + fn free_functions_return_valid_sampling() {
353 + let s = default_sampling_config();
354 + assert_eq!(s.temperature, Some(0.7));
355 +
356 + let s = deterministic_sampling_config();
357 + assert_eq!(s.temperature, Some(0.0));
358 +
359 + let s = mobile_sampling_config();
360 + assert_eq!(s.max_tokens, Some(128));
361 + }
362 +
363 + #[test]
364 + fn message_helpers() {
365 + let m = system_message("You are helpful.".into());
366 + assert_eq!(m.role, ChatRole::System);
367 + assert_eq!(m.content, "You are helpful.");
368 +
369 + let m = user_message("Hello".into());
370 + assert_eq!(m.role, ChatRole::User);
371 +
372 + let m = assistant_message("Hi!".into());
373 + assert_eq!(m.role, ChatRole::Assistant);
374 + }
375 +
376 + #[tokio::test]
377 + async fn onde_chat_engine_new_is_unloaded() {
378 + let engine = OndeChatEngine::new();
379 + assert!(!engine.is_loaded().await);
380 + let info = engine.info().await;
381 + assert_eq!(info.status, EngineStatus::Unloaded);
382 + assert_eq!(info.history_length, 0);
383 + }
384 +
385 + #[tokio::test]
386 + async fn onde_chat_engine_send_without_model_errors() {
387 + let engine = OndeChatEngine::new();
388 + let result = engine.send_message("hello".into()).await;
389 + assert!(result.is_err());
390 + }
391 +
392 + #[tokio::test]
393 + async fn onde_chat_engine_history_empty() {
394 + let engine = OndeChatEngine::new();
395 + assert!(engine.history().await.is_empty());
396 + assert_eq!(engine.clear_history().await, 0);
397 + }
398 +
399 + #[tokio::test]
400 + async fn onde_chat_engine_unload_when_none() {
401 + let engine = OndeChatEngine::new();
402 + assert!(engine.unload_model().await.is_none());
403 + }
404 +}
src/inference/mod.rs new
+72
@@ -0,0 +1,72 @@
1 +//! On-device LLM inference powered by [mistral.rs](https://github.com/EricLBuehler/mistral.rs).
2 +//!
3 +//! This module provides a high-level, framework-agnostic API for running
4 +//! LLM chat inference on-device across macOS, iOS, Windows, Linux, and Android.
5 +//!
6 +//! # Modules
7 +//!
8 +//! - [`engine`] — The main [`ChatEngine`](engine::ChatEngine) that wraps mistral.rs
9 +//! with conversation history, sampling config, and model lifecycle management.
10 +//! - [`types`] — Shared types ([`ChatMessage`](types::ChatMessage),
11 +//! [`SamplingConfig`](types::SamplingConfig), [`InferenceResult`](types::InferenceResult), etc.)
12 +//! used by the engine and any UI layer.
13 +//! - [`models`] — Model ID constants and rich metadata for all supported models.
14 +//! - [`token`] — HuggingFace token resolution (build-time literal or on-disk cache).
15 +//!
16 +//! # Quick Start
17 +//!
18 +//! ```rust,ignore
19 +//! use onde::inference::engine::ChatEngine;
20 +//! use onde::inference::types::*;
21 +//!
22 +//! // Create engine and load the platform-appropriate default model.
23 +//! let engine = ChatEngine::new();
24 +//! engine.load_gguf_model(
25 +//! GgufModelConfig::platform_default(),
26 +//! Some("You are a helpful assistant.".into()),
27 +//! None, // platform-aware sampling defaults
28 +//! ).await?;
29 +//!
30 +//! // Multi-turn chat (history is managed automatically).
31 +//! let reply = engine.send_message("What is Rust's ownership model?").await?;
32 +//! println!("{}", reply.text);
33 +//!
34 +//! let follow_up = engine.send_message("Can you give me an example?").await?;
35 +//! println!("{}", follow_up.text);
36 +//!
37 +//! // One-shot generation (does NOT modify conversation history).
38 +//! let enhanced = engine.generate(
39 +//! vec![ChatMessage::user("Expand this into a detailed prompt: a cat in space")],
40 +//! Some(SamplingConfig::deterministic()),
41 +//! ).await?;
42 +//! println!("{}", enhanced.text);
43 +//!
44 +//! // Streaming inference.
45 +//! let mut rx = engine.stream_message("Tell me a story.").await?;
46 +//! while let Some(chunk) = rx.recv().await {
47 +//! if !chunk.delta.is_empty() {
48 +//! print!("{}", chunk.delta);
49 +//! }
50 +//! if chunk.done {
51 +//! break;
52 +//! }
53 +//! }
54 +//!
55 +//! // Cleanup.
56 +//! engine.unload_model().await;
57 +//! ```
58 +
59 +pub mod engine;
60 +pub mod ffi;
61 +pub mod models;
62 +pub mod token;
63 +pub mod types;
64 +
65 +// Re-export the most commonly used items at the `inference::` level for
66 +// ergonomic imports like `use onde::inference::{ChatEngine, ChatMessage, ...}`.
67 +pub use engine::ChatEngine;
68 +pub use ffi::{OndeChatEngine, StreamChunkListener};
69 +pub use types::{
70 + ChatMessage, ChatRole, EngineInfo, EngineStatus, GgufModelConfig, InferenceError,
71 + InferenceResult, IsqModelConfig, SamplingConfig, StreamChunk,
72 +};
src/inference/models.rs new
+120
@@ -0,0 +1,120 @@
1 +/// Pre-quantized Qwen 2.5 1.5B Instruct (GGUF Q4_K_M) — lightest mobile option (~941 MB).
2 +/// Fits comfortably on both iOS (iPhone 16e, 8 GB RAM) and Android memory-constrained devices.
3 +/// Note: the 3B variant (~1.93 GB) caused OOM on iPhone 16e because iOS gives apps only ~2-3 GB;
4 +/// the 1.5B variant at ~941 MB leaves comfortable headroom for KV cache, activations, and the app.
5 +/// bartowski's GGUF embeds the full tokenizer and Qwen2.5 chat template, so on iOS/macOS no
6 +/// separate tok_model_id download is needed. On Android an explicit tok_model_id is required.
7 +pub const BARTOWSKI_QWEN25_1_5B_INSTRUCT_GGUF: &str = "bartowski/Qwen2.5-1.5B-Instruct-GGUF";
8 +/// The specific GGUF filename to download from the bartowski 1.5B repo.
9 +pub const QWEN25_1_5B_GGUF_FILE: &str = "Qwen2.5-1.5B-Instruct-Q4_K_M.gguf";
10 +/// Base model repo used for the HF tokenizer on Android.
11 +/// On Android, GGUF loading requires an explicit tokenizer source; on iOS/macOS the tokenizer
12 +/// embedded in the GGUF file is used instead to avoid an extra network download.
13 +pub const QWEN25_1_5B_TOK_MODEL_ID: &str = "Qwen/Qwen2.5-1.5B-Instruct";
14 +
15 +/// Pre-quantized Qwen 2.5 Coder 1.5B Instruct (GGUF Q4_K_M) — dedicated coding model (~941 MB).
16 +///
17 +/// Uses the `qwen2` GGUF architecture, identical to Qwen2.5-1.5B-Instruct, so it loads through
18 +/// the existing `quantized_qwen.rs` path in mistral.rs without any code changes.
19 +///
20 +/// Strongly preferred over the general-purpose 1.5B for coding tasks: trained on 5.5T tokens of
21 +/// code and math data, with fill-in-the-middle (FIM) and repo-level code understanding.
22 +/// Same memory footprint as the general 1.5B (~941 MB) but dramatically better at code.
23 +///
24 +/// bartowski's GGUF embeds the full tokenizer and chat template, so on iOS/macOS no separate
25 +/// tok_model_id download is needed. On Android an explicit tok_model_id is required.
26 +pub const BARTOWSKI_QWEN25_CODER_1_5B_INSTRUCT_GGUF: &str =
27 + "bartowski/Qwen2.5-Coder-1.5B-Instruct-GGUF";
28 +/// The specific GGUF filename to download from the bartowski Coder 1.5B repo.
29 +pub const QWEN25_CODER_1_5B_GGUF_FILE: &str = "Qwen2.5-Coder-1.5B-Instruct-Q4_K_M.gguf";
30 +/// Base model repo used for the HF tokenizer on Android.
31 +pub const QWEN25_CODER_1_5B_TOK_MODEL_ID: &str = "Qwen/Qwen2.5-Coder-1.5B-Instruct";
32 +
33 +/// HuggingFace repo for the full-precision Qwen 2.5 Coder 7B Instruct model.
34 +///
35 +/// Used by ISQ pipelines (`TextModelBuilder`) which download the safetensors weights
36 +/// directly and quantise them in-situ on the device. Unlike the bartowski GGUF
37 +/// variants, this repo ships the original BF16 weights; mistral.rs handles
38 +/// quantisation to Q4K or Q8_0 at load time via the `--isq` flag.
39 +///
40 +/// Requires ~8 GB RAM during the load phase (4-bit); ~12 GB for 8-bit.
41 +/// Metal-accelerated on macOS; CPU fallback available but very slow.
42 +pub const QWEN25_CODER_7B_INSTRUCT: &str = "Qwen/Qwen2.5-Coder-7B-Instruct";
43 +
44 +/// Pre-quantized Qwen 2.5 Coder 3B Instruct (GGUF Q4_K_M) — best coding quality on macOS (~1.93 GB).
45 +///
46 +/// Same `qwen2` architecture as the 3B general model; ideal for macOS desktops where the extra
47 +/// quality headroom over the 1.5B is worthwhile. Not recommended for iOS (OOM risk).
48 +pub const BARTOWSKI_QWEN25_CODER_3B_INSTRUCT_GGUF: &str =
49 + "bartowski/Qwen2.5-Coder-3B-Instruct-GGUF";
50 +/// The specific GGUF filename to download from the bartowski Coder 3B repo.
51 +pub const QWEN25_CODER_3B_GGUF_FILE: &str = "Qwen2.5-Coder-3B-Instruct-Q4_K_M.gguf";
52 +/// Base model repo used for the HF tokenizer on Android.
53 +pub const QWEN25_CODER_3B_TOK_MODEL_ID: &str = "Qwen/Qwen2.5-Coder-3B-Instruct";
54 +
55 +/// Pre-quantized Qwen 2.5 3B Instruct (GGUF Q4_K_M) — balanced option (~1.93 GB).
56 +/// Ideal for macOS desktops and Android devices with sufficient RAM.
57 +/// Not recommended as default on iOS: the 3B variant caused OOM on iPhone 16e (8 GB RAM)
58 +/// because iOS gives apps only ~2-3 GB; use the 1.5B variant for iOS instead.
59 +/// No in-situ quantization needed; loads directly at quantized size.
60 +pub const BARTOWSKI_QWEN25_3B_INSTRUCT_GGUF: &str = "bartowski/Qwen2.5-3B-Instruct-GGUF";
61 +/// The specific GGUF filename to download from the bartowski 3B repo.
62 +pub const QWEN25_3B_GGUF_FILE: &str = "Qwen2.5-3B-Instruct-Q4_K_M.gguf";
63 +/// Base model repo used for the HF tokenizer (tokenizer.json + tokenizer_config.json).
64 +pub const QWEN25_3B_TOK_MODEL_ID: &str = "Qwen/Qwen2.5-3B-Instruct";
65 +
66 +/// All model IDs that the Onde inference engine supports.
67 +/// Used by `list_local_hf_models` to filter the HuggingFace cache
68 +/// to only show models that can actually be used for generation.
69 +pub const SUPPORTED_MODELS: &[&str] = &[
70 + BARTOWSKI_QWEN25_1_5B_INSTRUCT_GGUF,
71 + BARTOWSKI_QWEN25_3B_INSTRUCT_GGUF,
72 +];
73 +
74 +/// Rich metadata for a supported model, used by the frontend to display
75 +/// unavailable models that can be downloaded.
76 +pub struct SupportedModelInfo {
77 + /// The full HuggingFace model identifier, e.g. "bartowski/Qwen2.5-1.5B-Instruct-GGUF".
78 + pub id: &'static str,
79 + /// Human-friendly display name for the model.
80 + pub name: &'static str,
81 + /// Organisation or publisher display name.
82 + pub org: &'static str,
83 + /// Short description of the model's purpose / capabilities.
84 + pub description: &'static str,
85 + /// Approximate total size in bytes when fully downloaded.
86 + /// Used by the progress monitor to estimate download percentage.
87 + ///
88 + /// These values are computed by summing every file in the model's
89 + /// HuggingFace repository (via the `/api/models/{id}?blobs=true`
90 + /// endpoint's `siblings[].size` fields).
91 + pub expected_size_bytes: u64,
92 +}
93 +
94 +/// Complete list of supported models with display metadata.
95 +///
96 +/// When adding a new model, add its constant ID to [`SUPPORTED_MODELS`] **and**
97 +/// a corresponding entry here so the frontend can show it in the model list UI.
98 +pub const SUPPORTED_MODEL_INFO: &[SupportedModelInfo] = &[
99 + SupportedModelInfo {
100 + id: BARTOWSKI_QWEN25_1_5B_INSTRUCT_GGUF,
101 + name: "Qwen 2.5 1.5B (GGUF)",
102 + org: "Qwen / Alibaba",
103 + description: "Lightest pre-quantized chat model — ideal for iOS & Android (~941 MB). \
104 + Fits comfortably within iOS memory limits (iPhone 16e, 8 GB RAM).",
105 + // Qwen2.5-1.5B-Instruct-Q4_K_M.gguf from bartowski repo.
106 + // Exact file size from HuggingFace API siblings[].size.
107 + expected_size_bytes: 986_048_768,
108 + },
109 + SupportedModelInfo {
110 + id: BARTOWSKI_QWEN25_3B_INSTRUCT_GGUF,
111 + name: "Qwen 2.5 3B (GGUF)",
112 + org: "Qwen / Alibaba",
113 + description:
114 + "Pre-quantized chat model for macOS & Android — balanced quality and size (~1.93 GB). \
115 + Not recommended as default on iOS due to memory constraints.",
116 + // Qwen2.5-3B-Instruct-Q4_K_M.gguf from bartowski repo.
117 + // Exact file size from HuggingFace API siblings[].size.
118 + expected_size_bytes: 1_929_903_264,
119 + },
120 +];
src/inference/token.rs new
+108
@@ -0,0 +1,108 @@
1 +//! Hugging Face token resolution for on-device model loading.
2 +//!
3 +//! On iOS there is no way to run `mistralrs login` or place a token file at
4 +//! `~/.cache/huggingface/token`, so we embed the token at **build time**.
5 +//!
6 +//! ## How it works
7 +//!
8 +//! 1. The consuming app's `build.rs` sets the `HF_TOKEN` env var (e.g. by
9 +//! loading a `.env` file with `dotenvy`, or via CI secrets).
10 +//! 2. `option_env!("HF_TOKEN")` in this module picks it up at compile time
11 +//! and bakes it into the binary.
12 +//!
13 +//! This works regardless of whether the build is triggered from a terminal
14 +//! (where shell env vars are available) or from Xcode (which strips the
15 +//! shell environment).
16 +//!
17 +//! ## Setup
18 +//!
19 +//! Set the `HF_TOKEN` environment variable before building, or add it to
20 +//! a `.env` file that your app's `build.rs` loads:
21 +//!
22 +//! ```text
23 +//! HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
24 +//! ```
25 +//!
26 +//! Then build normally — no extra env vars needed.
27 +//!
28 +//! ## Runtime behaviour
29 +//!
30 +//! [`hf_token_source`] returns:
31 +//!
32 +//! 1. `TokenSource::Literal(token)` — if `HF_TOKEN` was present at compile time.
33 +//! 2. `TokenSource::CacheToken` — otherwise (reads `~/.cache/huggingface/token`;
34 +//! works on macOS after `mistralrs login`).
35 +
36 +#[cfg(any(
37 + target_os = "macos",
38 + target_os = "ios",
39 + target_os = "tvos",
40 + target_os = "windows",
41 + target_os = "android"
42 +))]
43 +use mistralrs::TokenSource;
44 +
45 +/// Token baked in at compile time (if the `HF_TOKEN` env var was set).
46 +///
47 +/// `option_env!` is evaluated by `rustc` during compilation — the resulting
48 +/// value is a `Option<&'static str>` embedded directly in the binary.
49 +/// This means the token never needs to exist on the device's filesystem.
50 +const BUILD_TIME_HF_TOKEN: Option<&str> = option_env!("HF_TOKEN");
51 +
52 +/// Resolve the best available [`TokenSource`] for Hugging Face Hub requests.
53 +///
54 +/// Prefer the compile-time literal when available (required on iOS), otherwise
55 +/// fall back to the on-disk cache token (the default on macOS where
56 +/// `mistralrs login` has been run).
57 +#[cfg(any(
58 + target_os = "macos",
59 + target_os = "ios",
60 + target_os = "tvos",
61 + target_os = "windows",
62 + target_os = "android"
63 +))]
64 +pub fn hf_token_source() -> TokenSource {
65 + match BUILD_TIME_HF_TOKEN {
66 + Some(token) if !token.is_empty() => {
67 + log::debug!("Using build-time HF_TOKEN for Hugging Face authentication.");
68 + TokenSource::Literal(token.to_string())
69 + }
70 + _ => {
71 + log::debug!(
72 + "No build-time HF_TOKEN found; falling back to cached token \
73 + (~/.cache/huggingface/token)."
74 + );
75 + TokenSource::CacheToken
76 + }
77 + }
78 +}
79 +
80 +#[cfg(test)]
81 +mod tests {
82 + use super::*;
83 +
84 + #[test]
85 + fn build_time_token_is_option() {
86 + // This just ensures the constant compiles and is the expected type.
87 + let _: Option<&str> = BUILD_TIME_HF_TOKEN;
88 + }
89 +
90 + #[test]
91 + #[cfg(any(
92 + target_os = "macos",
93 + target_os = "ios",
94 + target_os = "tvos",
95 + target_os = "windows",
96 + target_os = "android"
97 + ))]
98 + fn token_source_returns_valid_variant() {
99 + let source = hf_token_source();
100 + // We can't know at test time whether HF_TOKEN was set, but we can
101 + // verify it returns *something* without panicking.
102 + let display = format!("{}", source);
103 + assert!(
104 + display.starts_with("literal:") || display == "cache",
105 + "Unexpected token source: {display}"
106 + );
107 + }
108 +}
src/inference/types.rs new
+477
@@ -0,0 +1,477 @@
1 +//! Shared types for the on-device LLM inference engine.
2 +//!
3 +//! These types are intentionally framework-agnostic — they do **not** depend on
4 +//! any UI framework. They carry UniFFI derive annotations so that
5 +//! UniFFI bindgen can generate Swift / Kotlin bindings automatically.
6 +//!
7 +//! ## UniFFI compatibility notes
8 +//!
9 +//! - `usize` is **not** supported by UniFFI → we use `u64` / `u32` instead.
10 +//! - Enums use `#[derive(uniffi::Enum)]`.
11 +//! - Plain data structs use `#[derive(uniffi::Record)]`.
12 +//! - Error enums use `#[derive(uniffi::Error)]`.
13 +
14 +use serde::{Deserialize, Serialize};
15 +
16 +// ── Message role ─────────────────────────────────────────────────────────────
17 +
18 +/// Role of a message in a chat conversation.
19 +///
20 +/// Mirrors the standard OpenAI / mistral.rs role taxonomy but is decoupled
21 +/// from `mistralrs::TextMessageRole` so that callers don't need a direct
22 +/// dependency on mistral.rs.
23 +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, uniffi::Enum)]
24 +#[serde(rename_all = "lowercase")]
25 +pub enum ChatRole {
26 + System,
27 + User,
28 + Assistant,
29 +}
30 +
31 +impl std::fmt::Display for ChatRole {
32 + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
33 + match self {
34 + ChatRole::System => write!(f, "system"),
35 + ChatRole::User => write!(f, "user"),
36 + ChatRole::Assistant => write!(f, "assistant"),
37 + }
38 + }
39 +}
40 +
41 +// ── Chat message ─────────────────────────────────────────────────────────────
42 +
43 +/// A single message in a conversation.
44 +#[derive(Debug, Clone, Serialize, Deserialize, uniffi::Record)]
45 +pub struct ChatMessage {
46 + pub role: ChatRole,
47 + pub content: String,
48 +}
49 +
50 +impl ChatMessage {
51 + pub fn system(content: impl Into<String>) -> Self {
52 + Self {
53 + role: ChatRole::System,
54 + content: content.into(),
55 + }
56 + }
57 +
58 + pub fn user(content: impl Into<String>) -> Self {
59 + Self {
60 + role: ChatRole::User,
61 + content: content.into(),
62 + }
63 + }
64 +
65 + pub fn assistant(content: impl Into<String>) -> Self {
66 + Self {
67 + role: ChatRole::Assistant,
68 + content: content.into(),
69 + }
70 + }
71 +}
72 +
73 +// ── Sampling configuration ───────────────────────────────────────────────────
74 +
75 +/// Sampling parameters for text generation.
76 +///
77 +/// All fields are optional — `None` means "use the engine default".
78 +///
79 +/// Note: `usize` is replaced with `u64` for UniFFI compatibility.
80 +#[derive(Debug, Clone, Serialize, Deserialize, uniffi::Record)]
81 +pub struct SamplingConfig {
82 + /// Sampling temperature (higher = more random). Typical range: 0.0–2.0.
83 + pub temperature: Option<f64>,
84 + /// Nucleus (top-p) sampling threshold. Typical value: 0.9–0.95.
85 + pub top_p: Option<f64>,
86 + /// Top-k sampling limit.
87 + pub top_k: Option<u64>,
88 + /// Min-p sampling threshold.
89 + pub min_p: Option<f64>,
90 + /// Maximum number of tokens to generate.
91 + pub max_tokens: Option<u64>,
92 + /// Frequency penalty (penalise tokens proportional to occurrence count).
93 + pub frequency_penalty: Option<f32>,
94 + /// Presence penalty (penalise tokens that appeared at all).
95 + pub presence_penalty: Option<f32>,
96 +}
97 +
98 +impl Default for SamplingConfig {
99 + /// Sensible creative-chat defaults.
100 + fn default() -> Self {
101 + Self {
102 + temperature: Some(0.7),
103 + top_p: Some(0.95),
104 + top_k: None,
105 + min_p: None,
106 + max_tokens: Some(512),
107 + frequency_penalty: None,
108 + presence_penalty: None,
109 + }
110 + }
111 +}
112 +
113 +impl SamplingConfig {
114 + /// Deterministic sampling (temperature = 0, greedy decoding).
115 + pub fn deterministic() -> Self {
116 + Self {
117 + temperature: Some(0.0),
118 + top_p: None,
119 + top_k: None,
120 + min_p: None,
121 + max_tokens: Some(512),
122 + frequency_penalty: None,
123 + presence_penalty: None,
124 + }
125 + }
126 +
127 + /// Conservative mobile defaults — lower max_tokens for faster response on
128 + /// constrained devices (CPU-only ARM SoCs doing ~1-3 tok/sec).
129 + pub fn mobile() -> Self {
130 + Self {
131 + temperature: Some(0.7),
132 + top_p: Some(0.95),
133 + top_k: None,
134 + min_p: None,
135 + max_tokens: Some(128),
136 + frequency_penalty: None,
137 + presence_penalty: None,
138 + }
139 + }
140 +
141 + /// Sampling config tuned for coding assistants.
142 + ///
143 + /// Key differences from the general-purpose default:
144 + ///
145 + /// - **`frequency_penalty = 1.15`** — the primary fix for the repeated-token
146 + /// loop you saw with small GGUF models. Penalises each token proportional
147 + /// to how many times it has already appeared in the output, making it
148 + /// progressively harder to emit the same word again. 1.15 is aggressive
149 + /// enough to kill echo loops without hurting code token repetition
150 + /// (identifiers, keywords) at typical response lengths.
151 + ///
152 + /// - **`presence_penalty = 0.1`** — a light nudge away from any token that
153 + /// has appeared at all, encouraging lexical variety across the response.
154 + ///
155 + /// - **`temperature = 0.2`** — low randomness keeps generated code
156 + /// deterministic and syntactically correct. High temperature causes
157 + /// hallucinated variable names and broken syntax in code blocks.
158 + ///
159 + /// - **`top_p = 0.95`** — nucleus sampling retained to avoid degenerate
160 + /// greedy outputs while still keeping the distribution tight.
161 + ///
162 + /// - **`max_tokens = 1024`** — doubled vs. the chat default to allow
163 + /// complete function / class explanations without truncation.
164 + pub fn coding() -> Self {
165 + Self {
166 + temperature: Some(0.2),
167 + top_p: Some(0.95),
168 + top_k: None,
169 + min_p: None,
170 + max_tokens: Some(1024),
171 + frequency_penalty: Some(1.15),
172 + presence_penalty: Some(0.1),
173 + }
174 + }
175 +
176 + /// Like [`coding`](Self::coding) but capped at 256 tokens for mobile
177 + /// devices where latency matters more than completeness.
178 + pub fn coding_mobile() -> Self {
179 + Self {
180 + max_tokens: Some(256),
181 + ..Self::coding()
182 + }
183 + }
184 +}
185 +
186 +// ── GGUF model configuration ─────────────────────────────────────────────────
187 +
188 +/// Configuration for loading a GGUF model via mistral.rs.
189 +///
190 +/// This is the primary model format for on-device mobile/desktop inference
191 +/// because GGUF files are pre-quantized and load at their compressed size,
192 +/// avoiding the large memory spike of in-situ quantization.
193 +#[derive(Debug, Clone, Serialize, Deserialize, uniffi::Record)]
194 +pub struct GgufModelConfig {
195 + /// HuggingFace model repository ID, e.g. `"bartowski/Qwen2.5-1.5B-Instruct-GGUF"`.
196 + pub model_id: String,
197 + /// GGUF filename(s) within the repository, e.g. `["Qwen2.5-1.5B-Instruct-Q4_K_M.gguf"]`.
198 + pub files: Vec<String>,
199 + /// Optional: explicit tokenizer model ID (required on Android where GGUF
200 + /// embedded tokenizers are not supported by the candle backend).
201 + pub tok_model_id: Option<String>,
202 + /// Human-friendly display name, e.g. `"Qwen 2.5 1.5B"`.
203 + pub display_name: String,
204 + /// Approximate memory footprint description, e.g. `"~941 MB (GGUF Q4_K_M)"`.
205 + pub approx_memory: String,
206 +}
207 +
208 +// ── ISQ model configuration ──────────────────────────────────────────────────
209 +
210 +/// Configuration for loading an ISQ (in-situ quantised) model via mistral.rs
211 +/// `TextModelBuilder`.
212 +///
213 +/// Unlike GGUF models, ISQ models are loaded directly from a HuggingFace
214 +/// safetensors repo and quantised in-situ on the device. They require more
215 +/// RAM during the load phase but support a wider range of architectures
216 +/// (e.g. DeepSeek-Coder-V2-Lite, Qwen2.5-Coder-7B).
217 +///
218 +/// **macOS-only for now** — ISQ with Metal acceleration requires the `metal`
219 +/// feature on mistral.rs. The struct is still defined on all platforms so
220 +/// that types.rs compiles everywhere without `cfg` gates on the record
221 +/// (UniFFI requires all record types to be reachable without feature flags).
222 +#[derive(Debug, Clone, Serialize, Deserialize, uniffi::Record)]
223 +pub struct IsqModelConfig {
224 + /// HuggingFace model repository ID, e.g. `"Qwen/Qwen2.5-Coder-7B-Instruct"`.
225 + pub model_id: String,
226 + /// Number of quantisation bits. `4` → Q4K (AFQ4 on Metal), `8` → Q8_0 (AFQ8 on Metal).
227 + /// Any value outside `{4, 8}` defaults to 4-bit.
228 + pub isq_bits: u8,
229 + /// Human-friendly display name, e.g. `"Qwen 2.5 Coder 7B"`.
230 + pub display_name: String,
231 + /// Approximate memory footprint description, e.g. `"~4.5 GB (Q4K, Metal)"`.
232 + pub approx_memory: String,
233 +}
234 +
235 +impl IsqModelConfig {
236 + /// Qwen 2.5 Coder 7B Instruct — ISQ 4-bit, Metal-accelerated (~4.5 GB).
237 + ///
238 + /// Best macOS desktop coding quality. Loaded from the official HF repo
239 + /// and quantised in-situ; requires ~8 GB RAM during the load phase.
240 + pub fn qwen25_coder_7b_isq4() -> Self {
241 + Self {
242 + model_id: super::models::QWEN25_CODER_7B_INSTRUCT.into(),
243 + isq_bits: 4,
244 + display_name: "Qwen 2.5 Coder 7B (ISQ 4-bit)".into(),
245 + approx_memory: "~4.5 GB (ISQ Q4K, Metal)".into(),
246 + }
247 + }
248 +
249 + /// Qwen 2.5 Coder 7B Instruct — ISQ 8-bit, Metal-accelerated (~9 GB).
250 + ///
251 + /// Higher quality than the 4-bit variant; requires ~12 GB RAM on load.
252 + /// Suitable for Macs with 16 GB+ unified memory.
253 + pub fn qwen25_coder_7b_isq8() -> Self {
254 + Self {
255 + model_id: super::models::QWEN25_CODER_7B_INSTRUCT.into(),
256 + isq_bits: 8,
257 + display_name: "Qwen 2.5 Coder 7B (ISQ 8-bit)".into(),
258 + approx_memory: "~9 GB (ISQ Q8K, Metal)".into(),
259 + }
260 + }
261 +}
262 +
263 +// ── Inference result ─────────────────────────────────────────────────────────
264 +
265 +/// The result of a chat inference request.
266 +#[derive(Debug, Clone, Serialize, Deserialize, uniffi::Record)]
267 +pub struct InferenceResult {
268 + /// The generated assistant reply text.
269 + pub text: String,
270 + /// Wall-clock duration of the inference in seconds.
271 + pub duration_secs: f64,
272 + /// Human-readable duration string (e.g. `"2m 3.1s"` or `"4.5s"`).
273 + pub duration_display: String,
274 + /// Finish reason reported by the model (e.g. `"stop"`, `"length"`).
275 + pub finish_reason: String,
276 +}
277 +
278 +// ── Streaming chunk ──────────────────────────────────────────────────────────
279 +
280 +/// A single streaming token chunk from the inference engine.
281 +#[derive(Debug, Clone, Serialize, Deserialize, uniffi::Record)]
282 +pub struct StreamChunk {
283 + /// The token text delta (may be empty for the final chunk).
284 + pub delta: String,
285 + /// Whether this is the final chunk in the stream.
286 + pub done: bool,
287 + /// Finish reason (only set on the final chunk).
288 + pub finish_reason: Option<String>,
289 +}
290 +
291 +// ── Engine status ────────────────────────────────────────────────────────────
292 +
293 +/// Lifecycle status of the inference engine.
294 +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, uniffi::Enum)]
295 +#[serde(rename_all = "snake_case")]
296 +pub enum EngineStatus {
297 + /// No model loaded.
298 + Unloaded,
299 + /// Model is being downloaded / loaded into memory.
300 + Loading,
301 + /// Model is loaded and ready to accept requests.
302 + Ready,
303 + /// Model is currently running inference.
304 + Generating,
305 + /// An error occurred (model may or may not still be loaded).
306 + Error,
307 +}
308 +
309 +impl std::fmt::Display for EngineStatus {
310 + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
311 + match self {
312 + EngineStatus::Unloaded => write!(f, "unloaded"),
313 + EngineStatus::Loading => write!(f, "loading"),
314 + EngineStatus::Ready => write!(f, "ready"),
315 + EngineStatus::Generating => write!(f, "generating"),
316 + EngineStatus::Error => write!(f, "error"),
317 + }
318 + }
319 +}
320 +
321 +// ── Engine info snapshot ─────────────────────────────────────────────────────
322 +
323 +/// A point-in-time snapshot of the engine's state, suitable for status UIs.
324 +///
325 +/// Note: `history_length` is `u64` (not `usize`) for UniFFI compatibility.
326 +#[derive(Debug, Clone, Serialize, Deserialize, uniffi::Record)]
327 +pub struct EngineInfo {
328 + pub status: EngineStatus,
329 + /// Display name of the currently loaded model, if any.
330 + pub model_name: Option<String>,
331 + /// Approximate memory footprint description, if a model is loaded.
332 + pub approx_memory: Option<String>,
333 + /// Number of conversation turns in the current history.
334 + pub history_length: u64,
335 +}
336 +
337 +// ── Error type ───────────────────────────────────────────────────────────────
338 +
339 +/// Errors that can occur during inference engine operations.
340 +///
341 +/// Exposed to Swift / Kotlin as a sealed class / enum via `uniffi::Error`.
342 +#[derive(Debug, thiserror::Error, uniffi::Error)]
343 +pub enum InferenceError {
344 + #[error("No model loaded — call `load_model` first")]
345 + NoModelLoaded,
346 +
347 + #[error("Model is already loaded: {model_name}")]
348 + AlreadyLoaded { model_name: String },
349 +
350 + #[error("Failed to build model: {reason}")]
351 + ModelBuild { reason: String },
352 +
353 + #[error("Inference failed: {reason}")]
354 + Inference { reason: String },
355 +
356 + #[error("Model loading was cancelled")]
357 + Cancelled,
358 +
359 + #[error("{reason}")]
360 + Other { reason: String },
361 +}
362 +
363 +// ── Duration formatting helper ───────────────────────────────────────────────
364 +
365 +/// Format a `std::time::Duration` as `Xm Ys` or just `Ys` when under a minute.
366 +pub fn format_duration(d: std::time::Duration) -> String {
367 + let total_secs = d.as_secs_f64();
368 + let mins = (total_secs / 60.0).floor() as u64;
369 + let secs = total_secs - (mins as f64 * 60.0);
370 + if mins > 0 {
371 + format!("{}m {:.1}s", mins, secs)
372 + } else {
373 + format!("{:.1}s", secs)
374 + }
375 +}
376 +
377 +#[cfg(test)]
378 +mod tests {
379 + use super::*;
380 +
381 + #[test]
382 + fn chat_message_constructors() {
383 + let sys = ChatMessage::system("You are helpful.");
384 + assert_eq!(sys.role, ChatRole::System);
385 + assert_eq!(sys.content, "You are helpful.");
386 +
387 + let user = ChatMessage::user("Hello");
388 + assert_eq!(user.role, ChatRole::User);
389 +
390 + let asst = ChatMessage::assistant("Hi there!");
391 + assert_eq!(asst.role, ChatRole::Assistant);
392 + }
393 +
394 + #[test]
395 + fn chat_role_display() {
396 + assert_eq!(ChatRole::System.to_string(), "system");
397 + assert_eq!(ChatRole::User.to_string(), "user");
398 + assert_eq!(ChatRole::Assistant.to_string(), "assistant");
399 + }
400 +
401 + #[test]
402 + fn chat_role_serde_rename() {
403 + // Verify that #[serde(rename_all = "lowercase")] is applied by
404 + // checking the Display impl matches the expected serialised form.
405 + // (Full serde_json round-trip is tested in integration tests that
406 + // already depend on serde_json.)
407 + assert_eq!(format!("{}", ChatRole::System), "system");
408 + assert_eq!(format!("{}", ChatRole::User), "user");
409 + assert_eq!(format!("{}", ChatRole::Assistant), "assistant");
410 + }
411 +
412 + #[test]
413 + fn sampling_config_defaults() {
414 + let cfg = SamplingConfig::default();
415 + assert_eq!(cfg.temperature, Some(0.7));
416 + assert_eq!(cfg.top_p, Some(0.95));
417 + assert_eq!(cfg.max_tokens, Some(512));
418 + }
419 +
420 + #[test]
421 + fn sampling_config_deterministic() {
422 + let cfg = SamplingConfig::deterministic();
423 + assert_eq!(cfg.temperature, Some(0.0));
424 + assert!(cfg.top_p.is_none());
425 + }
426 +
427 + #[test]
428 + fn sampling_config_mobile() {
429 + let cfg = SamplingConfig::mobile();
430 + assert_eq!(cfg.max_tokens, Some(128));
431 + }
432 +
433 + #[test]
434 + fn engine_status_display() {
435 + assert_eq!(EngineStatus::Ready.to_string(), "ready");
436 + assert_eq!(EngineStatus::Generating.to_string(), "generating");
437 + }
438 +
439 + #[test]
440 + fn format_duration_under_minute() {
441 + let d = std::time::Duration::from_secs_f64(4.567);
442 + assert_eq!(format_duration(d), "4.6s");
443 + }
444 +
445 + #[test]
446 + fn format_duration_over_minute() {
447 + let d = std::time::Duration::from_secs_f64(125.3);
448 + assert_eq!(format_duration(d), "2m 5.3s");
449 + }
450 +
451 + #[test]
452 + fn inference_error_display() {
453 + let err = InferenceError::NoModelLoaded;
454 + assert_eq!(err.to_string(), "No model loaded — call `load_model` first");
455 +
456 + let err = InferenceError::ModelBuild {
457 + reason: "out of memory".into(),
458 + };
459 + assert_eq!(err.to_string(), "Failed to build model: out of memory");
460 +
461 + let err = InferenceError::AlreadyLoaded {
462 + model_name: "Qwen 2.5".into(),
463 + };
464 + assert_eq!(err.to_string(), "Model is already loaded: Qwen 2.5");
465 + }
466 +
467 + #[test]
468 + fn engine_info_history_is_u64() {
469 + let info = EngineInfo {
470 + status: EngineStatus::Ready,
471 + model_name: Some("Test".into()),
472 + approx_memory: None,
473 + history_length: 42,
474 + };
475 + assert_eq!(info.history_length, 42u64);
476 + }
477 +}
src/lib.rs new
+66
@@ -0,0 +1,66 @@
1 +//! # Onde
2 +//!
3 +//! **On-device chat inference for cross-platform apps.**
4 +//!
5 +//! Run LLM chat locally — no cloud, no latency, no data leaving the device.
6 +//!
7 +//! Onde wraps [mistral.rs](https://github.com/EricLBuehler/mistral.rs) with a
8 +//! unified API for model discovery, HuggingFace Hub downloads, cache
9 +//! management, and GPU acceleration across every platform.
10 +//!
11 +//! Built by [Onde Inference](https://ondeinference.com)
12 +//!
13 +//! ## Modules
14 +//!
15 +//! - [`hf_cache`] — HuggingFace Hub cache inspection, repair, and model
16 +//! download with a framework-agnostic progress-callback API.
17 +//! - [`inference`] — Chat inference engine, UniFFI FFI wrapper, model metadata,
18 +//! and HuggingFace token resolution.
19 +//!
20 +//! ## Re-exports
21 +//!
22 +//! `mistralrs`, `hf_hub`, and `mistralrs_core` are re-exported so that apps
23 +//! depending on `onde` do not need their own direct dependency on those crates.
24 +//! Access them as `onde::mistralrs`, `onde::hf_hub`, and `onde::mistralrs_core`.
25 +//!
26 +//! ## Example
27 +//!
28 +//! ```rust,ignore
29 +//! use onde::inference::ChatEngine;
30 +//! use onde::inference::GgufModelConfig;
31 +//!
32 +//! let engine = ChatEngine::new();
33 +//! engine
34 +//! .load_gguf_model(
35 +//! GgufModelConfig::platform_default(),
36 +//! Some("You are a helpful assistant.".into()),
37 +//! None,
38 +//! )
39 +//! .await?;
40 +//!
41 +//! let result = engine.send_message("Hello!").await?;
42 +//! println!("{}", result.text);
43 +//! ```
44 +
45 +pub mod hf_cache;
46 +pub mod inference;
47 +
48 +uniffi::setup_scaffolding!();
49 +
50 +// Re-export mistralrs for every platform that onde supports.
51 +// Apps use `onde::mistralrs::Model` etc. instead of declaring a direct dep.
52 +#[cfg(any(
53 + target_os = "macos",
54 + target_os = "ios",
55 + target_os = "tvos",
56 + target_os = "windows",
57 + target_os = "linux",
58 + target_os = "android"
59 +))]
60 +pub use mistralrs;
61 +
62 +// Android-only re-exports needed for the GLOBAL_HF_CACHE workaround.
63 +#[cfg(target_os = "android")]
64 +pub use hf_hub;
65 +#[cfg(target_os = "android")]
66 +pub use mistralrs_core;
tvos_chkstk.s new
+23
@@ -0,0 +1,23 @@
1 +// tvos_chkstk.s — stub for ___chkstk_darwin on tvOS
2 +//
3 +// On macOS and iOS, ___chkstk_darwin is provided by libSystem as a stack
4 +// probing function. On arm64 Apple platforms the kernel grows the stack
5 +// automatically via guard pages, so the probe is effectively a no-op that
6 +// just returns. tvOS does NOT export this symbol, but aws-lc-sys assembly
7 +// references it unconditionally.
8 +//
9 +// This file provides the missing symbol so the linker succeeds.
10 +// It is only compiled for tvOS targets (see build.rs).
11 +//
12 +// Symbol naming:
13 +// Mach-O adds one leading underscore to C symbols automatically.
14 +// The linker error says: Undefined symbol "___chkstk_darwin"
15 +// That means the C-level name is "__chkstk_darwin" (two underscores).
16 +// In assembly we write the raw Mach-O symbol = "___chkstk_darwin" (three underscores).
17 +
18 +.text
19 +.globl ___chkstk_darwin
20 +.p2align 2
21 +
22 +___chkstk_darwin:
23 + ret
uniffi-bindgen/.gitignore new
+2
@@ -0,0 +1,2 @@
1 +target/
2 +Cargo.lock
uniffi-bindgen/Cargo.toml new
+12
@@ -0,0 +1,12 @@
1 +[package]
2 +name = "onde-uniffi-bindgen"
3 +version = "0.1.0"
4 +edition = "2021"
5 +publish = false
6 +
7 +[[bin]]
8 +name = "uniffi-bindgen"
9 +path = "uniffi-bindgen.rs"
10 +
11 +[dependencies]
12 +uniffi = { version = "=0.31.0", features = ["cli"] }
uniffi-bindgen/uniffi-bindgen.rs new
+3
@@ -0,0 +1,3 @@
1 +fn main() {
2 + uniffi::uniffi_bindgen_main()
3 +}