@ondeinference / onde / commits / f12567e

Switch to onde-mistralrs crates for publish; update SDK release docs

- Replace git mistralrs deps with onde-mistralrs crates in Cargo.toml for all platforms - Add detailed crates.io publish workflow, onde-mistralrs strategy, and GresIQ/HF_TOKEN secrets handling to sdk-regressions skill - Update README with model license table and clarify copyright - Update Cargo.lock for onde-mistralrs and candle-core version changes

keypair34 committed Apr 26, 2026 at 13:50 UTC f12567eba7f6329bc71e659bb0baceeb4e8721cb
4 files changed +526 -356
.agents/skills/sdk-regressions/SKILL.md
+213 -2
@@ -1,6 +1,6 @@
1 ---
2 name: sdk-regressions
3 -description: Cross-compilation regression checks for the onde crate. Covers macOS, iOS, tvOS (nightly), Android, Windows, and Linux target verification. Run before merging any change that touches platform-gated code, Cargo.toml, .cargo/config.toml, or build.rs.
3 +description: Cross-compilation regression checks for the onde crate. Covers macOS, iOS, tvOS (nightly), Android, Windows, and Linux target verification. Run before merging any change that touches platform-gated code, Cargo.toml, .cargo/config.toml, or build.rs. Also documents the onde-mistralrs fork publishing strategy and cargo publish workflow.
4 allowed-tools: Read, Edit, Glob, Grep, Terminal
5 user-invocable: true
6 ---
@@ -25,6 +25,9 @@ macOS can silently break iOS, tvOS, or Android because:
25 - Assembly stubs (`tvos_chkstk.s`) and `build.rs` gating
26 - `.cargo/config.toml` sets `+fp16` rustflags on 5 Apple targets
27
28 +See also: **[`cargo publish` workflow](#cargo-publish-workflow)** at the bottom
29 +of this file — must be followed when releasing a new `onde` version.
30 +
31 ---
32
33 ## Tier 1 — Must Pass on Every PR (stable toolchain)
@@ -303,6 +306,214 @@ Any other match means a sandboxed path is using the wrong subdirectory.
306
307 ---
308
309 +---
310 +
311 +## Publishing to crates.io
312 +
313 +### Why we don't use upstream `mistralrs` directly
314 +
315 +The short version: our PRs aren't merged yet.
316 +
317 +`onde` depends on a personal fork of Eric Buehler's `mistral.rs`, kept at
318 +`github.com/setoelkahfi/mistral.rs` on the `fix/all-platform-fixes` branch.
319 +The fork is ahead of upstream by ~41 commits carrying fixes that only matter
320 +for Apple and Android targets — things upstream hasn't needed to care about:
321 +
322 +| Fix | Status |
323 +|-----|--------|
324 +| iOS Metal 3.0 support | PR open |
325 +| Android 32-bit memory limit constants | PR open |
326 +| HF_HOME propagation in diffusion/flux pipelines | PR open |
327 +| `metallib` link step `--sdk` / `-std` flags | PR open |
328 +
329 +Dropping the fork and pointing at upstream `mistralrs 0.8.1` would silently
330 +break iOS on entry-level devices and Android on 32-bit targets. Don't do it
331 +until those PRs land.
332 +
333 +**Legal note:** The fork is a derivative of Eric Buehler's MIT-licensed
334 +`mistral.rs`. His `LICENSE` file — including `Copyright (c) 2024 Eric Buehler`
335 +— must remain intact and unmodified in the fork at all times. See
336 +[`legal-and-trademarks/SKILL.md`](../legal-and-trademarks/SKILL.md) for full
337 +attribution obligations.
338 +
339 +### The `onde-mistralrs` workaround
340 +
341 +`cargo publish` strips every `git =` field before uploading to crates.io.
342 +That means a git-only dep is useless to anyone who installs `onde` from the
343 +registry — they can't resolve it.
344 +
345 +The solution: publish the fork to crates.io under the name `onde-mistralrs`
346 +(and `onde-mistralrs-core`, etc.), then reference it with `package =` so the
347 +rest of the code still compiles unchanged with `use mistralrs::...`:
348 +
349 +```toml
350 +# In onde/Cargo.toml — each platform target section looks like this.
351 +# Cargo resolves it as "onde-mistralrs" from crates.io,
352 +# but the Rust code sees it as `mistralrs`.
353 +mistralrs = { version = "0.8.2", package = "onde-mistralrs", features = ["metal"] }
354 +```
355 +
356 +This is the only supported publish path until the upstream PRs merge. Don't
357 +swap it for plain `mistralrs` from crates.io without checking every platform
358 +target first.
359 +
360 +---
361 +
362 +### Step 1 — check if `onde-mistralrs` needs a new publish
363 +
364 +First, see how far the fork has drifted from upstream:
365 +
366 +```bash
367 +curl -s "https://api.github.com/repos/setoelkahfi/mistral.rs/compare/EricLBuehler:mistral.rs:master...setoelkahfi:mistral.rs:fix%2Fall-platform-fixes" \
368 + | python3 -c "import sys,json; d=json.load(sys.stdin); print('ahead:', d['ahead_by'], 'behind:', d['behind_by'])"
369 +```
370 +
371 +If there are new commits since the last `onde-mistralrs` publish, or if
372 +upstream cut a new release that the fork has rebased onto, bump the workspace
373 +version in the fork and publish. Sub-crates must go in dependency order:
374 +
375 +```bash
376 +# 1. Edit the root Cargo.toml in setoelkahfi/mistral.rs
377 +# bump version, e.g. 0.8.1 → 0.8.2
378 +
379 +# 2. Publish in order — each one depends on the previous:
380 +cargo publish -p onde-mistralrs-macros
381 +cargo publish -p onde-mistralrs-paged-attn
382 +cargo publish -p onde-mistralrs-quant
383 +cargo publish -p onde-mistralrs-core
384 +cargo publish -p onde-mistralrs
385 +
386 +# 3. Give the index ~30 s to propagate before moving on.
387 +```
388 +
389 +---
390 +
391 +### Step 2 — update `onde/Cargo.toml`
392 +
393 +Every platform target section has its own `onde-mistralrs` line. They all need
394 +to move to the new version together — leaving one target on an older version
395 +causes a dependency conflict.
396 +
397 +```bash
398 +# All occurrences should print the same version number:
399 +grep 'onde-mistralrs' onde/Cargo.toml
400 +```
401 +
402 +---
403 +
404 +### Step 3 — dry-run first
405 +
406 +```bash
407 +cd onde
408 +cargo publish --dry-run --allow-dirty 2>&1 | tail -20
409 +```
410 +
411 +A clean run ends with `Finished` and no errors. Common things that go wrong:
412 +
413 +| Error | What to do |
414 +|-------|------------|
415 +| `all dependencies must have a version` | add `version = "..."` to the dep |
416 +| `no matching package named onde-mistralrs` | publish the fork first (step 1) |
417 +| `git specification will be removed` | expected — cargo is just telling you the git field gets stripped |
418 +| `files in working directory contain changes` | add `--allow-dirty` |
419 +
420 +---
421 +
422 +### Step 4 — publish `onde`, then update `sigit`
423 +
424 +```bash
425 +cargo publish
426 +```
427 +
428 +Then bump the version in `sigit/Cargo.toml`:
429 +
430 +```toml
431 +# git = is used for local dev; crates.io consumers resolve via version =
432 +onde = { version = "0.1.9", git = "https://github.com/ondeinference/onde", branch = "development" }
433 +```
434 +
435 +Keep both fields. The `git =` field takes precedence during local development;
436 +crates.io strips it and falls back to `version =` for anyone installing from
437 +the registry.
438 +
439 +---
440 +
441 +### GresIQ credentials and `HF_TOKEN`
442 +
443 +People building `onde` from crates.io don't need to do anything about secrets.
444 +Here's why.
445 +
446 +`build.rs` passes credentials to the compiler via `option_env!()`. That macro
447 +resolves at compile time — if the env var isn't set, it compiles to `None` and
448 +the feature that needs it is silently disabled. No crash, no partial state, no
449 +mystery error at runtime.
450 +
451 +```rust
452 +// pulse/client.rs — GresIQ telemetry (Onde Inference internal infrastructure)
453 +const EMBEDDED_API_KEY_DEV: Option<&str> = option_env!("GRESIQ_API_KEY_DEV");
454 +const EMBEDDED_API_SECRET_DEV: Option<&str> = option_env!("GRESIQ_API_SECRET_DEV");
455 +
456 +// token.rs — HuggingFace Hub authentication
457 +const BUILD_TIME_HF_TOKEN: Option<&str> = option_env!("HF_TOKEN");
458 +```
459 +
460 +Who gets what:
461 +
462 +| Who is building | Env vars present? | What happens |
463 +|-----------------|-------------------|--------------|
464 +| Our CI (official builds) | yes, via GitHub secrets | telemetry on; HF token baked into binary |
465 +| Someone who ran `cargo add onde` | no | telemetry off; HF token read from `~/.cache/huggingface/token` at runtime |
466 +| iOS/tvOS builds via XCFramework | yes, via `.env` file | HF token baked in — required because there's no writable filesystem on device |
467 +
468 +The GresIQ credentials are Onde Inference's own infrastructure secrets — they
469 +exist so the SDK can phone home for usage telemetry on official builds. External
470 +consumers don't have them, don't need them, and won't be affected by not having
471 +them.
472 +
473 +Never put `GRESIQ_API_KEY_*` or `HF_TOKEN` in `README.md`, `.env.example`, or
474 +any committed file. The `.env` in the crate root is git-ignored for a reason:
475 +
476 +```bash
477 +# onde/.env — git-ignored, never commit
478 +GRESIQ_API_KEY_DEV=...
479 +GRESIQ_API_SECRET_DEV=...
480 +GRESIQ_API_KEY_PRODUCTION=...
481 +GRESIQ_API_SECRET_PRODUCTION=...
482 +HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
483 +```
484 +
485 +---
486 +
487 +### Working against the git fork locally
488 +
489 +If you're making changes to the fork and want `onde` to pick them up without
490 +publishing, uncomment the patch block in `onde/Cargo.toml`:
491 +
492 +```toml
493 +[patch.crates-io]
494 +onde-mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", package = "mistralrs" }
495 +onde-mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", package = "mistralrs-core" }
496 +```
497 +
498 +Re-comment it before running `cargo publish`. Cargo strips `[patch.crates-io]`
499 +from the published manifest automatically, so it won't affect registry
500 +consumers either way — but leaving it active makes the dry-run confusing.
501 +
502 +---
503 +
504 +### Pre-publish checklist
505 +
506 +- [ ] all `onde-mistralrs` version numbers in `onde/Cargo.toml` are the same
507 +- [ ] `onde/Cargo.toml` `[package] version` is bumped
508 +- [ ] `sigit/Cargo.toml` `onde` version matches
509 +- [ ] `[patch.crates-io]` git override is commented out
510 +- [ ] `cargo publish --dry-run --allow-dirty` passes clean
511 +- [ ] `.env` is not listed in the dry-run's packaged file output
512 +- [ ] Eric Buehler's `LICENSE` in the `mistral.rs` fork is unmodified
513 +
514 +---
515 +
516 *Last updated: July 2025 — added hf-hub/mistralrs-core iOS/tvOS deps,
307 -cache path consistency checks, App Group convention.*
517 +cache path consistency checks, App Group convention, onde-mistralrs publish
518 +strategy, GresIQ/HF_TOKEN secrets documentation.*
519
Cargo.lock
+269 -320
@@ -590,14 +590,14 @@ dependencies = [
590
591 [[package]]
592 name = "candle-core"
593 -version = "0.10.2"
593 +version = "0.9.2"
594 source = "registry+https://github.com/rust-lang/crates.io-index"
595 -checksum = "6bd9895436c1ba5dc1037a19935d084b838db066ff4e15ef7dded020b7c12a4a"
595 +checksum = "c15b675b80d994b2eadb20a4bbe434eabeb454eac3ee5e2b4cf6f147ee9be091"
596 dependencies = [
597 "byteorder",
598 "candle-metal-kernels",
599 "candle-ug",
600 - "float8",
600 + "float8 0.6.1",
601 "gemm 0.19.0",
602 "half",
603 "libm",
@@ -611,16 +611,15 @@ dependencies = [
611 "rayon",
612 "safetensors 0.7.0",
613 "thiserror 2.0.18",
614 - "tokenizers 0.22.2",
614 "yoke 0.8.2",
615 "zip",
616 ]
617
618 [[package]]
619 name = "candle-metal-kernels"
621 -version = "0.10.2"
620 +version = "0.9.2"
621 source = "registry+https://github.com/rust-lang/crates.io-index"
623 -checksum = "4b6b5a4cae6b4e1ab0efcee4dc05272d11b374a3d1ba121b3a961e36be54ab60"
622 +checksum = "2fdfe9d06de16ce49961e49084e5b79a75a9bdf157246e7c7b6328e87a7aa25d"
623 dependencies = [
624 "half",
625 "objc2",
@@ -633,9 +632,9 @@ dependencies = [
632
633 [[package]]
634 name = "candle-nn"
636 -version = "0.10.2"
635 +version = "0.9.2"
636 source = "registry+https://github.com/rust-lang/crates.io-index"
638 -checksum = "a9317a09d6530b758990ed7f625ac69ff43653bc9ee28b0464644ad1169ada87"
637 +checksum = "3045fa9e7aef8567d209a27d56b692f60b96f4d0569f4c3011f8ca6715c65e03"
638 dependencies = [
639 "candle-core",
640 "candle-metal-kernels",
@@ -651,9 +650,9 @@ dependencies = [
650
651 [[package]]
652 name = "candle-ug"
654 -version = "0.10.2"
653 +version = "0.9.2"
654 source = "registry+https://github.com/rust-lang/crates.io-index"
656 -checksum = "ca0fc3167cbc99c8ec1be618cb620aa21dca95038f118c3579a79370e3dc5f77"
655 +checksum = "c22d62be69068bf58987a45f690612739d8d2ea1bf508c1b87dc6815a019575d"
656 dependencies = [
657 "ug",
658 "ug-metal",
@@ -885,15 +884,6 @@ dependencies = [
884 "unicode-segmentation",
885 ]
886
888 -[[package]]
889 -name = "convert_case"
890 -version = "0.10.0"
891 -source = "registry+https://github.com/rust-lang/crates.io-index"
892 -checksum = "633458d4ef8c78b72454de2d54fd6ab2e60f9e02be22f3c6104cdc8a4e0fceb9"
893 -dependencies = [
894 - "unicode-segmentation",
895 -]
896 -
887 [[package]]
888 name = "core-foundation"
889 version = "0.9.4"
@@ -976,17 +966,15 @@ checksum = "d0a5c400df2834b80a4c3327b3aad3a4c4cd4de0629063962b03235697506a28"
966
967 [[package]]
968 name = "crossterm"
979 -version = "0.29.0"
969 +version = "0.25.0"
970 source = "registry+https://github.com/rust-lang/crates.io-index"
981 -checksum = "d8b9f2e4c67f833b660cdb0a3523065869fb35570177239812ed4c905aeff87b"
971 +checksum = "e64e6c0fbe2c17357405f7c758c1ef960fce08bdfb2c03d88d2a18d7e09c4b67"
972 dependencies = [
983 - "bitflags 2.11.1",
973 + "bitflags 1.3.2",
974 "crossterm_winapi",
985 - "derive_more",
986 - "document-features",
987 - "mio",
975 + "libc",
976 + "mio 0.8.11",
977 "parking_lot",
989 - "rustix",
978 "signal-hook",
979 "signal-hook-mio",
980 "winapi",
@@ -1253,7 +1241,6 @@ version = "2.1.1"
1241 source = "registry+https://github.com/rust-lang/crates.io-index"
1242 checksum = "799a97264921d8623a957f6c3b9011f3b5492f557bbb7a5a19b7fa6d06ba8dcb"
1243 dependencies = [
1256 - "convert_case 0.10.0",
1244 "proc-macro2",
1245 "quote",
1246 "rustc_version",
@@ -1339,15 +1326,6 @@ version = "1.1.1"
1326 source = "registry+https://github.com/rust-lang/crates.io-index"
1327 checksum = "c2db04e74f0a9a93103b50e90b96024c9b2bdca8bce6a632ec71b88736d3d359"
1328
1342 -[[package]]
1343 -name = "document-features"
1344 -version = "0.2.12"
1345 -source = "registry+https://github.com/rust-lang/crates.io-index"
1346 -checksum = "d4b8a88685455ed29a21542a33abd9cb6510b6b129abadabdcef0f4c55bc8f61"
1347 -dependencies = [
1348 - "litrs",
1349 -]
1350 -
1329 [[package]]
1330 name = "dotenvy"
1331 version = "0.15.7"
@@ -1592,9 +1570,9 @@ dependencies = [
1570
1571 [[package]]
1572 name = "float8"
1595 -version = "0.7.0"
1573 +version = "0.6.1"
1574 source = "registry+https://github.com/rust-lang/crates.io-index"
1597 -checksum = "c2d1f04709a8ac06e8e8042875a3c466cc4832d3c1a18dbcb9dba3c6e83046bc"
1575 +checksum = "719a903cc23e4a89e87962c2a80fdb45cdaad0983a89bd150bb57b4c8571a7d5"
1576 dependencies = [
1577 "half",
1578 "num-traits",
@@ -1602,6 +1580,15 @@ dependencies = [
1580 "rand_distr 0.5.1",
1581 ]
1582
1583 +[[package]]
1584 +name = "float8"
1585 +version = "0.7.0"
1586 +source = "registry+https://github.com/rust-lang/crates.io-index"
1587 +checksum = "c2d1f04709a8ac06e8e8042875a3c466cc4832d3c1a18dbcb9dba3c6e83046bc"
1588 +dependencies = [
1589 + "half",
1590 +]
1591 +
1592 [[package]]
1593 name = "fnv"
1594 version = "1.0.7"
@@ -2836,12 +2823,6 @@ version = "0.8.2"
2823 source = "registry+https://github.com/rust-lang/crates.io-index"
2824 checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0"
2825
2839 -[[package]]
2840 -name = "litrs"
2841 -version = "1.0.0"
2842 -source = "registry+https://github.com/rust-lang/crates.io-index"
2843 -checksum = "11d3d7f243d5c5a8b9bb5d6dd2b1602c0cb0b9db1621bafc7ed66e35ff9fe092"
2844 -
2826 [[package]]
2827 name = "llguidance"
2828 version = "1.7.4"
@@ -3095,235 +3076,25 @@ dependencies = [
3076
3077 [[package]]
3078 name = "mio"
3098 -version = "1.2.0"
3079 +version = "0.8.11"
3080 source = "registry+https://github.com/rust-lang/crates.io-index"
3100 -checksum = "50b7e5b27aa02a74bac8c3f23f448f8d87ff11f92d3aac1a6ed369ee08cc56c1"
3081 +checksum = "a4a650543ca06a924e8b371db273b2756685faae30f8487da1b56505a8f78b0c"
3082 dependencies = [
3083 "libc",
3084 "log",
3085 "wasi",
3105 - "windows-sys 0.61.2",
3106 -]
3107 -
3108 -[[package]]
3109 -name = "mistralrs"
3110 -version = "0.8.1"
3111 -source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#e53956bcd1b40c8221f79b919d78c1e496aba634"
3112 -dependencies = [
3113 - "anyhow",
3114 - "candle-core",
3115 - "candle-nn",
3116 - "clap 4.6.1",
3117 - "either",
3118 - "futures",
3119 - "image",
3120 - "indexmap 2.14.0",
3121 - "mistralrs-core",
3122 - "mistralrs-macros",
3123 - "rand 0.9.4",
3124 - "reqwest 0.13.2",
3125 - "schemars 1.2.1",
3126 - "serde",
3127 - "serde_json",
3128 - "thiserror 2.0.18",
3129 - "tokio",
3130 - "tracing",
3131 - "tracing-subscriber",
3132 - "walkdir",
3086 + "windows-sys 0.48.0",
3087 ]
3088
3089 [[package]]
3136 -name = "mistralrs-audio"
3137 -version = "0.8.1"
3138 -source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#e53956bcd1b40c8221f79b919d78c1e496aba634"
3139 -dependencies = [
3140 - "anyhow",
3141 - "apodize",
3142 - "hound",
3143 - "symphonia",
3144 -]
3145 -
3146 -[[package]]
3147 -name = "mistralrs-core"
3148 -version = "0.8.1"
3149 -source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#e53956bcd1b40c8221f79b919d78c1e496aba634"
3090 +name = "mio"
3091 +version = "1.2.0"
3092 +source = "registry+https://github.com/rust-lang/crates.io-index"
3093 +checksum = "50b7e5b27aa02a74bac8c3f23f448f8d87ff11f92d3aac1a6ed369ee08cc56c1"
3094 dependencies = [
3151 - "ahash",
3152 - "akin",
3153 - "anyhow",
3154 - "apodize",
3155 - "as-any",
3156 - "async-trait",
3157 - "base64 0.22.1",
3158 - "bm25",
3159 - "bytemuck",
3160 - "bytemuck_derive",
3161 - "candle-core",
3162 - "candle-metal-kernels",
3163 - "candle-nn",
3164 - "cfgrammar",
3165 - "chrono",
3166 - "clap 4.6.1",
3167 - "csv",
3168 - "derive-new",
3169 - "derive_more",
3170 - "dirs",
3171 - "either",
3172 - "float8",
3173 - "futures",
3174 - "galil-seiferas",
3175 - "half",
3176 - "hashbrown 0.16.1",
3177 - "hf-hub",
3178 - "hound",
3179 - "html2text",
3180 - "http",
3181 - "image",
3182 - "indexmap 2.14.0",
3183 - "indicatif 0.18.4",
3184 - "interprocess",
3185 - "itertools",
3095 "libc",
3187 - "llguidance",
3188 - "lrtable",
3189 - "minijinja",
3190 - "minijinja-contrib",
3191 - "mistralrs-audio",
3192 - "mistralrs-mcp",
3193 - "mistralrs-paged-attn",
3194 - "mistralrs-quant",
3195 - "mistralrs-vision",
3196 - "num-traits",
3197 - "objc",
3198 - "objc2-metal",
3199 - "openai-harmony",
3200 - "ordered-float",
3201 - "parking_lot",
3202 - "radix_trie",
3203 - "rand 0.9.4",
3204 - "rand_distr 0.5.1",
3205 - "rand_isaac",
3206 - "rayon",
3207 - "regex",
3208 - "regex-automata",
3209 - "reqwest 0.13.2",
3210 - "rubato",
3211 - "rust-mcp-schema",
3212 - "rustc-hash 2.1.2",
3213 - "rustfft",
3214 - "safetensors 0.7.0",
3215 - "schemars 1.2.1",
3216 - "scraper",
3217 - "serde",
3218 - "serde-big-array",
3219 - "serde-saphyr",
3220 - "serde_json",
3221 - "serde_plain",
3222 - "statrs",
3223 - "strum 0.27.2",
3224 - "symphonia",
3225 - "sysinfo",
3226 - "thiserror 2.0.18",
3227 - "tokenizers 0.21.4",
3228 - "tokio",
3229 - "tokio-rayon",
3230 - "tokio-tungstenite",
3231 - "toktrie",
3232 - "toktrie_hf_tokenizers",
3233 - "toml",
3234 - "tqdm",
3235 - "tracing",
3236 - "tracing-subscriber",
3237 - "urlencoding",
3238 - "uuid 1.23.1",
3239 - "variantly",
3240 - "vob",
3241 -]
3242 -
3243 -[[package]]
3244 -name = "mistralrs-macros"
3245 -version = "0.8.1"
3246 -source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#e53956bcd1b40c8221f79b919d78c1e496aba634"
3247 -dependencies = [
3248 - "darling 0.23.0",
3249 - "proc-macro2",
3250 - "quote",
3251 - "syn 2.0.117",
3252 -]
3253 -
3254 -[[package]]
3255 -name = "mistralrs-mcp"
3256 -version = "0.8.1"
3257 -source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#e53956bcd1b40c8221f79b919d78c1e496aba634"
3258 -dependencies = [
3259 - "anyhow",
3260 - "async-trait",
3261 - "futures-util",
3262 - "http",
3263 - "reqwest 0.13.2",
3264 - "rust-mcp-schema",
3265 - "serde",
3266 - "serde_json",
3267 - "tokio",
3268 - "tokio-tungstenite",
3269 - "tracing",
3270 - "utoipa",
3271 - "uuid 1.23.1",
3272 -]
3273 -
3274 -[[package]]
3275 -name = "mistralrs-paged-attn"
3276 -version = "0.8.1"
3277 -source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#e53956bcd1b40c8221f79b919d78c1e496aba634"
3278 -dependencies = [
3279 - "anyhow",
3280 - "candle-core",
3281 - "candle-metal-kernels",
3282 - "dispatch2",
3283 - "float8",
3284 - "half",
3285 - "objc2-foundation",
3286 - "objc2-metal",
3287 - "thiserror 2.0.18",
3288 -]
3289 -
3290 -[[package]]
3291 -name = "mistralrs-quant"
3292 -version = "0.8.1"
3293 -source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#e53956bcd1b40c8221f79b919d78c1e496aba634"
3294 -dependencies = [
3295 - "byteorder",
3296 - "candle-core",
3297 - "candle-metal-kernels",
3298 - "candle-nn",
3299 - "dispatch2",
3300 - "float8",
3301 - "half",
3302 - "hf-hub",
3303 - "lazy_static",
3304 - "memmap2",
3305 - "objc2-foundation",
3306 - "objc2-metal",
3307 - "paste",
3308 - "rayon",
3309 - "regex",
3310 - "safetensors 0.7.0",
3311 - "serde",
3312 - "serde_json",
3313 - "thiserror 2.0.18",
3314 - "tokio",
3315 - "tracing",
3316 - "yoke 0.8.2",
3317 -]
3318 -
3319 -[[package]]
3320 -name = "mistralrs-vision"
3321 -version = "0.8.1"
3322 -source = "git+https://github.com/setoelkahfi/mistral.rs?branch=fix%2Fall-platform-fixes#e53956bcd1b40c8221f79b919d78c1e496aba634"
3323 -dependencies = [
3324 - "candle-core",
3325 - "image",
3326 - "rayon",
3096 + "wasi",
3097 + "windows-sys 0.61.2",
3098 ]
3099
3100 [[package]]
@@ -3650,8 +3421,8 @@ dependencies = [
3421 "hf-hub",
3422 "home",
3423 "log",
3653 - "mistralrs",
3654 - "mistralrs-core",
3424 + "onde-mistralrs",
3425 + "onde-mistralrs-core",
3426 "reqwest 0.12.28",
3427 "serde",
3428 "serde_json",
@@ -3663,25 +3434,232 @@ dependencies = [
3434 ]
3435
3436 [[package]]
3666 -name = "onig"
3667 -version = "6.5.1"
3437 +name = "onde-mistralrs"
3438 +version = "0.7.1-alpha.1"
3439 source = "registry+https://github.com/rust-lang/crates.io-index"
3669 -checksum = "336b9c63443aceef14bea841b899035ae3abe89b7c486aaf4c5bd8aafedac3f0"
3440 +checksum = "76dd22e45632b11b3ce66a96687cb3901f4bd2e3f0523600ac84ed8aa62025ea"
3441 dependencies = [
3671 - "bitflags 2.11.1",
3442 + "anyhow",
3443 + "candle-core",
3444 + "candle-nn",
3445 + "clap 4.6.1",
3446 + "either",
3447 + "futures",
3448 + "image",
3449 + "indexmap 2.14.0",
3450 + "onde-mistralrs-core",
3451 + "onde-mistralrs-macros",
3452 + "rand 0.9.4",
3453 + "reqwest 0.13.2",
3454 + "schemars 1.2.1",
3455 + "serde",
3456 + "serde_json",
3457 + "thiserror 2.0.18",
3458 + "tokio",
3459 + "tracing",
3460 + "tracing-subscriber",
3461 + "walkdir",
3462 +]
3463 +
3464 +[[package]]
3465 +name = "onde-mistralrs-audio"
3466 +version = "0.7.1-alpha.1"
3467 +source = "registry+https://github.com/rust-lang/crates.io-index"
3468 +checksum = "3961d6fc770ccd7a0fa855b071ed90387b11cba18f5f61953cbd4608219bc8b1"
3469 +dependencies = [
3470 + "anyhow",
3471 + "apodize",
3472 + "hound",
3473 + "symphonia",
3474 +]
3475 +
3476 +[[package]]
3477 +name = "onde-mistralrs-core"
3478 +version = "0.7.1-alpha.1"
3479 +source = "registry+https://github.com/rust-lang/crates.io-index"
3480 +checksum = "07013d7ba1fef6c15ac49b5703bfa9fdfe98491ade97df81f8d6194b56f1996e"
3481 +dependencies = [
3482 + "ahash",
3483 + "akin",
3484 + "anyhow",
3485 + "apodize",
3486 + "as-any",
3487 + "async-trait",
3488 + "base64 0.22.1",
3489 + "bm25",
3490 + "bytemuck",
3491 + "bytemuck_derive",
3492 + "candle-core",
3493 + "candle-metal-kernels",
3494 + "candle-nn",
3495 + "cfgrammar",
3496 + "chrono",
3497 + "clap 4.6.1",
3498 + "csv",
3499 + "derive-new",
3500 + "derive_more",
3501 + "dirs",
3502 + "either",
3503 + "float8 0.6.1",
3504 + "futures",
3505 + "galil-seiferas",
3506 + "half",
3507 + "hashbrown 0.16.1",
3508 + "hf-hub",
3509 + "hound",
3510 + "html2text",
3511 + "http",
3512 + "image",
3513 + "indexmap 2.14.0",
3514 + "indicatif 0.18.4",
3515 + "interprocess",
3516 + "itertools",
3517 "libc",
3673 - "once_cell",
3674 - "onig_sys",
3518 + "llguidance",
3519 + "lrtable",
3520 + "minijinja",
3521 + "minijinja-contrib",
3522 + "num-traits",
3523 + "objc",
3524 + "objc2-metal",
3525 + "onde-mistralrs-audio",
3526 + "onde-mistralrs-mcp",
3527 + "onde-mistralrs-paged-attn",
3528 + "onde-mistralrs-quant",
3529 + "onde-mistralrs-vision",
3530 + "openai-harmony",
3531 + "ordered-float",
3532 + "parking_lot",
3533 + "radix_trie",
3534 + "rand 0.9.4",
3535 + "rand_distr 0.5.1",
3536 + "rand_isaac",
3537 + "rayon",
3538 + "regex",
3539 + "regex-automata",
3540 + "reqwest 0.13.2",
3541 + "rubato",
3542 + "rust-mcp-schema",
3543 + "rustc-hash 2.1.2",
3544 + "rustfft",
3545 + "safetensors 0.7.0",
3546 + "schemars 1.2.1",
3547 + "scraper",
3548 + "serde",
3549 + "serde-big-array",
3550 + "serde-saphyr",
3551 + "serde_json",
3552 + "serde_plain",
3553 + "statrs",
3554 + "strum 0.27.2",
3555 + "symphonia",
3556 + "sysinfo",
3557 + "thiserror 2.0.18",
3558 + "tokenizers",
3559 + "tokio",
3560 + "tokio-rayon",
3561 + "tokio-tungstenite",
3562 + "toktrie",
3563 + "toktrie_hf_tokenizers",
3564 + "toml",
3565 + "tqdm",
3566 + "tracing",
3567 + "tracing-subscriber",
3568 + "urlencoding",
3569 + "uuid 1.23.1",
3570 + "variantly",
3571 + "vob",
3572 ]
3573
3574 [[package]]
3678 -name = "onig_sys"
3679 -version = "69.9.1"
3575 +name = "onde-mistralrs-macros"
3576 +version = "0.7.1-alpha.1"
3577 source = "registry+https://github.com/rust-lang/crates.io-index"
3681 -checksum = "c7f86c6eef3d6df15f23bcfb6af487cbd2fed4e5581d58d5bf1f5f8b7f6727dc"
3578 +checksum = "0c82222350c6c2c57c6f08f1ff8fcfb7bc5662c6d2fe0bee43ba51f9fc8d7443"
3579 dependencies = [
3683 - "cc",
3684 - "pkg-config",
3580 + "darling 0.23.0",
3581 + "proc-macro2",
3582 + "quote",
3583 + "syn 2.0.117",
3584 +]
3585 +
3586 +[[package]]
3587 +name = "onde-mistralrs-mcp"
3588 +version = "0.7.1-alpha.1"
3589 +source = "registry+https://github.com/rust-lang/crates.io-index"
3590 +checksum = "1a89fc8e22b64a334b58ff27ab312b8a68a51c4c3bb70ba17fea3c0006dd02a5"
3591 +dependencies = [
3592 + "anyhow",
3593 + "async-trait",
3594 + "futures-util",
3595 + "http",
3596 + "reqwest 0.13.2",
3597 + "rust-mcp-schema",
3598 + "serde",
3599 + "serde_json",
3600 + "tokio",
3601 + "tokio-tungstenite",
3602 + "tracing",
3603 + "utoipa",
3604 + "uuid 1.23.1",
3605 +]
3606 +
3607 +[[package]]
3608 +name = "onde-mistralrs-paged-attn"
3609 +version = "0.7.1-alpha.1"
3610 +source = "registry+https://github.com/rust-lang/crates.io-index"
3611 +checksum = "6fc4bb2bf35dd3da67781c370559757bd8528f9002be701551319ab6f99d4a4b"
3612 +dependencies = [
3613 + "anyhow",
3614 + "candle-core",
3615 + "candle-metal-kernels",
3616 + "dispatch2",
3617 + "float8 0.7.0",
3618 + "half",
3619 + "objc2-foundation",
3620 + "objc2-metal",
3621 + "thiserror 2.0.18",
3622 +]
3623 +
3624 +[[package]]
3625 +name = "onde-mistralrs-quant"
3626 +version = "0.7.1-alpha.1"
3627 +source = "registry+https://github.com/rust-lang/crates.io-index"
3628 +checksum = "3d897de390f2a3b2ebdf0c9b93a969b40229d774b367b24b7461b1c30f38474b"
3629 +dependencies = [
3630 + "byteorder",
3631 + "candle-core",
3632 + "candle-metal-kernels",
3633 + "candle-nn",
3634 + "dispatch2",
3635 + "float8 0.6.1",
3636 + "half",
3637 + "hf-hub",
3638 + "lazy_static",
3639 + "memmap2",
3640 + "objc2-foundation",
3641 + "objc2-metal",
3642 + "paste",
3643 + "rayon",
3644 + "regex",
3645 + "safetensors 0.7.0",
3646 + "serde",
3647 + "serde_json",
3648 + "thiserror 2.0.18",
3649 + "tokio",
3650 + "tracing",
3651 + "yoke 0.8.2",
3652 +]
3653 +
3654 +[[package]]
3655 +name = "onde-mistralrs-vision"
3656 +version = "0.7.1-alpha.1"
3657 +source = "registry+https://github.com/rust-lang/crates.io-index"
3658 +checksum = "7abbab76130494cb2e9d24fdb12a3a92f1b5c2537ac40ff13885fd60d72670f2"
3659 +dependencies = [
3660 + "candle-core",
3661 + "image",
3662 + "rayon",
3663 ]
3664
3665 [[package]]
@@ -3838,12 +3816,6 @@ version = "0.2.17"
3816 source = "registry+https://github.com/rust-lang/crates.io-index"
3817 checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
3818
3841 -[[package]]
3842 -name = "pkg-config"
3843 -version = "0.3.33"
3844 -source = "registry+https://github.com/rust-lang/crates.io-index"
3845 -checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e"
3846 -
3819 [[package]]
3820 name = "plain"
3821 version = "0.2.3"
@@ -5089,7 +5061,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
5061 checksum = "b75a19a7a740b25bc7944bdee6172368f988763b744e3d4dfe753f6b4ece40cc"
5062 dependencies = [
5063 "libc",
5092 - "mio",
5064 + "mio 0.8.11",
5065 "signal-hook",
5066 ]
5067
@@ -5867,39 +5839,6 @@ dependencies = [
5839 "unicode_categories",
5840 ]
5841
5870 -[[package]]
5871 -name = "tokenizers"
5872 -version = "0.22.2"
5873 -source = "registry+https://github.com/rust-lang/crates.io-index"
5874 -checksum = "b238e22d44a15349529690fb07bd645cf58149a1b1e44d6cb5bd1641ff1a6223"
5875 -dependencies = [
5876 - "ahash",
5877 - "aho-corasick",
5878 - "compact_str",
5879 - "dary_heap",
5880 - "derive_builder",
5881 - "esaxx-rs",
5882 - "getrandom 0.3.4",
5883 - "itertools",
5884 - "log",
5885 - "macro_rules_attribute",
5886 - "monostate",
5887 - "onig",
5888 - "paste",
5889 - "rand 0.9.4",
5890 - "rayon",
5891 - "rayon-cond",
5892 - "regex",
5893 - "regex-syntax",
5894 - "serde",
5895 - "serde_json",
5896 - "spm_precompiled",
5897 - "thiserror 2.0.18",
5898 - "unicode-normalization-alignments",
5899 - "unicode-segmentation",
5900 - "unicode_categories",
5901 -]
5902 -
5842 [[package]]
5843 name = "tokio"
5844 version = "1.52.1"
@@ -5908,7 +5847,7 @@ checksum = "b67dee974fe86fd92cc45b7a95fdd2f99a36a6d7b0d431a231178d3d670bbcc6"
5847 dependencies = [
5848 "bytes",
5849 "libc",
5911 - "mio",
5850 + "mio 1.2.0",
5851 "parking_lot",
5852 "pin-project-lite",
5853 "signal-hook-registry",
@@ -5996,7 +5935,7 @@ dependencies = [
5935 "log",
5936 "serde",
5937 "serde_json",
5999 - "tokenizers 0.21.4",
5938 + "tokenizers",
5939 "toktrie",
5940 ]
5941
@@ -6087,7 +6026,8 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
6026 [[package]]
6027 name = "tqdm"
6028 version = "0.8.0"
6090 -source = "git+https://github.com/setoelkahfi/tqdm?branch=deps%2Fbump-crossterm#41e4182829136e6dadbdd1c36af824d19219147a"
6029 +source = "registry+https://github.com/rust-lang/crates.io-index"
6030 +checksum = "b316d5c2ac649ca856dacd487d0ebb94f3b746bada51355d93dd2c007ab62a2e"
6031 dependencies = [
6032 "anyhow",
6033 "crossterm",
@@ -6177,7 +6117,7 @@ version = "2.2.1"
6117 source = "registry+https://github.com/rust-lang/crates.io-index"
6118 checksum = "4fc20a25899c5ae43735253244214f3d5eb75dd28d9cc8491732294a985adb8c"
6119 dependencies = [
6180 - "convert_case 0.6.0",
6120 + "convert_case",
6121 "proc-macro2",
6122 "quote",
6123 "state",
@@ -7056,6 +6996,15 @@ dependencies = [
6996 "windows-targets 0.42.2",
6997 ]
6998
6999 +[[package]]
7000 +name = "windows-sys"
7001 +version = "0.48.0"
7002 +source = "registry+https://github.com/rust-lang/crates.io-index"
7003 +checksum = "677d2418bec65e3338edb076e806bc1ec15693c5d0104683f2efe857f61056a9"
7004 +dependencies = [
7005 + "windows-targets 0.48.5",
7006 +]
7007 +
7008 [[package]]
7009 name = "windows-sys"
7010 version = "0.52.0"
Cargo.toml
+25 -27
@@ -26,64 +26,51 @@ crate-type = ["lib", "cdylib", "staticlib"]
26 # in the Android sandbox).
27 [target.'cfg(target_os = "android")'.dependencies]
28 hf-hub = { version = "0.4", features = ["tokio", "rustls-tls"], default-features = false }
29 -# mistralrs = { version = "0.8.1", features = [] }
30 -mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = [] }
31 -# mistralrs-core = { version = "0.8.1", features = [] }
32 -mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = [] }
29 +mistralrs = { version = "0.8.2", package = "onde-mistralrs", features = [] }
30 +mistralrs-core = { version = "0.8.2", package = "onde-mistralrs-core", features = [] }
31
32 # iOS: Metal-accelerated inference; hf-hub + mistralrs-core needed for
33 # GLOBAL_HF_CACHE sandbox workaround (~/.cache is outside the container).
34 [target.'cfg(target_os = "ios")'.dependencies]
35 hf-hub = { version = "0.4", features = ["tokio", "rustls-tls"], default-features = false }
38 -# mistralrs = { version = "0.8.1", features = ["metal"] }
39 -mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
40 -# mistralrs-core = { version = "0.8.1", features = ["metal"] }
41 -mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
36 +mistralrs = { version = "0.8.2", package = "onde-mistralrs", features = ["metal"] }
37 +mistralrs-core = { version = "0.8.2", package = "onde-mistralrs-core", features = ["metal"] }
38
39 # Linux: CPU-only baseline (no GPU features).
40 # For NVIDIA GPU builds, patch this to features = ["cuda"] in CI before building.
41 [target.'cfg(target_os = "linux")'.dependencies]
46 -# mistralrs = { version = "0.8.1" }
47 -mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes" }
42 +mistralrs = { version = "0.8.2", package = "onde-mistralrs" }
43
44 # macOS: Metal-accelerated inference.
45 [target.'cfg(target_os = "macos")'.dependencies]
51 -# mistralrs = { version = "0.8.1", features = ["metal"] }
52 -mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
46 +mistralrs = { version = "0.8.2", package = "onde-mistralrs", features = ["metal"] }
47
48 # tvOS: Metal-accelerated inference (Apple TV has Apple GPU); hf-hub +
49 # mistralrs-core needed for GLOBAL_HF_CACHE sandbox workaround.
50 [target.'cfg(target_os = "tvos")'.dependencies]
51 hf-hub = { version = "0.4", features = ["tokio", "rustls-tls"], default-features = false }
58 -# mistralrs = { version = "0.8.1", features = ["metal"] }
59 -mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
60 -# mistralrs-core = { version = "0.8.1", features = ["metal"] }
61 -mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
52 +mistralrs = { version = "0.8.2", package = "onde-mistralrs", features = ["metal"] }
53 +mistralrs-core = { version = "0.8.2", package = "onde-mistralrs-core", features = ["metal"] }
54
55 # visionOS: Metal-accelerated inference (Apple Vision Pro has M2 GPU);
56 # hf-hub + mistralrs-core needed for GLOBAL_HF_CACHE sandbox workaround.
57 [target.'cfg(target_os = "visionos")'.dependencies]
58 hf-hub = { version = "0.4", features = ["tokio", "rustls-tls"], default-features = false }
67 -# mistralrs = { version = "0.8.1", features = ["metal"] }
68 -mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
69 -# mistralrs-core = { version = "0.8.1", features = ["metal"] }
70 -mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
59 +mistralrs = { version = "0.8.2", package = "onde-mistralrs", features = ["metal"] }
60 +mistralrs-core = { version = "0.8.2", package = "onde-mistralrs-core", features = ["metal"] }
61
62 # watchOS: Metal-accelerated inference (Apple Watch Series 4+ has Apple GPU);
63 # hf-hub + mistralrs-core needed for GLOBAL_HF_CACHE sandbox workaround.
64 # Memory constrained — default to 1.5B model.
65 [target.'cfg(target_os = "watchos")'.dependencies]
66 hf-hub = { version = "0.4", features = ["tokio", "rustls-tls"], default-features = false }
77 -# mistralrs = { version = "0.8.1", features = ["metal"] }
78 -mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
79 -# mistralrs-core = { version = "0.8.1", features = ["metal"] }
80 -mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", features = ["metal"] }
67 +mistralrs = { version = "0.8.2", package = "onde-mistralrs", features = ["metal"] }
68 +mistralrs-core = { version = "0.8.2", package = "onde-mistralrs-core", features = ["metal"] }
69
70 # Windows: CPU-only baseline (no GPU features).
71 # For NVIDIA GPU builds, patch this to features = ["cuda"] in CI before building.
72 [target.'cfg(target_os = "windows")'.dependencies]
85 -# mistralrs = { version = "0.8.1" }
86 -mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes" }
73 +mistralrs = { version = "0.8.2", package = "onde-mistralrs" }
74
75 [dependencies]
76 anyhow = "1"
@@ -112,9 +99,20 @@ dotenvy = "0.15"
99 # Required for UniFFI scaffolding code generation at build time.
100 uniffi = { version = "=0.31.0", features = ["build"] }
101
115 -# Patch sysctl with watchOS (target_os = "watchos") support until upstream merges it.
102 +# ── Local development patches ────────────────────────────────────────────────
103 +#
104 +# [patch.crates-io] is stripped by `cargo publish` and never seen by crates.io
105 +# consumers. These patches only apply when building from this local checkout.
106 +#
107 +# To develop against the mistral.rs git fork locally instead of the published
108 +# onde-mistralrs crates, uncomment the block below:
109 +#
110 +# [patch.crates-io]
111 +# onde-mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", package = "mistralrs" }
112 +# onde-mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", package = "mistralrs-core" }
113
114 # Patch sysctl with watchOS support until upstream merges it.
115 # PR: https://github.com/johalun/sysctl-rs/pull/74
116 +# NOTE: [patch.crates-io] is ignored by crates.io consumers — safe to keep.
117 [patch.crates-io]
118 sysctl = { git = "https://github.com/setoelkahfi/sysctl-rs.git", branch = "feature/watchos" }
README.md
+19 -7
@@ -54,13 +54,25 @@ Onde is dual-licensed under **MIT** and **Apache 2.0**. You may use it under eit
54
55 Models downloaded by Onde have their own licenses independent of this crate. By using Onde, you are also subject to the license of the model you load:
56
57 -| Model | License |
58 -|---|---|
59 -| Qwen 2.5 1.5B / 3B Instruct | [Qwen Community License](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct/blob/main/LICENSE) |
60 -| Qwen 2.5 Coder 1.5B / 3B / 7B | [Qwen Community License](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct/blob/main/LICENSE) |
57 +| Model | Size | License | Commercial use |
58 +|---|---|---|---|
59 +| Qwen 2.5 1.5B Instruct (GGUF Q4_K_M) | ~941 MB | [Qwen Community License](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct/blob/main/LICENSE) | ✅ with conditions¹ |
60 +| Qwen 2.5 3B Instruct (GGUF Q4_K_M) | ~1.93 GB | [Qwen Community License](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct/blob/main/LICENSE) | ✅ with conditions¹ |
61 +| Qwen 2.5 Coder 7B Instruct (GGUF Q4_K_M) | ~4.4 GB | [Qwen Community License](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/blob/main/LICENSE) | ✅ with conditions¹ |
62 +| Qwen 3 1.7B (GGUF Q4_K_M) | ~1.3 GB | [Apache 2.0](https://huggingface.co/Qwen/Qwen3-1.7B/blob/main/LICENSE) | ✅ |
63 +| Qwen 3 4B (GGUF Q4_K_M) | ~2.7 GB | [Apache 2.0](https://huggingface.co/Qwen/Qwen3-4B/blob/main/LICENSE) | ✅ |
64 +| Qwen 3 8B (GGUF Q4_K_M) | ~5 GB | [Apache 2.0](https://huggingface.co/Qwen/Qwen3-8B/blob/main/LICENSE) | ✅ |
65 +| Qwen 3 14B (GGUF Q4_K_M) | ~8.4 GB | [Apache 2.0](https://huggingface.co/Qwen/Qwen3-14B/blob/main/LICENSE) | ✅ |
66 +| DeepSeek Coder 6.7B Instruct (GGUF Q4_K_M) | ~3.8 GB | [DeepSeek License v1.0](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct/blob/main/LICENSE) | ✅ with conditions² |
67 +
68 +¹ **Qwen Community License conditions:** no training of competing models, attribution required, no misrepresentation of origin. Organisations with more than 100 million monthly active users must obtain a separate commercial licence from Alibaba Cloud.
69 +
70 +² **DeepSeek License v1.0 conditions:** use-based restrictions apply (see Attachment A of the license). Prohibits military use, generation of disinformation, and certain other uses. Governing law is PRC law.
71 +
72 +Onde's own license (MIT OR Apache-2.0) is independent of these model licenses. If you build an application on top of Onde, you are responsible for complying with the license of whichever model your users load.
73
74 ---
75
64 -<p align="center">
65 - <sub>© 2026 <a href="https://ondeinference.com">Onde Inference</a></sub>
66 -</p>
76 +## Copyright
77 +
78 +© 2026 [Onde Inference](https://ondeinference.com) (Splitfire AB).