@ondeinference / onde
Code Commits Issues Pulls Cloud Agent CI/CD
main
copilot/assess-inference-engine-complexity copilot/fix-onde-npm-sdk-v1-1-2-release development feature/diffusion-vision main
md 632 lines 22.6 KB
Rendered Raw
1 ---
2 name: sdk-regressions
3 description: Cross-compilation regression checks for the onde crate. Covers macOS, iOS, tvOS (nightly), Android, Windows, and Linux target verification. Run before merging any change that touches platform-gated code, Cargo.toml, .cargo/config.toml, or build.rs. Also documents the onde-mistralrs fork publishing strategy and cargo publish workflow.
4 allowed-tools: Read, Edit, Glob, Grep, Terminal
5 user-invocable: true
6 ---
7
8 # SKILL: SDK Cross-Compilation Regression Checks
9
10 > Always run these checks before merging changes that touch `onde/src/`,
11 > `onde/Cargo.toml`, `onde/.cargo/config.toml`, `onde/build.rs`, or any
12 > platform-gated dependency.
13
14 ---
15
16 ## Why This Exists
17
18 The `onde` crate compiles for **6 target OSes** with heavy `#[cfg(target_os)]`
19 branching (106+ occurrences in `engine.rs` alone). A change that compiles on
20 macOS can silently break iOS, tvOS, or Android because:
21
22 - Different `mistralrs` feature sets per platform (`["metal"]` vs `[]`)
23 - Platform-conditional dependencies (`hf-hub` only on Android)
24 - Tier-3 targets (tvOS) require nightly + `-Z build-std`
25 - Assembly stubs (`tvos_chkstk.s`) and `build.rs` gating
26 - `.cargo/config.toml` sets `+fp16` rustflags on 5 Apple targets
27
28 See also: **[`cargo publish` workflow](#cargo-publish-workflow)** at the bottom
29 of this file — must be followed when releasing a new `onde` version.
30
31 ---
32
33 ## Tier 1 — Must Pass on Every PR (stable toolchain)
34
35 These use the default stable toolchain and must always succeed:
36
37 ```bash
38 # macOS — host platform, Metal backend
39 cargo check
40
41 # iOS device — Metal backend, +fp16
42 cargo check --target aarch64-apple-ios
43
44 # iOS simulator — Metal backend, +fp16
45 cargo check --target aarch64-apple-ios-sim
46 ```
47
48 ### Quick one-liner
49
50 ```bash
51 cargo check && \
52 cargo check --target aarch64-apple-ios && \
53 cargo check --target aarch64-apple-ios-sim
54 ```
55
56 ---
57
58 ## Tier 2 — tvOS (nightly toolchain, `-Z build-std`)
59
60 tvOS targets are tier-3 in Rust — they are **not** in `rustup target list` and
61 require nightly with `-Z build-std` and the `rust-src` component.
62
63 ```bash
64 # tvOS device
65 cargo +nightly check -Z build-std --target aarch64-apple-tvos
66
67 # tvOS simulator
68 cargo +nightly check -Z build-std --target aarch64-apple-tvos-sim
69 ```
70
71 ### tvOS-specific regression risks
72
73 | Risk | What breaks | How to detect |
74 |------|-------------|---------------|
75 | `scripts/tvos_chkstk.s` deleted | `Undefined symbol: ___chkstk_darwin` at link time | `test -f scripts/tvos_chkstk.s` |
76 | `cc` build-dep removed from `Cargo.toml` | `build.rs` can't compile the assembly stub | `grep 'cc' Cargo.toml` |
77 | `build.rs` tvOS block removed | Same linker error | `grep 'tvos' build.rs` |
78 | nightly `rust-src` not installed | `-Z build-std` fails immediately | `rustup +nightly component list --installed \| grep rust-src` |
79 | `.cargo/config.toml` `+fp16` removed | Silent precision loss or candle kernel panics | `grep 'fp16' .cargo/config.toml` |
80
81 ---
82
83 ## Tier 3 — Android, Windows, Linux (stable, needs cross toolchains)
84
85 These check that CPU-only code paths compile. For this crate, `cargo check`
86 still runs C build scripts from dependencies such as `aws-lc-sys` and `ring`,
87 so absent cross GCC toolchains are real blockers, not harmless link-only
88 failures.
89
90 ```bash
91 # Android arm64 — CPU, requires hf-hub sandbox workaround
92 cargo check --target aarch64-linux-android
93
94 # Windows MSVC — CPU, exact CI target; run on Windows or a Windows CI runner
95 cargo check --target x86_64-pc-windows-msvc
96
97 # Linux — CPU
98 cargo check --target x86_64-unknown-linux-gnu
99 ```
100
101 On macOS hosts without cross GCC installed, expect Linux checks to fail looking
102 for `x86_64-linux-gnu-gcc`, and Windows GNU checks to fail looking for
103 `x86_64-w64-mingw32-gcc`. Use the Colima/Docker flow below instead of treating
104 those as code regressions.
105
106 ### macOS host fallback: Colima/Docker
107
108 Use this when the goal is to run the Linux and Windows cross-checks on a macOS
109 host and the host does not have cross GCC toolchains. On Apple Silicon, run a
110 native `linux/arm64` container. Avoid `linux/amd64` emulation for this crate;
111 the dependency build can crash under QEMU before reaching `onde` code.
112
113 Give Colima enough memory before running these checks:
114
115 ```bash
116 colima status
117 colima stop
118 colima start --memory 8
119 ```
120
121 Run Linux and the local Windows GNU proxy check in the container:
122
123 ```bash
124 docker run --rm --platform linux/arm64 \
125 -v "$PWD:/workspace" \
126 -v onde-cargo-registry:/usr/local/cargo/registry \
127 -v onde-cargo-git:/usr/local/cargo/git \
128 -w /workspace \
129 rust:1.92-bookworm \
130 bash -lc '
131 export PATH=/usr/local/cargo/bin:$PATH
132 apt-get update >/dev/null
133 DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
134 gcc-x86-64-linux-gnu \
135 libc6-dev-amd64-cross \
136 linux-libc-dev-amd64-cross \
137 gcc-mingw-w64-x86-64 \
138 binutils-mingw-w64-x86-64 >/dev/null
139 rustup target add x86_64-unknown-linux-gnu x86_64-pc-windows-gnu >/dev/null
140 CARGO_BUILD_JOBS=1 cargo check --target x86_64-unknown-linux-gnu
141 CARGO_BUILD_JOBS=1 cargo check --target x86_64-pc-windows-gnu
142 '
143 ```
144
145 `x86_64-pc-windows-gnu` exercises `#[cfg(target_os = "windows")]` and the
146 CPU-only Windows path locally, but it is not a substitute for the CI
147 `x86_64-pc-windows-msvc` check. The MSVC ABI check needs a Windows/MSVC
148 toolchain, normally GitHub Actions `windows-latest`.
149
150 If the container reaches `Checking onde ...` and then reports Rust errors, the
151 cross toolchain is working. Fix the crate error and rerun the check.
152
153 ---
154
155 ## Full Regression Suite (copy-paste)
156
157 ```bash
158 #!/usr/bin/env bash
159 set -euo pipefail
160
161 echo "═══ Tier 1: macOS + iOS (stable) ═══"
162 cargo check
163 cargo check --target aarch64-apple-ios
164 cargo check --target aarch64-apple-ios-sim
165
166 echo "═══ Tier 2: tvOS (nightly + build-std) ═══"
167 cargo +nightly check -Z build-std --target aarch64-apple-tvos
168 cargo +nightly check -Z build-std --target aarch64-apple-tvos-sim
169
170 echo "═══ Tier 3: Android / Windows / Linux ═══"
171 cargo check --target aarch64-linux-android
172 cargo check --target x86_64-pc-windows-msvc
173 cargo check --target x86_64-unknown-linux-gnu
174
175 echo "✅ All cross-compilation checks passed"
176 ```
177
178 ---
179
180 ## Required Rustup Targets
181
182 Install these once on a development machine:
183
184 ```bash
185 # Tier 1 (stable)
186 rustup target add aarch64-apple-darwin x86_64-apple-darwin
187 rustup target add aarch64-apple-ios aarch64-apple-ios-sim
188
189 # Tier 3 (stable)
190 rustup target add aarch64-linux-android armv7-linux-androideabi
191 rustup target add x86_64-linux-android i686-linux-android
192 rustup target add x86_64-pc-windows-msvc
193 rustup target add x86_64-pc-windows-gnu
194 rustup target add x86_64-unknown-linux-gnu
195
196 # Tier 2 — tvOS (nightly, no `rustup target add` — uses -Z build-std)
197 rustup toolchain install nightly
198 rustup +nightly component add rust-src
199 ```
200
201 ---
202
203 ## `.cargo/config.toml` — Critical Settings
204
205 These rustflags are set at the workspace level and **must not be removed**:
206
207 ```toml
208 [target.aarch64-apple-ios]
209 rustflags = ["-C", "target-feature=+fp16"]
210
211 [target.aarch64-apple-ios-sim]
212 rustflags = ["-C", "target-feature=+fp16"]
213
214 [target.aarch64-apple-tvos]
215 rustflags = ["-C", "target-feature=+fp16"]
216
217 [target.aarch64-apple-tvos-sim]
218 rustflags = ["-C", "target-feature=+fp16"]
219
220 [target.x86_64-apple-tvos]
221 rustflags = ["-C", "target-feature=+fp16"]
222 ```
223
224 Removing `+fp16` causes silent precision degradation or runtime panics in
225 candle's Metal/NEON kernels.
226
227 ---
228
229 ## Platform Dependency Matrix
230
231 | `target_os` | `mistralrs` features | Extra deps | GPU | Default model |
232 |-------------|---------------------|------------|-----|---------------|
233 | `macos` | `["metal"]` | — | Metal | Qwen 2.5 3B |
234 | `ios` | `["metal"]` | `hf-hub`, `mistralrs-core` | Metal | Qwen 2.5 1.5B |
235 | `tvos` | `["metal"]` | `hf-hub`, `mistralrs-core` | Metal | Qwen 2.5 1.5B |
236 | `android` | `[]` | `hf-hub`, `mistralrs-core` | CPU | Qwen 2.5 1.5B |
237 | `windows` | `[]` | — | CPU | Qwen 2.5 3B |
238 | `linux` | `[]` | — | CPU | Qwen 2.5 3B |
239
240 `hf-hub` and `mistralrs-core` are required on all **sandboxed** platforms
241 (iOS, tvOS, Android) for the `GLOBAL_HF_CACHE` workaround — `~/.cache` is
242 outside the app container on iOS/tvOS, and `dirs::home_dir()` panics on Android.
243 The re-exports in `src/lib.rs` (`pub use hf_hub; pub use mistralrs_core;`) are
244 gated to `#[cfg(any(target_os = "android", target_os = "ios", target_os = "tvos"))]`.
245
246 ---
247
248 ## Common Regression Patterns
249
250 ### 1. New model added but not to all 6 platform `cfg` blocks
251
252 `engine.rs` has separate `#[cfg(target_os = "...")]` blocks for model builder
253 logic. A new model constructor added only under `macos` will fail on `ios`.
254
255 **Detection:** Tier 1 checks catch iOS; Tier 2 catches tvOS; Tier 3 catches
256 Android/Windows/Linux.
257
258 ### 2. Dependency added without platform gating
259
260 Adding a dep that uses `SystemConfiguration.framework` (e.g., `hyper_util`
261 proxy detection) works on macOS but causes linker errors on iOS/tvOS unless the
262 podspec declares `s.frameworks = 'SystemConfiguration'`.
263
264 **Detection:** Tier 1 `cargo check` passes (no linking), but `flutter build ios`
265 fails. Must also test a full `flutter build` periodically.
266
267 ### 3. Fork branch divergence
268
269 All `mistralrs` deps point to a fork:
270 `git = "https://github.com/setoelkahfi/mistral.rs"`, branch
271 `fix/all-platform-fixes`. If that branch is rebased, deleted, or diverges from
272 upstream, **all** platforms break.
273
274 **Detection:** Any `cargo check` fails with "could not find branch".
275
276 ### 4. UniFFI version mismatch
277
278 `uniffi = "=0.31.0"` must be pinned identically in three places:
279 - `onde/Cargo.toml` dependencies
280 - `onde/Cargo.toml` build-dependencies
281 - `onde/uniffi-bindgen/Cargo.toml`
282
283 Mixing versions causes bindgen panics at codegen time.
284
285 **Detection:** `cargo build` in `uniffi-bindgen/` or `build.rs` panics.
286
287 ### 5. Android `home_dir()` sandbox panic
288
289 `dirs::home_dir()` panics in the Android sandbox. The `hf-hub` dep is added
290 explicitly on Android so `HF_HOME` can be set programmatically. Never call
291 `dirs::home_dir()` or `home::home_dir()` in code paths reachable on Android.
292
293 **Detection:** Tier 3 `cargo check --target aarch64-linux-android` catches
294 compile-time issues; runtime panics need an Android emulator test.
295
296 ---
297
298 ## Dart SDK Cross-Build Checks
299
300 When changes touch `sdk/dart/`, also verify the Flutter plugin builds:
301
302 ```bash
303 cd sdk/dart/example
304
305 # macOS (requires Xcode)
306 flutter build macos --debug
307
308 # iOS (requires Xcode + iOS SDK)
309 flutter build ios --debug --no-codesign
310 ```
311
312 ### Known environment pitfall
313
314 **Never export Android NDK toolchain binaries globally in `.zshrc`:**
315
316 ```bash
317 # ❌ BREAKS Xcode builds — NDK's ld/ar/strip shadow Apple's tools
318 export LD=$TOOLCHAIN/bin/ld
319 export AR=$TOOLCHAIN/bin/llvm-ar
320
321 # ✅ Safe — env vars only, no PATH/tool overrides
322 export ANDROID_NDK_HOME="$HOME/Library/Android/sdk/ndk/29.0.14206865"
323 ```
324
325 If Xcode invokes the NDK's `ld` instead of Apple's, you get:
326 `ld: error: unknown argument '-Xlinker'`
327
328 The fix: comment out the tool exports, use Cargo's
329 `sdk/dart/rust/.cargo/config.toml` for Android linker configuration instead.
330
331 ---
332
333 ## Cache Path Consistency (Cross-SDK)
334
335 All Onde-powered apps share downloaded models via the App Group
336 `group.com.ondeinference.apps`. The cache layout inside the shared container
337 **must be identical** across every SDK and app type:
338
339 ```
340 <group container>/
341 ├── models/ ← HF_HOME
342 │ └── hub/ ← HF_HUB_CACHE (GGUF files live here)
343 └── tmp/ ← TMPDIR (iOS sandbox restricts system TMPDIR)
344 ```
345
346 **The subdirectory is `models/`, NOT `huggingface/`.**
347
348 Files that set this path (all must agree):
349
350 | File | Variable | Expected value |
351 |------|----------|----------------|
352 | `onde/sdk/dart/rust/src/api.rs` (`configure_cache_dir`) | `hf_home` | `data_dir.join("models")` |
353 | `onde/src/hf_cache.rs` (`download_model` — general) | `hf_home` | `data_dir.join("models")` |
354 | `onde/src/hf_cache.rs` (`download_model` — Android) | `hf_home` | `resolved_app_data.join("models")` |
355 | Tauri apps (`setup_application_filesystem.rs`) | `models_home` | `container_dir.join("models")` |
356 | `onde/src/hf_cache.rs` (`hf_cache_dir` — fallback) | default | `~/.cache/huggingface/hub` (non-sandboxed only, OK to differ) |
357
358 **Regression check:** `grep -rn 'join("huggingface")' src/hf_cache.rs sdk/dart/rust/src/api.rs`
359 should return **only** the non-sandboxed fallback in `hf_cache_dir()` (line ~164).
360 Any other match means a sandboxed path is using the wrong subdirectory.
361
362 ---
363
364 ---
365
366 ## Publishing to crates.io
367
368 ### Why we don't use upstream `mistralrs` directly
369
370 The short version: our PRs aren't merged yet.
371
372 `onde` depends on a personal fork of Eric Buehler's `mistral.rs`, kept at
373 `github.com/setoelkahfi/mistral.rs` on the `fix/all-platform-fixes` branch.
374 The fork is ahead of upstream by ~41 commits carrying fixes that only matter
375 for Apple and Android targets — things upstream hasn't needed to care about:
376
377 | Fix | Status |
378 |-----|--------|
379 | iOS Metal 3.0 support | PR open |
380 | Android 32-bit memory limit constants | PR open |
381 | HF_HOME propagation in diffusion/flux pipelines | PR open |
382 | `metallib` link step `--sdk` / `-std` flags | PR open |
383
384 Dropping the fork and pointing at upstream `mistralrs 0.8.1` would silently
385 break iOS on entry-level devices and Android on 32-bit targets. Don't do it
386 until those PRs land.
387
388 **Legal note:** The fork is a derivative of Eric Buehler's MIT-licensed
389 `mistral.rs`. His `LICENSE` file — including `Copyright (c) 2024 Eric Buehler`
390 — must remain intact and unmodified in the fork at all times. See
391 [`legal-and-trademarks/SKILL.md`](../legal-and-trademarks/SKILL.md) for full
392 attribution obligations.
393
394 ### The `onde-mistralrs` workaround
395
396 > For the full inventory of every fork and `[patch.crates-io]` override in the
397 > stack (mistral.rs, candle, sysctl, mio, tqdm, core2) and how the
398 > git-vs-published tiers resolve differently, see the **forked-dependencies**
399 > skill. This section covers only the mistral.rs publish mechanics.
400
401 `cargo publish` strips every `git =` field before uploading to crates.io.
402 That means a git-only dep is useless to anyone who installs `onde` from the
403 registry — they can't resolve it.
404
405 The solution: publish the fork to crates.io under the name `onde-mistralrs`
406 (and `onde-mistralrs-core`, etc.), then reference it with `package =` so the
407 rest of the code still compiles unchanged with `use mistralrs::...`:
408
409 ```toml
410 # In onde/Cargo.toml — each platform target section looks like this.
411 # Cargo resolves it as "onde-mistralrs" from crates.io,
412 # but the Rust code sees it as `mistralrs`.
413 mistralrs = { version = "0.8.2", package = "onde-mistralrs", features = ["metal"] }
414 ```
415
416 This is the only supported publish path until the upstream PRs merge. Don't
417 swap it for plain `mistralrs` from crates.io without checking every platform
418 target first.
419
420 ---
421
422 ### Step 0 — check if the candle snapshot needs an `onde-candle-*` publish
423
424 Since 0.9.x, upstream mistral.rs pins `candle` to a git rev past the latest
425 released candle (e.g. rev `27f20fea`, 19 commits past `0.11.0`, adding
426 `QTensor::indexed_gemv`, `gemv_fused_shared_lhs`, `BarrierPool::execute_chunked`).
427 `cargo publish` strips git pins, so publish verification resolves candle from
428 crates.io and fails with E0599 "method not found" if the pinned rev has APIs the
429 released candle lacks.
430
431 When that happens, publish the pinned rev under `onde-candle-*` names from the
432 local candle checkout (`~/Repositories/candle`, branch pattern
433 `onde/candle-<version>-<rev>`):
434
435 ```bash
436 # Which candle crates actually diverged from the released version?
437 git diff --stat <release-bump-commit> <pinned-rev> -- candle-core candle-nn candle-metal-kernels
438
439 # Rename ONLY the diverged crates (0.11.0 era: candle-core + candle-nn; the
440 # rest are byte-identical, official crates serve them). candle-nn must be
441 # republished whenever candle-core is, even if unchanged, or the official
442 # candle-nn drags the official candle-core into the graph -> two candle-core
443 # crates -> type mismatch.
444 cargo publish -p onde-candle-core
445 cargo publish -p onde-candle-nn
446 ```
447
448 Then in the mistral.rs fork workspace `Cargo.toml`:
449
450 ```toml
451 candle-core = { version = "0.11.0", package = "onde-candle-core" }
452 candle-nn = { version = "0.11.0", package = "onde-candle-nn" }
453 candle-flash-attn-v3 = { version = "0.11.0" } # unchanged at rev -> official
454 candle-metal-kernels = { version = "0.11.0" } # unchanged at rev -> official
455 ```
456
457 ### Step 1 — check if `onde-mistralrs` needs a new publish
458
459 First, see how far the fork has drifted from upstream:
460
461 ```bash
462 curl -s "https://api.github.com/repos/setoelkahfi/mistral.rs/compare/EricLBuehler:mistral.rs:master...setoelkahfi:mistral.rs:fix%2Fall-platform-fixes" \
463 | python3 -c "import sys,json; d=json.load(sys.stdin); print('ahead:', d['ahead_by'], 'behind:', d['behind_by'])"
464 ```
465
466 If there are new commits since the last `onde-mistralrs` publish, or if
467 upstream cut a new release that the fork has rebased onto, bump the workspace
468 version in the fork and publish. As of 0.9.1 the set is **12 crates** (upstream
469 split out flash-attn/metal-compile/code-exec/sandbox, and vision/audio/mcp must
470 be onde-published too because crates.io only has upstream's old versions).
471 The rename lives in the workspace dep table (`package = "onde-..."` on each
472 entry, local dep keys unchanged so imports stay `mistralrs_*`).
473
474 ```bash
475 # 1. Edit the root Cargo.toml in setoelkahfi/mistral.rs
476 # bump version, e.g. 0.9.0 → 0.9.1
477
478 # 2. Publish in dependency order (leaves first):
479 cargo publish -p onde-mistralrs-macros
480 cargo publish -p onde-mistralrs-metal-compile
481 cargo publish -p onde-mistralrs-sandbox
482 cargo publish -p onde-mistralrs-flash-attn
483 cargo publish -p onde-mistralrs-vision
484 cargo publish -p onde-mistralrs-audio
485 cargo publish -p onde-mistralrs-mcp
486 cargo publish -p onde-mistralrs-paged-attn
487 cargo publish -p onde-mistralrs-quant
488 cargo publish -p onde-mistralrs-code-exec
489 cargo publish -p onde-mistralrs-core
490 cargo publish -p onde-mistralrs
491
492 # 3. cargo waits for index propagation automatically after each publish.
493 ```
494
495 Known publish-verification snags (hit during 0.9.1):
496
497 | Error | Fix |
498 |-------|-----|
499 | E0599 on candle APIs (`indexed_gemv`, ...) | Step 0 — publish `onde-candle-*` and repoint |
500 | `dependency tqdm does not specify a version` | tqdm must be a registry dep, not git — it is plain `tqdm = "0.8.1"` since the crossterm bump landed upstream |
501 | `core2 0.4.0 is yanked` | `cargo update -p bitstream-io` (4.10.0 dropped core2) |
502
503 ---
504
505 ### Step 2 — update `onde/Cargo.toml`
506
507 Every platform target section has its own `onde-mistralrs` line. They all need
508 to move to the new version together — leaving one target on an older version
509 causes a dependency conflict.
510
511 ```bash
512 # All occurrences should print the same version number:
513 grep 'onde-mistralrs' onde/Cargo.toml
514 ```
515
516 ---
517
518 ### Step 3 — dry-run first
519
520 ```bash
521 cd onde
522 cargo publish --dry-run --allow-dirty 2>&1 | tail -20
523 ```
524
525 A clean run ends with `Finished` and no errors. Common things that go wrong:
526
527 | Error | What to do |
528 |-------|------------|
529 | `all dependencies must have a version` | add `version = "..."` to the dep |
530 | `no matching package named onde-mistralrs` | publish the fork first (step 1) |
531 | `git specification will be removed` | expected — cargo is just telling you the git field gets stripped |
532 | `files in working directory contain changes` | add `--allow-dirty` |
533
534 ---
535
536 ### Step 4 — publish `onde`, then update `sigit`
537
538 ```bash
539 cargo publish
540 ```
541
542 Then bump the version in `sigit/Cargo.toml`:
543
544 ```toml
545 # git = is used for local dev; crates.io consumers resolve via version =
546 onde = { version = "0.1.9", git = "https://github.com/ondeinference/onde", branch = "development" }
547 ```
548
549 Keep both fields. The `git =` field takes precedence during local development;
550 crates.io strips it and falls back to `version =` for anyone installing from
551 the registry.
552
553 ---
554
555 ### GresIQ credentials and `HF_TOKEN`
556
557 People building `onde` from crates.io don't need to do anything about secrets.
558 Here's why.
559
560 `build.rs` passes credentials to the compiler via `option_env!()`. That macro
561 resolves at compile time — if the env var isn't set, it compiles to `None` and
562 the feature that needs it is silently disabled. No crash, no partial state, no
563 mystery error at runtime.
564
565 ```rust
566 // pulse/client.rs — GresIQ telemetry (Onde Inference internal infrastructure)
567 const EMBEDDED_API_KEY_DEV: Option<&str> = option_env!("GRESIQ_API_KEY_DEV");
568 const EMBEDDED_API_SECRET_DEV: Option<&str> = option_env!("GRESIQ_API_SECRET_DEV");
569
570 // token.rs — HuggingFace Hub authentication
571 const BUILD_TIME_HF_TOKEN: Option<&str> = option_env!("HF_TOKEN");
572 ```
573
574 Who gets what:
575
576 | Who is building | Env vars present? | What happens |
577 |-----------------|-------------------|--------------|
578 | Our CI (official builds) | yes, via GitHub secrets | telemetry on; HF token baked into binary |
579 | Someone who ran `cargo add onde` | no | telemetry off; HF token read from `~/.cache/huggingface/token` at runtime |
580 | iOS/tvOS builds via XCFramework | yes, via `.env` file | HF token baked in — required because there's no writable filesystem on device |
581
582 The GresIQ credentials are Onde Inference's own infrastructure secrets — they
583 exist so the SDK can phone home for usage telemetry on official builds. External
584 consumers don't have them, don't need them, and won't be affected by not having
585 them.
586
587 Never put `GRESIQ_API_KEY_*` or `HF_TOKEN` in `README.md`, `.env.example`, or
588 any committed file. The `.env` in the crate root is git-ignored for a reason:
589
590 ```bash
591 # onde/.env — git-ignored, never commit
592 GRESIQ_API_KEY_DEV=...
593 GRESIQ_API_SECRET_DEV=...
594 GRESIQ_API_KEY_PRODUCTION=...
595 GRESIQ_API_SECRET_PRODUCTION=...
596 HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
597 ```
598
599 ---
600
601 ### Working against the git fork locally
602
603 If you're making changes to the fork and want `onde` to pick them up without
604 publishing, uncomment the patch block in `onde/Cargo.toml`:
605
606 ```toml
607 [patch.crates-io]
608 onde-mistralrs = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", package = "mistralrs" }
609 onde-mistralrs-core = { git = "https://github.com/setoelkahfi/mistral.rs", branch = "fix/all-platform-fixes", package = "mistralrs-core" }
610 ```
611
612 Re-comment it before running `cargo publish`. Cargo strips `[patch.crates-io]`
613 from the published manifest automatically, so it won't affect registry
614 consumers either way — but leaving it active makes the dry-run confusing.
615
616 ---
617
618 ### Pre-publish checklist
619
620 - [ ] all `onde-mistralrs` version numbers in `onde/Cargo.toml` are the same
621 - [ ] `onde/Cargo.toml` `[package] version` is bumped
622 - [ ] `sigit/Cargo.toml` `onde` version matches
623 - [ ] `[patch.crates-io]` git override is commented out
624 - [ ] `cargo publish --dry-run --allow-dirty` passes clean
625 - [ ] `.env` is not listed in the dry-run's packaged file output
626 - [ ] Eric Buehler's `LICENSE` in the `mistral.rs` fork is unmodified
627
628 ---
629
630 *Last updated: July 2026 — added Colima/Docker Linux and Windows GNU
631 cross-check guidance for macOS hosts without cross GCC, plus the MSVC CI
632 caveat.*