Setup agentic engineering and fix ci
Seto Elkahfi committed
Jun 27, 2026 at 07:38 UTC
b74ee99376bb0fc075b45a30f134ec56dcbdb482
2 files changed
+79
-3
CLAUDE.md
new
+66
@@ -0,0 +1,66 @@
1
+# CLAUDE.md
2
+
3
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
+
5
+## What this is
6
+
7
+`onde-cli` is a native Rust terminal UI (binary name `onde`) for managing an Onde Inference account and running a local model pipeline: fine-tune a safetensors small language model with LoRA, merge the adapter, export to GGUF, test it in a local chat, upload it to Hugging Face, and assign it to an Onde app. All packaging (npm, PyPI, NuGet, pub.dev, Homebrew, crates.io) is thin wrappers around this one binary; the Rust crate is the source of truth.
8
+
9
+## Build, run, test
10
+
11
+```sh
12
+cargo build # debug build
13
+cargo run # launches the TUI
14
+cargo test # unit tests only (the heavy ones are #[ignore])
15
+cargo clippy --all-targets
16
+```
17
+
18
+The build **bakes credentials in at compile time** via `build.rs`, which reads `.env` (or the environment in CI). The build fails if any of these are missing: `ONDE_APP_ID`, `ONDE_APP_SECRET`, `GRESIQ_API_KEY`, `GRESIQ_API_SECRET`. `HF_TOKEN` is optional (defaults to empty). Changing `.env` triggers a rebuild. These are exposed in code as `env!(...)` constants in `src/app.rs`.
19
+
20
+On macOS the build pulls in candle's `metal` and `accelerate` features, so inference and training run on Metal; elsewhere they fall back to CPU.
21
+
22
+### Tests that load real models
23
+
24
+`src/gguf.rs` has two `#[ignore]` integration tests that export a ~1GB GGUF and run inference through `onde::mistralrs`. They need a Qwen3-0.6B model cached in the Onde App Group container and skip gracefully when it is absent.
25
+
26
+```sh
27
+cargo test gguf::tests::exported_qwen3_gguf_is_runnable -- --ignored --nocapture
28
+cargo test gguf::tests::finetune_merge_export_run -- --ignored --nocapture
29
+```
30
+
31
+Env knobs for these: `ONDE_TEST_MODEL_DIR`, `ONDE_TEST_DTYPE` (`f16`/`q8_0`), `ONDE_TEST_LR`.
32
+
33
+### Debugging the TUI
34
+
35
+`main.rs` redirects both stdout and stderr to `~/.cache/onde/debug.log` before ratatui takes the alternate screen, because `mistral.rs` writes to both fds and would otherwise tear up the TUI. To see what the app or the inference engine is doing, tail that log; `println!`/`eprintln!`/`log::*` all land there, not on screen.
36
+
37
+## Architecture
38
+
39
+### TUI event loop (`app.rs`, `ui.rs`, `main.rs`)
40
+
41
+The whole app is one `App` struct plus a `Screen` enum acting as a state machine. `app::run` owns a single `tokio::sync::mpsc` channel of `AuthEvent`s and a `crossterm` `EventStream`, multiplexed with `tokio::select!`. Keystrokes mutate `App` synchronously; anything slow (network calls, downloads, fine-tune, merge, GGUF export, chat inference) is spawned as a background tokio task or OS thread that streams progress back as `AuthEvent`s, which `App::apply` folds into state. `ui.rs` is a pure render of `App` and intentionally does not depend on the SDK directly (it re-exports `OndeApp`/`OndeModel` through `app.rs`). When adding a feature, the pattern is: add a `Screen` variant, a key handler, an `AuthEvent` variant, and a background task that emits progress.
42
+
43
+Background work uses two task kinds deliberately: network/IO uses `tokio::spawn`; CPU-heavy tensor work (`finetune`, `merge`, `gguf`) uses `std::thread::spawn` so it does not starve the async runtime.
44
+
45
+### The local model pipeline
46
+
47
+These modules form the fine-tune-to-deploy chain, each running on a background thread and streaming a `*Progress` enum:
48
+
49
+- `finetune.rs` — hand-written LoRA trainer (candle). Builds a Qwen forward pass (RMSNorm, RoPE, GQA, optional Qwen3 QK-norm), trains LoRA A/B on q/v projections in F32, writes `lora_adapter.safetensors`. Gradients are sanitized (non-finite elements zeroed) and globally norm-clipped before each AdamW step; a step is skipped entirely if its global grad norm is non-finite. Skipping this guard corrupts every weight after the first bad step.
50
+- `merge.rs` — folds the LoRA adapter back into base weights (`W + scale·(B@A)`), writes a merged `model.safetensors` plus copied config/tokenizer.
51
+- `gguf.rs` — **hand-rolled GGUF writer** (no llama.cpp). Conventions that must hold for mistral.rs/candle to load and run the file correctly: tensor dims are written innermost-first (reverse of safetensors shape, because candle reverses on read); `head_dim` comes from config and is emitted as `attention.key_length`/`value_length` (Qwen3 decouples it from `hidden_size/num_heads`); `token_type` is an INT32 array; `general.architecture` is chosen from `model_type` so Qwen3 routes to candle's `quantized_qwen3` loader.
52
+- `chat.rs` — loads a local GGUF via `onde::mistralrs::GgufModelBuilder` (the same engine the Onde SDK uses) and streams a multi-turn chat, so a model can be tested before publishing.
53
+- `hf_upload.rs` / `hf_clone.rs` / `hf_search.rs` / `hf.rs` — Hugging Face Hub: upload the GGUF, check/create a repo, search models, and resolve/merge the local HF cache (incl. the macOS Onde App Group container).
54
+
55
+### Account / deploy side (`gresiq.rs`, `token.rs`, `project.rs`)
56
+
57
+`gresiq.rs` wraps `smbcloud-gresiq-sdk` for apps and the model catalog. "Deploying" a model means `assign_model(app_id, model_id)` against a catalog entry; the end app (e.g. `rumilearnpersian`) then fetches that assignment through the Onde SDK's `load_assigned_model` and downloads the GGUF. The CLI can only assign models that already exist in the GresIQ catalog. `token.rs` persists the auth token; `project.rs` manages per-project fine-tune workspaces under `~/.onde`.
58
+
59
+### The `onde` dependency
60
+
61
+The `onde` crate provides the inference engine (`onde::mistralrs`, a vendored mistral.rs) and `onde::inference::models::SUPPORTED_MODEL_INFO` (the supported-model catalog the inference picker mirrors). It is normally the published crate; `Cargo.toml` has commented `path`/`[patch.crates-io]` blocks for developing against local checkouts of `onde`, the `smbcloud-*` crates, and `candle`. Never commit with those uncommented.
62
+
63
+## Conventions
64
+
65
+- Git: merge feature/release/hotfix branches with `--no-ff` (explicit merge commits). Tag the merge commit that holds the final release state. See `.agents/skills/git/SKILL.md`.
66
+- Distribution changes: see `.agents/skills/distribution/SKILL.md` before touching any wrapper package; keep all channel versions aligned with the Rust crate version.
src/gguf.rs
+13
-3
@@ -1078,7 +1078,10 @@ mod tests {
1078
});
1079
let adapter_path = adapter_path.expect("fine-tune must produce an adapter");
1080
eprintln!("[e2e] fine-tune done, last loss {last_loss}");
1081
- assert!(last_loss.is_finite(), "training loss is NaN/Inf — training diverged");
1081
+ assert!(
1082
+ last_loss.is_finite(),
1083
+ "training loss is NaN/Inf — training diverged"
1084
+ );
1085
1086
// 2. Merge.
1087
let merged_dir = work.join("merged");
@@ -1158,8 +1161,15 @@ mod tests {
1161
Ok("f16") => GgufDtype::F16,
1162
_ => GgufDtype::Q8_0,
1163
};
1161
- eprintln!("[test] model_dir={} dtype={}", model_dir.display(),
1162
- if matches!(dtype, GgufDtype::F16) { "f16" } else { "q8_0" });
1164
+ eprintln!(
1165
+ "[test] model_dir={} dtype={}",
1166
+ model_dir.display(),
1167
+ if matches!(dtype, GgufDtype::F16) {
1168
+ "f16"
1169
+ } else {
1170
+ "q8_0"
1171
+ }
1172
+ );
1173
1174
let out_dir = std::env::temp_dir().join("onde-gguf-test");
1175
std::fs::create_dir_all(&out_dir).unwrap();