Add cloud-agent product plan and roadmap (internal)

new file mode 100644 index 0000000..fb8c6bd --- /dev/null +++ b/docs/product/cloud-agent-plan.md @@ -0,0 +1,419 @@ +# siGit Cloud Agent — product plan and roadmap + +Status: draft v1 (2026-06-26). Owner: product/eng. Audience: internal (private repo). + +A GitHub-Copilot-coding-agent-style product built into sigit.si and siGit Code +Cloud: you give it a task against a repo you host on sigit.si, an isolated AWS +sandbox runs the siGit Code agent loop against that repo, and it returns a branch +plus a proposed pull request. This document is the strategy, the architecture, and +the phased roadmap. + +--- + +## 1. What we are building (and what it is not) + +**siGit Cloud Agent** is an asynchronous, server-side coding agent. The user +describes a task ("add pagination to the repos list", "fix the failing +auth test", "upgrade Rails to 8.1"), points it at one of their sigit.si repos and +a base branch, and walks away. The platform provisions an ephemeral AWS sandbox, +clones the repo into it, runs siGit Code headless against the task with cloud +inference, lets it edit files and run build/test commands, then pushes a head +branch back to sigit.si and opens a pull request for human review. + +It is the cloud, autonomous sibling of the two things siGit already ships: + +- **siGit Code** (public Rust CLI / ACP agent) runs *locally / on the desktop*, + interactively, driven by the developer at their keyboard. +- **siGit Cloud Agent** runs *in our cloud*, asynchronously, driven by a task and + reviewed afterward through a PR. + +Reference points in the market: GitHub Copilot coding agent (assign an issue, it +opens a PR), OpenAI Codex cloud, Cursor background agents, Devin. The +differentiator for us is that we already own the whole vertical: the git host, the +agent, the inference, the account, and the billing all live inside siGit. We are +not bolting an agent onto someone else's platform; we are completing a platform we +already run. + +What it is **not**, in v1: it is not an autonomous merger (a human reviews and +merges), not a long-lived persistent dev environment (sandboxes are ephemeral per +run), and not a chat product (the chat tier already exists; this is the +task-to-PR product on top of it). + +--- + +## 2. Why siGit is unusually well positioned + +The expensive parts of a cloud-agent product already exist in this repo and the +sibling repos. The new work is mostly orchestration and isolation, not net-new +agent or inference plumbing. + +| Capability the product needs | Already exists | Where | +|---|---|---| +| A git host the agent can clone from and push to | yes | `git_http_controller` (smart-HTTP, bare repos on disk, `Repository#disk_path`) | +| The coding agent itself (edit/run/test loop, tool calling) | yes | `sigit` (public Rust CLI/ACP agent), `onde-cloud` tool-call mapping | +| Hosted inference with auth, identity masking, metering | yes | `Api::V1::ChatCompletionsController` → `OndeCloudService` → Onde Cloud | +| Accounts and a trust boundary that mints scoped tokens | yes | smbCloud auth, `Api::V1` sessions/me, `git_token` exchange | +| Subscription gating + monthly metering | yes | `Subscription`, `CloudUsage`, `User#entitled_to_cloud?` | + +What is **missing** and must be built (see roadmap): + +1. **An execution plane on AWS** (the sandbox machine) and the orchestration to + drive it. This is the heart of the project. +2. **A control plane in Rails**: an `AgentRun` resource, its lifecycle, log + streaming, and web UI. +3. **Pull requests** (and ideally issues). sigit.si has repos, blobs, commits, and + stars, but no `PullRequest` model today. The agent's output is a PR, so a + minimal PR/diff/review surface is a hard dependency. This can be scoped down to + a "compare and open PR" view for v1. +4. **Per-run scoped credentials**: short-lived git-push and inference tokens minted + server-side, budget-bounded, never long-lived in the sandbox. +5. **A new metering/pricing dimension**: agent runs consume sandbox compute *and* + inference tokens, so COGS has two drivers, not one. + +--- + +## 3. Architecture + +Two planes plus the existing inference path. Control plane is Rails (sigit-si); +execution plane is AWS; inference reuses the existing `/api/v1/chat/completions` +proxy unchanged. + +``` +┌─ Control plane (Rails, sigit-si) ───────────────────────────────────────────┐ +│ Web UI: repo "Agent" tab → run form, live transcript, diff, "Open PR" │ +│ AgentRunsController + Api::V1::AgentRunsController (create/show/cancel/log) │ +│ AgentRun model (lifecycle state machine) │ +│ AgentRunJob → provisions sandbox, monitors, collects result │ +│ Mints per-run scoped tokens (git push + inference), enforces caps/metering │ +└──────────────────────────────────────────────────────────────────────────────┘ + │ RunTask (aws-sdk) ▲ SSE/webhook: logs, status, diff + ▼ │ +┌─ Execution plane (AWS) ─────────────────────────────────────────────────────┐ +│ Ephemeral sandbox (Fargate task v1 → Firecracker microVM at scale) │ +│ ├─ clones repo from sigit.si over smart-HTTP (scoped git token) │ +│ ├─ runs siGit Code headless against the task │ +│ │ OPENAI_BASE_URL=https://sigit.si/api/v1 (per-run inference token) │ +│ ├─ edits files, runs build/test in restricted shell │ +│ └─ pushes head branch back to sigit.si (scoped git token) │ +│ Private subnet, egress allowlist (sigit.si + package registries only) │ +│ Hard caps: wall-clock, CPU/mem, token budget, max tool calls │ +└──────────────────────────────────────────────────────────────────────────────┘ + │ /v1/chat/completions (per-run token) + ▼ +┌─ Inference (unchanged) ─────────────────────────────────────────────────────┐ +│ Api::V1::ChatCompletionsController → OndeCloudService → Onde Cloud → upstream│ +│ Existing entitlement gate, allowance metering, and identity masking apply. │ +└──────────────────────────────────────────────────────────────────────────────┘ +``` + +### 3.1 Control plane (Rails) + +**`AgentRun` model** (new table). Belongs to `user` and `repository`. Fields: + +- `status`: `queued`, `provisioning`, `running`, `pushing`, `needs_input`, + `completed`, `failed`, `canceled` (state machine; one-way transitions logged). +- `task_prompt` (text), `base_branch`, `head_branch` (generated, e.g. + `agent/<run-id>-<slug>`), `pull_request_id` (nullable until pushed). +- `sandbox_ref` (ECS task ARN / microVM id), `region`. +- Budgets and accounting: `token_budget`, `tokens_used`, `wall_clock_limit_s`, + `started_at`, `finished_at`, `exit_reason`. +- `transcript_url` (S3 pointer for the full log), plus a tail kept in Postgres for + the live view. + +**Controllers**: a web `AgentRunsController` (HTML, Turbo) under the repo, and an +`Api::V1::AgentRunsController` so the CLI and desktop can trigger and follow runs. +Actions: `create`, `index`, `show`, `cancel`, `messages#create` (steer a running +agent), and a `logs` SSE endpoint that relays the live transcript (reuse the +`ActionController::Live` pattern already used for chat streaming). + +**`AgentRunJob`** (Active Job, on the existing DB-backed queue): transitions the +run to `provisioning`, calls AWS to start the sandbox, persists the sandbox ref, +then hands off to monitoring. Cancellation and timeout both tear the sandbox down. + +**Web UI**: a new "Agent" (or "Tasks") tab on the repository page. A run form +(task prompt, base branch, optional model tier). A live transcript panel (Turbo +Streams fed by the SSE relay). On completion, a diff view and an "Open pull +request" action. + +### 3.2 Execution plane (AWS) — the sandbox machine + +This is the core AWS decision and the riskiest surface, because the sandbox runs +build and test commands over user code. + +**Runtime choice.** + +- **v1: AWS Fargate (ECS) ephemeral tasks.** One task per run. Scales to zero, pay + per second, decent container isolation, no servers to manage, and `RunTask` is a + single SDK call from a Rails job. Fast to ship. This is the recommendation for + v1. +- **At scale: Firecracker microVMs.** For stronger isolation of untrusted code and + for snapshot/restore warm pools (sub-second starts), move the sandbox to + Firecracker microVMs on bare-metal EC2 (the model E2B / Modal / Codex-style + sandboxes use). More operational weight; defer until run volume and the threat + model justify it. gVisor or Kata on EC2 is a middle option if Fargate isolation + proves insufficient before we are ready for Firecracker. + +**The sandbox image.** A container that bundles headless siGit Code plus a base +toolchain (git, common language runtimes). The agent boots, reads the run spec +from an injected env/file, clones, works, and pushes. Per-language base images (or +a `.sigit/agent.yml` setup step, see Phase 3) keep cold builds fast. + +**Orchestration.** v1 keeps it simple: the Rails `AgentRunJob` calls ECS `RunTask` +directly via `aws-sdk-ecs`, passing the run spec as container overrides, and polls +task status (or receives EventBridge task-state-change events into a webhook). If +the lifecycle grows (retries, multi-step, fan-out), promote to Step Functions. +Avoid Step Functions on day one; it is premature. + +**Networking and isolation (load-bearing for safety).** + +- Sandbox runs in a **private subnet**. Egress through a NAT restricted by an + **allowlist**: sigit.si (git + inference) and an explicit set of package + registries (rubygems, npm, pypi, crates, etc.). Everything else is denied. This + is the primary control against data exfiltration, SSRF against internal + services, and crypto-mining abuse. +- **No inbound.** The sandbox is not reachable from the internet. +- Per-run IAM role scoped to only what the task needs; no broad account access + inside the sandbox. + +**Credentials (mint short-lived, never long-lived).** The sandbox receives: + +- a **git token scoped to the single repo and ideally the single head branch**, + valid for the run only (extends the existing `git_token` exchange); +- an **inference token** minted per run, carrying the user's entitlement and a + hard token budget, pointed at `https://sigit.si/api/v1`. + +Both expire when the run ends. The sandbox never holds `app_secret` or any +long-lived credential, mirroring the existing public-client rule. + +**Logs and artifacts.** The agent streams transcript chunks back to Rails (the SSE +relay) for the live view and writes the full transcript and build logs to S3 +(pointer stored on `AgentRun`). CloudWatch captures infra-level logs. + +### 3.3 Inference path (reused as-is) + +The sandboxed agent sets `OPENAI_BASE_URL=https://sigit.si/api/v1` and uses the +per-run inference token as its bearer. That flows through the **existing** +`Api::V1::ChatCompletionsController`: entitlement gate, allowance metering, and the +`SIGIT_IDENTITY_PROMPT` identity masking all apply with no change. This is a major +reason the project is tractable: the autonomous agent is just another client of an +inference endpoint we already operate and protect. + +This also gives a clean answer to the existing "inference token ↔ Onde Cloud auth" +open item: for cloud-agent runs the token is minted server-side with a budget, so +there is no public client holding credentials at all. + +### 3.4 Git and PR flow + +The agent pushes the head branch via the existing `git-receive-pack` endpoint. On +push, the platform creates (or links) the PR record. Because **no PR model exists +yet**, scope for v1: + +- minimal `PullRequest` model (base/head branch, repo, author, status, title, + body), a diff/compare view (we already render blobs and commits, so the diff + renderer is incremental), and "open / close / merge" actions for the reviewer; +- the agent fills in title and body from its summary. PR prose must stay neutral + and must never name the upstream model or provider (same rule as chat). + +Issues (assign-an-issue-to-the-agent) are a Phase 3 surface and depend on an issue +model that also does not exist yet. + +--- + +## 4. Safety, guardrails, and abuse + +Principal-engineer non-negotiables, because this executes code on our infra on +behalf of users: + +- **Hard caps per run**: wall-clock timeout, CPU/memory limits, inference token + budget (tied to `CloudUsage`/a new `AgentUsage`), max tool calls, max sandbox + lifetime. A run that blows any cap is killed and marked `failed` with a reason. +- **Network egress allowlist** (section 3.2). The single most important control. +- **Ephemeral, scoped credentials** only. Nothing long-lived in the sandbox. +- **Human-in-the-loop by default**: the agent proposes a PR; it does not merge. No + auto-merge in v1. +- **Concurrency limits per plan**: caps simultaneous runs per user to bound spend + and abuse. +- **Identity hygiene**: transcripts, PR titles/bodies, and error messages stay + neutral; never disclose the upstream model or provider (existing rule extends to + agent output). +- **Cancellation is real**: cancel tears down the sandbox and revokes the run's + tokens. +- **Idempotency**: run creation and sandbox start are idempotent so retries cannot + double-spend. + +A short threat-model doc is a Phase 0 deliverable (exfiltration, SSRF to internal +metadata endpoints, resource abuse / mining, secret leakage from the user's own +repo, prompt injection from repo contents steering the agent). + +--- + +## 5. Pricing and packaging + +Agent runs cost us **sandbox compute (Fargate per-second) + inference tokens**, so +metering must capture both and price above blended COGS. The numbers below are a +working model with the inputs we cannot yet read from this repo marked +`[FILL FROM PHASE 0]`. The metering schema and billing copy should be designed +against this formula now; the dollar figures get populated once the Phase 0 +dogfood produces measured per-run token counts. + +### 5.1 Unit economics: cost per run (COGS) + +``` +cost_per_run = sandbox_cost + inference_cost + +sandbox_cost = vcpu_count * $0.04048/vcpu-hr * hours + + mem_gb * $0.004445/gb-hr * hours (Fargate, us-east-1) + +inference_cost = tokens_per_run / 1_000_000 * blended_upstream_cost_per_mtok +``` + +Worked example, sandbox side (known, fixed): 1 vCPU + 2 GB for a 20-minute run: + +``` +(1 * 0.04048 + 2 * 0.004445) * (20/60) = $0.0165 per run +``` + +Sandbox compute is rounding error. **Inference dominates**, and it is the unknown: + +``` +tokens_per_run = [FILL FROM PHASE 0] (expect 200K – 2M+) +blended_upstream_cost_per_mtok = [FILL: Onde/upstream blended $/Mtok] + +inference_cost_per_run = (tokens_per_run / 1e6) * blended_cost_per_mtok +cost_per_run = 0.0165 + inference_cost_per_run +``` + +Conclusion that holds regardless of the blanks: **price on metered inference per +run; treat sandbox-minutes as a kill-switch guardrail, not a billing axis.** + +### 5.2 Packaging (allowance model, mirrors `Subscription::CLOUD_ALLOWANCE`) + +Agent runs are an included monthly bundle per plan, with overage or an add-on for +heavy use. Suggested `Subscription` constant shape: + +```ruby +# Included siGit Cloud Agent runs per billing period. Tunable pricing knob. +AGENT_RUNS = { "pro" => [FILL], "team" => [FILL] }.freeze # e.g. pro 20, team 75 +TRIAL_AGENT_RUNS = [FILL] # e.g. 2 +AGENT_OVERAGE_PER_RUN = [FILL] # $/run past the bundle +``` + +| Plan | Price (today) | Included agent runs/mo | Overage | Notes | +|---|---|---|---|---| +| Free | $0 | 0 | n/a | local siGit Code only, no cloud agent | +| Trial | $0 (14 days) | `[FILL: ~2]` | n/a | felt-value, hard run cap | +| Pro | $20/mo (existing) | `[FILL: ~20]` | `[FILL: $/run]` | bundle sized so COGS < ~X% of $20 | +| Team | (existing) | `[FILL: ~75]` | `[FILL: $/run]` | higher bundle + concurrency | + +Sizing rule for the bundle: pick included-runs so that **bundle COGS stays under a +target fraction of plan price** (e.g. included-runs * cost_per_run ≤ 40% of MRR), +leaving margin for the chat allowance those plans already include. With +`cost_per_run` from 5.1 unknown, the bundle count is the lever set last. + +### 5.3 Metering + +Add `AgentUsage` keyed by `(user, billing_period_key)` (parallel to `CloudUsage`), +recording per period: `runs_count`, `agent_seconds`, `tokens_used`. Runs decrement +the plan's run bundle; tokens already flow through the existing cloud allowance and +its 429 cap, so a single run can never escape the token budget. `GET +/api/v1/billing` gains `agent_runs_used` + `agent_runs_allowance` alongside the +existing cloud fields. + +### 5.4 Guardrails (worst-case cost containment) + +- **Per-run token budget** minted into the run token, independent of the monthly + allowance, so one pathological run is bounded. +- **Per-run wall-clock + sandbox-lifetime cap** (kills runaway compute). +- **Per-plan concurrency cap** bounds simultaneous spend. +- **Trial run cap** (`TRIAL_AGENT_RUNS`) prevents trial-driven upstream bills, + mirroring `TRIAL_ALLOWANCE`. + +### 5.5 What Phase 0 must measure to finalize this + +1. **`tokens_per_run`** distribution (p50/p90/p99) over 20–30 real dogfood runs. +2. **`blended_upstream_cost_per_mtok`** (from the Onde/upstream cost, internal). +3. Resulting **`cost_per_run`** distribution, then back-solve included-run bundles + per tier against the target-margin rule in 5.2. + +Until 1 and 2 are real, every dollar above is a placeholder by design. + +--- + +## 6. Roadmap (phased) + +Estimates are calendar weeks for a small team; adjust to staffing. Each phase ends +with a demoable, dogfoodable increment. + +### Phase 0 — Foundations and spikes (2–3 weeks) +- Decide sandbox runtime: **Fargate for v1** (documented path to Firecracker). +- Build the headless siGit Code container image; prove end to end **manually**: a + container clones a sigit.si repo, runs the agent against a task with inference via + `/api/v1`, edits files, runs a test, and pushes a branch. +- Per-run scoped token minting (git + inference) in Rails. +- Threat model + egress allowlist design. +- Add `aws-sdk-ecs`/`aws-sdk-core` to the Gemfile; stand up the VPC/subnet/NAT and + the task definition in IaC. +- **Exit:** one task goes from prompt to pushed branch by hand. + +### Phase 1 — Private alpha, single happy path (3–4 weeks) +- `AgentRun` model + state machine; web `AgentRunsController`; `AgentRunJob` calling + ECS `RunTask`. +- Repo "Agent" tab: run form, live transcript via SSE/Turbo, final diff view. +- Branch push + a minimal compare view (full PR model can lag one phase). +- Caps: wall-clock timeout, concurrency = 1, token budget tied to metering. +- Internal-only feature flag; dogfood on our own repos. +- **Exit:** team members trigger runs from the web and review the diff. + +### Phase 2 — Beta, productized (4–6 weeks) +- Minimal `PullRequest` model + diff/review UI so agent output is a real PR. +- Steerability: follow-up messages to a running agent, cancel, re-run. +- Metering + pricing surface: `AgentUsage`, plan caps, billing page copy. +- Hardening: enforce egress allowlist, isolation review, token scoping, per-plan + concurrency, abuse controls. +- Trigger surfaces: `sigit cloud run` in the CLI and a desktop entry point, both + hitting `Api::V1::AgentRunsController`. +- Cold-start work (prebaked base images; warm pool if needed). +- **Exit:** invite-only beta with real external users and real billing. + +### Phase 3 — GA and advanced (ongoing) +- Issues + assign-an-issue-to-the-agent (needs an issue model). +- Custom environments: repo-level `.sigit/agent.yml` (setup steps, allowed + commands, language matrix), the analog of Copilot's environment customization. +- Firecracker microVM sandboxes + snapshot warm pools for isolation and sub-second + starts at volume. +- Dependency/repo caching (S3/EFS layers) for faster, cheaper runs. +- Parallel runs, plan-then-execute, multi-file refactors at scale. +- Observability and an eval harness (task success rate, PR acceptance rate, cost + per accepted PR) to drive model-tier and prompt tuning. + +--- + +## 7. Key decisions and open dependencies + +- **PRs/issues do not exist on sigit.si yet.** A minimal PR surface is on the + critical path (Phase 2); issues gate the issue-to-PR flow (Phase 3). Confirm + scope early. +- **Sandbox isolation level.** Fargate for v1; reassess against the threat model + before opening to untrusted external users at volume; Firecracker is the scale + answer. +- **Inference token model.** Mint per-run, server-side, budget-bounded. This also + closes the standing "inference token ↔ Onde Cloud auth" gap for this path. +- **COGS visibility.** Must meter sandbox time *and* tokens from day one, or + pricing flies blind. +- **No AWS SDK in the app yet.** Adds a new infra dependency and IAM/VPC footprint + to own and secure. +- **Naming.** "siGit Cloud Agent" is a working name; align with the existing + brand rules (`siGit Code`, `siGit Code Cloud`, `Onde Cloud`) before anything + ships to users. + +--- + +## 8. North-star and success metrics + +- **North star:** accepted agent PRs per active paying user per month. +- **Quality:** task success rate (run produces a mergeable PR without human + fixes), PR acceptance rate, median time-to-PR. +- **Economics:** cost per accepted PR (sandbox + tokens), gross margin per plan. +- **Reliability/safety:** zero egress-allowlist escapes, zero credential leaks, + sandbox p95 cold start, run failure rate by cause. +```