main
md 437 lines 22.9 KB
Rendered Raw
1 # siGit Code Cloud Agent — product plan and roadmap
2
3 Status: draft v2 (2026-06-26). Owner: product/eng. Audience: internal (private repo).
4 Naming and scope per [product-overview.md](product-overview.md); this is the
5 detailed spec for the **siGit Code Cloud Agent** product defined there.
6
7 A GitHub-Copilot-coding-agent-style product: you delegate a task against a Git
8 repository, an isolated AWS sandbox runs siGit Code autonomously against that repo,
9 and it returns a branch plus a proposed pull request. The first repo target is
10 **sigit.si**; the agent is **Git-host agnostic** behind a Git-host adapter so the
11 same agent later runs against **GitHub, GitLab, and other Git hosts**. Scope is
12 **Git only** (no SVN, Mercurial, or other version control systems). This document
13 is the strategy, the architecture, and the phased roadmap.
14
15 ---
16
17 ## 1. What we are building (and what it is not)
18
19 **siGit Code Cloud Agent** is an asynchronous, server-side coding agent. The user
20 delegates a task or issue ("add pagination to the repos list", "fix the failing
21 auth test", "upgrade Rails to 8.1"), points it at one of their repos and a base
22 branch, and walks away. The platform provisions an ephemeral AWS sandbox, clones
23 the repo into it, runs siGit Code headless against the task with hosted inference,
24 lets it edit files and run build/test commands, then pushes a head branch and
25 opens a pull request for human review.
26
27 It is the autonomous sibling of the other two things siGit ships (see
28 [product-overview.md](product-overview.md)):
29
30 - **siGit Code** (Rust CLI / ACP agent) runs *locally / on the desktop*,
31 interactively, driven by the developer at their keyboard.
32 - **siGit Code Cloud** is the *hosted chat* (interactive, synchronous). Its work
33 unit is a chat **session**.
34 - **siGit Code Cloud Agent** runs *in our cloud*, asynchronously, driven by a task
35 and reviewed afterward through a PR. Its work unit is an agent **run** (never a
36 "session", which belongs to the chat product).
37
38 **MVP trigger (decided):** from a repo on sigit.si, the user describes a task and
39 the agent produces a PR. Formal issue-assignment (assign a tracked issue to the
40 agent, the Copilot headline flow) is a fast-follow that depends on an Issues
41 feature, and on the Git-host adapters for GitHub/GitLab issues.
42
43 Reference points in the market: GitHub Copilot coding agent (assign an issue, it
44 opens a PR), OpenAI Codex cloud, Cursor background agents, Devin. The
45 differentiator for us is that we already own the whole vertical: the git host, the
46 agent, the inference, the account, and the billing all live inside siGit. We are
47 not bolting an agent onto someone else's platform; we are completing a platform we
48 already run.
49
50 What it is **not**, in v1: it is not an autonomous merger (a human reviews and
51 merges), not a long-lived persistent dev environment (sandboxes are ephemeral per
52 run), and not a chat product (the chat tier already exists; this is the
53 task-to-PR product on top of it).
54
55 ---
56
57 ## 2. Why siGit is unusually well positioned
58
59 The expensive parts of a cloud-agent product already exist in this repo and the
60 sibling repos. The new work is mostly orchestration and isolation, not net-new
61 agent or inference plumbing.
62
63 | Capability the product needs | Already exists | Where |
64 |---|---|---|
65 | A git host the agent can clone from and push to | yes | `git_http_controller` (smart-HTTP, bare repos on disk, `Repository#disk_path`) |
66 | The coding agent itself (edit/run/test loop, tool calling) | yes | `sigit` (public Rust CLI/ACP agent), `onde-cloud` tool-call mapping |
67 | Hosted inference with auth, identity masking, metering | yes | `Api::V1::ChatCompletionsController``OndeCloudService` → Onde Cloud |
68 | Accounts and a trust boundary that mints scoped tokens | yes | smbCloud auth, `Api::V1` sessions/me, `git_token` exchange |
69 | Subscription gating + monthly metering | yes | `Subscription`, `CloudUsage`, `User#entitled_to_cloud?` |
70
71 What is **missing** and must be built (see roadmap):
72
73 1. **An execution plane on AWS** (the sandbox machine) and the orchestration to
74 drive it. This is the heart of the project.
75 2. **A control plane in Rails**: an `AgentRun` resource, its lifecycle, log
76 streaming, and web UI.
77 3. **Pull requests** (and ideally issues). sigit.si has repos, blobs, commits, and
78 stars, but no `PullRequest` model today. The agent's output is a PR, so a
79 minimal PR/diff/review surface is a hard dependency. This can be scoped down to
80 a "compare and open PR" view for v1.
81 4. **Per-run scoped credentials**: short-lived git-push and inference tokens minted
82 server-side, budget-bounded, never long-lived in the sandbox.
83 5. **A new metering/pricing dimension**: agent runs consume sandbox compute *and*
84 inference tokens, so COGS has two drivers, not one.
85
86 ---
87
88 ## 3. Architecture
89
90 Two planes plus the existing inference path. Control plane is Rails (sigit-si);
91 execution plane is AWS; inference reuses the existing `/api/v1/chat/completions`
92 proxy unchanged.
93
94 ```
95 ┌─ Control plane (Rails, sigit-si) ───────────────────────────────────────────┐
96 │ Web UI: repo "Agent" tab → run form, live transcript, diff, "Open PR" │
97 │ AgentRunsController + Api::V1::AgentRunsController (create/show/cancel/log) │
98 │ AgentRun model (lifecycle state machine) │
99 │ AgentRunJob → provisions sandbox, monitors, collects result │
100 │ Mints per-run scoped tokens (git push + inference), enforces caps/metering │
101 └──────────────────────────────────────────────────────────────────────────────┘
102 │ RunTask (aws-sdk) ▲ SSE/webhook: logs, status, diff
103 ▼ │
104 ┌─ Execution plane (AWS) ─────────────────────────────────────────────────────┐
105 │ Ephemeral sandbox (Fargate task v1 → Firecracker microVM at scale) │
106 │ ├─ clones repo from sigit.si over smart-HTTP (scoped git token) │
107 │ ├─ runs siGit Code headless against the task │
108 │ │ OPENAI_BASE_URL=https://sigit.si/api/v1 (per-run inference token) │
109 │ ├─ edits files, runs build/test in restricted shell │
110 │ └─ pushes head branch back to sigit.si (scoped git token) │
111 │ Private subnet, egress allowlist (sigit.si + package registries only) │
112 │ Hard caps: wall-clock, CPU/mem, token budget, max tool calls │
113 └──────────────────────────────────────────────────────────────────────────────┘
114 │ /v1/chat/completions (per-run token)
115
116 ┌─ Inference (unchanged) ─────────────────────────────────────────────────────┐
117 │ Api::V1::ChatCompletionsController → OndeCloudService → Onde Cloud → upstream│
118 │ Existing entitlement gate, allowance metering, and identity masking apply. │
119 └──────────────────────────────────────────────────────────────────────────────┘
120 ```
121
122 ### 3.1 Control plane (Rails)
123
124 **`AgentRun` model** (new table). Belongs to `user` and `repository`. Fields:
125
126 - `status`: `queued`, `provisioning`, `running`, `pushing`, `needs_input`,
127 `completed`, `failed`, `canceled` (state machine; one-way transitions logged).
128 - `task_prompt` (text), `base_branch`, `head_branch` (generated, e.g.
129 `agent/<run-id>-<slug>`), `pull_request_id` (nullable until pushed).
130 - `sandbox_ref` (ECS task ARN / microVM id), `region`.
131 - Budgets and accounting: `token_budget`, `tokens_used`, `wall_clock_limit_s`,
132 `started_at`, `finished_at`, `exit_reason`.
133 - `transcript_url` (S3 pointer for the full log), plus a tail kept in Postgres for
134 the live view.
135
136 **Controllers**: a web `AgentRunsController` (HTML, Turbo) under the repo, and an
137 `Api::V1::AgentRunsController` so the CLI and desktop can trigger and follow runs.
138 Actions: `create`, `index`, `show`, `cancel`, `messages#create` (steer a running
139 agent), and a `logs` SSE endpoint that relays the live transcript (reuse the
140 `ActionController::Live` pattern already used for chat streaming).
141
142 **`AgentRunJob`** (Active Job, on the existing DB-backed queue): transitions the
143 run to `provisioning`, calls AWS to start the sandbox, persists the sandbox ref,
144 then hands off to monitoring. Cancellation and timeout both tear the sandbox down.
145
146 **Web UI**: a new "Agent" (or "Tasks") tab on the repository page. A run form
147 (task prompt, base branch, optional model tier). A live transcript panel (Turbo
148 Streams fed by the SSE relay). On completion, a diff view and an "Open pull
149 request" action.
150
151 ### 3.2 Execution plane (AWS) — the sandbox machine
152
153 This is the core AWS decision and the riskiest surface, because the sandbox runs
154 build and test commands over user code.
155
156 **Runtime choice.**
157
158 - **v1: AWS Fargate (ECS) ephemeral tasks.** One task per run. Scales to zero, pay
159 per second, decent container isolation, no servers to manage, and `RunTask` is a
160 single SDK call from a Rails job. Fast to ship. This is the recommendation for
161 v1.
162 - **At scale: Firecracker microVMs.** For stronger isolation of untrusted code and
163 for snapshot/restore warm pools (sub-second starts), move the sandbox to
164 Firecracker microVMs on bare-metal EC2 (the model E2B / Modal / Codex-style
165 sandboxes use). More operational weight; defer until run volume and the threat
166 model justify it. gVisor or Kata on EC2 is a middle option if Fargate isolation
167 proves insufficient before we are ready for Firecracker.
168
169 **The sandbox image.** A container that bundles headless siGit Code plus a base
170 toolchain (git, common language runtimes). The agent boots, reads the run spec
171 from an injected env/file, clones, works, and pushes. Per-language base images (or
172 a `.sigit/agent.yml` setup step, see Phase 3) keep cold builds fast.
173
174 **Orchestration.** v1 keeps it simple: the Rails `AgentRunJob` calls ECS `RunTask`
175 directly via `aws-sdk-ecs`, passing the run spec as container overrides, and polls
176 task status (or receives EventBridge task-state-change events into a webhook). If
177 the lifecycle grows (retries, multi-step, fan-out), promote to Step Functions.
178 Avoid Step Functions on day one; it is premature.
179
180 **Networking and isolation (load-bearing for safety).**
181
182 - Sandbox runs in a **private subnet**. Egress through a NAT restricted by an
183 **allowlist**: sigit.si (git + inference) and an explicit set of package
184 registries (rubygems, npm, pypi, crates, etc.). Everything else is denied. This
185 is the primary control against data exfiltration, SSRF against internal
186 services, and crypto-mining abuse.
187 - **No inbound.** The sandbox is not reachable from the internet.
188 - Per-run IAM role scoped to only what the task needs; no broad account access
189 inside the sandbox.
190
191 **Credentials (mint short-lived, never long-lived).** The sandbox receives:
192
193 - a **git token scoped to the single repo and ideally the single head branch**,
194 valid for the run only (extends the existing `git_token` exchange);
195 - an **inference token** minted per run, carrying the user's entitlement and a
196 hard token budget, pointed at `https://sigit.si/api/v1`.
197
198 Both expire when the run ends. The sandbox never holds `app_secret` or any
199 long-lived credential, mirroring the existing public-client rule.
200
201 **Logs and artifacts.** The agent streams transcript chunks back to Rails (the SSE
202 relay) for the live view and writes the full transcript and build logs to S3
203 (pointer stored on `AgentRun`). CloudWatch captures infra-level logs.
204
205 ### 3.3 Inference path (reused as-is)
206
207 The sandboxed agent sets `OPENAI_BASE_URL=https://sigit.si/api/v1` and uses the
208 per-run inference token as its bearer. That flows through the **existing**
209 `Api::V1::ChatCompletionsController`: entitlement gate, allowance metering, and the
210 `SIGIT_IDENTITY_PROMPT` identity masking all apply with no change. This is a major
211 reason the project is tractable: the autonomous agent is just another client of an
212 inference endpoint we already operate and protect.
213
214 This also gives a clean answer to the existing "inference token ↔ Onde Cloud auth"
215 open item: for cloud-agent runs the token is minted server-side with a budget, so
216 there is no public client holding credentials at all.
217
218 ### 3.4 Git and PR flow
219
220 The agent pushes the head branch via the existing `git-receive-pack` endpoint. On
221 push, the platform creates (or links) the PR record. Because **no PR model exists
222 yet**, scope for v1:
223
224 - minimal `PullRequest` model (base/head branch, repo, author, status, title,
225 body), a diff/compare view (we already render blobs and commits, so the diff
226 renderer is incremental), and "open / close / merge" actions for the reviewer;
227 - the agent fills in title and body from its summary. PR prose must stay neutral
228 and must never name the upstream model or provider (same rule as chat).
229
230 Issues (assign-an-issue-to-the-agent) are a Phase 3 surface and depend on an issue
231 model that also does not exist yet.
232
233 ---
234
235 ## 4. Safety, guardrails, and abuse
236
237 Principal-engineer non-negotiables, because this executes code on our infra on
238 behalf of users:
239
240 - **Hard caps per run**: wall-clock timeout, CPU/memory limits, inference token
241 budget (tied to `CloudUsage`/a new `AgentUsage`), max tool calls, max sandbox
242 lifetime. A run that blows any cap is killed and marked `failed` with a reason.
243 - **Network egress allowlist** (section 3.2). The single most important control.
244 - **Ephemeral, scoped credentials** only. Nothing long-lived in the sandbox.
245 - **Human-in-the-loop by default**: the agent proposes a PR; it does not merge. No
246 auto-merge in v1.
247 - **Concurrency limits per plan**: caps simultaneous runs per user to bound spend
248 and abuse.
249 - **Identity hygiene**: transcripts, PR titles/bodies, and error messages stay
250 neutral; never disclose the upstream model or provider (existing rule extends to
251 agent output).
252 - **Cancellation is real**: cancel tears down the sandbox and revokes the run's
253 tokens.
254 - **Idempotency**: run creation and sandbox start are idempotent so retries cannot
255 double-spend.
256
257 A short threat-model doc is a Phase 0 deliverable (exfiltration, SSRF to internal
258 metadata endpoints, resource abuse / mining, secret leakage from the user's own
259 repo, prompt injection from repo contents steering the agent).
260
261 ---
262
263 ## 5. Pricing and packaging
264
265 Agent runs cost us **sandbox compute (Fargate per-second) + inference tokens**, so
266 metering must capture both and price above blended COGS. The numbers below are a
267 working model with the inputs we cannot yet read from this repo marked
268 `[FILL FROM PHASE 0]`. The metering schema and billing copy should be designed
269 against this formula now; the dollar figures get populated once the Phase 0
270 dogfood produces measured per-run token counts.
271
272 ### 5.1 Unit economics: cost per run (COGS)
273
274 ```
275 cost_per_run = sandbox_cost + inference_cost
276
277 sandbox_cost = vcpu_count * $0.04048/vcpu-hr * hours
278 + mem_gb * $0.004445/gb-hr * hours (Fargate, us-east-1)
279
280 inference_cost = tokens_per_run / 1_000_000 * blended_upstream_cost_per_mtok
281 ```
282
283 Worked example, sandbox side (known, fixed): 1 vCPU + 2 GB for a 20-minute run:
284
285 ```
286 (1 * 0.04048 + 2 * 0.004445) * (20/60) = $0.0165 per run
287 ```
288
289 Sandbox compute is rounding error. **Inference dominates**, and it is the unknown:
290
291 ```
292 tokens_per_run = [FILL FROM PHASE 0] (expect 200K – 2M+)
293 blended_upstream_cost_per_mtok = [FILL: Onde/upstream blended $/Mtok]
294
295 inference_cost_per_run = (tokens_per_run / 1e6) * blended_cost_per_mtok
296 cost_per_run = 0.0165 + inference_cost_per_run
297 ```
298
299 Conclusion that holds regardless of the blanks: **price on metered inference per
300 run; treat sandbox-minutes as a kill-switch guardrail, not a billing axis.**
301
302 ### 5.2 Packaging (allowance model, mirrors `Subscription::CLOUD_ALLOWANCE`)
303
304 Agent runs are an included monthly bundle per plan, with overage or an add-on for
305 heavy use. Suggested `Subscription` constant shape:
306
307 ```ruby
308 # Included siGit Code Cloud Agent runs per billing period. Tunable pricing knob.
309 AGENT_RUNS = { "pro" => [FILL], "team" => [FILL] }.freeze # e.g. pro 20, team 75
310 TRIAL_AGENT_RUNS = [FILL] # e.g. 2
311 AGENT_OVERAGE_PER_RUN = [FILL] # $/run past the bundle
312 ```
313
314 | Plan | Price (today) | Included agent runs/mo | Overage | Notes |
315 |---|---|---|---|---|
316 | Free | $0 | 0 | n/a | local siGit Code only, no cloud agent |
317 | Trial | $0 (14 days) | `[FILL: ~2]` | n/a | felt-value, hard run cap |
318 | Pro | $20/mo (existing) | `[FILL: ~20]` | `[FILL: $/run]` | bundle sized so COGS < ~X% of $20 |
319 | Team | (existing) | `[FILL: ~75]` | `[FILL: $/run]` | higher bundle + concurrency |
320
321 Sizing rule for the bundle: pick included-runs so that **bundle COGS stays under a
322 target fraction of plan price** (e.g. included-runs * cost_per_run ≤ 40% of MRR),
323 leaving margin for the chat allowance those plans already include. With
324 `cost_per_run` from 5.1 unknown, the bundle count is the lever set last.
325
326 ### 5.3 Metering
327
328 Add `AgentUsage` keyed by `(user, billing_period_key)` (parallel to `CloudUsage`),
329 recording per period: `runs_count`, `agent_seconds`, `tokens_used`. Runs decrement
330 the plan's run bundle; tokens already flow through the existing cloud allowance and
331 its 429 cap, so a single run can never escape the token budget. `GET
332 /api/v1/billing` gains `agent_runs_used` + `agent_runs_allowance` alongside the
333 existing cloud fields.
334
335 ### 5.4 Guardrails (worst-case cost containment)
336
337 - **Per-run token budget** minted into the run token, independent of the monthly
338 allowance, so one pathological run is bounded.
339 - **Per-run wall-clock + sandbox-lifetime cap** (kills runaway compute).
340 - **Per-plan concurrency cap** bounds simultaneous spend.
341 - **Trial run cap** (`TRIAL_AGENT_RUNS`) prevents trial-driven upstream bills,
342 mirroring `TRIAL_ALLOWANCE`.
343
344 ### 5.5 What Phase 0 must measure to finalize this
345
346 1. **`tokens_per_run`** distribution (p50/p90/p99) over 20–30 real dogfood runs.
347 2. **`blended_upstream_cost_per_mtok`** (from the Onde/upstream cost, internal).
348 3. Resulting **`cost_per_run`** distribution, then back-solve included-run bundles
349 per tier against the target-margin rule in 5.2.
350
351 Until 1 and 2 are real, every dollar above is a placeholder by design.
352
353 ---
354
355 ## 6. Roadmap (phased)
356
357 Estimates are calendar weeks for a small team; adjust to staffing. Each phase ends
358 with a demoable, dogfoodable increment.
359
360 ### Phase 0 — Foundations and spikes (2–3 weeks)
361 - Decide sandbox runtime: **Fargate for v1** (documented path to Firecracker).
362 - Build the headless siGit Code container image; prove end to end **manually**: a
363 container clones a sigit.si repo, runs the agent against a task with inference via
364 `/api/v1`, edits files, runs a test, and pushes a branch.
365 - Per-run scoped token minting (git + inference) in Rails.
366 - Threat model + egress allowlist design.
367 - Add `aws-sdk-ecs`/`aws-sdk-core` to the Gemfile; stand up the VPC/subnet/NAT and
368 the task definition in IaC.
369 - **Exit:** one task goes from prompt to pushed branch by hand.
370
371 ### Phase 1 — Private alpha, single happy path (3–4 weeks)
372 - `AgentRun` model + state machine; web `AgentRunsController`; `AgentRunJob` calling
373 ECS `RunTask`.
374 - Repo "Agent" tab: run form, live transcript via SSE/Turbo, final diff view.
375 - Branch push + a minimal compare view (full PR model can lag one phase).
376 - Caps: wall-clock timeout, concurrency = 1, token budget tied to metering.
377 - Internal-only feature flag; dogfood on our own repos.
378 - **Exit:** team members trigger runs from the web and review the diff.
379
380 ### Phase 2 — Beta, productized (4–6 weeks)
381 - Minimal `PullRequest` model + diff/review UI so agent output is a real PR.
382 - Steerability: follow-up messages to a running agent, cancel, re-run.
383 - Metering + pricing surface: `AgentUsage`, plan caps, billing page copy.
384 - Hardening: enforce egress allowlist, isolation review, token scoping, per-plan
385 concurrency, abuse controls.
386 - Trigger surfaces: `sigit cloud run` in the CLI and a desktop entry point, both
387 hitting `Api::V1::AgentRunsController`.
388 - Cold-start work (prebaked base images; warm pool if needed).
389 - **Exit:** invite-only beta with real external users and real billing.
390
391 ### Phase 3 — GA and advanced (ongoing)
392 - Issues + assign-an-issue-to-the-agent (needs an issue model).
393 - Custom environments: repo-level `.sigit/agent.yml` (setup steps, allowed
394 commands, language matrix), the analog of Copilot's environment customization.
395 - Firecracker microVM sandboxes + snapshot warm pools for isolation and sub-second
396 starts at volume.
397 - Dependency/repo caching (S3/EFS layers) for faster, cheaper runs.
398 - Parallel runs, plan-then-execute, multi-file refactors at scale.
399 - Observability and an eval harness (task success rate, PR acceptance rate, cost
400 per accepted PR) to drive model-tier and prompt tuning.
401
402 ---
403
404 ## 7. Key decisions and open dependencies
405
406 - **PRs/issues do not exist on sigit.si yet.** A minimal PR surface is on the
407 critical path (Phase 2); issues gate the issue-to-PR flow (Phase 3). Confirm
408 scope early.
409 - **Sandbox isolation level.** Fargate for v1; reassess against the threat model
410 before opening to untrusted external users at volume; Firecracker is the scale
411 answer.
412 - **Inference token model.** Mint per-run, server-side, budget-bounded. This also
413 closes the standing "inference token ↔ Onde Cloud auth" gap for this path.
414 - **COGS visibility.** Must meter sandbox time *and* tokens from day one, or
415 pricing flies blind.
416 - **No AWS SDK in the app yet.** Adds a new infra dependency and IAM/VPC footprint
417 to own and secure.
418 - **Naming (resolved).** The product is **siGit Code Cloud Agent** (see
419 [product-overview.md](product-overview.md)). "session" is reserved for siGit Code
420 Cloud chat; the agent's work unit is a **run**. Do not name the agent or its runs
421 "sessions".
422 - **Git-host scope.** Git only (no SVN, Mercurial, or other VCS). sigit.si is the
423 first Git host, not the only one. Keep the git clone / branch-push / PR-open
424 behind a `GitHostAdapter` seam from day one (sigit.si now; GitHub, GitLab, other
425 Git hosts later) so the agent is not welded to our own forge.
426
427 ---
428
429 ## 8. North-star and success metrics
430
431 - **North star:** accepted agent PRs per active paying user per month.
432 - **Quality:** task success rate (run produces a mergeable PR without human
433 fixes), PR acceptance rate, median time-to-PR.
434 - **Economics:** cost per accepted PR (sandbox + tokens), gross margin per plan.
435 - **Reliability/safety:** zero egress-allowlist escapes, zero credential leaks,
436 sandbox p95 cold start, run failure rate by cause.
437 ```