The best way to control Claude Code, Codex and more from any computer or phone

main
md 548 lines 27.2 KB
Rendered Raw
1 # RFC: Git Building Blocks
2
3 ## Current implementation
4
5 Git contributes native Effect route and storage layers. Applications own their
6 API schema and server. The public `Git.ApiLive` is composed beside application
7 routes; `Git.InternalApiLive` is mounted separately from user authentication.
8
9 ```ts
10 const PublicRoutes = Layer.mergeAll(AppApiLive, Git.ApiLive).pipe(
11 Layer.provide(Authentication.layer),
12 );
13 const Routes = Layer.mergeAll(PublicRoutes, Git.InternalApiLive).pipe(
14 Layer.provide(Git.ApiHandlersLive),
15 // repository, registry, hasher, blob, and platform layers
16 );
17 const fetch = yield* HttpRouter.toHttpEffect(Routes);
18 ```
19
20 Applications can also build every group against their own `AppApi`, including
21 native `HttpApiMiddleware` that provides a typed user. Raw protocol handlers use
22 `@alchemy.run/git/Http` to decode streaming requests and encode Git responses, and
23 `Git.Engine` for repository operations. `preparePush` stages and validates objects;
24 application effects authorize refs and inspect content; `commitPush` advances refs.
25 Ref writes and merges have scoped preparation methods too. No policy callback
26 service is installed. See [DESIGN.md §8](./DESIGN.md#8-auth-model-nothing-inside-the-engine)
27 and [Engine operations](https://alchemy.run/git/blocks/engine).
28
29 ## Historical proposal
30
31 The original proposal below is retained as design history. Its `Git.Server`,
32 `Git.ServerLive`, and Git-specific authentication services were removed; those
33 examples are superseded by the composition above. The storage design discussion
34 remains useful, but the historical API names are not current exports.
35
36 **On an AWS-native assembly (the DynamoDB question):** a `RefStore` on
37 DynamoDB pay-per-request is *conditionally* feasible — conditional writes
38 give ref CAS directly, and `TransactWriteItems` (100-item atomic cap)
39 covers `commitPush` **only if** staged-object visibility moves from
40 per-row flips to a push-epoch marker (commit = one item flip; membership
41 checks join against committed pushes). That redesign is the honest
42 price. But a RefStore alone is not AWS parity: the protocol engine,
43 ObjectStore index, commit-graph walks, alarms (compaction/bundles/GC),
44 and the push serialization point all live in the DO today — an AWS
45 assembly means Lambda-hosted planes, DynamoDB-backed stores with
46 optimistic transactions replacing `transactionSync`, and EventBridge/SQS
47 replacing alarms. Feasible, PPR-friendly on cost, and exactly what the
48 storage-agnostic contracts of §3.3 point at — but it is a Phase-D-scale
49 port, not a store swap. Until then `BlobStoreS3` serves the real hybrid:
50 Cloudflare compute, S3 bytes.
51
52 Re-thinks distribution of `@alchemy.run/git`. Today users import a finished
53 appliance:
54
55 ```ts
56 const git = yield* GitService(); // one worker, one bucket, our routes, our auth
57 ```
58
59 The proposal: distribute **git building blocks** — Effect `Context.Service`
60 contracts with swappable `Layer` implementations — that users assemble into
61 *their own* Worker. The package ships **no Worker and no `GitService()`**:
62 the top-level block is `Git.Server`, a `Context.Service` exposing `fetch`.
63 Users construct the `Cloudflare.Worker` themselves, build one layer graph,
64 and wire `fetch` in.
65
66 ```ts
67 import * as Git from "@alchemy.run/git";
68 import * as BetterAuth from "@alchemy.run/better-auth";
69
70 // One layer graph, one Effect.provide. Every block is swappable.
71 const GitLive = Git.ServerLive.pipe(
72 Layer.provide(Git.ReposDurableObject), // refs + objects in Durable Objects
73 Layer.provide(Git.RegistryDurableObject),
74 Layer.provide(Git.PullStoreDurableObject), // pull requests
75 Layer.provide(Git.BlobStoreR2(Bucket)), // packs & clone bundles in R2
76 Layer.provide(
77 Git.AuthBetterAuth({
78 // your rules, in git's vocabulary — this line is branch protection
79 authorize: ({ user, repo, action }) =>
80 action._tag === "Fetch"
81 ? repo.public || user != null
82 : action._tag === "Push"
83 ? action.updates.every((u) =>
84 u.ref === "refs/heads/main"
85 ? user?.id === repo.owner
86 : user != null,
87 )
88 : user?.id === repo.owner,
89 }),
90 ),
91 Layer.provide(BetterAuth.CloudflareD1(Database)),
92 );
93
94 export default class GitHost extends Cloudflare.Worker<GitHost>()(
95 "git",
96 { main: import.meta.url },
97 Effect.gen(function* () {
98 const git = yield* Git.Server;
99 return { fetch: git.fetch };
100 }).pipe(Effect.provide(GitLive)),
101 ) {}
102 ```
103
104 ---
105
106 ## 1. Why
107
108 1. **Swappability.** R2 is an implementation detail that currently leaks
109 through six modules (`ObjectStore` exposes the raw
110 `ReadWriteBucketClient`; `PackSource`, `IncomingBody`, `Compact`,
111 `Bundle`, `Purge` all import the R2 client type; the Worker binds the
112 bucket directly for the bundle splice). Nothing about pack storage is
113 R2-specific — it needs ranged reads, streaming multipart writes, and
114 delete. S3 satisfies that through alchemy's *existing* AWS S3
115 capabilities.
116 2. **Partial adoption.** Some users want only the wire plane behind their
117 own auth; some want the REST plane inside an existing HttpApi; some want
118 the GitHub facade and nothing else. Today it's all-or-nothing.
119 3. **Auth is the sharpest customization demand.** DESIGN §8 already calls
120 the `Credentials` service "the v2 identity seam". Making `Auth` a
121 first-class swappable service is how OAuth/JWT/SSO happens without
122 forking the package.
123 4. It's the alchemy thesis applied to ourselves: infrastructure as
124 composable, typed Effect programs — not appliances.
125
126 ## 2. One layer graph, two runtime contexts
127
128 A git repo needs one serialization point: refs CAS, push staging, pull
129 numbering. On Cloudflare that is a Durable Object, and code that needs
130 `transactionSync` must run *inside* it. But — and this is the load-bearing
131 simplification — that is a **runtime** fact, not an **authorship** fact.
132 The DO classes live in the same script as the Worker, so the user authors
133 ONE layer graph; alchemy's DurableObject machinery captures the composed
134 layers and constructs the DO-internal sub-graph per instance on first
135 event (exactly how `GitRepoLive.pipe(Layer.provide(R2Binding))` works
136 today).
137
138 ```
139 ONE authored layer graph (the user's GitLive)
140 │
141 ┌─────────────────────┴──────────────────────┐
142 Worker runtime Repo DO runtime
143 Wire/Rest/GitHubCompat routes protocol engine
144 Registry resolution · Auth ObjectStore · RefStore
145 BlobStore (bundle splice reads) PullStore · TokenStore
146 jobs · BlobStore (packs/spill)
147 ```
148
149 Consequences:
150
151 - Providing `Git.BlobStoreS3(bucket)` **once** satisfies every consumer on
152 both sides. There is no per-DO `layer:` option, no DO subclassing, and
153 no manual class re-export: providing `Git.ReposDurableObject` /
154 `Git.RegistryDurableObject` pulls the DO classes into the bundle through
155 the layer graph, and alchemy's generated worker entry exports them for
156 workerd (verify during Phase C that the generated-entry path covers
157 package-shipped DO classes; if not, that is a Phase C work item in
158 alchemy core, not user ceremony).
159 - The two runtime contexts still construct *separate instances* of a
160 layer's service (a Worker-side BlobStore client and a DO-side one).
161 That is invisible to the author but matters for layer implementors:
162 construction must be cheap and context-free (the isolate-scope rules
163 from AGENTS.md apply on both sides).
164 - Swappable-per-repo storage is out of scope: one graph means one
165 configuration per deployment, which is the sane default anyway.
166
167 ## 3. Service taxonomy
168
169 **Naming convention: one flat `Git` namespace, no nesting.** Contracts are
170 bare names (`Git.BlobStore`, `Git.Auth`, `Git.Server`);
171 implementations are `{Contract}{Impl}` (`Git.BlobStoreR2`,
172 `Git.BlobStoreS3`, `Git.AuthTokens`, `Git.ReposDurableObject`,
173 `Git.RegistryDurableObject`, `Git.PullStoreDurableObject`,
174 `Git.ServerLive` — all plain layers in one graph). Mirrors the package-level
175 flat-export convention (`ReadBucketBinding`, `GitAuthLive`) rather than
176 the `Cloudflare.KV.*` sub-namespacing.
177
178 ### 3.1 `BlobStore` — the R2 ↔ S3 seam (the headline)
179
180 Immutable, content-addressed bulk bytes. Everything the pack plane needs
181 and nothing more:
182
183 ```ts
184 export class BlobStore extends Context.Service<BlobStore, {
185 /** Ranged read; `undefined` = whole object. `null` = missing. */
186 readonly get: (
187 key: string,
188 range?: { readonly offset: number; readonly length: number },
189 ) => Effect.Effect<BlobBody | null, BlobStoreError, RuntimeContext>;
190 /** Whole-object write with known length (streams allowed). */
191 readonly put: (
192 key: string,
193 body: Uint8Array | Stream.Stream<Uint8Array, BlobStoreError>,
194 options: { readonly contentLength: number },
195 ) => Effect.Effect<void, BlobStoreError, RuntimeContext>;
196 /** Streaming write of unknown length (push-body spill). */
197 readonly multipart: (
198 key: string,
199 ) => Effect.Effect<BlobMultipart, BlobStoreError, RuntimeContext>;
200 readonly delete: (
201 keys: string | ReadonlyArray<string>,
202 ) => Effect.Effect<void, BlobStoreError, RuntimeContext>;
203 /** Prefix listing — purge/GC only, never on a serving path. */
204 readonly list: (
205 prefix: string,
206 ) => Stream.Stream<BlobMeta, BlobStoreError, RuntimeContext>;
207 }>()("alchemy/Git/BlobStore") {}
208
209 export interface BlobBody {
210 readonly size: number;
211 readonly bytes: Effect.Effect<Uint8Array, BlobStoreError>;
212 readonly stream: Stream.Stream<Uint8Array, BlobStoreError>;
213 }
214
215 export interface BlobMultipart {
216 readonly uploadPart: (n: number, part: Uint8Array) =>
217 Effect.Effect<void, BlobStoreError>;
218 readonly complete: Effect.Effect<void, BlobStoreError>;
219 readonly abort: Effect.Effect<void, BlobStoreError>;
220 }
221 ```
222
223 Implementations shipped:
224
225 | Layer | Rides on | Notes |
226 |---|---|---|
227 | `Git.BlobStoreR2(bucket)` | `Cloudflare.R2.ReadWriteBucket` | today's behavior, extracted |
228 | `Git.BlobStoreS3(bucket)` | `AWS.S3` capabilities (`GetObject`/`PutObject`/multipart) | already exist as alchemy bindings |
229 | `Git.BlobStoreMemory()` | Map | tests; kills the R2-shaped mocks |
230
231 Uniform-part-size and part-count limits differ per store (R2: uniform
232 parts, 10k max; S3: 5 MiB min, 10k max) — the contract mandates the
233 *intersection* (uniform parts ≥ 5 MiB except last), which `IncomingBody`
234 already satisfies.
235
236 Consumers refactored onto it: `ObjectStore` (stops leaking `bucket`),
237 `PackSource` (`r2RandomAccess` → `blobRandomAccess`), `IncomingBody`,
238 `Jobs/Compact`, `Jobs/Bundle`, `Jobs/Purge`, the Worker's bundle splice.
239
240 ### 3.2 `Auth` — authentication + domain-specific authorization
241
242 **Why not "AuthPolicy" or "AuthStore"?** The contract *decides* and stores
243 nothing: token secrets live behind `TokenStore`, sessions live in the auth
244 system's own database.
245
246 **The contract is domain-specific — no imposed role ladder.** An earlier
247 draft had users return `"read" | "write" | "admin"` scopes; that imposes
248 *our default token implementation's* vocabulary on everyone. Instead, the
249 engine asks questions in the domain's own terms and the user answers
250 yes/no. The engine is the only party that knows what is actually being
251 attempted — actions are protocol facts (a receive-pack's parsed ref
252 commands, a merge's PR number), which is also why auth cannot live in a
253 fronting middleware:
254
255 ```ts
256 /** What the caller is attempting — the engine's vocabulary, not ours. */
257 export type GitAction =
258 | { readonly _tag: "Fetch" } // clone/fetch, REST reads
259 | { readonly _tag: "Push"; // receive-pack, ref writes
260 readonly updates: ReadonlyArray<RefUpdate> } // (per-branch rules!)
261 | { readonly _tag: "CreatePull"; readonly base: string; readonly head: string }
262 | { readonly _tag: "UpdatePull"; readonly number: number }
263 | { readonly _tag: "MergePull"; readonly number: number }
264 | { readonly _tag: "ReadRepo" }
265 | { readonly _tag: "UpdateRepo" } // visibility, default branch
266 | { readonly _tag: "DeleteRepo" }
267 | { readonly _tag: "CreateRepo"; readonly owner: string }
268 | { readonly _tag: "ManageTokens" };
269
270 export class Auth extends Context.Service<Auth, {
271 /** AUTHENTICATION: who is calling? (never fails closed on anonymous) */
272 readonly authenticate: (
273 headers: Readonly<Record<string, string | undefined>>,
274 ) => Effect.Effect<Actor, GitAuthError, RuntimeContext>;
275 /** AUTHORIZATION: may this actor perform this action on this repo? */
276 readonly authorize: (input: {
277 readonly actor: Actor;
278 /** Repo state the engine supplies (visibility, owner) — `null` for
279 * repo-less actions like CreateRepo. */
280 readonly repo: RepoContext | null;
281 readonly action: GitAction;
282 }) => Effect.Effect<boolean, GitAuthError, RuntimeContext>;
283 }>()("alchemy/Git/Auth") {}
284 ```
285
286 Users express *their* model in *their* vocabulary — org roles, per-branch
287 protection, PR-only workflows — because the action carries the facts:
288
289 ```ts
290 const MyAuth = Git.AuthBetterAuth({
291 authorize: ({ user, repo, action }) => {
292 switch (action._tag) {
293 case "Fetch":
294 case "ReadRepo":
295 return repo?.public || user != null;
296 case "Push":
297 // protected main: only the repo owner may update it directly
298 return action.updates.every((u) =>
299 u.ref === "refs/heads/main" ? user?.id === repo.owner : user != null,
300 );
301 case "MergePull":
302 return user?.id === repo.owner;
303 default:
304 return user?.id === repo.owner;
305 }
306 },
307 });
308 ```
309
310 **Worked backwards from `packages/better-auth`** (the exercise that shaped
311 this): issuance and auth UX stay *outside* the blocks — the user mounts
312 `auth.fetch` at `/api/auth/*` next to `git.fetch` in their worker. Per-
313 request decisions happen *inside* via this contract: the adapter
314 authenticates browser sessions with `auth.getSession(headers)` and git-CLI
315 Basic-auth PATs with the Better Auth API-key plugin (git cannot do cookies
316 or redirects — the password field carries the key), then runs the user's
317 `authorize` with the resolved user.
318
319 **Where the old scope ladder went**: inside `Git.AuthTokens`. The default
320 implementation stores `read|write|admin` on its tokens and implements
321 `authorize` by mapping each `GitAction` to a required scope internally —
322 its business, invisible to the contract. Public-repo anonymous reads are
323 likewise just the default `authorize` behavior (`Fetch`/`ReadRepo` on
324 `repo.public`), no longer a special engine rule; `readOnly` write
325 rejection stays in the DO because it is repo state independent of the
326 caller.
327
328 **Enforcement across the runtime boundary** (as implemented): the
329 Worker runs `authenticate` once per request and forwards the resulting
330 `Actor` to the DO over the trusted internal channel — the DO trusts
331 identity, never re-derives it, and the wire path keeps the
332 `ADMIN_HEADER` discipline (worker strips inbound forgeries, mints the
333 header only after its own timing-safe verify; pinned by a forged-header
334 401 test). `authorize` runs at the site that owns the action's facts:
335 the Worker for registry-level actions (`CreateRepo`, `ListRepos`, with
336 `repo: null`), the Repo DO for everything per-repo — including
337 `git-receive-pack`, which authorizes twice: once at entry with
338 `Push { updates: [] }` ("may this caller push at all?", cheap rejection
339 before the pack streams) and again after command parsing with the real
340 `updates`, which is where per-branch policies bite. The DO enriches
341 `token` actors from its tokens table before asking (identity is the
342 engine's job; the scope's *meaning* is `AuthTokens`'s). Both runtime
343 contexts resolve the same `Auth` layer from the one graph (§2), so
344 `authorize` must be a fast, pure decision — it sits on the DO hot path.
345
346 **Why have `Auth` at all, instead of "write your own auth over the top"?**
347 A fronting middleware can only allow or deny whole requests. It cannot see
348 the actions — which ref a push updates, which PR a merge targets — without
349 reimplementing the protocol, and it cannot stamp identity into domain
350 objects without smuggling it through headers, which is an auth seam
351 anyway, just untyped. `Auth` IS auth-over-the-top in its minimal typed
352 form: two functions. Implementing it from scratch is a page of code;
353 `Git.AuthTokens` makes fresh deploys secure by default;
354 `Git.AuthBetterAuth` is optional sugar. Coarse gating (rate limits, IP
355 allowlists) still composes by wrapping `git.fetch`.
356
357 ### 3.3 Storage services — defined by domain semantics, not SQL
358
359 The repo's storage services are **storage-agnostic domain contracts**.
360 There is no shared `SqlClient`/`RepoSql` service and no SQL anywhere in a
361 contract — SQLite-in-the-DO is an implementation detail of the *default
362 layers*, invisible to consumers and to alternative implementations.
363
364 - **`ObjectStore`** — content-addressed object index + hot bytes:
365 `getMeta(Batch)` / `readContent` / `insertStagedBatch` /
366 `missingObjects` / `promoteStaged` / compaction bookkeeping. (Bulk cold
367 bytes already live behind `BlobStore`.)
368 - **`RefStore`** — ref reads, HEAD, and the repo's **atomic commit
369 points** expressed as domain operations, not exposed transactions:
370
371 ```ts
372 /** Atomically: verify CAS expectations, flip staged objects live,
373 * apply ref updates, run registered stamps (e.g. mark a pull
374 * merged). Implementations MUST make this a single atomic unit. */
375 readonly commitPush: (input: CommitPushInput) =>
376 Effect.Effect<CommitPushResult, RefConflict | ..., RuntimeContext>;
377 ```
378
379 Atomicity is a *documented obligation of the contract* — the default
380 layer discharges it with the DO's `transactionSync`; a hypothetical
381 FoundationDB/Postgres layer would use its own transactions. Cross-
382 service atomicity (refs + staged objects + pull stamping) is handled by
383 making the composite operation itself the contract surface — never by
384 leaking a transaction handle across services.
385 - **`PullStore`** — pull rows + per-repo numbering (`create` allocates
386 the next number atomically; how is the layer's business).
387 - **`TokenStore`** — mint/verify/revoke (storage behind the default
388 `Git.AuthTokens`).
389
390 Default layers ship as flat `Git.{Service}DurableObject` implementations,
391 all provided in the user's single layer graph. The near-term win is honesty and
392 testability (contracts mockable without SQL fixtures; each default layer
393 unit-tested in isolation); the long-term win is that "swap the metadata
394 store" is now a layer, not a fork.
395
396 ### 3.4 Worker-site services
397
398 - **`Registry`** — `owner/name → repoId` resolution + listing (contract
399 over the Registry DO; a single-tenant "static registry" layer becomes
400 possible for users who want exactly one repo and no registry DO).
401 - **HTTP groups** — `Git.Protocol`, `Git.Repos`, `Git.Refs`, `Git.Objects`,
402 `Git.Pulls`, and `Git.GitHub`. Each is independently mountable in an
403 Effect `HttpApi`. `Git.Handlers` exposes the corresponding handler
404 objects; `Git.ApiHandlersLive` builds them over `RegistryStore`,
405 `RepoStore`, `BlobStore`, and `Hasher`.
406
407 ## 4. `Git.Server` — the top-level block
408
409 There is no `GitService()` and no shipped Worker. The package's largest
410 unit is a service:
411
412 ```ts
413 // Open default: all six groups, with the shared handler implementation.
414 const GitLive = Git.ServerLive.pipe(
415 Layer.provide(Git.ReposDurableObject),
416 Layer.provide(Git.RegistryDurableObject),
417 Layer.provide(Git.HasherInline),
418 Layer.provide(Git.BlobStoreR2(GitObjects)),
419 );
420
421 // A custom management API: select a group, attach middleware, and implement it.
422 class ManagementApi extends HttpApi.make("git-management")
423 .add(Git.Repos)
424 .middleware(Authenticated) {}
425
426 const ManagementLive = HttpApiBuilder.group(ManagementApi, "repos", (h) =>
427 Effect.map(Git.Handlers, (git) => h.handleAll(git.repos)),
428 );
429
430 const ManagementServer = Git.Server.layer(ManagementApi, ManagementLive).pipe(
431 Layer.provide(Git.ApiHandlersLive),
432 Layer.provide(AuthenticatedLive),
433 // provide the storage and hasher layers as above
434 );
435 ```
436
437 `Git.Server` exposes `fetch`, so users can mount it whole, nest it under
438 another router, or register the exported API groups directly.
439 `Server.layer` mounts `Git.InternalApi` separately from application
440 middleware. A host that builds its own router can pair `InternalApi`
441 with `Git.InternalLive`. Users own the `Cloudflare.Worker`, its bindings,
442 domains, assets, and Durable Object exports.
443
444 Consequences accepted: this is a **breaking change** — `GitService()` and
445 the internal `GitWorker` class are deleted, and the example app becomes
446 the reference assembly (its own Worker + `Git.ServerLive` + default
447 layers + its one-origin SPA front door).
448
449 ## 5. Migration phases
450
451 Each phase lands green on the full suite before the next starts.
452
453 - **Phase A — `BlobStore`.** Define the contract + `R2`/`Memory` layers;
454 refactor the six consumers and the Worker splice onto it; delete the
455 `bucket` leak from `ObjectStore`. Pure refactor, no behavior change.
456 Then add `Git.BlobStoreS3` + a live S3-flavored test (deploys the DO with
457 the S3 layer against a real bucket).
458 - **Phase B — `Auth` + DO-site services.** Extract `RefStore` /
459 `PullStore` / `TokenStore` / `ObjectStore`-as-service; `RepoObject`
460 becomes choreography. `Auth` replaces the scattered
461 `parseBasicOrBearer`/`verifyAdminKey` call sites.
462 - **Phase C — plane split + `Git.Server`.** Native HTTP groups exported
463 individually; `Git.Server` service + `Git.ServerLive`/`Git.Server.layer(api, groups)`;
464 DO classes re-exported by users (no subclass factory); **`GitService()`
465 and the shipped `GitWorker` deleted**; the example app rewritten as the reference
466 assembly; tests re-pointed at a fixture assembly; docs (`@layer` pages
467 per block, assembly guide). All documentation and examples compose with
468 `Layer.mergeAll`/`Layer.provide` into a single layer handed to one
469 `Effect.provide` — chained `Effect.provide` calls are a lint-level
470 error in this package.
471
472 ## 6. AWS-native assembly (no Durable Objects) — parity analysis
473
474 > **Status: DEFERRED.** Lambda cannot accept streaming/large request
475 > bodies (6 MB sync-invoke cap, base64 overhead → ~4.5 MB effective), so
476 > there is no serverless answer for `receive-pack` on AWS today. The
477 > practical AWS path is Durable Objects hosted on ECS (Atypus), which is
478 > separate ongoing work. This section is kept as the design record; the
479 > project focus is Cloudflare-native assemblies.
480
481 What the Repo DO actually provides, and the AWS-native replacement for
482 each. The pure `Protocol/` codec layer, `BlobStore` (S3 layer shipped), and
483 the HTTP planes (Effect `HttpApi`/`HttpRouter` — portable to alchemy's
484 Lambda hosts) need **no** work. The rest:
485
486 | DO capability | AWS-native replacement | difficulty |
487 |---|---|---|
488 | refs CAS | DynamoDB conditional writes | easy |
489 | `commitPush` atomicity (refs + staged flip + pull stamp) | `TransactWriteItems` (100-item cap) — **requires the push-epoch redesign** | medium |
490 | push semaphore | mostly unnecessary: staging is content-addressed and idempotent; losers fail at the commit CAS. Optional lease lock for resource hygiene | easy |
491 | object index + hot bytes (SQLite rows) | metadata items in DynamoDB (`BatchGetItem` ×100 for `missingObjects`/`getMetaBatch`); bytes in S3 from day one (DynamoDB's 400 KB item cap rules out the 1 MiB inline tier) | medium |
492 | commit graph + merge-base walks (10k sub-ms SELECTs) | batched-frontier BFS over `BatchGetItem`; acceptable for PR-sized divergence, painful for deep walks — **reachability bitmaps get promoted from optimization to prerequisite** | hard |
493 | wire-plane hosting (streaming HTTP both directions) | **the real blocker, and it is not DynamoDB**: Function URLs stream *responses* (up to 200 MB, bandwidth-capped past 6 MB) but **not requests** — the request body is buffered into the invocation event under the 6 MB sync-invoke cap (binary is base64-encoded into that JSON, so ~4.5 MB effective), which breaks any non-trivial `git push`. Options: (a) Fargate/App Runner container for the wire plane (no caps, long-lived), (b) a two-phase push (presign → S3 → commit) which stock git clients cannot speak, or (c) accept a push cap. Clones are fine — response streaming covers upload-pack, and presigned S3 bundle URLs make the serving plane *better* than the DO splice (zero compute) | hard |
494 | alarms (compact / bundle / gc / purge) | SQS delayed messages or EventBridge Scheduler one-shots enqueued at the commit point | easy |
495 | Registry DO | DynamoDB table, conditional create | easy |
496 | tokens / pulls tables | DynamoDB tables (PPR) | easy |
497
498 **Single-table design is mandatory.** The DynamoDB stores above are one
499 table, not five: everything keys as `PK = REPO#<repoId>` with typed sort
500 keys (`REF#refs/heads/main`, `OBJ#<oid>`, `PUSH#<pushId>`,
501 `PULL#<n>`, `META`), plus `OWNER#<owner>` / `NAME#<name>` items for the
502 registry and `TOKEN#<hash>` for tokens. This is not a style preference:
503 `TransactWriteItems` spans items freely but the whole `commitPush`
504 transaction (ref CAS + push flip + pull stamp) staying in one table keeps
505 capacity, backup, and IAM surface singular, and the per-repo partition
506 key gives every store the same locality the DO gave for free. The
507 storage-agnostic contracts (§3.3) never see any of this — key schema is
508 an implementation detail of the `*Dynamo` layers, exactly as SQL never
509 appears in the contracts today.
510
511 **The push-epoch redesign** (the load-bearing prerequisite): today a
512 commit flips thousands of staged object rows live inside one SQLite
513 transaction — impossible under a 100-item transaction cap. Instead,
514 objects reference their `pushId` permanently and visibility is a single
515 `pushes` item flipping to `committed`; membership checks join against
516 committed pushes. One item flip = fits any transaction budget. Notably
517 this would *also* speed up the DO implementation (no bulk row flip), so
518 it is worth doing in Phase B regardless of AWS plans.
519
520 **Sequencing if AWS parity becomes real:** Phase B store extraction
521 (the prerequisite for everything — the contracts must exist before
522 Dynamo layers can implement them) → push-epoch redesign (benefits both
523 backends) → `RefStore`/`ObjectStore`/`PullStore`/`TokenStore`/`Registry`
524 Dynamo layers → Lambda-hosted REST + GitHub-compat planes (no size
525 issues, lands early) → reachability bitmaps → the wire-plane hosting
526 decision (Fargate container vs. capped Lambda) last, because it is the
527 only piece with no clean serverless answer.
528
529 Cost shape under pay-per-request: a 44k-object push ≈ 44k WCUs ≈ $0.05;
530 metadata reads are cheap; the closure-walk read amplification is the
531 number to watch, and bitmaps are what contain it.
532
533 ## 7. Open questions (want your calls)
534
535 1. **S3 priority**: is `Git.BlobStoreS3` a Phase-A deliverable (proves the
536 seam immediately) or is the seam + `Memory` enough until someone asks?
537 2. ~~Namespace~~ **Resolved**: one flat `Git` namespace —
538 `Git.BlobStoreS3`, `Git.ReposDurableObject`, `Git.ServerLive`; no
539 nested sub-namespaces.
540 3. **Registry optionality**: is the single-tenant/no-registry layer worth
541 designing for now, or YAGNI?
542 4. **How far to shrink `RepoObject`**: Phase B as scoped keeps the wire
543 protocol choreography inline. Extracting a `Protocol` service too is
544 possible but touches the most battle-tested code for the least
545 swappability value — I'd leave it.
546 5. ~~Does `Auth` absorb the identity feature?~~ **Resolved by the
547 better-auth exercise**: yes — `identify()` + the `grant` variant carry
548 identity, and PR authorship/merge attribution consume it in Phase B.