@devries-cameron20 / t3code Mirror · read-only
Fork 0

The best way to control Claude Code, Codex and more from any computer or phone

Mirrored from https://github.com/Camo67/t3code.git

Synced Nov 21, 07:15 UTC

main
md 175 lines 16.3 KB
Rendered Raw
1 # Provider review gate
2
3 Each row turns a rule from `AGENTS.md` or the skill into a check with a
4 command and an expected result. The rows come from the first review of the
5 GCP provider and the corrections made during it; on the pre-fix GCP code
6 these greps returned hundreds of hits each.
7
8 Run it at every wave boundary and before calling a provider done. Use a
9 fresh reviewer (a subagent with no stake in the code) and have it report,
10 for every item, the command it ran, the count, and the file:line of each
11 hit it did not accept. "Looks fine" is not an answer. Every accepted hit
12 needs a one-line reason.
13
14 `<P>` is the provider name. Paths are relative to the repo root:
15 `src/<P>` means `packages/<p>/src` (the provider package) and `test/<P>`
16 means `packages/integration-tests/test/<P>`. The greps find candidates;
17 read each hit.
18
19 ## 0. Coverage
20
21 Run this first; the other sections only judge the code that exists.
22
23 | Check | Command | Expect |
24 | --- | --- | --- |
25 | Every service cataloged | compare `ls submodules/distilled/packages/<pkg>/src/services` with `ls processes/<P>/catalog` | one catalog file per service |
26 | Nothing left unbuilt | count statuses in `processes/<P>/catalog/INDEX.md` | no `missing` or `implemented` rows; every `out-of-scope` and `blocked` row has a reason or exact error |
27 | Index matches the code | every `tested` row has a `Resource<...>("<P>.<Service>.<Name>")` in `src/<P>` and a registered provider in `Providers.ts` | yes |
28 | One test file per resource | every `tested` row has `test/<P>/<Service>/<Name>.test.ts` | yes |
29 | Operations classified | every operation in each distilled service module appears in its catalog file as `resource-lifecycle`, `binding`, `event-source`, or `internal` with a reason | yes; the provider's port of `scripts/audit-service.ts` prints no unclassified operations |
30 | Binding coverage | the audit's `missingBindings` per service, and `rg -l 'Binding\.Service<' src/<P> \| xargs -n1 dirname \| sort -u \| wc -l` against the services with `binding` operations | no missing bindings; every service with data-plane operations has bindings (AWS: 205 of 212) |
31 | Bindings through distilled | read each `*Http.ts` | wraps a `@distilled.cloud/<pkg>` operation; no `HttpClientRequest` |
32 | Non-resource deliverables | index rows for runtime, event sources, websites, examples | none `missing` |
33 | Skill freshness | `git log -1 --format=%H -- .agents/skills/alchemy-provider` vs the `Skill: <sha>` line at the top of `INDEX.md` | equal; a newer skill commit means re-read it and add rows for its new deliverables |
34 | Framework examples | `ls examples \| grep '^<p>-website-'` vs `ls examples \| grep '^hetzner-website-'` | one per framework the provider hosts (skip when it can't host websites) |
35 | Workload examples | `ls examples \| grep '^<p>-'` against the workload table in Step 10 | one per workload the cloud supports; each registered in `tsconfig.json` and `scripts/test-examples.ts`, with a passing `test/integ.test.ts` |
36 | Docs generated for everything | `ls website/src/content/docs/providers/<P>` after `pnpm docs:gen` | one page per resource and binding page group, matching the index |
37 | Out-of-scope list published | the overview page's "Out of scope" section | matches the `out-of-scope` rows |
38
39 ## How to compare against what good looks like
40
41 For each service, open the closest AWS or Cloudflare equivalent and compare
42 side by side: props naming, `diff`, `reconcile`, binding shape, test shape,
43 JSDoc examples. Where the new provider differs, the difference needs a
44 reason the platform forces. References:
45
46 | Concern | Reference |
47 | --- | --- |
48 | Small reconcile with labels, `list`, adoption | `packages/hetzner/src/SshKey.ts` |
49 | Multi-aspect reconcile | `packages/aws/src/DynamoDB/Table.ts`, `packages/aws/src/Kinesis/Stream.ts` |
50 | Read/Write/ReadWrite bindings, shared scaffolding unexported | `packages/cloudflare/src/R2/` |
51 | Bindings that reach any host | `packages/aws/src/AWSBindings.ts`, `packages/gcp/src/GCPBindings.ts`, `packages/aws/src/SQS/BindingHttp.ts` |
52 | Package shape | `packages/hetzner/` (`package.json`, `tsconfig.json`, `src/Contributions.ts`) |
53 | Bindings tested inside a deployed host | `packages/integration-tests/test/AWS/DynamoDB/Bindings.test.ts` + `handler.ts` |
54 | Framework websites on a CDN + functions | `packages/aws/src/Website/` (S3 + CloudFront + Lambda), `packages/cloudflare/src/Website/` |
55 | Framework websites on containers/VMs (only when the cloud has no CDN or functions) | `packages/fly/src/Website/`, `packages/hetzner/src/Website/` |
56
57 ## 1. Package
58
59 | Check | Command | Expect |
60 | --- | --- | --- |
61 | Exports generated | `pnpm exports:check` | up to date; no hand-written entries |
62 | Provider layers | `pnpm lint:providers` | no `unknown`/`any` requirement on `providers()` |
63 | Core untouched | `git diff origin/main --stat -- packages/alchemy/src` | empty; anything core needs is a `Contributions` field |
64 | Package boundary | `rg -n "from \"(@alchemy\.run/[\w-]+\|alchemy)/.*internal" src/<P>` and `rg -n "\.\./\.\./[a-z-]+/src" src/<P>` | 0; other packages reached only through their public exports |
65 | Contributions | read `src/<P>/Contributions.ts` and `package.json` `alchemy.contributes` | lazy `import()` fields only; listed in `contributes` |
66 | Globals | `ls src/<P>/globals.d.ts` and `rg -n 'packages/<p>/src/globals' .gitignore` | both present |
67 | Registration | `git grep -il hetzner -- ':!packages/hetzner' ':!submodules' ':!examples'` vs the same for `<p>` | every file that lists Hetzner lists `<P>` where it applies |
68 | Version | `grep '"version"' packages/<p>/package.json packages/aws/package.json` | equal (packages release in lockstep) |
69
70 ## 2. Effect code
71
72 | Check | Command | Expect |
73 | --- | --- | --- |
74 | Nested yields (`f(yield* x)`) | `rg -n 'yield\* [\w.]+\([^)]*yield\*' src/<P> test/<P> examples/<p>-*` | 0; bind to a named const first |
75 | `orDie` in lifecycle or binding init | `rg -n 'orDie' src/<P>` | only in credential and layer wiring (`Credentials.ts`, `Providers.ts`) |
76 | `Effect.promise` / swallowed causes | `rg -n 'Effect\.promise\|catchCause\|catchAllCause' src/<P>` | 0; use `Effect.tryPromise` with a typed error |
77 | Environment reads in resource code | `rg -n 'process\.env' src/<P>` | only `AuthProvider.ts` CI credentials |
78 | Widening casts | `rg -n 'as unknown as\|as any' src/<P>` | 0, or a comment naming the compiler limit |
79
80 ## 3. Errors
81
82 | Check | Command | Expect |
83 | --- | --- | --- |
84 | Forbidden treated as missing | `rg -n '"Forbidden"' src/<P> test/<P>` | never in a not-found/gone path; a 403-for-missing API gets its own distilled tag |
85 | Message string matching | `rg -n 'message.*(includes\|match\|test)\(\|/[^/]*not found[^/]*/i' src/<P>` | 0; patch distilled with a matcher |
86 | Hand-written HTTP | `rg -n 'HttpClientRequest\.(get\|post\|put\|delete\|head\|patch)\(' src/<P>` | every hit is a presigned/SAS URL from the API or the user's own deployed endpoint, with a comment saying so; data-plane and credential calls go through distilled |
87 | Catch-all errors handled | `rg -n 'Unknown\w*Error\|HttpError' src/<P>` | 0 |
88 | Lists swallowing errors | `rg -n 'orElseSucceed\|catch\w*\(.*\[\]' src/<P>` | 0 on any list/read path |
89 | Single-page lists | read every `list` and `findBy*` helper | paginated (`.items(...)` or a page-token loop) |
90
91 ## 4. Lifecycle
92
93 | Check | How | Expect |
94 | --- | --- | --- |
95 | One operation waiter | `rg -c 'waitFor\w*Operation\|pollOperation' src/<P>` | one definition in a shared module, many call sites |
96 | Realistic wait budgets | read each budget against the provider's documented create time | minutes for databases, clusters, environments; never a test-style 10 × 5s |
97 | Ready before return | read each `reconcile` | returns after the API reports ready, not after the create call |
98 | SQL migrations | for each resource hosting a SQL schema, `rg -n 'runMigrations' src/<P>` | a `migrations` prop driven by `@alchemy.run/sql/Migrations` (see **SQL database migrations** in `AGENTS.md`), with a live test that adds a file and redeploys |
99 | Identity change replaces | for each user-chosen name/id prop, read `diff` | `replace`; a rename test proves the old one is deleted |
100 | Ownership markers | `rg -n 'alchemy' src/<P> \| rg -v 'label\|tag\|import'` | markers only in labels/tags, never in user-visible fields |
101 | Copied helpers | `rg -c 'const (isNotFound\|isMissing\|ignoreMissing\|jsonEqual)' src/<P>` | one shared copy each |
102 | `output === undefined` branches | `rg -n 'output === undefined' src/<P>` | none inside `reconcile` |
103
104 ## 5. Bindings and runtimes
105
106 | Check | How | Expect |
107 | --- | --- | --- |
108 | Bind, not host.bind | `rg -n 'host\.bind\(' src/<P>` and `rg -ln 'Bindings\.Bind' src/<P>` | bindings call `<P>Bindings.Bind`; no direct `host.bind` and no allowlist of host types |
109 | Foreign hosts | read `<P>Bindings.ts` | a non-native host gets a created identity + key bound into its env; `withCredentials` reads it at runtime |
110 | Other providers' types | `rg -n '"(AWS\|Cloudflare\|GCP\|Azure\|Fly\|Hetzner\|Railway)\.[A-Z]' src/<P>` minus `<P>`'s own prefix | 0; host access goes through `RuntimeContext` and core host services |
111 | Deploy-only reads | `rg -n 'Environment\.current' src/<P>` in binding files | every hit inside the `!globalThis.__ALCHEMY_RUNTIME__` block |
112 | Native bundles fold | `rg -ln 'HostBundle' src/<P>` | applied by every host bundler the provider owns |
113 | External-host suite | `ls test/<P>/ExternalHost` | a Cloudflare Worker (or other foreign host) calls the provider's bindings |
114 | Least privilege | list every role/permission each binding grants (`rg -n 'roles/\|permission' src/<P>`) | scoped to the bound resource; no `admin`/`editor`/`owner` unless the JSDoc says why |
115 | Revocation | read the host's grant sync | revokes only grants Alchemy added; pre-existing grants survive |
116 | Secrets | read each binding that hands out a credential | `Redacted`, never a plain env var or log line |
117 | Argument shapes | type test | callables accept the resource, the Effect producing it, and the resolved value (`WriteTable(table)` and `WriteTable(yield* table)`) |
118 | Runtimes identified | `rg -n 'runtime' processes/<P>/catalog/INDEX.md` vs every resource that runs user code | each one marked |
119 | Both forms | for each runtime: `Platform(` in its file, and an `### Async` JSDoc section | effectful class and async `main` form both ship and are documented |
120 | InferEnv | `rg -n 'export type InferEnv' src/<P>` | exported, and used in every async example |
121 | Both forms tested | `ls test/<P>/**/fixtures` | an effectful and an async fixture per runtime, each deployed and called |
122 | Workflow engine | the catalog's engine row, and `rg -ln 'WorkflowEngine' src/<P>` | the cloud's durable engine has an Effect engine Layer passing the eight conformance cases from `AGENTS.md`, or the row is `out-of-scope` with the reason |
123 | Runtime coloring | read each runtime callable | requires `RuntimeContext`; no `WorkerEnvironment`-style leaks |
124 | Event sources | compare with AWS `SQS.QueueEventSource` / Cloudflare cron | contract in the source service, per-host implementation layers |
125 | User-facing config | read setup docs and examples | region/account from profile or a `<P>.Region("...")` layer, matching `AWS.Region`; no new env vars |
126
127 ## 6. Websites
128
129 Skip this section when the provider can't host websites; then `src/<P>/Website` doesn't exist.
130
131 | Check | Command | Expect |
132 | --- | --- | --- |
133 | Hosting choice | the catalog's website entry | names the cloud's static hosting, CDN, and function runtime, with a reason for each |
134 | No containers by default | `rg -n 'ContainerApp\|Run\.Service\|ECS\|Dockerfile\|image' src/<P>/Website` | 0, or the framework's JSDoc says why it needs a long-lived process and the cloud has no alternative |
135 | Static is compute-free | read `StaticSite.ts` and every assets-only path (Vite, SPA output) | uploads to object/asset storage; no server, function, or VM |
136 | CDN by default | read `FrameworkSite.ts` | the CDN is created without opting in; deploys purge or invalidate changed paths |
137 | Function target | `ls packages/frontend-frameworks/src/*/<p>.ts` | one per framework when SSR runs on functions; the Node target only on container/VM-only clouds |
138 | Live proof | each `Website/<Framework>.test.ts` | asserts the response came through the CDN and an SSR route ran on the function |
139
140 ## 7. Tests
141
142 | Check | Command | Expect |
143 | --- | --- | --- |
144 | Gating flags | `rg -oh 'process\.env\.\w+' test/<P> \| sort \| uniq -c` | one provider-wide slow flag, entitlement flags, and ids of external systems a test needs; never sizes, regions, or other config as env vars |
145 | Lifecycle by default | `rg -c 'skipIf' test/<P>` against the file count | most lifecycles ungated |
146 | Accept-anything assertions | `rg -n 'expect\(\[' test/<P>` | each entitlement probe asserts one specific tag |
147 | Always-true assertions | `rg -n 'Array\.isArray\(.*\?\? \[\]\)\|toBeDefined\(\)' test/<P>` | each one asserts a real value |
148 | Early exits | `rg -n '^\s+return;$' test/<P>` | 0 |
149 | Bindings as the host | every `Bindings.test.ts` | deploys a fixture host and calls the binding over HTTP; at least one suite runs the binding on a foreign host (e.g. a Cloudflare Worker) |
150 | Redeploy coverage | tests for anything that syncs grants or bindings on the host | deploy twice and assert the grant survives |
151 | Leaks | after a full run, list owned resources in the test account | none |
152 | Slow test cases | the runner's `✓`/`✗` result lines, which end with each test's duration (`pnpm test:integration test/<P> ... \| rg '[✓✗]'`) | no failure-path case runs longer than a few minutes |
153 | Blocked rows | `rg -n 'blocked' processes/<P>/catalog/INDEX.md` | each needs payment, vendor approval, or hardware and names the exact request; quotas and preview features were requested through the API |
154 | Suite shape | how the suite was run | one process, whole provider directory; quotas shared with semaphores, never batched runs |
155
156 ## 8. Docs, naming, and scope
157
158 | Check | Command | Expect |
159 | --- | --- | --- |
160 | Undeclared names in examples | `rg -n 'existing\.' src/<P>` and read every `**Example:**` | every identifier is declared in the example |
161 | Binding examples | count `@binding` blocks vs those showing the host and `Effect.provide(...)` | all of them |
162 | Resource examples | every `@resource` has at least one example | yes |
163 | Runtime JSDoc | `rg -c '\*\*Example:\*\*' <runtime file>` and its `###` headings next to `packages/aws/src/Lambda/Function.ts` | ten or more examples; intro, cautions, each form, configuration, packaging, a bindings section per service, event sources |
164 | Fat-head JSDoc | the same for each fat-head resource next to `packages/aws/src/S3/Bucket.ts` / `packages/cloudflare/src/R2/Bucket.ts` | five or more sections: creating, each configurable feature, binding to a runtime, runtime operations, event sources where they apply |
165 | Long-tail JSDoc | sample 20 long-tail files | intro with the cloud's type name, one complete creation example, a caution per replacing prop |
166 | Prop JSDoc | `rg -n '^ [a-zA-Z]+\??:' src/<P>/<Service>/<Resource>.ts` minus documented ones | every prop and attribute documented, with `@default` where there is one |
167 | Namespace casing | `ls src/<P>` | PascalCase |
168 | Singular names | read the list of exported resource names | correct English (`Index`, `Process`, `Batch`) |
169 | Scope | read the service list | no end-user data, OAuth-scoped, retired, or duplicate APIs; out-of-scope list on the overview page |
170 | PR description | read it | a `###` section with a snippet for every runtime, event source, websites, and each fat-head resource; long tail in one line; no snippet claims behavior no test proves |
171 | Docs build | `pnpm docs:check-jsdoc && pnpm docs:gen && pnpm docs:check` | clean |
172 | Hub structure | `ls website/src/content/docs/<p>/*/` next to `ls website/src/content/docs/aws/*/` | Overview, Setup, `tutorial/part-1..5`, `compute/choosing-a-runtime`, sections by concern; no `guides/`; every AWS section has a counterpart unless the provider has nothing for it |
173 | Hub snippets | read every fenced block with `import` lines | a complete file with a `// path` first line; no `...`, no module-scope `yield*`; code matches an `examples/<p>-*` project or a JSDoc example |
174 | Hub voice | `rg -n 'Alchemy (mints\|synthesizes\|reconciles)\|this page\|three ideas\|under the hood' website/src/content/docs/<p>` | 0; second person, one or two lines per snippet |
175 | Hub wiring | sidebar in `website/astro.config.ts`, `GROUP_ICONS`, `docs-tabs.ts`, `ProviderDirectory.astro`, `generate-llms-txt.ts` | every section listed in each; moved pages redirected in `website/src/worker.ts` |