The best way to control Claude Code, Codex and more from any computer or phone
| 1 | --- |
| 2 | name: alchemy-provider |
| 3 | description: Bring up a new Alchemy cloud provider (its own `@alchemy.run/<provider>` package under packages/<provider>) end to end — credentials, a distilled SDK, resources, bindings, live tests, and the patch loop that feeds live-test failures back into distilled — or extend an existing provider with a new service, or bring an existing provider up to the bar (gap audit, then waves). Use for "add a <X> provider", "create an alchemy provider for <X>", "port <X> to alchemy", "add <service> to the <X> provider", "bring the <X> provider up to par", or any work that needs a distilled SDK plus Alchemy resources tested against a live API. |
| 4 | --- |
| 5 | |
| 6 | # Building an Alchemy provider |
| 7 | |
| 8 | A provider is built on a flywheel. Each turn of it makes both the SDK and |
| 9 | the provider more correct: |
| 10 | |
| 11 | ``` |
| 12 | 1. distilled SDK ──> 2. alchemy resources ──> 3. live tests |
| 13 | ^ │ |
| 14 | └──── 4. distilled patches <── wrong error / wrong schema |
| 15 | ``` |
| 16 | |
| 17 | The live API is the only authority. A resource is done when its test |
| 18 | passes against the real provider, and every mismatch that test surfaced is |
| 19 | fixed in distilled rather than worked around in alchemy. |
| 20 | |
| 21 | **The target is the whole provider.** "Create a provider for X" means every |
| 22 | resource in every service the distilled SDK covers is either implemented |
| 23 | and live-tested, or listed as out of scope with a reason from step 2. A |
| 24 | handful of foundation resources is the first wave, never the deliverable. |
| 25 | Cost is not a scoping reason: expensive resources are implemented like any |
| 26 | other (see step 6 for running their tests). The user narrows scope; the |
| 27 | agent does not. |
| 28 | |
| 29 | **Coverage is 1:1 with the API, operation by operation.** Every |
| 30 | operation in every in-scope service ends up as exactly one of: part of a |
| 31 | resource's lifecycle, a binding, an event source, or an `internal` row |
| 32 | with a reason. A service with resources but no bindings is half done. |
| 33 | AWS is the bar: 2,846 binding services across 205 of its 212 services. |
| 34 | GCP reached 57 of 161 services and Azure 4 of 160; both were reported as |
| 35 | done. Every call goes through distilled. When distilled is missing an |
| 36 | operation, patch distilled (step 1); never work around it in alchemy. |
| 37 | |
| 38 | The whole provider also means its non-resource deliverables: a binding |
| 39 | for every data-plane operation (step 5), every runtime in both forms, event |
| 40 | sources, and a workflow engine when the cloud runs code, framework websites on the cloud's CDN and functions (step 7, when the |
| 41 | cloud hosts websites), examples, the docs hub, and generated docs |
| 42 | (step 10). They are rows in the same index as resources, with the same |
| 43 | statuses. Never write "a later phase" into an agent contract without |
| 44 | adding those rows: on Azure, the contract said "bindings are a later |
| 45 | phase — do not build them", the phase never came, and 1,193 resources |
| 46 | shipped with zero bindings, no runtime, no examples, and docs for 4 |
| 47 | services. |
| 48 | |
| 49 | `AGENTS.md` owns the doctrines this skill relies on; read these sections |
| 50 | before writing code and follow them as written: |
| 51 | |
| 52 | - **Package boundaries** — every provider is its own workspace package; |
| 53 | core never imports it, and it reaches core only through `alchemy/*`. |
| 54 | - **Reconciler doctrine** — the observe → ensure → sync shape of `reconcile`. |
| 55 | - **Typed Error Doctrine (distilled)** — every error alchemy handles is a |
| 56 | typed tag in the SDK. |
| 57 | - **Speed doctrine: never wait on a hang** — bounded retries, `timeout` on |
| 58 | every test run, the three-iteration budget. |
| 59 | - **Read/Write/ReadWrite binding convention**, **Runtime-only methods**, |
| 60 | and **Isolate scope vs request scope** — bindings and runtimes. |
| 61 | - **Documentation Generation** — JSDoc is the only docs source. |
| 62 | - **The Resource Factory Process** — how to fan the work out to agents. |
| 63 | |
| 64 | Those rules existed while the GCP provider was built, and its first review |
| 65 | still found the same violations in hundreds of files. Reading the rules is |
| 66 | not enough: [review.md](review.md) turns each of them into a check with a |
| 67 | command and an expected result. Run that review at every wave boundary and |
| 68 | before step 10; a provider with unexplained hits is not done. |
| 69 | |
| 70 | ## Bringing an existing provider up to the bar |
| 71 | |
| 72 | When the provider already exists (a PR in progress, or a package on |
| 73 | `main`), start with a gap audit instead of step 0: |
| 74 | |
| 75 | 1. Merge `origin/main` into the branch and type-check the package (see |
| 76 | **Branch, checkpoints, and PRs**). |
| 77 | 2. Run every check in [review.md](review.md) against the code as it is, |
| 78 | and port `scripts/audit-service.ts` (step 2) to get per-service |
| 79 | operation coverage. |
| 80 | 3. Rebuild `INDEX.md` from the results: one row per missing resource, |
| 81 | binding, runtime form, event source, workflow engine, website |
| 82 | rework, example, docs page, and JSDoc gap, plus one per review hit |
| 83 | that needs a fix (hand-written HTTP, `host.bind`, a container |
| 84 | website). Existing rows keep their status only when review.md passes |
| 85 | them. |
| 86 | 4. Do the distilled work first. Missing data-plane modules, credentials |
| 87 | providers, and manual specs block the bindings that depend on them. |
| 88 | 5. Then run the waves from step 4 onward, with the same contracts as a |
| 89 | new provider. |
| 90 | |
| 91 | Report the audit to the user before the first wave: counts per category |
| 92 | and the planned waves. |
| 93 | |
| 94 | ## Orchestration |
| 95 | |
| 96 | Fan-out work (the gap audit, the catalog in step 2, implementation waves, |
| 97 | fix rounds in step 8, the review gate in step 9) runs as a **workflow** |
| 98 | when the harness has one: Claude Code's `Workflow` tool (load its |
| 99 | `workflow-authoring` skill before writing the script), Grok's `workflow` |
| 100 | tool (its `create-workflow` skill), or the equivalent elsewhere. Run a |
| 101 | workflow only when the user asked for one, since it spends many agents. A workflow runs the whole wave as one |
| 102 | script with bounded parallelism, enforces an agent budget, and journals |
| 103 | every finished agent's result, so a crash costs only the agents still |
| 104 | running. When no workflow tool exists, launch each agent as a detached |
| 105 | top-level thread (`t3_thread_launch` with `workspaceStrategy: "worktree"` |
| 106 | in T3 Code), never a `delegate_task` subagent thread; the coordinator reads |
| 107 | each thread's result with `t3_thread_read`. Either way, |
| 108 | each agent gets the task contract from **What every agent task prompt must |
| 109 | include** in `AGENTS.md`, owns one distilled service, and returns the |
| 110 | structured result the coordinator turns into index statuses. |
| 111 | |
| 112 | Coordinator duties during a long run: |
| 113 | |
| 114 | - **Size agents to finish within an hour.** A service with more than about |
| 115 | 25 resources runs as a chain of agents over resource batches. Agents that |
| 116 | run for hours hit the model's idle timeout and lose their context. |
| 117 | - **Read the workflow state, not only file activity.** Each status check |
| 118 | counts agents by state and reads the error of every failed or cancelled |
| 119 | one. A model-API outage (`auth_unavailable`, idle timeouts) looks like a |
| 120 | quiet period from the file system; on Azure it went unnoticed for three |
| 121 | hours. Relaunch only the rows still `missing`, with an assess-first |
| 122 | contract, once the cause is gone. |
| 123 | - **Offer periodic status updates** when the run will take more than an |
| 124 | hour, and include the index counts in each. |
| 125 | |
| 126 | ## Branch, checkpoints, and PRs |
| 127 | |
| 128 | - Work in a fresh worktree on a branch from `origin/main`, with |
| 129 | `submodules/distilled` at the commit alchemy pins plus your distilled |
| 130 | branch. Never start from whatever branch the worktree happens to be on. |
| 131 | - Open **draft** PRs in alchemy and distilled after the first wave that |
| 132 | passes live, and push a checkpoint at every wave boundary. The |
| 133 | description carries the current index counts. |
| 134 | - Each checkpoint: rebase onto `origin/main` (distilled onto the pin |
| 135 | `main` uses), type-check, push, and confirm the PR's checks actually ran. |
| 136 | A PR with merge conflicts runs no `pull_request` workflows, so a stale |
| 137 | green check means nothing. Dependency bumps on `main` (Effect, Node |
| 138 | types) break new code; catch them at the checkpoint. |
| 139 | - When a rebase or pull brings in a newer version of this skill, re-read |
| 140 | the whole skill and review.md before the next wave. Add an index row for |
| 141 | every new deliverable, then build those rows and run review.md with a |
| 142 | fresh reviewer. |
| 143 | - `pnpm exec tsrs -b` reads distilled's built declarations (`lib/`), while |
| 144 | the test runner reads its `src/`. After distilled patches or a pin bump, |
| 145 | rebuild that package before the type check, or new error types show up |
| 146 | as false type errors. |
| 147 | - After merging `main`, type-check the provider package on its own first |
| 148 | (`cd packages/<provider> && pnpm exec tsrs -b`), then the workspace. A |
| 149 | newer distilled pin is the usual break: a shape that became open |
| 150 | (`[key: string]: unknown`) or a renamed error tag. |
| 151 | - Commit with the repo's hooks. If signing or a hook hangs, ask the user; |
| 152 | if they approve skipping it, run `pnpm exec oxfmt` yourself and say the |
| 153 | commits are unsigned. |
| 154 | |
| 155 | ## Step 0 — get credentials first |
| 156 | |
| 157 | **Always ask the user for credentials for the upstream provider before |
| 158 | anything else**, even when the request does not mention testing. Live tests |
| 159 | are the whole point of the loop, and a provider built without them is |
| 160 | unverified. Ask in one message, and name exactly what is needed: |
| 161 | |
| 162 | - the auth method the provider supports (API token, service account key, |
| 163 | OAuth app, access key pair) and the minimum scopes or roles the test |
| 164 | resources need; |
| 165 | - a dedicated test account, project, or organization — never a production |
| 166 | one, because tests create and nuke resources; |
| 167 | - account identifiers the API needs (org id, project id, default region); |
| 168 | - known limits of that account (plan tier, quotas, features that need an |
| 169 | upgrade), so gated tests can be planned up front. |
| 170 | |
| 171 | Store what the user gives you in an Alchemy profile, never in the repo: |
| 172 | `alchemy profile edit --profile testing --add <Provider>` (use the profile |
| 173 | name the user asks for). Tests select it with `--profile <name>`. Never |
| 174 | print a secret, write it to a tracked file, or put it in a PR, commit, or |
| 175 | log. If the user has no credentials, give them the exact steps to create |
| 176 | them (console path or CLI commands, roles to grant), offer to do it in the |
| 177 | browser if they allow it, and keep working on the catalog and code |
| 178 | meanwhile. Until credentials arrive, mark every live test as blocked in |
| 179 | your report; do not claim anything is tested. |
| 180 | |
| 181 | ## Step 1 — the distilled SDK |
| 182 | |
| 183 | Every provider talks to its API through a distilled SDK |
| 184 | (`@distilled.cloud/<pkg>` in `submodules/distilled/packages/<pkg>`). Alchemy |
| 185 | never calls the API with a hand-written HTTP client. |
| 186 | |
| 187 | - **No SDK yet:** follow the `distilled-sdk` skill in the submodule |
| 188 | (`submodules/distilled/.agents/skills/distilled-sdk/SKILL.md`). It covers |
| 189 | sourcing the spec, the mirror, convert/generate, the README, and the |
| 190 | distilled PR. |
| 191 | - **SDK exists but is wrong** (missing typed error, nullable field, wrong |
| 192 | response schema, unmarked secret): follow the `distilled-sdk-patch` |
| 193 | skill (`submodules/distilled/.agents/skills/distilled-sdk-patch/SKILL.md`). |
| 194 | - **SDK is behind its spec:** follow `distilled-sdk-update`. |
| 195 | |
| 196 | **That includes the data plane.** Bindings call the cloud's data APIs |
| 197 | (put a blob, send a message, read a secret), and those go through |
| 198 | distilled exactly like the management API that creates the resources. |
| 199 | AWS is the model: `sqs.sendMessage`, `s3.putObject` and |
| 200 | `dynamodb.getItem` are distilled operations, and AWS bindings hold no |
| 201 | HTTP code. Check data-plane coverage when you check the SDK. If the SDK |
| 202 | covers only the management API, extend it before writing a binding: |
| 203 | |
| 204 | - **The spec has a data-plane model** (Azure's |
| 205 | `specification/<service>/data-plane/`, GCP's data APIs in discovery |
| 206 | docs): extend the converter to include it, as its own service module |
| 207 | (`storage-queue`, `keyvault-secrets`). |
| 208 | - **The API has no published spec** (Kudu zip deploy, a CLI-only |
| 209 | endpoint): write the model by hand in the package's `manual-specs/`. |
| 210 | - **The protocol isn't HTTP-shaped** (XML bodies, SAS signing, a token |
| 211 | audience per data plane): add it to the distilled protocol, not to |
| 212 | alchemy. |
| 213 | - **Credentials from the runtime's identity endpoint** (managed identity, |
| 214 | metadata server, container credentials) are a distilled credentials |
| 215 | provider, like AWS's `fromContainerMetadata`. A binding never fetches |
| 216 | its own token. |
| 217 | |
| 218 | On Azure, the converter skips `data-plane/` (`scripts/convert.ts`), so |
| 219 | the agent hand-wrote XML request builders for Storage queues and blobs, |
| 220 | Key Vault secrets, Service Bus, Event Hubs, a static-website uploader, and |
| 221 | a managed-identity token cache: 22 raw request sites, with no typed |
| 222 | errors and no tests against a spec. Don't repeat it. Stop and extend |
| 223 | distilled. |
| 224 | |
| 225 | Raw `HttpClient` calls are allowed only for: |
| 226 | |
| 227 | - URLs the API hands back to you to use as-is (a presigned or SAS upload |
| 228 | URL, a signed download URL); |
| 229 | - the user's own deployed endpoints (calling a Worker, Function, or |
| 230 | service the stack deployed). |
| 231 | |
| 232 | Each one carries a comment saying which case it is. |
| 233 | |
| 234 | If a skill file is missing at the pinned submodule commit, read it from |
| 235 | distilled `main`: `git -C submodules/distilled fetch origin main && git -C |
| 236 | submodules/distilled show origin/main:.agents/skills/<skill>/SKILL.md`. |
| 237 | |
| 238 | Distilled changes land as their own PR in `alchemy-run/distilled`. After |
| 239 | it merges, move the submodule pin in alchemy to the merged commit |
| 240 | (`git -C submodules/distilled checkout <sha>` then commit the gitlink) and |
| 241 | run `pnpm install` so the workspace links the package. |
| 242 | |
| 243 | ## Step 2 — catalog and scope |
| 244 | |
| 245 | Catalog **every** service module in the SDK |
| 246 | (`ls submodules/distilled/packages/<pkg>/src/services`) before writing |
| 247 | resources; for a large SDK, fan the catalog out as a workflow (see Orchestration). Each service gets |
| 248 | `processes/<Provider>/catalog/<service>.md` in that section's format: |
| 249 | resources, props with replacement rules, attributes, |
| 250 | lifecycle-to-operation mapping, bindings, testability, and priority. |
| 251 | `processes/` is gitignored, so the catalog is the local order book; the |
| 252 | committed record of scope is the overview page below. |
| 253 | |
| 254 | Each catalog file also lists **every operation** in the service module, |
| 255 | classified the way `scripts/audit-service.ts` classifies AWS: |
| 256 | |
| 257 | | Category | Meaning | Becomes | |
| 258 | | --- | --- | --- | |
| 259 | | `resource-lifecycle` | create/get/update/delete/list of the resource itself | a resource's `reconcile`/`read`/`delete`/`list` | |
| 260 | | `binding` | data-plane work on or inside a resource (put an object, send a message, query a table, read a secret, invoke a function) | a `Binding.Service` + `<Op>Http.ts` | |
| 261 | | `event-source` | the resource triggers runtimes (notifications, streams, subscriptions) | an event source | |
| 262 | | `internal` | not useful from user code (long-running-operation polling, deprecated aliases) | nothing, with a reason | |
| 263 | |
| 264 | Read operations a user calls at runtime (get an item, list objects, |
| 265 | describe a queue's depth) are bindings even when a lifecycle uses them |
| 266 | too. Port `scripts/audit-service.ts` to the provider before the first |
| 267 | wave (it reads the distilled module and the package and prints the gaps), |
| 268 | and run it per service at every wave boundary. |
| 269 | |
| 270 | `processes/<Provider>/catalog/INDEX.md` has one row per resource and one per binding with a |
| 271 | status of `missing`, `implemented`, `tested`, `blocked` (with the exact |
| 272 | error), or `out-of-scope` (with the reason). Build waves from it and repeat |
| 273 | until no row is `missing` or `implemented`. Every progress report and the |
| 274 | final report quote its counts per status; "done" while any in-scope row is |
| 275 | `missing` is a false claim. |
| 276 | |
| 277 | The first line of `INDEX.md` records the skill version it was built |
| 278 | against: `Skill: <sha>`, from `git log -1 --format=%H -- |
| 279 | .agents/skills/alchemy-provider`. Update it after re-reading a newer skill. |
| 280 | |
| 281 | Decide scope per resource and record the out-of-scope list in the |
| 282 | provider's website overview under an "Out of scope" section: |
| 283 | |
| 284 | - **In scope:** infrastructure the user configures and owns (databases, |
| 285 | buckets, queues, networks, keys, functions, DNS). |
| 286 | - **Out of scope** (the only valid reasons): end-user data APIs (mail, calendar, documents, videos), |
| 287 | APIs that need end-user OAuth scopes, deprecated or retired APIs, |
| 288 | billing/subscription objects, closed-beta endpoints, and duplicates of |
| 289 | another service. |
| 290 | |
| 291 | Fix public names now; renaming after release is a breaking change. |
| 292 | Service namespaces are PascalCase (`AccessContextManager`), and resource |
| 293 | names are correct English singulars (`Index`, not `Indexe`; `Process`, not |
| 294 | `Processe`). Audit generated names by eye before the first PR. |
| 295 | |
| 296 | ## Step 3 — the provider package |
| 297 | |
| 298 | A provider is its own workspace package, `@alchemy.run/<provider>` in |
| 299 | `packages/<provider>`, following **Package boundaries** in `AGENTS.md`. |
| 300 | Copy the closest small package: `packages/axiom` (token auth, flat |
| 301 | resources) or `packages/hetzner` (labels, `list`, nuke ordering, bindings, |
| 302 | a hosted runtime, websites). A package holds: |
| 303 | |
| 304 | | File | Purpose | |
| 305 | | --- | --- | |
| 306 | | `package.json` | name, shared alchemy version, publish metadata, `exports`/`publishConfig` (generated, below), `alchemy.contributes`, dependencies | |
| 307 | | `tsconfig.json` | composite project; references the distilled package, `distilled/core`, and `../alchemy` | |
| 308 | | `scripts/build.ts` | copied verbatim from a sibling package | |
| 309 | | `src/Contributions.ts` | `Contributions.make({ name, auth, … })` — what the CLI, the Alchemist API, and the test harness load without the user's stack | |
| 310 | | `src/AuthProvider.ts` | `makeStoredAuthProvider` fields, CI env vars, `<Provider>Auth` layer | |
| 311 | | `src/Credentials.ts` | `fromAuthProvider()` = `lazyCredentials<Resolved, ConfigError>()(...)` → the distilled `Credentials` layer | |
| 312 | | `src/Providers.ts` | `ProviderCollection` + `providers()` layer (provider layers, credentials, `FetchHttpClient`, auth, profile and credential stores) | |
| 313 | | `src/<Provider>Bindings.ts` | how bindings reach the cloud from any host (step 5) | |
| 314 | | `src/globals.d.ts` | `import type {} from "alchemy/Phase";` so `globalThis.__ALCHEMY_RUNTIME__` type-checks | |
| 315 | | `src/index.ts` | re-exports every resource, binding, and layer; `export * as <Service> from "./<Service>/index.ts"` | |
| 316 | | `src/Labels.ts` / tags helper | ownership labels, if the API supports labels or tags | |
| 317 | | `src/Bootstrap.ts` | process entry for a hosted runtime, if the cloud runs code (source-only export) | |
| 318 | | `README.md` | npm readme: short intro, install line, one complete type-checked example, absolute `https://alchemy.run` links | |
| 319 | |
| 320 | `Contributions.ts` is imported on every CLI start, so it holds metadata and |
| 321 | `Contributions.lazy(() => import(...), pick)` fields only. Contribute `auth` |
| 322 | always; add `stateStores`, `cli`, `testServices`, and `devEnv` when the |
| 323 | provider has them (`packages/aws/src/Contributions.ts` uses all of them). |
| 324 | Declare it in `package.json`: |
| 325 | |
| 326 | ```json |
| 327 | "alchemy": { "contributes": ["@alchemy.run/<provider>/Contributions"] } |
| 328 | ``` |
| 329 | |
| 330 | Never hand-edit `exports`. Run `pnpm exports:sync` after adding or |
| 331 | removing a service directory; CI runs `pnpm exports:check`. Modules other |
| 332 | packages must not import are named `internal.ts` or live under |
| 333 | `internal/`. Dependencies: `@distilled.cloud/<pkg>` and |
| 334 | `@distilled.cloud/core` are `dependencies`; `alchemy` and `effect` are |
| 335 | peer + dev dependencies; another `@alchemy.run/*` package is a dependency |
| 336 | only when every user needs it, otherwise an optional peer. |
| 337 | |
| 338 | Register the package everywhere the repo enumerates providers. Find the |
| 339 | full list by grepping for an existing package (`git grep -il hetzner -- |
| 340 | ':!packages/hetzner' ':!submodules' ':!examples'`); at the time of |
| 341 | writing it is: |
| 342 | |
| 343 | - `package.json` — the `build:pkg` filter list and a root |
| 344 | `@alchemy.run/<provider>` devDependency (the CLI finds `Contributions` |
| 345 | through the project's dependencies); |
| 346 | - `tsconfig.json` — project references to the package and its distilled |
| 347 | package; |
| 348 | - `packages/integration-tests/package.json` and `tsconfig.json` — the |
| 349 | package, its distilled package, and the project reference; |
| 350 | - `.github/workflows/pkg.yml` — the `@alchemy.run[Collapsed]` group; |
| 351 | - `.gitignore` — `!packages/<provider>/src/globals.d.ts`; |
| 352 | - `stacks/nuke.ts` — `<Provider>.providers()`; |
| 353 | - `scripts/with-fake-profiles.sh` — fake CI credentials; |
| 354 | - `scripts/generate-api-reference.ts` and `scripts/fix-api-jsdocs.ts` — |
| 355 | the package's `src` root and provider prefix; |
| 356 | - `packages/integration-tests/test/Bundle/ProviderBootstraps.test.ts` — |
| 357 | every `Bootstrap` the package ships; |
| 358 | - `website/astro.config.ts`, `website/src/components/ProviderDirectory.astro`, |
| 359 | `website/scripts/generate-llms-txt.ts`, and |
| 360 | `website/src/content/docs/<provider>/` (`index.mdx`, `setup.mdx`) — |
| 361 | sidebar, directory card, llms.txt, overview, and setup page. |
| 362 | |
| 363 | Core (`packages/alchemy`) never changes to add a provider. If a provider |
| 364 | seems to need a core change (a new auth registry entry, a CLI command, a |
| 365 | state store), it is a `Contributions` field instead. |
| 366 | |
| 367 | Configuration such as region, project, or account comes from the |
| 368 | credential/profile or from a layer the user provides (the AWS |
| 369 | `AWS.Region(...)` pattern). Do not invent environment variables for it; |
| 370 | env vars exist only for CI credentials and for values the runtime platform |
| 371 | itself sets. |
| 372 | |
| 373 | If `providers()` wires hundreds of resources, memoize it |
| 374 | (`let cached; export const providers = () => (cached ??= make())`) and keep |
| 375 | nested `Layer.mergeAll` groups of about 90; an unmemoized collection |
| 376 | rebuilds every layer per call and runs the test process out of memory. |
| 377 | `pnpm lint:providers` fails when a `providers()` layer leaks an `unknown` |
| 378 | requirement. |
| 379 | |
| 380 | ## Step 4 — resources |
| 381 | |
| 382 | Implement each resource per **Reconciler doctrine**, co-locating the |
| 383 | contract and provider in `packages/<provider>/src/<Service>/<Resource>.ts`. These |
| 384 | are the mistakes that most often slip through review (each has a check in |
| 385 | [review.md](review.md)): |
| 386 | |
| 387 | - **Block until ready.** `reconcile` returns only after the API reports the |
| 388 | resource usable: poll the long-running operation to completion and then |
| 389 | read the ready state. Use one shared operation waiter per provider with a |
| 390 | per-resource time budget that matches reality (minutes for databases and |
| 391 | clusters). Post-ready propagation delay is acceptable; returning early is |
| 392 | not. |
| 393 | - **Identity changes replace.** If a user-chosen name or id is part of the |
| 394 | resource's identity, `diff` returns `replace` when it changes; otherwise |
| 395 | the old resource leaks. |
| 396 | - **Missing is not forbidden.** Only a typed not-found tag means "gone". |
| 397 | A permission error treated as not-found drops the row from state and |
| 398 | leaks the resource. If the API returns 403 for missing resources, patch |
| 399 | distilled with a specific tag for that case. |
| 400 | - **Lists fail loudly and paginate.** A `list` that swallows errors reports |
| 401 | "nothing here" to nuke and hides leaks. Use the SDK's paginated |
| 402 | `.items(...)` stream. |
| 403 | - **Ownership markers go in labels or tags.** When the API has neither, |
| 404 | document the fallback marker in the resource JSDoc; never put markers in |
| 405 | fields users see or that change behaviour. |
| 406 | - **Shared helpers live once per provider** (operation waiter, not-found |
| 407 | helper, label diffing). Copies per service drift apart. |
| 408 | - **Enable what the account needs.** Clouds that require a service to be |
| 409 | switched on per account (Azure resource provider registration, GCP API |
| 410 | enablement) get an idempotent enable step inside `reconcile`, so a fresh |
| 411 | account deploys without manual setup. |
| 412 | - **Clean up after a failed create.** `read` and `delete` work when `olds` |
| 413 | or parts of `output` are missing, so the next run's opening `destroy` |
| 414 | removes whatever a crashed run left behind. Prove it once by interrupting |
| 415 | a create. |
| 416 | - **SQL databases take `migrations`.** Every resource that hosts a SQL |
| 417 | schema applies migrations on deploy through `@alchemy.run/sql/Migrations`; |
| 418 | the provider only writes the cloud-specific call, built on its executor |
| 419 | helpers (`withPgExecutor`, `makeStatementExecutor`, `splitSqlScript`). Prefer the |
| 420 | cloud's HTTPS data API, then a direct or connector connection, then an |
| 421 | in-network runner, and take credentials from the platform before asking |
| 422 | for `migrationsUser` / `migrationsPassword`. Follow **SQL database |
| 423 | migrations** in `AGENTS.md`; a SQL database shipped without migrations is |
| 424 | incomplete. |
| 425 | - **Implement `list`** filtered to Alchemy-owned resources, and declare |
| 426 | `nuke: { dependsOn: [...] }` where delete order matters, so |
| 427 | `alchemy unsafe nuke` can clean the test account. |
| 428 | |
| 429 | ## Step 5 — bindings and runtimes |
| 430 | |
| 431 | Follow the binding sections of `AGENTS.md`, then copy the AWS and GCP |
| 432 | packages: they are the reference for how bindings reach their host, and |
| 433 | most questions are answered by reading the closest existing binding. The |
| 434 | pattern was set by these PRs; read their descriptions when a case isn't |
| 435 | obvious from the code: |
| 436 | |
| 437 | | PR | What it settled | |
| 438 | | --- | --- | |
| 439 | | #2260 | bindings state what they need; a `<P>Bindings` module decides how the host gets it (`packages/aws/src/AWSBindings.ts`) | |
| 440 | | #2347 | `Bind` / `withCredentials` effectful constructors, identity from real resources, `__ALCHEMY_HOST_IDENTITY__` bundle fold (`packages/gcp/src/GCPBindings.ts`) | |
| 441 | | #2267 | deploy-only services read behind the runtime guard; `*Http` bindings on local resources fail with a typed error (`packages/cloudflare/src/HttpBindingDev.ts`) | |
| 442 | | #2244, #2315 | no branching on another provider's host type; reach the host through `RuntimeContext` and core host services (`alchemy/WebhookHost`) | |
| 443 | |
| 444 | **Cover every `binding` operation** in the catalog. Follow the AWS |
| 445 | shape: one `Binding.Service` per operation (`SQS.SendMessage`, |
| 446 | `DynamoDB.GetItem`), each with an `<Op>Http.ts` layer built on the |
| 447 | distilled operation, which injects the bound resource's identifier and |
| 448 | grants only that operation on that resource. Where the API has clear |
| 449 | access levels (object storage, key-value stores), also ship the |
| 450 | Read/Write/ReadWrite services from `AGENTS.md` on top of the per-operation ones. |
| 451 | |
| 452 | Least privilege depends on the cloud's permission model: |
| 453 | |
| 454 | - **Per-action permissions** (AWS IAM): grant the operation's actions on |
| 455 | the resource ARN, as `scripts/audit-service.ts` checks. |
| 456 | - **Role-based** (GCP IAM, Azure RBAC): grant the smallest predefined |
| 457 | role that covers the operation, on the resource, never the project or |
| 458 | subscription. If the smallest predefined role covers much more than the |
| 459 | operation, create a custom role with exactly the permissions or data |
| 460 | actions it needs, and reuse it across hosts. |
| 461 | |
| 462 | Every binding is a distilled call: the `<Op>Http.ts` layer wraps |
| 463 | `<service>.<operation>` from `@distilled.cloud/<pkg>`. If the operation |
| 464 | isn't in distilled (the data plane is missing, or the operation isn't |
| 465 | modelled), patch distilled first (step 1). |
| 466 | |
| 467 | **Declare** each capability as an interface plus factory: |
| 468 | |
| 469 | ```ts |
| 470 | export interface SendMessage extends Binding.Service< |
| 471 | SendMessage, |
| 472 | "AWS.SQS.SendMessage", |
| 473 | (queue: Queue) => Effect.Effect<(request: SendMessageRequest) => Effect.Effect<…>> |
| 474 | > {} |
| 475 | export const SendMessage = Binding.Service<SendMessage>("AWS.SQS.SendMessage"); |
| 476 | ``` |
| 477 | |
| 478 | **Reach any host.** A binding works on the provider's own compute and on |
| 479 | every other cloud's: a Lambda can bind a GCP topic, a Cloudflare Worker |
| 480 | an SQS queue. `src/<Provider>Bindings.ts` owns the decision and exports: |
| 481 | |
| 482 | - `HostBinding` — what a binding attaches: grants (IAM statements, role |
| 483 | bindings, role assignments) plus `env`. |
| 484 | - `Bind` — yields a tagged-template `bind`. On a host that runs as the |
| 485 | cloud's own identity (`runsAsRole`, `runsAsServiceAccount`; an Azure |
| 486 | managed identity) the grants go on that identity. Any other host gets an |
| 487 | identity built from the provider's own resources (IAM `User` + |
| 488 | `AccessKey` + `Role` on AWS, `ServiceAccount` + `ServiceAccountKey` on |
| 489 | GCP), the grants go on it, and its key goes into the host's env through |
| 490 | `CurrentRuntimeContext`. With no host (a script, plan-time `execute`) it |
| 491 | does nothing. In a host bundle it is a no-op, so the identity resources |
| 492 | drop out. |
| 493 | - `withCredentials(operation)` — resolves a distilled operation once when |
| 494 | the layer is built: the host's ambient identity, the bound key read back |
| 495 | from `RuntimeContext`, or the deployer's credentials. |
| 496 | - `<identity>HostBundle` — the rolldown option the provider's own host |
| 497 | bundlers apply (`Lambda/FunctionBundle.ts`, `CloudFunctions/FunctionSource.ts`, |
| 498 | the `ImageSource.ts` files). It sets `__ALCHEMY_HOST_IDENTITY__` (declared |
| 499 | in core's `Phase.ts`) so the created-key path drops out of native bundles. |
| 500 | |
| 501 | A binding layer then reads: |
| 502 | |
| 503 | ```ts |
| 504 | Effect.gen(function* () { |
| 505 | const publish = yield* GCPBindings.withCredentials(pubsub.publishProjectsTopics); |
| 506 | const bind = yield* GCPBindings.Bind; |
| 507 | return Effect.fn(function* (topic: Topic) { |
| 508 | if (!globalThis.__ALCHEMY_RUNTIME__) { |
| 509 | yield* bind`GCP.PubSub.WriteTopic(${topic})`({ iam: [/* scoped to topic */] }); |
| 510 | } |
| 511 | return (message) => publish({ topic: …, body: … }); |
| 512 | }); |
| 513 | }); |
| 514 | ``` |
| 515 | |
| 516 | Rules the reference code follows: |
| 517 | |
| 518 | - Never call `host.bind(...)` from a binding, and never allowlist the |
| 519 | provider's own host types; both lose foreign hosts. The exceptions are |
| 520 | host-only attachments that make no sense elsewhere, and they get their |
| 521 | own helper (`AWSBindings.bindVpc`, the Cloud SQL socket mount in |
| 522 | `GCP.SQL.Connect`). |
| 523 | - Never name another provider's resource type (`host.Type === |
| 524 | "Cloudflare.Worker"`). Write env through `RuntimeContext` (`runtime.set` |
| 525 | at deploy, `runtime.get` at runtime; secrets stay `Redacted`), and use |
| 526 | core host services such as `WebhookHost` for anything else a host must |
| 527 | provide. |
| 528 | - Services that only exist at deploy (`<P>Environment.current`, the |
| 529 | account or project id for a grant) are read inside the |
| 530 | `!globalThis.__ALCHEMY_RUNTIME__` block. Read at layer build, they crash |
| 531 | a foreign host at init. |
| 532 | - Per-service grant helpers are constructors too (`GrantOnQueue` in |
| 533 | `packages/aws/src/SQS/BindingHttp.ts`), and shared scaffolding lives in |
| 534 | an unexported `BindingHttp.ts` so each `<Op>Http.ts` is a one-line |
| 535 | `Layer.effect(Cap, make…({ … }))`. |
| 536 | - A provider with local emulation fails an HTTP binding to a locally |
| 537 | emulated resource with a typed error naming the native binding and |
| 538 | `Alchemy.remote()` (`HttpBindingUnsupportedInDev`). |
| 539 | |
| 540 | Document any grant broader than the bound resource (a condition on a |
| 541 | project-level grant, say) in the binding's JSDoc, with the reason. |
| 542 | |
| 543 | ### Runtimes |
| 544 | |
| 545 | Find every **runtime** in the catalog: a resource that runs the user's |
| 546 | code (functions, containers, services, workers, jobs, VMs that run a |
| 547 | program). Mark each one `runtime` in `INDEX.md`. Each ships in two |
| 548 | forms from the same resource: |
| 549 | |
| 550 | **Effectful** — a class built with core's `Platform` (`alchemy/Platform`). |
| 551 | `main: import.meta.url` makes the file both the bundle entry and the |
| 552 | program. Construction binds resources, and the returned handlers run per |
| 553 | request: |
| 554 | |
| 555 | ```ts |
| 556 | export default class Api extends GCP.Function<Api>()( |
| 557 | "Api", |
| 558 | { main: import.meta.url }, |
| 559 | Effect.gen(function* () { |
| 560 | const db = yield* GCP.Firestore.ReadWriteDatabase(Links); |
| 561 | return { |
| 562 | fetch: Effect.gen(function* () { |
| 563 | const doc = yield* db.get("links/home"); |
| 564 | return yield* HttpServerResponse.json(doc?.fields ?? {}); |
| 565 | }).pipe(Effect.orDie), |
| 566 | }; |
| 567 | }).pipe(Effect.provide(GCP.Firestore.ReadWriteDatabaseHttp)), |
| 568 | ) {} |
| 569 | ``` |
| 570 | |
| 571 | **Async** — the same resource called as a plain function, with `main` |
| 572 | pointing at an ordinary handler file and no Effect. Alchemy bundles that |
| 573 | file as-is, with no Effect runtime in it. The `env` prop takes resources, |
| 574 | `Config` values, and literals, and `<Provider>.InferEnv<typeof Fn>` |
| 575 | types the handler's `env` argument: |
| 576 | |
| 577 | ```ts |
| 578 | // alchemy.run.ts |
| 579 | export const Ping = Railway.Function("Ping", { |
| 580 | project: Site, |
| 581 | main: "./src/ping.ts", |
| 582 | env: { DATABASE_URL: db.connectionUri }, |
| 583 | }); |
| 584 | export type PingEnv = Railway.InferEnv<typeof Ping>; |
| 585 | ``` |
| 586 | |
| 587 | ```ts |
| 588 | // src/ping.ts |
| 589 | import type { PingEnv } from "../alchemy.run.ts"; |
| 590 | |
| 591 | export default { |
| 592 | async fetch(_request: Request, env: PingEnv) { |
| 593 | return new Response(env.DATABASE_URL ? "ok" : "missing"); |
| 594 | }, |
| 595 | }; |
| 596 | ``` |
| 597 | |
| 598 | Copy both forms from the runtime closest to yours: |
| 599 | |
| 600 | | Runtime | File | Notes | |
| 601 | | --- | --- | --- | |
| 602 | | Cloudflare Worker | `packages/cloudflare/src/Workers/Worker.ts`, `InferEnv.ts` | the full reference: `env` entries become native bindings | |
| 603 | | Railway Function | `packages/railway/src/Function.ts`, `InferEnv.ts` | bindings are env vars, so `InferEnv` maps to `string` | |
| 604 | | GCP Cloud Run, Hetzner Service | `packages/gcp/src/Run/Service.ts`, `packages/hetzner/src/Service.ts` | `Platform` over a container or process | |
| 605 | | AWS Lambda | `packages/aws/src/Lambda/Function.ts` | async form exists but has no `InferEnv`; don't copy that gap | |
| 606 | |
| 607 | Every runtime needs: |
| 608 | |
| 609 | - **Both forms**, each with its own JSDoc section (`### Effect …` and |
| 610 | `### Async …`) showing the stack declaration and the handler file. |
| 611 | - **`InferEnv`** exported from the package, so async users get a typed |
| 612 | `env` without writing the type by hand. |
| 613 | - **The `HostBinding` contract** on the resource, so effectful bindings |
| 614 | from this and other providers can attach grants and env. |
| 615 | - **The host bundle option** (`<identity>HostBundle`) on the effectful |
| 616 | bundle, and a source-only `Bootstrap` export for the process entry |
| 617 | (`packages/hetzner/src/Bootstrap.ts`, `packages/aws/src/Bootstrap/`). |
| 618 | - **Event sources** for what the cloud triggers it with (queues, topics, |
| 619 | storage events, schedules), shaped like AWS `SQS.QueueEventSource` and |
| 620 | Cloudflare cron. Their contract lives in the source service and each |
| 621 | runtime provides an implementation layer. |
| 622 | - **A Website target** when the runtime is how the cloud serves |
| 623 | server-rendered pages (step 7). |
| 624 | - **Tests for both forms**: a fixture per form under |
| 625 | `test/<Provider>/<Service>/fixtures/` (`effect-worker.ts` and |
| 626 | `async-worker.ts` in `test/Cloudflare/Images/` are the pattern), each |
| 627 | deployed and called over HTTP. |
| 628 | |
| 629 | ### Workflow engine |
| 630 | |
| 631 | If the cloud has a durable execution engine, the provider ships an Effect |
| 632 | workflow engine on it, so `alchemy/Workflows` code runs there unchanged. |
| 633 | Follow **Effect workflow engines** in `AGENTS.md`: it covers the DX to |
| 634 | keep, the two engine shapes, the shared encoding module, and the eight |
| 635 | conformance cases. |
| 636 | |
| 637 | | Cloud | Engine | Status | |
| 638 | | --- | --- | --- | |
| 639 | | Cloudflare | `Cloudflare.Workflows.NativeEngine` (Cloudflare Workflows), `DurableObjectEngine` | done, in-process | |
| 640 | | AWS | `AWS.Lambda.WorkflowFunction` + `DurableFunctionEngine` (Lambda durable functions) | done, host + remote | |
| 641 | | Azure | Durable Functions on the Durable Task Scheduler (`Azure.DurableTask.Scheduler` exists) | missing | |
| 642 | | GCP | no code-first durable runtime; Cloud Workflows is a YAML orchestrator | decide and record in the catalog before building | |
| 643 | |
| 644 | Research the cloud's engine in step 2 and record it in the catalog: its |
| 645 | durable step, durable sleep, external-event or callback primitive, how a |
| 646 | run is interrupted, and its limits. Then decide the shape: |
| 647 | |
| 648 | - **In-process** when the runtime that starts runs can also host the |
| 649 | durable construct (Cloudflare). The engine is a Layer provided next to |
| 650 | the workflow Layers. |
| 651 | - **Host + remote** when runs need a dedicated durable runtime (Lambda |
| 652 | durable functions, and most likely Azure Durable Functions). Ship a |
| 653 | `<Runtime>.WorkflowFunction`-style host that lists its workflows, and a |
| 654 | `<Runtime>Engine(...hosts)` Layer that routes to it and grants access. |
| 655 | |
| 656 | If the cloud has no durable engine, or only one that can't run code |
| 657 | steps, write `out-of-scope` with the reason on the engine's index row. |
| 658 | Don't build a database-backed engine of your own. |
| 659 | |
| 660 | The engine counts as a runtime: it gets its own index row, the |
| 661 | conformance suite, an `examples/<provider>-effect-workflows` example, and |
| 662 | a `compute/effect-workflows.mdx` page in the docs hub. |
| 663 | |
| 664 | ## Step 6 — live tests |
| 665 | |
| 666 | Tests live in `packages/integration-tests/test/<Provider>/` (paths below |
| 667 | are relative to `packages/integration-tests`). Write them per the test |
| 668 | sections of `AGENTS.md` (`test.provider`, |
| 669 | `stack.destroy()` at start and end, deterministic names, out-of-band |
| 670 | verification through distilled, typed wait-until-gone). Run with the |
| 671 | profile from step 0: |
| 672 | |
| 673 | ```sh |
| 674 | pnpm test:integration test/<Provider>/<Service>/<Resource>.test.ts --profile testing --retry 0 |
| 675 | ``` |
| 676 | |
| 677 | What a test must prove: |
| 678 | |
| 679 | - **Every resource has its own test file** |
| 680 | (`test/<Provider>/<Service>/<Resource>.test.ts`). Deploying a resource |
| 681 | as a dependency inside another resource's test does not count, because |
| 682 | its own update, replace, and delete paths never run. |
| 683 | - **Lifecycle runs by default.** Create, update, and (where applicable) |
| 684 | replace against the real API. Slow lifecycles are gated behind one |
| 685 | provider-wide flag with a realistic timeout; entitlement-bound ones keep |
| 686 | an ungated probe that asserts the one specific typed tag the account |
| 687 | returns. |
| 688 | - **Bindings run inside a deployed host** (a fixture Worker, Function, or |
| 689 | container calling the binding over HTTP) so the IAM grant or native |
| 690 | binding is exercised with the host's identity, not the deployer's. Test |
| 691 | both paths of `Bind`: the provider's own host, and a foreign host using |
| 692 | the created key in `test/<Provider>/ExternalHost/CloudflareWorker.test.ts` |
| 693 | (copy `test/GCP/ExternalHost/`). If the provider hosts code, also add |
| 694 | its host to the other providers' external-host suites |
| 695 | (`test/Cloudflare/ExternalHost/Lambda.test.ts` is the pattern). |
| 696 | - **Every binding has a test.** One `describe("<BindingName>")` per |
| 697 | binding, all driving the same deployed fixture host |
| 698 | (`test/AWS/DynamoDB/Bindings.test.ts` + `handler.ts` is the pattern). |
| 699 | A binding no test calls from a host is not done. |
| 700 | - **Assertions can fail.** No always-true checks, no lists of acceptable |
| 701 | tags, no early `return` that skips the assertions. |
| 702 | |
| 703 | **Expensive resources** (dedicated clusters, large VMs or databases, |
| 704 | reserved capacity, anything billed per hour at a noticeable rate) are |
| 705 | implemented and get full tests, but you do not run those tests on your own. |
| 706 | Gate them behind the provider-wide slow flag, mark them `blocked: cost` in |
| 707 | the index, and list them for the user with an estimate of what one run |
| 708 | costs. Run them only after the user says so. |
| 709 | |
| 710 | Scarce account quotas (networks, clusters, IP addresses) are shared with a |
| 711 | semaphore in a test helper, or moved to less-used regions. The suite still |
| 712 | runs in one process: `pnpm test:integration test/<Provider> --profile testing`. |
| 713 | |
| 714 | Unblock account prerequisites in the dedicated test account yourself as |
| 715 | soon as a test hits them. Request quota increases and register preview |
| 716 | features through the provider's API (anything that raises spend needs the |
| 717 | user's go-ahead from step 0 or step 6's cost rule), |
| 718 | and put any setup a fresh account would need again into the test helper. |
| 719 | Do one-time console work (free trials, free organizations, sign-ups) in the |
| 720 | browser with whatever browser-automation skill the harness has, and ask the user only to sign in. |
| 721 | A row stays `blocked` only when it needs payment, a vendor's approval, |
| 722 | hardware, or is skipped under the long-tail rule below; report each one |
| 723 | with the exact request that would unblock it. |
| 724 | |
| 725 | Failure-path tests assert the typed error and finish in minutes. Give them |
| 726 | a short timeout and never let them wait out a slow provisioning or delete; |
| 727 | a single test case that runs for an hour is a bug in the test. |
| 728 | |
| 729 | Stop chasing the long tail after the three-iteration budget from the |
| 730 | **Speed doctrine** in `AGENTS.md`, or as soon as the user says so. When the |
| 731 | remaining failures are slow provisioning, platform behavior, or test |
| 732 | infrastructure the cloud needs (a reachable endpoint, a custom image), |
| 733 | let the current run finish, then skip-gate each failing case with its |
| 734 | exact error, mark the row `blocked: skipped` in the index, and list those |
| 735 | rows in the next report so the user can reopen any. When the user sets a |
| 736 | deadline, cap test fixing at half of it and spend the rest on the other |
| 737 | deliverables (bindings, runtime, websites, examples, docs, PR). On Azure, |
| 738 | fix rounds on one-hour tests kept running after the user asked to skip |
| 739 | them, until the user set a one-hour deadline to wrap up. |
| 740 | |
| 741 | ## Step 7 — framework websites |
| 742 | |
| 743 | **Skip this step when the provider can't host a website.** Most |
| 744 | providers can't: observability, payments, databases, and auth (Axiom, |
| 745 | Stripe, Turso, PlanetScale) have no web hosting, so they ship no |
| 746 | `Website` namespace and get no "N/A" rows in the index. This step is |
| 747 | for providers that serve HTTP or static files. |
| 748 | |
| 749 | Those providers ship `<Provider>.Website.<Framework>` composites, so a |
| 750 | user deploys an Astro or Next.js app with one call: |
| 751 | |
| 752 | ```ts |
| 753 | const site = yield* AWS.Website.Astro("Site", { rootDir: "./web" }); |
| 754 | ``` |
| 755 | |
| 756 | ### Use the cloud's web hosting, not a container |
| 757 | |
| 758 | A website deploys onto the infrastructure the cloud built for websites: |
| 759 | static assets on a CDN, and server rendering on the cloud's serverless |
| 760 | functions behind that same CDN. Putting the whole build in a container |
| 761 | or an always-on VM, with no CDN, is the lazy default. It is not done, |
| 762 | even though it works for every framework on the first try. |
| 763 | |
| 764 | Azure's first pass did exactly that. All 14 frameworks, the static-only |
| 765 | ones included, became a Docker image on Container Apps. A follow-up moved |
| 766 | static builds to a Storage static website and server builds to an |
| 767 | always-on App Service plan, with Front Door optional. That is still a |
| 768 | VM instead of functions, and no CDN by default. GCP's composites still |
| 769 | deploy a Node image to Cloud Run for every framework, `StaticSite` |
| 770 | included. Copy neither. |
| 771 | |
| 772 | Pick the hosting per cloud before writing code, and record the choice |
| 773 | and its reason in the catalog: |
| 774 | |
| 775 | | Kind of provider | Static assets | Server rendering | Reference | |
| 776 | | --- | --- | --- | --- | |
| 777 | | Hyperscaler with a CDN and functions | object storage behind the CDN | serverless functions as the CDN's dynamic origin | `packages/aws/src/Website/` (S3 + CloudFront + Lambda, `SsrSite.ts`) | |
| 778 | | Edge platform | the platform's asset hosting | the platform's functions | `packages/cloudflare/src/Website/` (Workers + static assets) | |
| 779 | | Platform with its own function runtime | the platform's asset hosting | its functions, through a framework target | `packages/neon/src/Website/` (`frontend-frameworks/<framework>/neon.ts`) | |
| 780 | | Containers or VMs only | served by the same Node server, behind the platform's CDN when it has one | the Node server | `packages/fly/src/Website/`, `packages/hetzner/src/Website/`, `packages/railway/src/Website/` (`Cdn.ts`) | |
| 781 | |
| 782 | What that means for the clouds still to be built: |
| 783 | |
| 784 | - **Azure:** Static Web Apps, or Blob static website behind Front Door, |
| 785 | with server rendering on Azure Functions (Flex Consumption). Not |
| 786 | Container Apps, not an App Service plan. |
| 787 | - **GCP:** a Cloud Storage bucket behind Cloud CDN on an external |
| 788 | HTTPS load balancer, with Cloud Run or Cloud Functions as the |
| 789 | serverless origin for server rendering. Assets-only sites |
| 790 | never run a container. |
| 791 | |
| 792 | Rules: |
| 793 | |
| 794 | - An assets-only build (Vite, StaticSite, SPA output) never runs compute. |
| 795 | - The CDN is on by default, not behind a `cdn: true` flag. Assets are |
| 796 | uploaded with content hashes and cache headers, and a deploy |
| 797 | invalidates or purges what changed (`AssetDeployment.ts` + |
| 798 | `CloudFront.Invalidation` on AWS). |
| 799 | - Server rendering runs on functions that scale to zero. A container or |
| 800 | VM is acceptable only when the cloud has no function runtime, or when a |
| 801 | framework needs a long-lived process. Document that framework's reason |
| 802 | in its JSDoc. |
| 803 | - Functions need a framework target: a new |
| 804 | `packages/frontend-frameworks/src/<framework>/<provider>.ts` next to |
| 805 | `aws.ts`, `cloudflare.ts`, and `neon.ts`, producing a handler for the |
| 806 | cloud's function runtime. Only container/VM providers use the Node |
| 807 | target (`@alchemy.run/frontend-frameworks/<framework>/node`) through |
| 808 | core's `alchemy/Website/Server`. |
| 809 | - Before choosing a container, check whether the cloud's docs have a |
| 810 | hosting page for the framework (Azure Static Web Apps has one for |
| 811 | Next.js, Nuxt, Astro, and SvelteKit). If they do, use what it uses. |
| 812 | |
| 813 | ### Layout |
| 814 | |
| 815 | Ship the same framework set as the existing providers (`ls |
| 816 | packages/aws/src/Website` and `packages/hetzner/src/Website`: Astro, |
| 817 | Foldkit, Nextjs, Nuxt, Octane, ReactRouter, SolidStart, StaticSite, |
| 818 | SvelteKit, TanStackStart, Vinext, Vite, Vocs, Waku): |
| 819 | |
| 820 | - `FrameworkSite.ts` owns `makeFrameworkSite`, which builds through the |
| 821 | framework integration (`@alchemy.run/frontend-frameworks/<framework>`) |
| 822 | and deploys the output onto the hosting above. Its shared props |
| 823 | (`rootDir`, `memo`, `dev`, `env`, `assets`, `domain`, `tags`) keep the |
| 824 | names and meaning the sibling providers use. |
| 825 | - One small file per framework that only sets the framework specifier, the |
| 826 | deploy target, and the framework's own option block. |
| 827 | - `StaticSite.ts` for assets-only sites, and `index.ts` exported from the |
| 828 | provider barrel as `export * as Website from "./Website/index.ts"`. |
| 829 | |
| 830 | Each framework gets a live |
| 831 | `packages/integration-tests/test/<Provider>/Website/<Framework>.test.ts` |
| 832 | that asserts the response comes through the CDN (a cache header, or the |
| 833 | CDN hostname) and that a server-rendered route hits the function; a |
| 834 | `<Framework>.local.test.ts` for `alchemy dev`; and a shared |
| 835 | `PropSurface.test.ts`. It also gets an `examples/<provider>-website-<framework>` |
| 836 | example listed in `scripts/test-examples.ts`, and a page under |
| 837 | `website/src/content/docs/<provider>/frontend/`. |
| 838 | |
| 839 | ## Step 8 — close the loop |
| 840 | |
| 841 | Every failure the live run exposes is classified before it is fixed, in |
| 842 | this order: **provider bug > distilled patch > test fix**. |
| 843 | |
| 844 | | Symptom | Fix | |
| 845 | | --- | --- | |
| 846 | | `Unknown<Pkg>Error`, or a status/message you want to branch on | typed error patch (`distilled-sdk-patch`) | |
| 847 | | decode failure, field null or missing on the wire | nullable/optional patch | |
| 848 | | secret returned as a plain string | sensitive patch | |
| 849 | | resource leaks, returns before ready, wrong replace | provider fix in alchemy | |
| 850 | | test asserts something the API does not promise | test fix | |
| 851 | |
| 852 | After a patch, regenerate only that package (`pnpm generate <pkg>` in the |
| 853 | submodule) and rerun the test immediately — the test runner reads distilled |
| 854 | from `src/`, so there is nothing to build. Collect the patches, prove each |
| 855 | with `pnpm patches:audit`, and ship them as a distilled PR, then bump the |
| 856 | pin (step 1). |
| 857 | |
| 858 | After a full run, list what is still alive in the test account |
| 859 | (`pnpm nuke --dry-run --profile testing --include <Provider>.*` or the |
| 860 | provider's list calls). A green test that |
| 861 | leaves resources behind is a provider bug. |
| 862 | |
| 863 | ## Step 9 — review gate |
| 864 | |
| 865 | Run every check in [review.md](review.md) with a fresh reviewer subagent |
| 866 | and fix what it finds. Report the result table to the user with the |
| 867 | counts, so they can see what was checked without re-reviewing by hand. |
| 868 | |
| 869 | ## Step 10 — docs, examples, and the PR |
| 870 | |
| 871 | - JSDoc on every resource, prop, attribute, and binding, to the bar in |
| 872 | [docs.md](docs.md#api-reference-jsdoc): runtimes as complete as |
| 873 | `AWS.Lambda.Function`, fat-head resources as complete as S3 Bucket or |
| 874 | R2 Bucket, and a creation example plus replacement cautions for the |
| 875 | long tail. Then run `pnpm docs:check-jsdoc` and `pnpm docs:gen`. |
| 876 | - The hand-written hub at `website/src/content/docs/<provider>/`, built |
| 877 | to [docs.md](docs.md): the AWS hub's structure (Overview, Setup, a |
| 878 | five-part tutorial, sections by concern), code taken from tested |
| 879 | examples, and the voice rules there. A hub of loose "guides" that walk |
| 880 | the examples is not done; GCP's had to be rebuilt. |
| 881 | - The example set described under **Examples** below. |
| 882 | - Type-check with `pnpm exec tsrs -b`, run `pnpm exports:check` and |
| 883 | `pnpm lint:providers`, and format with `pnpm exec oxfmt`. |
| 884 | - Open the alchemy PR per **Pull Request Conventions**, linking the |
| 885 | distilled PRs it depends on. The description shows the provider's DX, |
| 886 | not a list of what was built: |
| 887 | - One or two lines on what the package is, then a few bullets of |
| 888 | counts (services, resources, live test files, gated tests). |
| 889 | - A `###` section for every runtime (each form), the event sources, |
| 890 | and the websites. |
| 891 | - A `###` section for each **fat-head** resource: the 10–20 that most |
| 892 | apps on this cloud use (resource groups or projects, identity and |
| 893 | roles, object storage, the main SQL and document databases, queues |
| 894 | and topics, secrets, the container registry, networking and DNS, |
| 895 | logs). Pick them from the cloud's own quickstarts, not from the |
| 896 | catalog's order. Group a resource with the ones it's always used |
| 897 | with (a namespace and its queue) and its bindings. |
| 898 | - Each section is one or two sentences, then one snippet copied from |
| 899 | that resource's JSDoc example, so the code is already checked. |
| 900 | - One closing line names the long tail and points to the generated |
| 901 | reference. Don't list it. |
| 902 | |
| 903 | Don't show a binding running in a host unless a test deploys it that |
| 904 | way. On Azure, the description was going to show bindings inside |
| 905 | `Azure.Functions.Function`, which isn't a registered binding host, so |
| 906 | those bindings did nothing. |
| 907 | |
| 908 | ### Examples |
| 909 | |
| 910 | Examples are how users learn the provider, and the docs hub copies its |
| 911 | code from them. Ship a comprehensive set, not one or two. Azure's first |
| 912 | pass shipped `azure-storage` (a Stack with no runtime) and one website; |
| 913 | AWS, Cloudflare, Fly, and Hetzner ship one example per framework plus one |
| 914 | per workload. |
| 915 | |
| 916 | **Every web framework gets its own example**, for every provider that |
| 917 | hosts websites: `examples/<provider>-website-<framework>`, one per |
| 918 | `<Provider>.Website.<Framework>` composite (Astro, Foldkit, Nextjs, Nuxt, |
| 919 | Octane, ReactRouter, SolidStart, SvelteKit, TanStackStart, Vinext, Vite, |
| 920 | Vocs, Waku, plus `<provider>-static-site`). `ls examples | grep |
| 921 | hetzner-website-` is the complete set to match. |
| 922 | |
| 923 | **Then one example per workload the cloud supports.** Go down this list |
| 924 | and add an example for every row the provider has, and skip the rest: |
| 925 | |
| 926 | | Workload | Shows | AWS / GCP / Cloudflare reference | |
| 927 | | --- | --- | --- | |
| 928 | | Serverless function, effectful | an HTTP handler with a binding to storage or a database | `aws-lambda`, `gcp-cloud-function`, `cloudflare-worker` | |
| 929 | | Serverless function, async | the same with a plain handler and `InferEnv` | `cloudflare-worker-async` | |
| 930 | | HTTP API | Effect HTTP API or RPC on the default runtime | `aws-lambda-httpapi`, `aws-lambda-rpc`, `gcp-cloud-run-api` | |
| 931 | | Queue | a producer and a consumer through an event source | `gcp-pubsub-worker`, `gcp-pubsub-fanout` | |
| 932 | | Scheduled job | a cron trigger | `gcp-cron`, `gcp-scheduled-job` | |
| 933 | | Storage events | a function triggered by an upload | `gcp-storage-uploads`, `gcp-eventarc-firestore` | |
| 934 | | Durable workflows | `alchemy/Workflows` on the provider's engine (Step 5): start, signal, read a run | no example yet; copy the conformance fixtures in `packages/integration-tests/test/Cloudflare/Workers/fixtures/durable-workflow/` | |
| 935 | | Containers | a container service | `aws-ecs`, `fly-service` | |
| 936 | | Kubernetes | a Deployment on the cloud's managed cluster | `aws-eks`, `gcp-gke` | |
| 937 | | VMs | a VM running a program | `aws-ec2`, `hetzner-server` | |
| 938 | | SQL database | a database with migrations, from a runtime | `aws-rds`, `gcp-cloud-sql-drizzle` | |
| 939 | | Cache | Redis from a runtime | `gcp-memorystore-redis`, `fly-redis` | |
| 940 | | Secrets | a secret read by a runtime | `gcp-secrets-kms` | |
| 941 | | AI | the cloud's model service | `aws-bedrock-ai`, `gcp-vertex-ai` | |
| 942 | | Service to service | two runtimes calling each other | `gcp-service-to-service` | |
| 943 | | Local dev | the same app under `alchemy dev` | `aws-dev`, `cloudflare-dev` | |
| 944 | |
| 945 | Name examples `<provider>-<workload>` in kebab case. Each one is a |
| 946 | workspace project with the layout of `examples/gcp-pubsub-worker`: |
| 947 | |
| 948 | - `package.json` (private, `deploy` / `dev` / `destroy` / `test` scripts, |
| 949 | `workspace:*` dependencies), `tsconfig.json` referencing |
| 950 | `packages/alchemy` and the provider package, and a `README.md`. |
| 951 | - `alchemy.run.ts`, with the runtime and resources in `src/`. |
| 952 | - `test/integ.test.ts` with `@alchemy.run/test/Bun`: deploy in `beforeAll`, |
| 953 | call the deployed app over HTTP, verify cloud state through distilled, |
| 954 | destroy in `afterAll`. Websites and `dev`-capable apps also get |
| 955 | `test/dev.test.ts`. |
| 956 | |
| 957 | Register every example in the root `tsconfig.json` references and in |
| 958 | `scripts/test-examples.ts`, and run its test against the testing profile |
| 959 | before you call it done. Track examples as rows in `INDEX.md` like any |
| 960 | other deliverable. |
| 961 | |
| 962 | A draft PR for an early wave is fine; its description states the index |
| 963 | counts and that the provider is incomplete. |
| 964 | |
| 965 | A provider is done when: |
| 966 | |
| 967 | - every SDK service is cataloged and every operation is classified; |
| 968 | - every in-scope resource has its own passing live test (or is |
| 969 | `blocked: cost` / `blocked: skipped` and the user has the list); |
| 970 | - every `binding` operation is a least-privilege binding over distilled, |
| 971 | goes through `<Provider>Bindings.Bind`, reaches foreign hosts, and is |
| 972 | called from a deployed host in a test; |
| 973 | - no alchemy code makes a hand-written API call (review.md's |
| 974 | "Hand-written HTTP" check is clean); |
| 975 | - every runtime ships both forms with `InferEnv`, its event sources, and |
| 976 | tests for each form; |
| 977 | - the cloud's durable engine has an Effect workflow engine that passes |
| 978 | the conformance cases, or is out of scope with a reason; |
| 979 | - the framework websites (if the cloud hosts websites) deploy to its CDN |
| 980 | and functions; |
| 981 | - the example set (one per framework, one per workload) deploys and |
| 982 | passes; |
| 983 | - JSDoc meets the bar in [docs.md](docs.md), the reference docs are |
| 984 | generated, and the hub matches [docs.md](docs.md); |
| 985 | - the PR description shows the runtimes and the fat head; |
| 986 | - the review gate is clean, the test account is clean after a full run, |
| 987 | and every SDK mismatch found along the way is a merged distilled patch. |