The best way to control Claude Code, Codex and more from any computer or phone

main
mdx 330 lines 12.2 KB
Raw
1 ---
2 title: State Store
3 description: How Alchemy persists resource state between deploys to compute diffs and track infrastructure.
4 ---
5
6 import StateStoreBootstrap from "../../../components/marketing-islands/StateStoreBootstrap.tsx";
7
8 Alchemy persists resource state between deploys so it can compute
9 diffs — comparing the desired state in your code against the current
10 state of your infrastructure.
11
12 ## How state is stored
13
14 Each resource's state is keyed by its **fully qualified name** (FQN),
15 which includes the namespace path and
16 [logical ID](/infrastructure-as-code/resource#logical-id). State is
17 scoped by [stack name](/infrastructure-as-code/stack) and
18 [**stage**](/environments/stages), so different stacks and
19 environments are fully isolated.
20
21 A resource's persisted state includes:
22
23 - **Resource type** — e.g. `Cloudflare.R2.Bucket`
24 - **Input properties** — the props you passed when creating it
25 - **Output attributes** — the values returned after creation
26 - **Instance ID** — a unique identifier for this instance
27 - **Lifecycle status** — `created`, `updating`, `deleting`, etc.
28 - **[Bindings](/infrastructure-as-effects/binding)** — data attached by policies and event sources
29
30 ## Local state
31
32 By default, state is stored on disk in the `.alchemy/` directory:
33
34 ```
35 .alchemy/
36 state/
37 MyApp/
38 dev_sam/
39 Bucket.json
40 Worker.json
41 ```
42
43 Add `.alchemy/` to your `.gitignore`:
44
45 ```sh
46 echo ".alchemy/" >> .gitignore
47 ```
48
49 Local state works for solo development. Each developer gets their own
50 state via their default stage (`dev_$USER`).
51
52 ## Remote state
53
54 For teams and CI, configure a remote state store so all deploys share
55 the same state. Alchemy includes a Cloudflare-backed store:
56
57 ```typescript
58 Alchemy.Stack(
59 "MyApp",
60 {
61 providers: Cloudflare.providers(),
62 state: Cloudflare.state(),
63 },
64 Effect.gen(function* () {
65 // ...
66 }),
67 );
68 ```
69
70 ## Cloudflare state store
71
72 `Cloudflare.state()` is the recommended state store for projects
73 already running on Cloudflare. It persists state in a Worker backed
74 by a Durable Object with embedded SQLite, with the auth token and
75 encryption key kept in your account's Secrets Store.
76
77 ### First-run bootstrap
78
79 The first time you run `alchemy deploy`, `plan`, or `dev` against a
80 stack configured with `Cloudflare.state()`, Alchemy can't find the
81 state-store Worker on your account and deploys it, once, with its
82 supporting resources:
83
84 <StateStoreBootstrap client:visible />
85
86 What gets created:
87
88 - **`Api`** — the state-store Worker itself (a Durable Object with
89 embedded SQLite, enabled on a `workers.dev` subdomain)
90 - **`AlchemyStateStoreToken`** — bearer token used by the CLI to
91 authenticate to the Worker, stored in your Secrets Store
92 - **`StateStoreEncryptionKey`** — symmetric key used to encrypt
93 resource state at rest inside the Durable Object's SQLite
94 - **`StateStoreSecrets`** — the Secrets Store that holds the two
95 secrets above, scoped to your Cloudflare account
96 - **`StateStoreAuthTokenValue` / `StateStoreEncryptionKeyValue`** —
97 the generated random values bound into the secrets above
98
99 These resources are reused across every stack and stage that uses
100 `Cloudflare.state()` on this account — no duplication per project.
101 Subsequent runs detect the existing Worker and use it.
102
103 In CI (`CI=true`) a missing store fails the run instead: a deploy there
104 should find its store provisioned, not create one as a side effect.
105 Provision it once ahead of time. Without `--backend`, the command
106 provisions the store your `alchemy.run.ts` configures:
107
108 ```sh
109 alchemy state bootstrap --profile <your-ci-profile>
110 ```
111
112 `AWS.state()` behaves the same way: it creates its DynamoDB table on
113 first use, except in CI, where `alchemy state bootstrap --backend aws`
114 provisions it. See [AWS state store](#aws-state-store).
115
116 `GCP.state()` does the same with a Cloud Storage bucket in your project,
117 `alchemy-state-<project>`. In CI, provision it with
118 `alchemy state bootstrap --backend gcp`.
119
120 ### Where credentials live
121
122 After the bootstrap, Alchemy writes the state-store URL and bearer
123 token to a credentials file under your Alchemy profile directory
124 (`~/.alchemy/profiles/<profile>/state-stores/cloudflare.json`). On
125 CI, set `CI=true`; the credentials are resolved from the Secrets
126 Store on every run via a short-lived edge-preview Worker, so nothing
127 needs to be persisted to disk.
128
129 ### More than one store per account
130
131 By default the state-store Worker is named `alchemy-state-store`. Pass
132 `workerName` for a separate store, for example one per team. It gets its own
133 Worker, its own secrets in the Secrets Store, and its own cached credentials:
134
135 ```typescript
136 state: Cloudflare.state({ workerName: "team-a-state" }),
137 ```
138
139 Address it from the CLI with `--name`, which takes the Worker name:
140
141 ```sh
142 alchemy state list --backend cloudflare --name team-a-state
143 ```
144
145 `AWS.state({ name })` does the same with a separate DynamoDB table,
146 `alchemy-state-<name>`; pass `tableName` to choose the full table name
147 yourself. `GCP.state({ name })` uses `alchemy-state-<name>-<project>`.
148
149 ### Pin the account
150
151 The store lives in the Cloudflare account your profile resolves. If your
152 credentials reach several accounts, pin it, so logging in to a different one
153 fails instead of reading or deploying the store there:
154
155 ```typescript
156 state: Cloudflare.state({ accountId: "c8eeff0e..." }),
157 ```
158
159 ### Serve it on your own domain
160
161 The store answers on its `workers.dev` URL. Some networks can't reach
162 `workers.dev`, so give it a hostname in one of your zones instead. Alchemy
163 attaches it to the store's Worker as a custom domain and uses it from then on:
164
165 ```typescript
166 state: Cloudflare.state({ domain: "alchemy-state.example.com" }),
167 ```
168
169 ## AWS state store
170
171 `AWS.state()` keeps state in one on-demand DynamoDB table per account and region, `alchemy-state`. A deploy loads a stage's records with one query instead of reading an object per resource.
172
173 ```typescript
174 Alchemy.Stack(
175 "MyApp",
176 {
177 providers: AWS.providers(),
178 state: AWS.state(),
179 },
180 Effect.gen(function* () {
181 // ...
182 }),
183 );
184 ```
185
186 A DynamoDB item holds at most 400 KB. A record larger than `overflowThreshold` (300 KiB by default) is written to an S3 overflow bucket, `alchemy-state-overflow-<account>-<region>-an`, and its item keeps the object's key.
187
188 The store creates its table, with deletion protection, and the overflow bucket on first use, except in CI. Provision them ahead of time with:
189
190 ```sh
191 alchemy state bootstrap --backend aws --profile <your-ci-profile>
192 ```
193
194 `AWS.state({ name: "team-a" })` uses a separate table, `alchemy-state-team-a`. `kmsKeyId` encrypts the table and the overflow bucket with your own KMS key.
195
196 The credentials that deploy need DynamoDB access to the table and S3 access to the overflow bucket. The first deploy of a stage also lists the old S3 store's bucket, to check the stage isn't still there; `migrateFrom: false` skips that.
197
198 ### Migrating from the S3 store
199
200 Earlier versions kept AWS state in an S3 bucket, `alchemy-state-<account>-<region>-an`. A stage still there fails to deploy until you move it. Run this in the project, so it migrates the store your entrypoint configures:
201
202 ```sh
203 alchemy state migrate --profile <your-profile>
204 ```
205
206 The command moves every stage, or the ones `--stack` and `--stage` select. It closes each moved stage's S3 objects in the bucket policy, so every older alchemy, say in a CI job you haven't upgraded, fails at its first read instead of deploying from the old copy. The S3 objects stay where they are. A stage you create in DynamoDB afterwards is locked in S3 too, so a teammate still on the S3 store can't create it a second time.
207
208 A stage another deploy is running is skipped and reported. Run the command again once that deploy finishes; it picks up where it stopped, including after a crash.
209
210 To go back to an older alchemy, move the stages back first:
211
212 ```sh
213 alchemy state migrate --rollback
214 ```
215
216 Roll back with the version you migrated with, before downgrading. State written by a newer version may use fields an older one can't read.
217
218 ### Keeping state in S3
219
220 `AWS.state({ backend: "s3" })` keeps using the S3 bucket. So does any config that passes `bucketName`, `prefix` or `encryption`, which only the S3 store takes. Bootstrap it with `--backend aws-s3`.
221
222 ## Postgres state store
223
224 `postgresState()` persists state in a Postgres database you already run, and it locks: each `(stack, stage)` holds a session advisory lock for the duration of a run, so two concurrent deploys of the same stage cannot interleave.
225
226 ```typescript
227 import { postgresState } from "@alchemy.run/sql/PostgresState";
228 import * as Config from "effect/Config";
229
230 Alchemy.Stack(
231 "MyApp",
232 {
233 providers: Prisma.providers(),
234 state: postgresState({ url: Config.Redacted("STATE_DATABASE_URL") }),
235 },
236 Effect.gen(function* () {
237 // ...
238 }),
239 );
240 ```
241
242 It is imported from `@alchemy.run/sql/PostgresState` rather than the `State` barrel because it depends on the optional `@effect/sql-pg` driver, which must stay out of bundled worker code. Install `@effect/sql-pg` to use it.
243
244 ## State and lifecycle
245
246 State transitions track the lifecycle of each resource:
247
248 | Status | Meaning |
249 | ----------- | ---------------------------------------------------------------- |
250 | `creating` | Create operation in progress |
251 | `created` | Resource exists and is healthy |
252 | `updating` | Update operation in progress |
253 | `updated` | Resource was updated successfully |
254 | `deleting` | Delete operation in progress |
255 | `replacing` | Resource is being replaced (new one created, old pending delete) |
256
257 These intermediate states (`creating`, `updating`, `deleting`) exist
258 because state is persisted before the cloud operation completes. If
259 the process crashes mid-operation, Alchemy uses the intermediate
260 state to recover on the next deploy.
261
262 ## In-memory state (testing)
263
264 For tests, Alchemy provides an in-memory state store so tests don't
265 touch the filesystem:
266
267 ```typescript
268 import * as TestState from "@alchemy.run/test/TestState";
269
270 // Seed with existing resource state
271 const state = TestState.state({
272 Bucket: {
273 /* ... */
274 },
275 });
276 ```
277
278 The test harness (`@alchemy.run/test/Bun` and `@alchemy.run/test/Vitest`)
279 handles state setup automatically.
280
281 ## Use a store without a stack file
282
283 A provider's store is reachable from your profile alone, so scripts, CI jobs
284 and one-off tools can read and manage state with no `alchemy.run.ts`. From the
285 CLI, pass `--backend` to any `state` command:
286
287 ```sh
288 alchemy state bootstrap --backend cloudflare --profile prod # create or upgrade it
289 alchemy state list --backend cloudflare --profile prod
290 alchemy state logs --backend cloudflare --name team-a --tail
291 ```
292
293 From code, use the `provider` source of [`Alchemist.State`](/cli/api#state):
294
295 ```typescript
296 import * as Alchemist from "alchemy/Alchemist";
297 import * as Effect from "effect/Effect";
298
299 const source = { type: "provider", backend: "cloudflare", profile: "prod" } as const;
300
301 const program = Effect.gen(function* () {
302 yield* Alchemist.State.bootstrap(source);
303 const state = yield* Alchemist.State.store(source);
304 return yield* state.listStacks();
305 }).pipe(Effect.provide(Alchemist.layer()), Effect.scoped);
306 ```
307
308 `Alchemist.State.layer(source)` provides the same store as a `State` layer.
309 The backend is the provider's name: `cloudflare`, `aws` or `gcp`. Pass `name`
310 for a [named store](#more-than-one-store-per-account).
311
312 Only the profile and `name` pick the store this way. Options set in the stack
313 file, such as a [pinned account](#pin-the-account) or a
314 [custom domain](#serve-it-on-your-own-domain), aren't read; use the
315 `configured` source (the CLI default, without `--backend`) when they matter.
316
317 ## Writing your own state store
318
319 A state store is just an Effect `Layer` that provides the `State`
320 service — back it with Postgres, S3, Redis, DynamoDB, or any other
321 backend. See [Custom State Store](/state-store/custom-state-store) for
322 the full walkthrough.
323
324 ## Where next
325
326 - [Custom State Store](/state-store/custom-state-store) — implement the
327 `StateService` interface end-to-end.
328 - [Testing Providers](/testing/testing-providers) — the in-memory store in practice.
329 - [References](/infrastructure-as-code/references) — reading another
330 stack's outputs.