The best way to control Claude Code, Codex and more from any computer or phone

main
mdx 439 lines 14.5 KB
Raw
1 ---
2 title: Custom State Store
3 description: Build a Postgres-backed Alchemy state store step by step — implement the StateService interface, plug it into a stack, and test it end-to-end.
4 ---
5
6 A state store is just an Effect `Layer` that provides Alchemy's
7 `State` service. The built-in stores (`localState()` on disk,
8 `Cloudflare.state()` on Cloudflare) cover most cases — but if you
9 already run Postgres, S3, Redis, or DynamoDB, you can back state
10 with that instead.
11
12 This guide builds a Postgres-backed state store from scratch. By the
13 end you'll have a `postgresState()` layer you can drop into any
14 `Alchemy.Stack`, and a test that round-trips a resource through it.
15 (If you just want a Postgres store, use the built-in one from `@alchemy.run/sql/PostgresState`, which adds advisory locking — this guide builds a simplified version to teach the interface.)
16
17 See [State Store](/state-store) for an overview of what
18 state Alchemy persists and why.
19
20 ## Scaffold the layer
21
22 A state store is a `Layer.Layer<State, never, R>` where `R` is any
23 ambient services your implementation needs. The `State` service does
24 **not** hold a `StateService` directly — it holds an
25 `Effect<StateService>` that lazily builds one:
26
27 ```typescript
28 class State extends Context.Service<State, Effect.Effect<StateService>>()(
29 "alchemy/State",
30 ) {}
31 ```
32
33 This indirection matters. Commands like `alchemy profile show` provide the
34 state layer but never touch state, so a store must not connect to its
35 backend (open a DB connection, deploy a worker, prompt for
36 credentials) just because the layer was built. Wrapping the builder
37 in [`Effect.cached`](https://effect.website/docs/getting-started/control-flow/#caching)
38 **defers** initialization until the first `yield* State` and **caches**
39 it so subsequent accesses reuse the same `StateService`.
40
41 Start with an empty `StateService` so the types compile, then fill it
42 in one method at a time.
43
44 Create `src/postgres-state.ts`:
45
46 ```typescript
47 // src/postgres-state.ts
48 import * as Effect from "effect/Effect";
49 import * as Layer from "effect/Layer";
50 import { State, type StateService } from "alchemy/State";
51
52 export interface PostgresStateProps {
53 connectionString: string;
54 }
55
56 export const postgresState = (props: PostgresStateProps) =>
57 // `Effect.cached` returns an `Effect<Effect<StateService>>`; yielding
58 // it inside `Layer.effect` hands `State` the deferred, memoized
59 // initializer. The body below only runs on first `yield* State`.
60 Layer.effect(State, Effect.cached(makePostgresState(props)));
61
62 const makePostgresState = (_props: PostgresStateProps) =>
63 Effect.gen(function* () {
64 const service: StateService = {
65 listStacks: () => Effect.die("not implemented"),
66 listStages: () => Effect.die("not implemented"),
67 list: () => Effect.die("not implemented"),
68 get: () => Effect.die("not implemented"),
69 set: () => Effect.die("not implemented"),
70 delete: () => Effect.die("not implemented"),
71 deleteStack: () => Effect.die("not implemented"),
72 getReplacedResources: () => Effect.die("not implemented"),
73 };
74 return service;
75 });
76 ```
77
78 You now have a layer Alchemy will accept — it just dies on every
79 operation. Next we'll wire up the database.
80
81 ## Connect to Postgres
82
83 We'll use the [`postgres`](https://github.com/porsager/postgres)
84 package and create one schema with a single `state` table keyed by
85 `(stack, stage, fqn)`. Acquire the connection with `Effect.acquireRelease`
86 so it's released when the stack tears down — this requires a `Scope`,
87 so the layer becomes `Layer.scoped`.
88
89 Because initialization is deferred, capture the scope at layer-build
90 time and provide it to the cached initializer, so the connection's
91 release is still tied to the stack's lifetime even though it opens
92 lazily on first use:
93
94 ```diff lang="typescript"
95 // src/postgres-state.ts
96 import * as Effect from "effect/Effect";
97 import * as Layer from "effect/Layer";
98 +import * as Scope from "effect/Scope";
99 +import postgres from "postgres";
100 import { State, type StateService } from "alchemy/State";
101
102 export interface PostgresStateProps {
103 connectionString: string;
104 }
105
106 export const postgresState = (props: PostgresStateProps) =>
107 - Layer.effect(State, Effect.cached(makePostgresState(props)));
108 + Layer.scoped(
109 + State,
110 + Effect.gen(function* () {
111 + // Capture the layer's scope so the deferred initializer can
112 + // register the connection's release on it.
113 + const context = yield* Effect.context<Scope.Scope>();
114 + const make = makePostgresState(props).pipe(
115 + Effect.provideContext(context),
116 + );
117 + return yield* Effect.cached(make);
118 + }),
119 + );
120
121 const makePostgresState = (props: PostgresStateProps) =>
122 Effect.gen(function* () {
123 + const sql = yield* Effect.acquireRelease(
124 + Effect.sync(() => postgres(props.connectionString)),
125 + (sql) => Effect.promise(() => sql.end()),
126 + );
127 +
128 + yield* Effect.promise(
129 + () => sql`
130 + create table if not exists alchemy_state (
131 + stack text not null,
132 + stage text not null,
133 + fqn text not null,
134 + value jsonb not null,
135 + primary key (stack, stage, fqn)
136 + )
137 + `,
138 + );
139 +
140 const service: StateService = {
141 listStacks: () => Effect.die("not implemented"),
142 // ...
143 };
144 return service;
145 });
146 ```
147
148 ## Implement `set` and `get`
149
150 These are the two hot-path methods Alchemy calls during every
151 deploy.
152
153 A few things to know before writing them:
154
155 - **Use `encodeState` / `reviveState`** from `alchemy/State` when
156 serializing. They handle `Secret` values, `Date`s, and other
157 non-JSON types in `ResourceState`.
158 - **Return `undefined` from `get`** for missing rows — don't error.
159 `StateStoreError` is for transport failures only.
160 - **Map thrown errors** through `StateStoreError` so Alchemy can
161 surface them with context.
162
163 ```diff lang="typescript"
164 // src/postgres-state.ts
165 import * as Effect from "effect/Effect";
166 import * as Layer from "effect/Layer";
167 import postgres from "postgres";
168 -import { State, type StateService } from "alchemy/State";
169 +import {
170 + State,
171 + StateStoreError,
172 + encodeState,
173 + reviveState,
174 + type StateService,
175 +} from "alchemy/State";
176
177 // ...
178
179 const makePostgresState = (props: PostgresStateProps) =>
180 Effect.gen(function* () {
181 const sql = yield* Effect.acquireRelease(
182 Effect.sync(() => postgres(props.connectionString)),
183 (sql) => Effect.promise(() => sql.end()),
184 );
185
186 // ... create table ...
187
188 + const run = <A>(thunk: () => Promise<A>) =>
189 + Effect.tryPromise({
190 + try: thunk,
191 + catch: (cause) =>
192 + new StateStoreError({
193 + message: cause instanceof Error ? cause.message : String(cause),
194 + cause: cause instanceof Error ? cause : undefined,
195 + }),
196 + });
197 +
198 const service: StateService = {
199 listStacks: () => Effect.die("not implemented"),
200 listStages: () => Effect.die("not implemented"),
201 list: () => Effect.die("not implemented"),
202 - get: () => Effect.die("not implemented"),
203 - set: () => Effect.die("not implemented"),
204 + get: ({ stack, stage, fqn }) =>
205 + run(
206 + () => sql<{ value: string }[]>`
207 + select value::text from alchemy_state
208 + where stack = ${stack} and stage = ${stage} and fqn = ${fqn}
209 + `,
210 + ).pipe(
211 + Effect.map((rows) =>
212 + rows.length === 0
213 + ? undefined
214 + : JSON.parse(rows[0].value, reviveState),
215 + ),
216 + ),
217 +
218 + set: ({ stack, stage, fqn, value }) =>
219 + run(
220 + () => sql`
221 + insert into alchemy_state (stack, stage, fqn, value)
222 + values (${stack}, ${stage}, ${fqn}, ${sql.json(encodeState(value))})
223 + on conflict (stack, stage, fqn)
224 + do update set value = excluded.value
225 + `,
226 + ).pipe(Effect.as(value)),
227 +
228 delete: () => Effect.die("not implemented"),
229 deleteStack: () => Effect.die("not implemented"),
230 getReplacedResources: () => Effect.die("not implemented"),
231 };
232 return service;
233 });
234 ```
235
236 ## Implement `list`, `listStacks`, `listStages`
237
238 These power `alchemy destroy`, drift detection, and the CLI's
239 inspection commands. They're straightforward `select distinct`
240 queries:
241
242 ```diff lang="typescript"
243 const service: StateService = {
244 - listStacks: () => Effect.die("not implemented"),
245 - listStages: () => Effect.die("not implemented"),
246 - list: () => Effect.die("not implemented"),
247 + listStacks: () =>
248 + run(() => sql<{ stack: string }[]>`
249 + select distinct stack from alchemy_state order by stack
250 + `).pipe(Effect.map((rows) => rows.map((r) => r.stack))),
251 +
252 + listStages: (stack) =>
253 + run(() => sql<{ stage: string }[]>`
254 + select distinct stage from alchemy_state
255 + where stack = ${stack} order by stage
256 + `).pipe(Effect.map((rows) => rows.map((r) => r.stage))),
257 +
258 + list: ({ stack, stage }) =>
259 + run(() => sql<{ fqn: string }[]>`
260 + select fqn from alchemy_state
261 + where stack = ${stack} and stage = ${stage} order by fqn
262 + `).pipe(Effect.map((rows) => rows.map((r) => r.fqn))),
263 +
264 // get / set unchanged ...
265 delete: () => Effect.die("not implemented"),
266 deleteStack: () => Effect.die("not implemented"),
267 getReplacedResources: () => Effect.die("not implemented"),
268 };
269 ```
270
271 ## Implement `delete` and `deleteStack`
272
273 `deleteStack` deletes the entire stack when `stage` is omitted, or
274 just one stage when provided:
275
276 ```diff lang="typescript"
277 const service: StateService = {
278 // listStacks / listStages / list / get / set unchanged ...
279
280 - delete: () => Effect.die("not implemented"),
281 - deleteStack: () => Effect.die("not implemented"),
282 + delete: ({ stack, stage, fqn }) =>
283 + run(() => sql`
284 + delete from alchemy_state
285 + where stack = ${stack} and stage = ${stage} and fqn = ${fqn}
286 + `).pipe(Effect.asVoid),
287 +
288 + deleteStack: ({ stack, stage }) =>
289 + run(() =>
290 + stage === undefined
291 + ? sql`delete from alchemy_state where stack = ${stack}`
292 + : sql`delete from alchemy_state where stack = ${stack} and stage = ${stage}`,
293 + ).pipe(Effect.asVoid),
294 +
295 getReplacedResources: () => Effect.die("not implemented"),
296 };
297 ```
298
299 ## Implement `getReplacedResources`
300
301 When Alchemy replaces a resource, the old one is kept in the store
302 with `status: "replaced"` until its destroy operation succeeds.
303 `getReplacedResources` returns that backlog so the next deploy can
304 finish cleanup.
305
306 You can implement this on top of `list` + `get`, but it's cheaper to
307 push the filter into Postgres:
308
309 ```diff lang="typescript"
310 const service: StateService = {
311 // ...
312
313 - getReplacedResources: () => Effect.die("not implemented"),
314 + getReplacedResources: ({ stack, stage }) =>
315 + run(
316 + () => sql<{ value: string }[]>`
317 + select value::text from alchemy_state
318 + where stack = ${stack}
319 + and stage = ${stage}
320 + and value->>'status' = 'replaced'
321 + `,
322 + ).pipe(
323 + Effect.map((rows) =>
324 + rows.map((r) => JSON.parse(r.value, reviveState)),
325 + ),
326 + ),
327 };
328 ```
329
330 ## Plug it into a stack
331
332 Pass your layer as the `state` option, exactly like the built-in
333 stores:
334
335 ```diff lang="typescript"
336 // alchemy.run.ts
337 import * as Alchemy from "alchemy";
338 import * as Cloudflare from "@alchemy.run/cloudflare";
339 import * as Effect from "effect/Effect";
340 +import { postgresState } from "./src/postgres-state.ts";
341
342 export default Alchemy.Stack(
343 "MyApp",
344 {
345 providers: Cloudflare.providers(),
346 - state: Cloudflare.state(),
347 + state: postgresState({
348 + connectionString: process.env.DATABASE_URL!,
349 + }),
350 },
351 Effect.gen(function* () {
352 // resources...
353 }),
354 );
355 ```
356
357 ## Test the round-trip
358
359 Spin up a throwaway Postgres (Docker, Testcontainers, or a local
360 instance) and verify a resource round-trips through your store. Note
361 the double `yield*`: the first resolves the `State` service to its
362 `Effect<StateService>` initializer, the second runs it to get the
363 `StateService`.
364
365 ```typescript
366 // test/postgres-state.test.ts
367 import { describe, expect, it } from "vitest";
368 import * as Effect from "effect/Effect";
369 import { State } from "alchemy/State";
370 import { postgresState } from "../src/postgres-state.ts";
371
372 const layer = postgresState({
373 connectionString: process.env.TEST_DATABASE_URL!,
374 });
375
376 describe("postgresState", () => {
377 it("round-trips a resource", () =>
378 Effect.gen(function* () {
379 const state = yield* yield* State;
380
381 yield* state.set({
382 stack: "Test",
383 stage: "dev",
384 fqn: "Bucket",
385 value: {
386 id: "Bucket",
387 fqn: "Bucket",
388 status: "created",
389 kind: "Cloudflare.R2.Bucket",
390 props: { name: "my-bucket" },
391 output: { name: "my-bucket" },
392 } as any,
393 });
394
395 const got = yield* state.get({
396 stack: "Test",
397 stage: "dev",
398 fqn: "Bucket",
399 });
400 expect(got?.id).toBe("Bucket");
401
402 const fqns = yield* state.list({ stack: "Test", stage: "dev" });
403 expect(fqns).toContain("Bucket");
404
405 yield* state.deleteStack({ stack: "Test" });
406 const after = yield* state.list({ stack: "Test", stage: "dev" });
407 expect(after).toEqual([]);
408 }).pipe(Effect.provide(layer), Effect.scoped, Effect.runPromise));
409 });
410 ```
411
412 For end-to-end testing of a real stack against your store, see
413 [Testing a Stack](/testing/testing-a-stack).
414
415 ## Things to know
416
417 A few details that aren't obvious from the interface but matter in
418 production:
419
420 - **FQNs are arbitrary strings** containing `/` and other
421 characters. Postgres handles them fine as parameterized values,
422 but if your backend uses FQNs in keys (filenames, S3 keys, Redis
423 keys) you'll need to escape them — Alchemy ships
424 `encodeFqn` / `decodeFqn` from `alchemy/FQN`.
425 - **Concurrent writes happen** during a deploy as Alchemy applies
426 resources in parallel. Your backend needs row-level (or
427 equivalent) consistency on the `(stack, stage, fqn)` key.
428 - **State persists between deploys**, so schema changes to
429 `ResourceState` are effectively migrations. Be conservative with
430 the encoded shape, and use `reviveState` so future Alchemy
431 versions can deserialize what you wrote.
432
433 ## Reference implementations
434
435 If you'd rather start from one of the built-ins:
436
437 - [`LocalState.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/State/LocalState.ts) — filesystem-backed, the simplest end-to-end example
438 - [`HttpStateStore.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/State/HttpStateStore.ts) — HTTP client against the Alchemy state API
439 - [`InMemoryState.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/State/InMemoryState.ts) — minimal in-process map, useful as a template