The best way to control Claude Code, Codex and more from any computer or phone

main
mdx 646 lines 20.5 KB
Raw
1 ---
2 title: Custom Provider
3 description: Add support for a new cloud or third-party API by declaring a Resource type and implementing its lifecycle as an Effect Layer.
4 ---
5
6 Providers are Effect `Layer`s — adding support for a new cloud or
7 third-party API is "declare a type, implement a Layer". See
8 [Providers](/infrastructure-as-code/provider) for the operation contract.
9
10 This guide walks through building a Stripe `Product` provider
11 end-to-end: declaring props and attributes, defining the resource
12 type, implementing the lifecycle (`reconcile`, `delete`, `diff`,
13 `read`), bundling it into a `providers()` layer, and writing a
14 test.
15
16 See [Resource lifecycle](/infrastructure-as-code/resource-lifecycle) for the
17 semantics of when each lifecycle method fires.
18
19 ## Declare props and attributes
20
21 Every resource has two sides:
22
23 - **Input properties** — the desired configuration you pass in
24 - **Output attributes** — the values the cloud returns after creation
25
26 Start with two plain TypeScript types. Both are pure data, so
27 they're trivial to share between the provider and call sites.
28
29 Create `src/stripe/Product.ts`:
30
31 ```typescript
32 // src/stripe/Product.ts
33 export interface StripeProductProps {
34 name: string;
35 description?: string;
36 active?: boolean;
37 }
38
39 export interface StripeProductAttributes {
40 productId: string;
41 created: number;
42 }
43 ```
44
45 ## Declare the Resource type
46
47 A `Resource<Type, Props, Attributes>` is a phantom type that ties a
48 string `Type` to its props and attributes. The string `Type` (here
49 `"Stripe.Product"`) is what Alchemy uses to look up the provider at
50 plan time — it must be globally unique.
51
52 ```diff lang="typescript"
53 // src/stripe/Product.ts
54 +import { Resource } from "alchemy";
55 +
56 export interface StripeProductProps {
57 name: string;
58 description?: string;
59 active?: boolean;
60 }
61
62 export interface StripeProductAttributes {
63 productId: string;
64 created: number;
65 }
66 +
67 +export type StripeProduct = Resource<
68 + "Stripe.Product",
69 + StripeProductProps,
70 + StripeProductAttributes
71 +>;
72 ```
73
74 ## Declare the Resource constructor (the "tag")
75
76 `Resource<T>(type)` returns the value users actually call —
77 `StripeProduct("Pro", { ... })`. It also doubles as the **tag**
78 the provider Layer registers itself against, so by convention the
79 type and the value share the same name.
80
81 ```diff lang="typescript"
82 // src/stripe/Product.ts
83 import { Resource } from "alchemy";
84
85 // ... props / attributes / type unchanged ...
86
87 export type StripeProduct = Resource<
88 "Stripe.Product",
89 StripeProductProps,
90 StripeProductAttributes
91 >;
92 +
93 +export const StripeProduct = Resource<StripeProduct>("Stripe.Product");
94 ```
95
96 You can already use this constructor in a stack — but with no
97 provider registered, planning will fail with `Provider not found
98 for Stripe.Product`. Let's fix that.
99
100 ## Scaffold the provider layer
101
102 A provider layer is a `Layer<Provider<R>>` produced by
103 `Provider.succeed(ResourceClass, service)`. The service is an
104 object with `reconcile`, `delete`, and `list` (required) plus
105 optional hooks like `diff`, `read`, `precreate`.
106
107 Start with stubs so the types compile, then fill them in:
108
109 ```diff lang="typescript"
110 // src/stripe/Product.ts
111 -import { Resource } from "alchemy";
112 +import * as Provider from "alchemy/Provider";
113 +import { Resource } from "alchemy";
114 +import * as Effect from "effect/Effect";
115
116 // ... props / attributes / type / constructor unchanged ...
117
118 export const StripeProduct = Resource<StripeProduct>("Stripe.Product");
119 +
120 +export const StripeProductProvider = () =>
121 + Provider.succeed(
122 + StripeProduct,
123 + StripeProduct.Provider.of({
124 + reconcile: () => Effect.die("not implemented"),
125 + delete: () => Effect.die("not implemented"),
126 + list: () => Effect.succeed([]),
127 + }),
128 + );
129 ```
130
131 A few patterns worth knowing:
132
133 - **`Provider.succeed`** wraps a `ProviderService` into a
134 `Layer<Provider<StripeProduct>>`.
135 - **`StripeProduct.Provider.of({...})`** is a typed constructor —
136 it forces every method's input/output to match the resource's
137 props and attributes.
138 - **`Provider.effect`** exists for services that need a dependency
139 resolved once at construction (a logger, `FileSystem`) — never a
140 live one; see the caution below.
141
142 ## Credentials
143
144 Handlers that call an authenticated API resolve a lazy credentials
145 service — nothing touches the network until an operation actually runs.
146 The [Custom Auth Provider](/environments/custom-auth-provider) guide
147 builds the `StripeCredentials` service the snippets below import;
148 providers wrapping unauthenticated APIs skip this entirely.
149
150 ## Implement `reconcile`
151
152 `reconcile` is the single lifecycle method that converges the cloud's
153 actual state to the desired state (`news`). It runs on every apply, so
154 one body must handle three situations, distinguished by `output` and
155 `olds`:
156
157 | `output` | `olds` | Situation |
158 | ----------- | ----------- | -------------------------------------------- |
159 | `undefined` | `undefined` | Greenfield — nothing exists yet |
160 | defined | defined | Routine update |
161 | defined | `undefined` | Adoption — the engine imported it via `read` |
162
163 Every reconciler follows the same shape:
164
165 ```
166 1. Observe — derive the physical id; read live cloud state
167 2. Ensure — if missing, create it; tolerate AlreadyExists/etc.
168 3. Sync — for each mutable aspect: read observed, diff vs
169 desired, apply only the delta
170 4. Return — fresh Attributes
171 ```
172
173 :::warning[Don't branch the body on `output === undefined`]
174 Writing `if (output === undefined) { /* create */ } else { /* update */ }`
175 just renames the old `create`/`update` split. Write one flow that
176 converges from any starting point — trust observed cloud state, not `olds`.
177 :::
178
179 Let's build the Stripe reconciler step by step.
180
181 ### Resolve the client
182
183 ```diff lang="typescript"
184 +import * as Redacted from "effect/Redacted";
185 +import Stripe from "stripe";
186 +import { StripeCredentials } from "./Credentials.ts";
187 +
188 return StripeProduct.Provider.of({
189 - reconcile: () => Effect.die("not implemented"),
190 + reconcile: Effect.fn(function* ({ news, output }) {
191 + const { apiKey } = yield* yield* StripeCredentials;
192 + const stripe = new Stripe(Redacted.value(apiKey));
193 + }),
194 delete: () => Effect.die("not implemented"),
195 });
196 ```
197
198 The double-yield resolves credentials lazily, inside the handler — the
199 provider layer builds without ever touching the auth chain.
200
201 ### Observe
202
203 ```diff lang="typescript"
204 reconcile: Effect.fn(function* ({ news, output }) {
205 const { apiKey } = yield* yield* StripeCredentials;
206 const stripe = new Stripe(Redacted.value(apiKey));
207 +
208 + let product = output?.productId
209 + ? yield* Effect.tryPromise(() =>
210 + stripe.products.retrieve(output.productId),
211 + ).pipe(Effect.catchAll(() => Effect.succeed(undefined)))
212 + : undefined;
213 }),
214 ```
215
216 If a previous run cached a `productId`, look up its live state — a failed
217 `retrieve` just means "missing". `output` is a cache of identifiers, never
218 proof the resource still exists.
219
220 :::tip[Use `Effect.tryPromise`, not `Effect.promise`]
221 `Effect.tryPromise` surfaces SDK errors as Effect failures the lifecycle
222 machinery can report cleanly; `Effect.promise` turns rejections into
223 defects.
224 :::
225
226 ### Ensure
227
228 ```diff lang="typescript"
229 let product = output?.productId
230 ? // ... observe ...
231 : undefined;
232 +
233 + if (!product) {
234 + product = yield* Effect.tryPromise(() =>
235 + stripe.products.create({
236 + name: news.name,
237 + description: news.description,
238 + active: news.active,
239 + }),
240 + );
241 + }
242 }),
243 ```
244
245 Create only when observation came back empty — greenfield creates and
246 "the cached id points at nothing" recover through the same line.
247
248 ### Sync
249
250 ```diff lang="typescript"
251 if (!product) {
252 // ... ensure ...
253 }
254 +
255 + if (
256 + product.name !== news.name ||
257 + product.description !== (news.description ?? null) ||
258 + product.active !== (news.active ?? true)
259 + ) {
260 + product = yield* Effect.tryPromise(() =>
261 + stripe.products.update(product!.id, {
262 + name: news.name,
263 + description: news.description,
264 + active: news.active,
265 + }),
266 + );
267 + }
268 }),
269 ```
270
271 Compare each mutable field of the *observed* product against `news` and
272 write only when something drifted — a no-drift deploy makes zero write
273 calls.
274
275 ### Return fresh Attributes
276
277 ```diff lang="typescript"
278 if (/* drifted */) {
279 // ... sync ...
280 }
281 +
282 + return {
283 + productId: product.id,
284 + created: product.created,
285 + };
286 }),
287 ```
288
289 The returned Attributes are persisted to state and become the next run's
290 `output`.
291
292 ### Why this shape works
293
294 The body is idempotent by construction: Alchemy can retry it after a
295 failed state write and it re-converges. Adoption needs no special case —
296 `retrieve` finds the imported resource and the sync step rewrites whatever
297 drifted from `news`.
298
299 :::note[Bindings]
300 `reconcile` also receives `bindings` — data attached by upstream policies
301 and event sources (IAM grants, queue subscriptions). Read them here if
302 your resource is a binding target; see
303 [Binding](/infrastructure-as-effects/binding).
304 :::
305
306 ## Implement `delete`
307
308 `delete` runs when the resource is removed from code, replaced, or
309 when `alchemy destroy` runs.
310
311 :::tip[Treat 404 as success]
312 Like `reconcile`, `delete` **must be idempotent**. Alchemy may retry
313 it if persistence fails between the cloud call succeeding and state
314 being written. Catch "not found" errors and return `Effect.void`.
315 :::
316
317 ```diff lang="typescript"
318 return StripeProduct.Provider.of({
319 reconcile: Effect.fn(function* ({ news, output }) { /* ... */ }),
320 - delete: () => Effect.die("not implemented"),
321 + delete: Effect.fn(function* ({ output }) {
322 + const { apiKey } = yield* yield* StripeCredentials;
323 + const stripe = new Stripe(Redacted.value(apiKey));
324 + yield* Effect.tryPromise(() =>
325 + stripe.products.del(output.productId),
326 + ).pipe(
327 + Effect.catchAll((cause) =>
328 + cause instanceof Error && cause.message.includes("No such product")
329 + ? Effect.void
330 + : Effect.fail(cause),
331 + ),
332 + );
333 + }),
334 });
335 ```
336
337 ## Implement `diff` (optional)
338
339 Some property changes can't be applied in place. For Stripe
340 products the `name` is mutable but (hypothetically) the
341 `description` is not — changing it requires recreating the
342 product. Implement `diff` to tell Alchemy which kind of change to
343 plan.
344
345 `diff` runs at **plan time**, before `reconcile`, and returns one of:
346
347 - `{ action: "noop" }` — change is trivial, skip reconciling
348 - `{ action: "update", stables?: [...] }` — apply in place
349 - `{ action: "replace", deleteFirst?: boolean }` — destroy and
350 recreate
351 - `undefined` / `void` — fall back to default (plan an in-place
352 update)
353
354 :::note[`diff` runs during planning]
355 Props may still be unresolved `Output`s when `diff` is called —
356 they're only fully resolved at apply time. Always guard with
357 `isResolved(news)` and return `undefined` to defer to the default
358 path when you can't decide.
359 :::
360
361 ```diff lang="typescript"
362 +import { isResolved } from "alchemy/Diff";
363 +
364 // ...
365
366 return StripeProduct.Provider.of({
367 + diff: Effect.fn(function* ({ news, olds }) {
368 + if (!isResolved(news)) return undefined;
369 + if (news.description !== olds.description) {
370 + return { action: "replace" } as const;
371 + }
372 + return undefined;
373 + }),
374 reconcile: Effect.fn(function* ({ news, output }) { /* ... */ }),
375 delete: Effect.fn(function* ({ output }) { /* ... */ }),
376 });
377 ```
378
379 For attributes that are immutable across **all** updates (e.g.
380 the Stripe `productId`, an ARN), declare them in
381 `stables` at the top level:
382
383 ```diff lang="typescript"
384 return StripeProduct.Provider.of({
385 + stables: ["productId"],
386 diff: Effect.fn(function* ({ news, olds }) { /* ... */ }),
387 // ...
388 });
389 ```
390
391 ## Implement `read` (optional, for recovery and adoption)
392
393 The engine calls `read` whenever a resource has no prior state, both
394 to recover from interrupted reconciles *and* to import pre-existing
395 cloud resources into a fresh state store. Returning `undefined` tells
396 Alchemy the resource doesn't exist and should be created.
397
398 ```diff lang="typescript"
399 return StripeProduct.Provider.of({
400 stables: ["productId"],
401 diff: Effect.fn(function* ({ news, olds }) { /* ... */ }),
402 + read: Effect.fn(function* ({ output }) {
403 + if (!output?.productId) return undefined;
404 + const { apiKey } = yield* yield* StripeCredentials;
405 + const stripe = new Stripe(Redacted.value(apiKey));
406 + const product = yield* Effect.tryPromise(() =>
407 + stripe.products.retrieve(output.productId),
408 + ).pipe(
409 + Effect.catchAll(() => Effect.succeed(undefined)),
410 + );
411 + if (!product) return undefined;
412 + return { productId: product.id, created: product.created };
413 + }),
414 reconcile: Effect.fn(function* ({ news, output }) { /* ... */ }),
415 delete: Effect.fn(function* ({ output }) { /* ... */ }),
416 });
417 ```
418
419 This example only finds resources by an ID we previously saved — without
420 a prior `output` it returns `undefined`, and that's a fine default.
421
422 ### Ownership-aware reads
423
424 ```typescript
425 import { Unowned } from "alchemy/AdoptPolicy";
426
427 read: Effect.fn(function* ({ id, olds }) {
428 const live = yield* lookupByName(olds.name);
429 if (!live) return undefined;
430 const attrs = { productId: live.id, created: live.created };
431 // Compare tags/owner against this stack/stage/id.
432 return ownsResource(id, live.tags) ? attrs : Unowned(attrs);
433 }),
434 ```
435
436 If your provider can find an existing resource from props alone (tags,
437 deterministic naming), brand foreign ones with `Unowned` so the engine
438 gates the takeover behind `--adopt`. The full three-way return
439 semantics are in [Providers › read](/infrastructure-as-code/provider#read).
440
441 ## Implement `list`
442
443 `list` enumerates every product in the account, returning the same
444 `Attributes` shape as `read` — it powers account-wide operations
445 like [`alchemy unsafe nuke`](/cli/nuke) and must paginate
446 exhaustively (see [Providers › list](/infrastructure-as-code/provider#list)):
447
448 ```diff lang="typescript"
449 return StripeProduct.Provider.of({
450 stables: ["productId"],
451 - list: () => Effect.succeed([]),
452 + list: Effect.fn(function* () {
453 + const { apiKey } = yield* yield* StripeCredentials;
454 + const stripe = new Stripe(Redacted.value(apiKey));
455 + const products = yield* Effect.tryPromise(() =>
456 + stripe.products.list({ limit: 100 }).autoPagingToArray({ limit: 10_000 }),
457 + );
458 + return products.map((p) => ({ productId: p.id, created: p.created }));
459 + }),
460 // ...
461 });
462 ```
463
464 Resources with no enumeration API (singletons, existence-only
465 resources) keep the `Effect.succeed([])` stub instead.
466
467 ## Bundle into a `providers()` layer
468
469 Users expect the same one-line ergonomics as the built-ins
470 (`Cloudflare.providers()`, `AWS.providers()`). Bundle the resource
471 collection and every provider implementation into a single layer:
472
473 ```typescript
474 // src/stripe/Providers.ts
475 import * as Provider from "alchemy/Provider";
476 import * as Layer from "effect/Layer";
477 import { StripeProduct, StripeProductProvider } from "./Product.ts";
478
479 export class Providers extends Provider.ProviderCollection<Providers>()(
480 "Stripe",
481 ) {}
482
483 export const providers = () =>
484 Layer.effect(
485 Providers,
486 Provider.collection([StripeProduct]),
487 ).pipe(Layer.provide(StripeProductProvider()));
488 ```
489
490 `Layer.provide` wires each resource provider to the collection
491 privately. A provider with no credentials service is done here.
492
493 Ours isn't — the handlers' `StripeCredentials` requirement is still
494 unmet. The [Custom Auth Provider](/environments/custom-auth-provider)
495 guide finishes this bundle by `Layer.provideMerge`-ing in the
496 credential bridge and the auth-provider registration.
497
498 Re-export the public surface, dropping the redundant service prefix
499 (the namespaced-export convention: callers write `Stripe.Product`):
500
501 ```typescript
502 // src/stripe/index.ts
503 export { StripeProduct as Product } from "./Product.ts";
504 export { Providers, providers } from "./Providers.ts";
505 ```
506
507 Either way, users plug your providers in like any built-in — no
508 API key in sight:
509
510 ```typescript
511 // alchemy.run.ts
512 import * as Alchemy from "alchemy";
513 import * as Effect from "effect/Effect";
514 import * as Stripe from "./src/stripe";
515
516 export default Alchemy.Stack(
517 "MyApp",
518 { providers: Stripe.providers(), state: Alchemy.localState() },
519 Effect.gen(function* () {
520 const pro = yield* Stripe.Product("Pro", {
521 name: "Pro plan",
522 description: "Everything in Free, plus...",
523 });
524 return { productId: pro.productId };
525 }),
526 );
527 ```
528
529 Users connect the provider with `alchemy profile edit --add Stripe` (CI
530 reads env vars instead); an unconfigured deploy fails with that exact
531 command to run. See [Auth Providers](/environments/auth-providers).
532
533 To mix with another cloud, merge the layers:
534
535 ```typescript
536 import * as Layer from "effect/Layer";
537
538 providers: Layer.mergeAll(Cloudflare.providers(), Stripe.providers()),
539 ```
540
541 ## Test the lifecycle
542
543 Full `test.provider` reference and patterns: [Testing Providers](/testing/testing-providers).
544
545 Alchemy's test harness (`@alchemy.run/test/Vitest` or `@alchemy.run/test/Bun`)
546 configures providers + state once at the top of the file, then
547 exposes `test.provider(name, (stack) => ...)` for provider-level
548 tests. Each `test.provider` body receives a fresh in-memory scratch
549 stack with `.deploy(effect)` and `.destroy()` helpers.
550
551 :::note[Three things the harness gives you]
552 - **`Test.make({ providers })`** — wires your provider Layer into
553 every test in the file so the resource type resolves.
554 - **`stack.deploy(effect)`** — runs a sub-stack with an in-memory
555 state store. The same logical ID across two `stack.deploy` calls
556 triggers an `update` (or `replace`, depending on `diff`).
557 - **`stack.destroy()`** — replans against an empty desired state to
558 tear everything down. Call at the end of the test.
559 :::
560
561 Create `test/Product.test.ts`:
562
563 ```typescript
564 // test/Product.test.ts
565 import * as Test from "@alchemy.run/test/Vitest";
566 import { expect } from "@effect/vitest";
567 import * as Effect from "effect/Effect";
568 import Stripe from "stripe";
569 import * as StripeProvider from "../src/stripe";
570
571 // Configure providers once per file. Credentials resolve through the
572 // Auth Provider — see /environments/custom-auth-provider.
573 const { test } = Test.make({ providers: StripeProvider.providers() });
574
575 const stripe = new Stripe(process.env.STRIPE_API_KEY!);
576
577 test.provider(
578 "create, update, delete a product",
579 (stack) => Effect.gen(function* () {
580 // Create
581 const created = yield* stack.deploy(
582 Effect.gen(function* () {
583 return yield* StripeProvider.Product("TestProduct", {
584 name: "v1",
585 description: "first version",
586 });
587 }),
588 );
589 expect(created.productId).toBeDefined();
590
591 const live1 = yield* Effect.tryPromise(() =>
592 stripe.products.retrieve(created.productId),
593 );
594 expect(live1.name).toBe("v1");
595
596 // Update (in place)
597 const updated = yield* stack.deploy(
598 Effect.gen(function* () {
599 return yield* StripeProvider.Product("TestProduct", {
600 name: "v2",
601 description: "first version",
602 });
603 }),
604 );
605 expect(updated.productId).toBe(created.productId);
606
607 const live2 = yield* Effect.tryPromise(() =>
608 stripe.products.retrieve(updated.productId),
609 );
610 expect(live2.name).toBe("v2");
611
612 // Destroy
613 yield* stack.destroy();
614 }),
615 );
616 ```
617
618 To verify replacement semantics, change a `stables` field (or the
619 field your `diff` flags as `replace`) and assert that
620 `updated.productId !== created.productId`.
621
622 ## Reference implementations
623
624 If you'd rather start from a real provider:
625
626 - [`Axiom/VirtualField.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/axiom/src/VirtualField.ts) — minimal CRUD with `diff` and `read`
627 - [`Cloudflare/R2/Bucket.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/cloudflare/src/R2/Bucket.ts) — production provider with bindings and replace semantics
628
629 ## Where next
630
631 - [Local Providers](/infrastructure-as-code/local-provider) — give your
632 resource a second, dev-mode implementation that emulates it locally
633 (`ProviderLayer.dual`, `LocalProvider.make`).
634 - [Custom Auth Provider](/environments/custom-auth-provider) — build
635 the `StripeCredentials` service and login flow this guide
636 consumes.
637 - [Auth Providers](/environments/auth-providers) — how credentials
638 resolve: lazy Effects, Profiles, auto-refresh.
639 - [Actions](/infrastructure-as-code/action) — deploy-time work that isn't a
640 resource.
641 - [Testing Providers](/testing/testing-providers) — the harness
642 patterns behind `test.provider`.
643 - [Providers](/infrastructure-as-code/provider) — the operation contract this
644 guide implements.
645 - [Bindings](/infrastructure-as-effects/binding) — connecting Resources
646 to Functions.