The best way to control Claude Code, Codex and more from any computer or phone

main
md 467 lines 17.3 KB
Rendered Raw
1 ---
2 title: One reconcile, no create vs. update
3 date: 2026-05-04
4 draft: true
5 excerpt: We replaced `create` and `update` with a single `reconcile` function. Both halves were already converging on the same defensive code.
6 ---
7
8 Most IaC tools you've probably used are CRUD-based.
9 CloudFormation, Terraform, Pulumi, SST, and the CDK all
10 model a resource's lifecycle as four distinct operations:
11 **Create**, **Read**, **Update**, **Delete**. When you write
12 a provider, you write four functions, and the engine picks
13 which one to call based on whether prior state exists.
14
15 Kubernetes took a different approach. A controller doesn't
16 "create" or "update" — it *reconciles*. It looks at desired
17 state, looks at actual state, and does whatever it takes to
18 make them match. Same function, every time. But Kubernetes
19 pays for that simplicity with a **continuous control loop**:
20 controllers run forever, polling and reconciling on every
21 tick. There's no `plan` step, no `apply` step, no human in
22 the loop telling it "yes, do that now."
23
24 Alchemy wants both halves. We keep Terraform's *plan and
25 apply* model — you run `alchemy deploy`, the engine computes
26 a diff, shows you what's about to change, and applies it
27 once. But inside that apply step, each resource runs a
28 Kubernetes-style reconcile instead of a CRUD branch. One
29 shot, not a loop; but the function itself doesn't care
30 whether the resource is new, stale, drifted, or adopted —
31 it just makes the cloud match desired state and returns.
32
33 This post is about why we made that switch, and why the
34 providers got noticeably smaller for it.
35
36 ## What `create` and `update` actually look like
37
38 If you've written an SST or CDK custom resource, or a
39 Terraform provider, you've seen the shape. Two functions —
40 `create` provisions from nothing, `update` mutates toward
41 new props.
42
43 Let's build one up for a DynamoDB `Table`, which we'll keep
44 using as the running example. Alchemy is Effect-native, so
45 the lifecycle functions are written as `Effect.fn` generators
46 that `yield*` AWS SDK calls instead of awaiting promises —
47 but the *shape* is the same as any CRUD provider. Start
48 with the skeleton:
49
50 ```typescript
51 {
52 create: Effect.fn(function* ({ id, news }) {
53 // resource doesn't exist. provision it from `news`.
54 }),
55 update: Effect.fn(function* ({ id, news, olds, output }) {
56 // resource exists. mutate it from `olds` toward `news`.
57 }),
58 }
59 ```
60
61 `create` runs once, when the engine has no prior state.
62 `update` runs every time after, with the previous props
63 (`olds`), the new props (`news`), and the attributes the
64 resource returned last time (`output`).
65
66 ### Fill in `create`
67
68 DynamoDB's `CreateTable` call takes the whole table shape
69 in a single round trip — key schema, attributes, billing
70 mode, tags, all of it:
71
72 ```diff lang="typescript"
73 create: Effect.fn(function* ({ id, news }) {
74 + const tableName = yield* createTableName(id, news);
75 + const desiredTags = yield* createTags(id, news.tags);
76 +
77 + yield* dynamodb.createTable({
78 + TableName: tableName,
79 + KeySchema: toKeySchema(news),
80 + AttributeDefinitions: toAttributeDefinitions(news.attributes),
81 + BillingMode: news.billingMode ?? "PAY_PER_REQUEST",
82 + StreamSpecification: news.stream,
83 + Tags: createTagsList(desiredTags),
84 + });
85 +
86 + yield* waitForTableActive(tableName);
87 + return { tableName, tableArn: ... };
88 }),
89 ```
90
91 Notice `Tags` is a field on `CreateTable` itself — there's
92 no follow-up `TagResource` call. AWS designed it so a fresh
93 table comes out of the API fully tagged in one round trip.
94 This is the natural shape of `create`: take props, hand them
95 to the cloud, return attributes.
96
97 ### Fill in `update` — tags
98
99 After the table exists, AWS won't let you change tags via
100 `UpdateTable`. There are separate `TagResource` and
101 `UntagResource` APIs, so `update` has to *diff* old props
102 against new props:
103
104 ```diff lang="typescript"
105 update: Effect.fn(function* ({ id, news, olds, output }) {
106 + const desiredTags = yield* createTags(id, news.tags);
107 + const oldTags = yield* createTags(id, olds.tags);
108 + const { removed, upsert } = diffTags(oldTags, desiredTags);
109 +
110 + if (upsert.length > 0) {
111 + yield* dynamodb.tagResource({
112 + ResourceArn: output.tableArn,
113 + Tags: upsert,
114 + });
115 + }
116 + if (removed.length > 0) {
117 + yield* dynamodb.untagResource({
118 + ResourceArn: output.tableArn,
119 + TagKeys: removed,
120 + });
121 + }
122 }),
123 ```
124
125 This is the line we'll keep coming back to:
126 `diffTags(oldTags, desiredTags)`, where `oldTags` is derived
127 from `olds`. `update` is using `olds` as its baseline —
128 *the props we last wrote down* — rather than what's actually
129 attached to the table in AWS right now. Hold onto that.
130
131 ### Fill in `update` — stream config
132
133 Stream configuration is mutable, but via `UpdateTable`
134 rather than `CreateTable`:
135
136 ```diff lang="typescript"
137 update: Effect.fn(function* ({ id, news, olds, output }) {
138 // ...tag sync above...
139 + if (news.stream !== olds.stream) {
140 + yield* dynamodb.updateTable({
141 + TableName: output.tableName,
142 + StreamSpecification: news.stream,
143 + });
144 + yield* waitForTableActive(output.tableName);
145 + }
146 + return output;
147 }),
148 ```
149
150 Same shape: compare `news` to `olds`, fire the delta API if
151 they differ. Every mutable aspect of the table — tags,
152 stream, TTL, PITR, GSIs — is its own `if (news.X !== olds.X)`
153 branch.
154
155 That's a textbook CRUD provider — `create` is a single-shot
156 bootstrap that consumes every prop at once, `update` is a
157 per-aspect diff between `news` and `olds`. Clean.
158 Reasonable. It falls apart in three places.
159
160 ## Break #1 — adoption
161
162 A user has an existing DynamoDB table called `Orders` in
163 their account. They want to start managing it with Alchemy
164 without re-creating it:
165
166 ```sh
167 alchemy deploy --adopt
168 ```
169
170 The engine has no prior state for `Orders`. Whichever of
171 the two lifecycle functions we call, we end up writing the
172 same defensive code.
173
174 ### If we call `create`
175
176 The first `yield*` hits `dynamodb.createTable(...)`. AWS
177 rejects it with `ResourceInUseException`. So we catch it.
178 Now we're inside the `create` body holding a table we may
179 or may not own, and we need to figure out which:
180
181 ```typescript
182 yield* dynamodb.createTable({ ... }).pipe(
183 Effect.catchTag("ResourceInUseException", () =>
184 Effect.gen(function* () {
185 // It already exists — is it ours?
186 const liveTags = yield* dynamodb.listTagsOfResource({ ResourceArn });
187 if (yield* hasAlchemyTags(id, liveTags)) {
188 return; // ours — silent adopt, fall through to sync
189 }
190 // Foreign — gate on AdoptPolicy
191 const allowed = yield* AdoptPolicy;
192 if (!allowed) return yield* Effect.fail(new OwnedBySomeoneElse({ ... }));
193 }),
194 ),
195 );
196
197 // ...and now what? We passed `BillingMode`, `StreamSpecification`,
198 // `Tags`, GSIs — none of them got applied, because we didn't actually
199 // create the table. We have to read live state and diff every aspect
200 // against desired, applying deltas where they differ.
201 ```
202
203 By the time we're done, `create` contains:
204
205 1. A creation attempt.
206 2. A conflict-recovery branch with an ownership check.
207 3. An `AdoptPolicy` gate.
208 4. A per-aspect sync against live cloud state.
209
210 Steps 2-4 are exactly what `update` would have done.
211
212 ### If we call `update`
213
214 Wait — we *can* route here, sort of. The engine has a `read`
215 lifecycle op precisely for this: when there's no prior state
216 but `--adopt` is set, `read` fetches live attributes and
217 seeds `output` before any lifecycle call. So `output` is
218 populated.
219
220 But `olds` is still `undefined` — there are no "previous
221 props" for a table we just met. And `update`'s tag-sync
222 line was:
223
224 ```typescript
225 const oldTags = yield* createTags(id, olds.tags); // 💥 olds is undefined
226 ```
227
228 The fix is the same shape as in `create`: don't trust
229 `olds`. Read tags from the cloud, diff observed against
230 desired:
231
232 ```typescript
233 const observedTags = yield* dynamodb.listTagsOfResource({ ResourceArn });
234 const { removed, upsert } = diffTags(observedTags, desiredTags);
235 ```
236
237 And every other `if (news.X !== olds.X)` branch in
238 `update` has to be rewritten the same way: read the live
239 value from the cloud, diff observed against desired, fire
240 the delta API.
241
242 Compare that to step 4 of the `create` body above. It's
243 the same code. `create` ended up needing it because the
244 table already existed when it tried to make one; `update`
245 needs it because `olds` is gone. Different reasons,
246 identical implementation.
247
248 ### The point
249
250 Both branches converge on the same shape because **both
251 need to assume nothing and observe everything.** You can't
252 trust `olds` (it may be missing on adoption, or stale after
253 drift). You can't trust that `createTable` will succeed
254 (it'll conflict on adoption). The only thing that's
255 authoritative is what the cloud says right now.
256
257 Once `create` and `update` have both grown an "observe live
258 state, then converge per-aspect" prefix, the split is
259 purely cosmetic. This is the actual reconciler in
260 [`AWS/DynamoDB/Table.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/AWS/DynamoDB/Table.ts),
261 which collapses both into one flow:
262
263 ```typescript
264 let state = yield* readTableState(tableName);
265
266 if (state === undefined) {
267 yield* dynamodb.createTable({ /* ...full props, tags inline... */ }).pipe(
268 // Race: a peer reconciler created the table between our observe and create.
269 Effect.catchTag("ResourceInUseException", () => adoptExistingTable(tableName)),
270 );
271 state = yield* readTableState(tableName);
272 }
273
274 // From here on, every aspect (tags, GSIs, PITR, TTL, stream) is synced
275 // against `state` — the OBSERVED cloud state — not against `olds`.
276 ```
277
278 `create` is now just one step inside the flow ("if it
279 doesn't exist yet, make it"). The rest is sync. Adoption
280 just means `state` was already populated when the function
281 started — no separate code path.
282
283 ## Break #2 — partial creates
284
285 A Cloudflare `Worker` is more than a script. It's a script,
286 plus a workers.dev subdomain toggle, plus any custom
287 domains, plus the binding/migration metadata for Durable
288 Objects. The `create` body has to:
289
290 1. `PUT` the script (`putWorker`)
291 2. Toggle the `workers.dev` subdomain on or off
292 3. Attach each custom domain via `putDomain`
293 4. Persist returned IDs (domain IDs, zone IDs) into state
294
295 Now imagine the process gets `SIGKILL`'d between step 1 and
296 step 3 — laptop lid closes, CI runner gets evicted, network
297 times out. The script is uploaded. The domain isn't. State
298 hasn't been persisted, because we crashed before returning.
299
300 Next deploy, the engine still has no prior state, so it
301 routes to `create` again. `putWorker` runs and succeeds
302 (it's an upsert). Then `putDomain` runs and tries to attach
303 the hostname — except now Cloudflare may already know
304 about it from the partial run, or it may not, or it may be
305 attached to a *different* Worker because someone retried
306 elsewhere. The `create` body wasn't written for any of those
307 cases; it was written assuming a clean account.
308
309 The textbook fix is "make `create` idempotent." But the
310 moment you do that — catch `WorkerAlreadyExists`, query the
311 domain table to see what's attached, conditionally PUT —
312 your `create` body is *already* a reconciler. It's reading
313 observed state and applying deltas. It just happens to be
314 named `create`.
315
316 [`Cloudflare/Workers/Worker.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/Cloudflare/Workers/Worker.ts)
317 makes the reconciler explicit. `putWorker` is a true upsert;
318 `reconcileDomains` queries the live domain list and computes
319 attach/detach deltas. There's no "first time" path. Crash
320 anywhere, re-run, converge.
321
322 ## Break #3 — drift
323
324 Same Worker, fully deployed, state persisted, everyone
325 happy. Between deploys, a teammate opens the Cloudflare
326 dashboard and detaches `api.example.com` because they
327 were debugging something.
328
329 Next deploy, the engine has prior state, so it routes to
330 `update`. `update` compares `news.domain` to `olds.domain`,
331 sees no diff, and skips the domain logic. The cloud is
332 wrong, but `update` thinks everything is fine because
333 **`olds` is not the cloud — it's our cached belief about
334 the cloud.**
335
336 The fix, again, is to query live state instead of trusting
337 `olds`. From the real reconciler:
338
339 ```typescript
340 // Always query the live state of domains attached to *this*
341 // Worker rather than trusting `_previous` from local state.
342 // State may have been wiped, populated by another machine, or
343 // simply be out of date. Without this we PUT domains that are
344 // already registered to this same Worker and Cloudflare
345 // returns a confusing "hostname already in use" error.
346 const liveAll = yield* listDomains({ accountId, service: scriptName });
347 ```
348
349 Once you're querying live state and diffing against it,
350 `olds` has no role left to play. It's at best a hint to skip
351 a no-op API call, never the source of truth.
352
353 ## The split was always a state machine
354
355 In all three breaks, the underlying problem is the same:
356 `create` and `update` form a two-state machine where
357 `update` assumes `create`'s postcondition holds. The cloud
358 doesn't honor that assumption. Adoption skips `create`
359 entirely. Partial creates leave the postcondition unmet.
360 Drift invalidates the postcondition after the fact.
361
362 By the time you've handled all three, `create` reads
363 observed state before doing anything, and `update` reads
364 observed state before doing anything. The two functions
365 have converged on the same shape — just under different
366 names, with different sets of edge cases papered over.
367
368 ## The new shape
369
370 The provider surface is now three operations:
371
372 ```typescript
373 {
374 diff, // replace? update? skip?
375 read, // load actual cloud state into the store
376 reconcile, // converge cloud → desired
377 delete: ...,
378 }
379 ```
380
381 `read` is called first whenever there's no prior state. If
382 it finds a resource, the engine checks the
383 [`AdoptPolicy`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/AdoptPolicy.ts)
384 — set globally with `--adopt`, or per-scope with
385 `.pipe(adopt(true))`. If adoption is allowed (or the
386 resource carries our ownership tags), the engine seeds state
387 from `read`'s output and proceeds. If not, it fails with
388 `OwnedBySomeoneElse` and tells the user how to opt in.
389
390 Then `reconcile` runs. Its contract is one sentence:
391
392 > Given desired state and current state, make the current
393 > state match.
394
395 It receives `output: Attributes | undefined` and
396 `olds: Props | undefined`. All three combinations are valid:
397
398 | `output` | `olds` | meaning |
399 | ----------- | ----------- | -------------------------------- |
400 | `undefined` | `undefined` | greenfield create |
401 | defined | defined | routine update |
402 | defined | `undefined` | adoption — first time we own it |
403
404 The body must work for all three. It's written as one flow:
405
406 ```
407 1. Observe — derive the physical id; read live cloud state
408 2. Ensure — if missing, create it; tolerate AlreadyExists races
409 3. Sync — for each mutable aspect:
410 • read OBSERVED cloud state
411 • compute desired state
412 • diff observed vs. desired
413 • apply the delta (skip the API on no-op)
414 ```
415
416 The forbidden pattern is `if (output === undefined) {
417 /* create body */ } else { /* update body */ }`. That's
418 rename-and-branch — it brings back every false assumption
419 we just deleted.
420
421 ## Plan didn't change
422
423 This is important: the *plan* still talks about create and
424 update. Open [`Plan.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/Plan.ts)
425 and you'll find an `action` field on every node, with the
426 same values as before:
427
428 ```typescript
429 type Action = "create" | "update" | "replace" | "delete" | "noop";
430 ```
431
432 The planner still computes a diff and decides, per resource,
433 which of those actions applies. The terraform-style preview
434 you see in your terminal still says "Plan: 4 to create,
435 2 to update, 1 to replace, 0 to destroy."
436
437 What changed is what happens at *apply* time. Both `create`
438 and `update` actions now route into the same provider
439 function — `reconcile`. The distinction lives in the plan
440 (because users want to know "is this a fresh resource or a
441 modification?"), but it doesn't live in the provider. The
442 provider's job is identical either way: read live state,
443 converge it toward desired.
444
445 In other words, the create/update split was useful as a
446 *user-facing concept* — "tell me what's about to change" —
447 but harmful as a *provider-authoring concept*. We kept the
448 former and deleted the latter.
449
450 ## Where to go next
451
452 This is part 1 of two. Part 2 covers how we actually
453 migrated all 50+ providers from `create` + `update` to
454 `reconcile`: we used a single AI prompt run in parallel
455 across the codebase, with the existing per-resource tests as
456 the ground truth, and the apply/plan integration tests as
457 the safety net for the engine changes themselves.
458
459 - [Part 2 — Migrating 50+ providers in an afternoon](/blog/2026-05-06-reconcile-ai-migration)
460 - [`AdoptPolicy`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/AdoptPolicy.ts)
461 — the engine-side spec for `read` → `Unowned` → adopt/fail routing.
462 - Canonical reconcilers:
463 [`AWS/S3/Bucket.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/AWS/S3/Bucket.ts),
464 [`AWS/Kinesis/Stream.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/AWS/Kinesis/Stream.ts),
465 [`AWS/DynamoDB/Table.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/AWS/DynamoDB/Table.ts).
466 - [PR #179](https://github.com/alchemy-run/alchemy/issues/179)
467 — the migration itself, every provider in one diff.