@devries-cameron20 / t3code Mirror · read-only
Fork 0

The best way to control Claude Code, Codex and more from any computer or phone

Mirrored from https://github.com/Camo67/t3code.git

Synced Nov 21, 06:00 UTC

main
mdx 242 lines 11.4 KB
Raw
1 ---
2 title: Projection
3 description: How a projection consumes events into a store you own, in order per key, batched and replayable, and why it never sends commands.
4 ---
5
6 import DAG from "../../../components/DAG.astro";
7
8 A projection consumes events into a store you own: a database table, a search index, a warehouse, or a [`Domain.Stream`](/domain/stream). Where a [view](/domain/view) is a read model the platform stores and serves for you, a projection writes wherever you choose and leaves serving it to you.
9
10 It shares the view's routing and ordering, and differs from a [policy](/domain/policy) in one way that shapes everything else: a projection never sends commands. Its only effect is on its own store, so it can always be replayed.
11
12 ## When a view isn't the right store
13
14 A view answers one question about one key, and the model serves it. That covers most screens. Some questions don't fit that shape. An accountant wants every money movement across every account this quarter, filtered by amount and sorted by date. A search box wants to find a customer by any part of their name. A data team wants the whole history in their warehouse. Each needs a store built for its kind of query, a SQL table, a search index, a warehouse, and each is a store you run.
15
16 A projection is how events reach a store like that. It's still a read model in the CQRS sense: built from events, and never the source of truth. Add one to a model that already has history, and it receives that history first, so its store starts complete (see [Consumers added later receive the history](/domain/model#consumers-added-later-receive-the-history)).
17
18 <DAG
19 nodes={[
20 { id: "events", label: "Account events", type: "the record", kind: "event", layer: 0 },
21 { id: "view", label: "AccountSummary", type: "served for you", kind: "view", layer: 1 },
22 { id: "ledger", label: "Ledger", type: "into your store", kind: "projection", layer: 1 },
23 { id: "table", label: "ledger table", type: "query it your way", layer: 2 },
24 ]}
25 edges={[
26 { from: "events", to: "view", dashed: true },
27 { from: "events", to: "ledger", dashed: true },
28 { from: "ledger", to: "table" },
29 ]}
30 caption="The same events feed a view the model serves and a table you query yourself."
31 />
32
33 Reach for a view first. Reach for a projection when the query needs a store a view doesn't give you, or when the data has to leave the model altogether.
34
35 ## Declare what it consumes
36
37 A projection names its sources, the key that partitions it, and the events it consumes. In the example bank, a ledger records every money movement on an account.
38
39 ```typescript title="packages/domain/src/Ledger/Ledger.projection.ts"
40 export class Ledger extends Domain.Projection("Ledger", {
41 from: [Account],
42 key: Account,
43 on: [
44 MoneyDeposited,
45 MoneyWithdrawn,
46 TransferDebited,
47 TransferCredited,
48 TransferRefunded,
49 SettlementReceived,
50 ],
51 }) {}
52 ```
53
54 Add it to the model's `projections`, like a policy.
55
56 <DAG
57 nodes={[
58 { id: "account", label: "Account a-1", type: "MoneyWithdrawn", kind: "aggregate" },
59 { id: "ledger", label: "Ledger a-1", type: "in order", kind: "projection" },
60 { id: "table", label: "ledger table", type: "D1" },
61 ]}
62 edges={[
63 { from: "account", to: "ledger", dashed: true },
64 { from: "ledger", to: "table" },
65 ]}
66 caption="The account commits first. Shortly after, the projection writes the movement to a table outside the model."
67 />
68
69 ## Implement it as a Layer
70
71 The handler lives in a Layer, built once with everything it needs. It always receives a batch: an array of inputs for one key, in order.
72
73 ```typescript title="packages/domain/src/Ledger/Ledger.projection.ts"
74 export const LedgerLive = Ledger.toLayer(
75 Effect.gen(function* () {
76 const store = yield* LedgerStore;
77 return Effect.fn(function* (batch) {
78 yield* store.record(
79 batch.map(({ key, event, envelope }) => ({
80 id: `${envelope.stream}:${envelope.seq}`,
81 accountId: key.id,
82 kind: event._tag,
83 amount: event.amount,
84 balanceAfter: event.balanceAfter,
85 at: DateTime.toEpochMillis(envelope.at),
86 })),
87 );
88 });
89 }),
90 { batch: { size: 100, wait: "1 second" } },
91 );
92 ```
93
94 Each input carries the `key` it was routed to, the `event`, its `envelope`, the `source` it came from, and, for a view source, the view's `state` after the event.
95
96 ## Depend on a store, not a database
97
98 The projection asks for `LedgerStore`, a service you define in the domain package, rather than a D1 binding. The server package provides it with D1; a story provides it with memory.
99
100 ```typescript title="packages/domain/src/Ledger/LedgerStore.ts"
101 export class LedgerStore extends Context.Service<
102 LedgerStore,
103 {
104 readonly record: (rows: ReadonlyArray<LedgerRow>) => Effect.Effect<void, never, RuntimeContext>;
105 readonly forAccount: (
106 accountId: string,
107 ) => Effect.Effect<ReadonlyArray<LedgerRow>, never, RuntimeContext>;
108 }
109 >()("Bank/LedgerStore") {}
110 ```
111
112 Its methods require `RuntimeContext`, because a database binding only exists inside a running Worker. A projection's handler may require it; the platform runs handlers inside the runtime.
113
114 ## Make every write idempotent
115
116 Inputs arrive at least once. If a handler fails halfway, or its host restarts, the same inputs run again, so a write must be safe to repeat. The ledger keys each row by the event it came from, `stream:seq`, and ignores a row it already has.
117
118 ```typescript title="packages/server/src/Ledger/D1LedgerStore.ts"
119 "INSERT INTO ledger (id, account_id, kind, amount, balance_after, at) VALUES (?, ?, ?, ?, ?, ?) ON CONFLICT (id) DO NOTHING",
120 ```
121
122 The platform helps on its side too: every key keeps an input log, and a delivery it has already accepted is dropped on arrival.
123
124 ## Partition it by key
125
126 `key` decides which inputs are handled in order. Inputs for one key are handled one batch at a time, in the order they were accepted; different keys run in parallel.
127
128 <DAG
129 nodes={[
130 { id: "a1", label: "Account a-1", type: "events", kind: "aggregate", layer: 0 },
131 { id: "a2", label: "Account a-2", type: "events", kind: "aggregate", layer: 0 },
132 { id: "p1", label: "Ledger a-1", type: "one at a time", kind: "projection", layer: 1 },
133 { id: "p2", label: "Ledger a-2", type: "one at a time", kind: "projection", layer: 1 },
134 ]}
135 edges={[
136 { from: "a1", to: "p1", dashed: true },
137 { from: "a2", to: "p2", dashed: true },
138 ]}
139 caption="One ordered partition per key. Keys don't wait for each other."
140 />
141
142 A source keyed by the same aggregate routes itself. For any other source, `keyOf` names the key, by event tag or source name, and can send one event to several keys with named routes, exactly as a [view's `keyOf`](/domain/view#route-one-event-to-several-keys) does. Each input then carries its `route`.
143
144 The key can also be a string Schema, such as an org id, when there is no aggregate to partition by. `keyOf` then returns the value, and the handler gets it as `key`.
145
146 ```typescript
147 export class OrgLedger extends Domain.Projection("OrgLedger", {
148 from: [Account],
149 key: OrgId,
150 keyOf: { Account: ({ event }) => event.orgId },
151 on: [MoneyDeposited],
152 }) {}
153 ```
154
155 Within a key, events from one source stay in that source's order. Events from different sources have no order between them: the model is distributed on purpose, and there is no total order to give.
156
157 ## Batch it on its Layer
158
159 Batching changes how a projection runs, not what it means, so it lives on the Layer. `batch: { size, wait }` hands the handler up to `size` inputs of one key at once, waiting up to `wait` for a batch to fill. Without it, each input is handled on its own, as soon as it arrives.
160
161 ```typescript title="packages/domain/src/Ledger/Ledger.projection.ts"
162 { batch: { size: 100, wait: "1 second" } },
163 ```
164
165 The handler's code is the same either way: it always receives an array. Changing the batch size is pure tuning.
166
167 `version` is tuning too. Bump it and the projection runs again over the whole history on the next deploy, for example after its table was lost, or once the handler writes a new shape. Empty or migrate the table first if a replayed write wouldn't overwrite the old row.
168
169 ```typescript title="packages/domain/src/Ledger/Ledger.projection.ts"
170 { batch: { size: 100, wait: "1 second" }, version: 1 },
171 ```
172
173 ## It never sends commands
174
175 A projection's effects stay in its own store. That's what makes it safe to replay: run it again over the same inputs, and the only thing that happens is the same rows are written again.
176
177 A policy is the opposite. It acts on the world, through commands and external systems, so it handles each new event once and never replays. When you need both, split the work: the projection writes the store, and a policy on the same events sends the commands.
178
179 <DAG
180 nodes={[
181 { id: "event", label: "MoneyWithdrawn", type: "Account a-1", kind: "event" },
182 { id: "ledger", label: "Ledger", type: "replayable", kind: "projection" },
183 { id: "table", label: "ledger table", type: "your store" },
184 { id: "review", label: "FraudReview", type: "never replays", kind: "policy" },
185 { id: "freeze", label: "FreezeAccount", type: "acts on the world", kind: "command" },
186 ]}
187 edges={[
188 { from: "event", to: "ledger", dashed: true },
189 { from: "ledger", to: "table" },
190 { from: "event", to: "review", dashed: true },
191 { from: "review", to: "freeze" },
192 ]}
193 caption="The same event, consumed twice: once into a store that can be rebuilt, once into an action that can't be undone."
194 />
195
196 ## Read your own write
197
198 Your store is eventually consistent with the model. A caller who has just sent a command and wants to read the result from your store waits for the projection first.
199
200 ```typescript title="packages/domain/test/ledgerAfterWithdrawal.ts"
201 export const ledgerAfterWithdrawal = Effect.fn(function* (accountId: AccountId) {
202 const accounts = yield* Account;
203 const receipt = yield* accounts.send(accountId, new WithdrawMoney({ amount: 2_000 }));
204 yield* Ledger.caughtUp(receipt);
205 return yield* (yield* LedgerStore).forAccount(accountId);
206 });
207 ```
208
209 `caughtUp` waits until the projection has handled every event in the receipt that it consumes directly from that aggregate.
210
211 ## Serve it yourself
212
213 A projection's store is yours, so its API is too. The example bank serves the ledger from a plain route on its Worker, authorizing the caller itself.
214
215 ```typescript title="packages/server/src/BankWorker.ts"
216 const summary = yield* summaries.query(Account.ref(accountId)).pipe(Effect.orDie);
217 const owner = Option.getOrNull(summary)?.customerId;
218 if (actor.value._tag === "Customer" && owner !== actor.value.customerId) {
219 return HttpServerResponse.empty({ status: 404 });
220 }
221 return yield* HttpServerResponse.json(yield* ledger.forAccount(accountId));
222 ```
223
224 Clients that want to know when to refetch can follow the same events the projection consumed, through the Api.
225
226 To have the model serve a projection's output for you, with live updates and `validate`, write a [stream](/domain/stream) instead.
227
228 ## Test it
229
230 Projections run for real in a [story](/domain/story). Give the story a store you can read back.
231
232 ```typescript title="packages/domain/test/Bank.story.test.ts"
233 const ledger = memoryLedger();
234 yield* ledgerStory(
235 given(a1, opened(), new MoneyDeposited({ amount: 10_000, balanceAfter: 10_000 })),
236 as(sam),
237 when(a1, new WithdrawMoney({ amount: 2_500 })),
238 then(a1, new MoneyWithdrawn({ amount: 2_500, balanceAfter: 7_500 })),
239 ).pipe(Effect.provide(ledger.layer));
240 ```
241
242 Next, [Stream](/domain/stream): a store the model keeps and serves for a projection.