@devries-cameron20 / t3code Mirror · read-only
Fork 0

The best way to control Claude Code, Codex and more from any computer or phone

Mirrored from https://github.com/Camo67/t3code.git

Synced Nov 21, 06:00 UTC

main
mdx 82 lines 2.52 KB
Raw
1 ---
2 title: Storage events
3 description: Run code when an object lands in a Cloud Storage bucket with GCP.Storage.consumeBucketEvents.
4 ---
5
6 `consumeBucketEvents` runs a handler when objects are written to or
7 deleted from a Bucket. Uploads stay fast because the work happens in
8 the background, on another service.
9
10 ## Handle new uploads
11
12 ```typescript
13 // src/Indexer.ts
14 import * as GCP from "@alchemy.run/gcp";
15 import * as Effect from "effect/Effect";
16 import * as HttpServerResponse from "effect/http/HttpServerResponse";
17 import * as Stream from "effect/Stream";
18 import { Uploads } from "./resources.ts";
19
20 export default class Indexer extends GCP.Function<Indexer>()(
21 "Indexer",
22 { main: import.meta.url },
23 Effect.gen(function* () {
24 const bucket = yield* Uploads;
25 const uploads = yield* GCP.Storage.ReadBucket(bucket);
26
27 yield* GCP.Storage.consumeBucketEvents(
28 bucket,
29 { eventTypes: ["OBJECT_FINALIZE"], prefix: "uploads/" },
30 (events) =>
31 events.pipe(
32 Stream.runForEach((event) =>
33 Effect.gen(function* () {
34 const object = yield* uploads.get(event.object, {
35 generation: event.generation,
36 });
37 if (object === undefined) return;
38 yield* Effect.log(`${event.object}: ${object.body.byteLength} bytes`);
39 }),
40 ),
41 Effect.orDie,
42 ),
43 );
44
45 return { fetch: Effect.succeed(HttpServerResponse.text("ok")) };
46 }).pipe(
47 Effect.provide([GCP.Storage.BucketEventSourceLive, GCP.Storage.ReadBucketHttp]),
48 Effect.provide(GCP.Run.TopicEventSource),
49 ),
50 ) {}
51 ```
52
53 `OBJECT_FINALIZE` fires when an object is written. `prefix` limits
54 events to one folder.
55
56 Cloud Storage publishes each event to a Pub/Sub topic, and Pub/Sub
57 pushes it to the service. That's why the service provides
58 `GCP.Run.TopicEventSource` as well.
59
60 ## Read the exact version
61
62 Pass `event.generation` to `get`. If the object was overwritten or
63 deleted since the event, you get `undefined` and a later event owns
64 the work.
65
66 ## Retries
67
68 A failed handler fails the delivery, and Pub/Sub sends the event
69 again. Write results keyed by the object name so a second run
70 overwrites the first.
71
72 ## Example
73
74 [`examples/gcp-storage-uploads`](https://github.com/alchemy-run/alchemy/tree/main/examples/gcp-storage-uploads)
75 uploads through one service and indexes each file into Firestore from
76 another.
77
78 ## Where next
79
80 - [Cloud Storage](/gcp/data/storage) — buckets and their bindings.
81 - [Eventarc](/gcp/messaging/eventarc) — events from other Google
82 services.