@devries-cameron20 / t3code Mirror · read-only
Fork 0

The best way to control Claude Code, Codex and more from any computer or phone

Mirrored from https://github.com/Camo67/t3code.git

Synced Nov 21, 07:15 UTC

main
md 152 lines 5.47 KB
Rendered Raw
1 ---
2 title: 2.0.0-beta.47 - AI Gateway Models
3 date: 2026-06-01T03:00:00Z
4 category: release
5 excerpt: v2.0.0-beta.47 turns a Cloudflare AI Gateway into an Effect `LanguageModel` provider — `gateway.model({...})` returns a layer that slots straight into `generateText` / `streamText` — and fixes streaming methods on the built-in Worker/DO bridge so they can return a `Stream` directly.
6 ---
7
8 `v2.0.0-beta.47` makes a Cloudflare AI Gateway a first-class
9 [Effect AI](/cloudflare/ai/effect-ai) provider, and fixes streaming
10 return values on the built-in Worker/Durable Object method
11 bridge.
12
13 ## AI Gateway as a `LanguageModel` provider
14
15 Every Effect AI provider — OpenAI, Anthropic, Workers AI — ships
16 a `LanguageModel` layer; once it's in scope, `generateText`,
17 `streamText`, tool calls, and structured output are all
18 provider-agnostic. beta.47 makes `Cloudflare.AI.Gateway` one of
19 those providers.
20
21 Bind the gateway in the Init phase, then call `.model({...})`
22 with a Workers AI model id. It returns a `LanguageModel` layer
23 directly — the binding handles auth and the gateway URL, so you
24 pass neither an API key nor a `Layer.unwrap`:
25
26 ```typescript
27 const aiGateway = yield* Cloudflare.AIbind(Gateway);
28
29 const languageModel = aiGateway.model({
30 client: aiGateway,
31 model: "@cf/meta/llama-3.1-8b-instruct",
32 parameters: { temperature: 0.7, maxTokens: 1024 },
33 });
34 ```
35
36 You then provide that layer to your handler, and provide
37 `Cloudflare.AI.GatewayBindingLive` once at the bottom of the Init
38 chain so the binding resolves at runtime:
39
40 ```typescript
41 export default Cloudflare.Worker(
42 "Worker",
43 { main: import.meta.filename },
44 Effect.gen(function* () {
45 const aiGateway = yield* Cloudflare.AIbind(Gateway);
46 const languageModel = aiGateway.model({
47 client: aiGateway,
48 model: "@cf/meta/llama-3.1-8b-instruct",
49 });
50
51 return {
52 fetch: Effect.gen(function* () {
53 const response = yield* LanguageModel.generateText({
54 prompt: "Say hello.",
55 }).pipe(Effect.orDie);
56 return yield* HttpServerResponse.json({ text: response.text });
57 }).pipe(Effect.provide(languageModel)),
58 };
59 }).pipe(Effect.provide(Cloudflare.AI.GatewayBindingLive)),
60 );
61 ```
62
63 Under the hood `makeLanguageModel` translates Effect AI requests
64 into Workers AI `ai.run(...)` calls — SSE parsing for streaming,
65 schema-typed usage reconstruction, tool calls, and structured
66 output all routed through the gateway, so you also get its
67 caching, rate limiting, retries, and unified request log for
68 free.
69
70 The [Effect AI guide](/cloudflare/ai/effect-ai) covers the provider
71 pattern (reading keys with `Config.redacted`, chat persistence),
72 and the [AI Gateway tutorial](/cloudflare/ai/ai-gateway)
73 walks the gateway end to end including streaming.
74 ([#389](https://github.com/alchemy-run/alchemy/pull/389))
75
76 ## Implicit Worker and Durable Object RPC can return a `Stream`
77
78 An Implicit Worker or Durable Object RPC method can now **return a
79 `Stream` directly** — no `Effect<Stream>` wrapper. Value methods
80 stay `Effect`s; the client stub reads each naturally:
81
82 A Durable Object exposing both kinds of method — `get` returns a
83 value, `tick` returns a `Stream`:
84
85 ```diff lang="typescript"
86 // src/counter.ts
87 export class Counter extends Cloudflare.DurableObject<Counter>()(
88 "Counter",
89 Effect.gen(function* () {
90 return Effect.gen(function* () {
91 const state = yield* Cloudflare.DurableObjectState;
92 return {
93 get: () => state.storage.get<number>("n"), // Effect → value
94 tick: (n: number) =>
95 - Effect.succeed(Stream.iterate(0, (i) => i + 1).pipe(Stream.take(n))),
96 + Stream.iterate(0, (i) => i + 1).pipe(Stream.take(n)), // Stream
97 };
98 });
99 }),
100 ) {}
101 ```
102
103 A Worker binds the namespace and pipes the `tick` Stream straight
104 onto the HTTP response:
105
106 ```typescript
107 // src/worker.ts
108 export default class Worker extends Cloudflare.Worker<Worker>()(
109 "Worker",
110 { main: import.meta.filename },
111 Effect.gen(function* () {
112 const counters = yield* Counter;
113 return {
114 fetch: Effect.gen(function* () {
115 const stream = counters
116 .getByName("default")
117 .tick(5)
118 .pipe(Stream.map((i) => `${i}\n`), Stream.encodeText);
119 return HttpServerResponse.stream(stream);
120 }),
121 };
122 }),
123 ) {}
124 ```
125
126 Previously a returned `Stream` was fed into the Effect runtime
127 and crashed with `Fiber.runLoop: Not a valid effect`; the fix
128 lifts it into the success channel in both `WorkerBridge` and
129 `DurableObjectBridge`.
130
131 ([#478](https://github.com/alchemy-run/alchemy/pull/478))
132
133 ## Also in this release
134
135 - **`login` no longer evaluates your stack** — `bun alchemy
136 login` used to eval the stack (which could fail on a missing
137 `Config` or build the state store before auth was configured,
138 bricking the flow). Providers/state are now read statically and
139 state-store init is deferred until actually used
140 ([#492](https://github.com/alchemy-run/alchemy/pull/492)).
141 - **`props.env` works in local Workers** — `Config` values
142 declared via `env` weren't being wired into the local Worker
143 provider under `alchemy dev`. Thanks
144 **[John Royal](https://github.com/jdrydn)**
145 ([#473](https://github.com/alchemy-run/alchemy/pull/473)).
146
147 ## Where to go next
148
149 - [Effect AI guide](/cloudflare/ai/effect-ai)
150 - [AI Gateway tutorial](/cloudflare/ai/ai-gateway)
151 - [CHANGELOG](https://github.com/alchemy-run/alchemy/blob/main/CHANGELOG.md#v200-beta47)
152 - [Compare v2.0.0-beta.46 → v2.0.0-beta.47](https://github.com/alchemy-run/alchemy/compare/v2.0.0-beta.46...v2.0.0-beta.47)