The best way to control Claude Code, Codex and more from any computer or phone

main
mdx 216 lines 7.55 KB
Raw
1 ---
2 title: Bedrock & Effect AI
3 description: Drive Effect's LanguageModel service with Amazon Bedrock models from a Lambda Function — IAM-scoped bindings, streaming, and tool calling over the Converse API.
4 sidebar:
5 order: 1
6 ---
7
8 Effect's AI surface gives you a single `LanguageModel` service that
9 every model provider ships as a `Layer`. Once that layer is in scope,
10 the rest of the surface (`generateText`, `streamText`, `Toolkit`,
11 `Chat`, …) is provider-agnostic.
12
13 `AWS.Bedrock.LanguageModel` builds that layer on top of Amazon
14 Bedrock. It speaks Bedrock's [Converse
15 API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html)
16 — the unified messages API that works across all conversational
17 foundation models (Amazon Nova, Anthropic Claude, Meta Llama,
18 Mistral, …) — so one binding covers every model on Bedrock. Requests
19 are signed with the Function's IAM role: there is no API key to
20 provision, store, or rotate.
21
22 ## Bind a model
23
24 `yield* AWS.Bedrock.LanguageModel(model)` in the Function's constructor
25 phase returns a `LanguageModel` `Layer` and grants the Function
26 `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream`
27 scoped to exactly that model:
28
29 ```typescript
30 // src/Api.ts
31 import * as AWS from "@alchemy.run/aws";
32 import * as Effect from "effect/Effect";
33 import { LanguageModel } from "effect/ai";
34 import * as HttpServerResponse from "effect/http/HttpServerResponse";
35
36 export default class Api extends AWS.Lambda.Function<Api>()(
37 "Api",
38 { main: import.meta.url, functionUrl: true },
39 Effect.gen(function* () {
40 const model = yield* AWS.Bedrock.LanguageModel(
41 "us.amazon.nova-micro-v1:0",
42 { parameters: { maxTokens: 1024, temperature: 0.7 } },
43 );
44
45 return {
46 fetch: Effect.gen(function* () {
47 const response = yield* LanguageModel.generateText({
48 prompt: "Say hello.",
49 }).pipe(Effect.orDie);
50 return yield* HttpServerResponse.json({ text: response.text });
51 }).pipe(Effect.provide(model)),
52 };
53 }).pipe(Effect.provide(AWS.Bedrock.LanguageModelHttp)),
54 ) {}
55 ```
56
57 `AWS.Bedrock.LanguageModelHttp` is the binding's implementation
58 layer. Provide it once at the bottom of the constructor's layer chain — it composes
59 the `Converse` and `ConverseStream` bindings, so it is bundled into
60 the Lambda and signs requests with the invocation role's credentials.
61
62 A model reference may be a foundation-model id
63 (`anthropic.claude-sonnet-4-20250514-v1:0`), a cross-region inference
64 profile id (`us.amazon.nova-micro-v1:0`), or a full Bedrock ARN.
65 Model access is an account entitlement — enable the model in the
66 Bedrock console (Model access) before invoking, otherwise calls fail
67 with `AccessDeniedException`. Many newer models are only invocable
68 through their cross-region inference profile id, not the bare
69 foundation-model id.
70
71 ## Stream text
72
73 `streamText` maps Bedrock's `ConverseStream` events onto Effect AI's
74 stream parts (`text-start`, `text-delta`, …, `finish`), one part per
75 Bedrock event:
76
77 ```typescript
78 import * as Stream from "effect/Stream";
79
80 const parts = LanguageModel.streamText({
81 prompt: "Write a haiku about infrastructure.",
82 }).pipe(Stream.provide(model));
83
84 yield* parts.pipe(
85 Stream.runForEach((part) =>
86 part.type === "text-delta"
87 ? Effect.sync(() => process.stdout.write(part.delta))
88 : Effect.void,
89 ),
90 );
91 ```
92
93 The terminal `finish` part carries the mapped stop reason
94 (`end_turn` → `stop`, `tool_use` → `tool-calls`, `max_tokens` →
95 `length`, …) and real token usage from the stream's `metadata` event,
96 including prompt-cache read/write counts.
97
98 ## Call tools
99
100 Effect AI toolkits work unchanged — tool schemas are translated to
101 Converse `toolConfig` and Bedrock's structured tool-use blocks are
102 decoded back into typed tool calls and results:
103
104 ```typescript
105 import * as Schema from "effect/Schema";
106 import { Tool, Toolkit } from "effect/ai";
107
108 const GetWeather = Tool.make("get_weather", {
109 description: "Get the current weather for a city.",
110 parameters: Schema.Struct({ city: Schema.String }),
111 success: Schema.Struct({ temperatureF: Schema.Number }),
112 });
113 const WeatherToolkit = Toolkit.make(GetWeather);
114
115 const response = yield* LanguageModel.generateText({
116 prompt: "What's the weather in Seattle?",
117 toolkit: WeatherToolkit,
118 }).pipe(
119 Effect.provide(
120 WeatherToolkit.toLayer({
121 get_weather: ({ city }) => Effect.succeed({ temperatureF: 72 }),
122 }),
123 ),
124 Effect.provide(model),
125 );
126
127 response.toolResults; // typed results, executed by your handlers
128 ```
129
130 ## Configure parameters at runtime
131
132 Binding is the IAM boundary — *which models* the Function may call
133 is fixed at deploy time. Everything else is a runtime decision. The
134 `parameters` passed to the binding are only defaults; scope per-call
135 overrides onto any call with `AWS.Bedrock.withModelParameters`:
136
137 ```typescript
138 const response = yield* LanguageModel.generateText({ prompt }).pipe(
139 AWS.Bedrock.withModelParameters({ temperature: 0, maxTokens: 64 }),
140 );
141 ```
142
143 It works on streams the same way:
144
145 ```typescript
146 const parts = LanguageModel.streamText({ prompt }).pipe(
147 Stream.provide(model),
148 AWS.Bedrock.withModelParameters({ temperature: 0.9 }),
149 );
150 ```
151
152 The override is fiber-scoped (a `Context.Reference`), so it applies
153 to exactly the calls inside the piped region — concurrent requests
154 with different settings never interfere. Defined fields win over the
155 binding's defaults; omitted fields fall through.
156
157 `parameters` covers the Converse API's portable `inferenceConfig`
158 (`maxTokens`, `temperature`, `topP`, `stopSequences`). Anything
159 model-specific passes through `additionalModelRequestFields`
160 verbatim — e.g. `{ top_k: 50 }` for Anthropic models.
161
162 ## Multiple models
163
164 Bind a list of models to grant IAM for all of them in one layer —
165 the first is the default, and `modelId` picks per call:
166
167 ```typescript
168 // Construction: IAM for both models, Nova Micro is the default
169 const model = yield* AWS.Bedrock.LanguageModel([
170 "us.amazon.nova-micro-v1:0",
171 "us.anthropic.claude-sonnet-4-20250514-v1:0",
172 ]);
173
174 // runtime: route this request to Claude, tuned for this call
175 const response = yield* LanguageModel.generateText({ prompt }).pipe(
176 AWS.Bedrock.withModelParameters({
177 modelId: "us.anthropic.claude-sonnet-4-20250514-v1:0",
178 maxTokens: 2048,
179 }),
180 );
181 ```
182
183 Separate `LanguageModel(...)` calls also work — each is its own
184 layer with its own grant — when you'd rather select a model by
185 providing a different layer per route.
186
187 ## Why not an OpenAI-compatible endpoint?
188
189 Bedrock also exposes an OpenAI-compatible Chat Completions endpoint,
190 but it authenticates with long-lived Bedrock API keys and only covers
191 a subset of models. The Converse-backed binding uses the Function's
192 IAM role (SigV4) with a least-privilege grant scoped to the bound
193 model, works with every conversational model on Bedrock, and needs no
194 secret material at all.
195
196 ## Lower-level access
197
198 The `LanguageModel` binding is built on the raw Converse bindings,
199 which remain available when you want the wire-level API:
200
201 - [`AWS.Bedrock.Converse`](/providers/aws/bedrock/converse) — single-shot
202 messages API
203 - [`AWS.Bedrock.ConverseStream`](/providers/aws/bedrock/conversestream) —
204 streaming event API
205 - [`AWS.Bedrock.InvokeModel`](/providers/aws/bedrock/invokemodel) — raw
206 model-native payloads
207
208 ## Where to go next
209
210 - [Effect AI documentation](https://effect.website/docs/ai/introduction)
211 — the API surface (`generateText`, `streamText`, `Toolkit`,
212 structured outputs).
213 - [Lambda](/aws/compute/lambda) — the Function runtime this binding
214 attaches to.
215 - [Effect AI on Cloudflare](/cloudflare/ai/effect-ai) — the same
216 `LanguageModel` programs running on Workers AI.