The best way to control Claude Code, Codex and more from any computer or phone

main
mdx 102 lines 2.96 KB
Raw
1 ---
2 title: Vertex AI
3 description: Call Gemini on Vertex AI from Cloud Run with the GenerateContent binding — no API key, just the service's own identity.
4 ---
5
6 `GCP.AIPlatform.GenerateContent` binds a Gemini model to your service.
7 The service calls Vertex AI as itself, so there's no API key to store
8 or rotate.
9
10 ## Bind a model
11
12 ```typescript
13 // src/Chat.ts
14 import * as GCP from "@alchemy.run/gcp";
15 import * as Effect from "effect/Effect";
16 import { HttpServerRequest } from "effect/http/HttpServerRequest";
17 import * as HttpServerResponse from "effect/http/HttpServerResponse";
18
19 export default class Chat extends GCP.Function<Chat>()(
20 "Chat",
21 { main: import.meta.url },
22 Effect.gen(function* () {
23 const gemini = yield* GCP.AIPlatform.GenerateContent("gemini-2.5-flash");
24
25 return {
26 fetch: Effect.gen(function* () {
27 const request = yield* HttpServerRequest;
28 const { prompt } = (yield* request.json) as { prompt: string };
29
30 const response = yield* gemini.generate({
31 contents: [{ role: "user", parts: [{ text: prompt }] }],
32 });
33
34 const text = (response.candidates?.[0]?.content?.parts ?? [])
35 .filter((part) => !part.thought)
36 .map((part) => part.text ?? "")
37 .join("");
38 return yield* HttpServerResponse.json({ text });
39 }).pipe(Effect.orDie),
40 };
41 }).pipe(Effect.provide(GCP.AIPlatform.GenerateContentHttp)),
42 ) {}
43 ```
44
45 The binding grants the service `roles/aiplatform.user`. Gemini 2.5 can
46 return "thought" parts, so filter them out of the answer.
47
48 ## Pick a region
49
50 The model runs on the `global` endpoint by default, which sends each
51 request wherever there's capacity. Pin a region instead:
52
53 ```typescript
54 const gemini = yield* GCP.AIPlatform.GenerateContent({
55 model: "gemini-2.5-flash",
56 location: "us-central1",
57 });
58 ```
59
60 ## Tune a request
61
62 System instructions and generation settings go on each call:
63
64 ```typescript
65 const response = yield* gemini.generate({
66 contents: [{ role: "user", parts: [{ text: prompt }] }],
67 systemInstruction: { parts: [{ text: "Answer in one sentence." }] },
68 generationConfig: { temperature: 0.2, maxOutputTokens: 256 },
69 });
70 ```
71
72 ## Handle rejected prompts
73
74 A blocked prompt or a bad setting fails with `BadRequest`. Return it to
75 the caller instead of a bare 500:
76
77 ```typescript
78 .pipe(
79 Effect.catchTag("BadRequest", (error) =>
80 HttpServerResponse.json({ error: error.message }, { status: 400 }),
81 ),
82 Effect.orDie,
83 )
84 ```
85
86 ## Keep it private
87
88 A public chat endpoint spends money on anyone's prompts. Leave
89 `invokerIamDisabled` off and call it from your own services with
90 [`InvokeService`](/gcp/networking/private-services).
91
92 ## Example
93
94 [`examples/gcp-vertex-ai`](https://github.com/alchemy-run/alchemy/tree/main/examples/gcp-vertex-ai)
95 is a chat endpoint with a live test.
96
97 ## Where next
98
99 - [Cloud Run](/gcp/compute/cloud-run) — the service the model is bound
100 to.
101 - [AIPlatform reference](/providers/gcp/aiplatform#generatecontent) —
102 every Vertex AI resource and binding.