The best way to control Claude Code, Codex and more from any computer or phone

main
md 79 lines 3.18 KB
Rendered Raw
1 # gcp-vertex-ai
2
3 A public chat endpoint on Cloud Run that answers with Gemini on Vertex AI —
4 the GCP counterpart of [`aws-bedrock-ai`](../aws-bedrock-ai).
5
6 | Route | Does |
7 | ------------ | -------------------------------------------------------------------- |
8 | `GET /` | health check |
9 | `POST /chat` | `{ prompt, system?, temperature?, maxOutputTokens? }` → `{ text, finishReason, modelVersion, usage }` |
10
11 ## Architecture
12
13 - `src/Chat.ts` — an Effect-native `GCP.Function` (a Cloud Run service) with
14 `main: import.meta.url` and `invokerIamDisabled: true`. Its init phase binds
15 `gemini-2.5-flash` with `GCP.AIPlatform.GenerateContent`; `fetch` calls it.
16 - `alchemy.run.ts` — the stack; outputs the service URL, name, and runtime
17 service account.
18
19 No API key exists anywhere. The container calls Vertex AI as its own runtime
20 service account, using tokens from the Cloud Run metadata server.
21
22 ## Bindings and IAM
23
24 | Binding | Grants |
25 | ---------------------------------------------------------------- | ---------------------------------------------------------------- |
26 | `GCP.AIPlatform.GenerateContent("gemini-2.5-flash")` (`GenerateContentHttp`) | `roles/aiplatform.user` to the service's runtime service account, on the project |
27 | `invokerIamDisabled: true` on the service | no IAM check on invoke — the URL is public |
28
29 Alchemy mints a dedicated runtime service account for the service; the grant
30 goes to that account only and is revoked on destroy. Vertex AI publisher
31 models have no resource-level IAM policy, so the role sits on the project.
32
33 The model id is a plain string because publisher models are not resources
34 you create. Pass `{ model, location }` to pin a region; the default location
35 is `global`:
36
37 ```ts
38 const gemini = yield* GCP.AIPlatform.GenerateContent({
39 model: "gemini-2.5-flash",
40 location: "us-central1",
41 });
42 const answer = yield* gemini.text("Write a haiku about rain.");
43 ```
44
45 ## Deploy
46
47 Credentials come from your alchemy profile: run `alchemy profile` once and pick GCP (*Service account JSON* for a key file, or *Stored* for an access token or key kept in `~/.alchemy/credentials`, plus a default region), then deploy with `--profile <name>`.
48
49 The Vertex AI API (`aiplatform.googleapis.com`) must be enabled on the
50 project, and Docker must be running for the local image build.
51
52 ```sh
53 pnpm deploy --profile <name>
54 ```
55
56 ```sh
57 curl -X POST "$url/chat" -H 'content-type: application/json' \
58 -d '{"prompt":"Write a haiku about Cloud Run"}'
59 ```
60
61 A fresh `roles/aiplatform.user` grant can take a minute or two to reach
62 Vertex AI, during which `/chat` answers `500`.
63
64 ## Test
65
66 ```sh
67 ALCHEMY_PROFILE=<name> bun test
68 ```
69
70 Deploys the stack, checks the project IAM policy for the grant, sends
71 `Reply with exactly: pong` to `/chat` and expects `pong` back, then destroys
72 the stack and verifies the service is gone and the grant revoked. Skipped
73 when Docker is not running.
74
75 ## Destroy
76
77 ```sh
78 pnpm destroy --profile <name>
79 ```