@devries-cameron20 / t3code Mirror · read-only
Fork 0

The best way to control Claude Code, Codex and more from any computer or phone

Mirrored from https://github.com/Camo67/t3code.git

Synced Nov 21, 06:00 UTC

main
mdx 104 lines 4.6 KB
Raw
1 ---
2 title: AI Gateway setup
3 description: Buy Neon AI Gateway credits and use a model from an Effect Function.
4 ---
5
6 Use Neon AI Gateway from an Effect Function with a model your organization can
7 access. Complete [Setup](/neon/setup) first.
8
9 ## Buy AI Gateway credits
10
11 [Buy AI Gateway credits in the Neon Console](https://console.neon.tech/app/billing)
12 before sending model requests. AI Gateway requires a paid Neon plan; follow
13 [Neon's purchase guide](https://neon.com/docs/ai-gateway/prepaid-credits) for the
14 steps and current requirements.
15
16 Choose a Chat Completions-compatible model from the
17 [model catalog](https://neon.com/docs/ai-gateway/models) that your organization
18 can access. The examples use the supported `aws-us-east-2` region.
19
20 ## Declare the branch gateway
21
22 Inside your stack, using an existing `branch`:
23
24 ```typescript
25 import * as Neon from "@alchemy.run/neon";
26
27 const gateway = yield* Neon.AIGateway("AI", { branch });
28 ```
29
30 This discovers the endpoint without sending an inference request or changing
31 billing. See the [resource reference](/providers/neon/aigateway) for explicit
32 project/branch IDs and an optional credential reference.
33
34 ## Bind an Effect model
35
36 Inside a Function's initialization Effect:
37
38 ```typescript
39 const ai = yield* Neon.QueryAIGateway(gateway);
40 ```
41
42 Provide `Neon.QueryAIGatewayHttp` on the application. It uses the injected
43 credential for a same-branch production Neon Function and manages a tracked
44 `ai_gateway:invoke` credential for a Worker, Lambda, local Function, or cross-branch
45 caller. An explicit credential on `Neon.AIGateway` overrides automatic injection.
46 Every path uses HTTP; Neon does not expose a Cloudflare-style native gateway
47 binding. The deployment management key is never the inference credential.
48
49 ## Verify one small request
50
51 Inside an authenticated request handler, use a Chat Completions model ID your
52 organization can access. The following handler excerpt assumes `modelId` is that
53 configured string:
54
55 ```typescript
56 import * as Effect from "effect/Effect";
57 import { LanguageModel } from "effect/ai";
58
59 const reply = yield* LanguageModel.generateText({
60 prompt: "Reply with hello.",
61 }).pipe(
62 Effect.provide(ai.model({
63 model: modelId,
64 parameters: { maxTokens: 16 },
65 })),
66 );
67 ```
68
69 Call the deployed route once, confirm it returns model text, and check the
70 organization's usage. The request consumes credits; it is not a deployment-time
71 probe. Some models use `maxCompletionTokens` or need a larger reasoning budget;
72 use the catalog's supported settings rather than sending both token-limit fields.
73 Protect the route with authentication and rate limits before exposing it to users.
74
75 After basic inference works, exercise the actual features you need: streamed
76 output, tools or structured results, and cancelling a stream. Endpoint discovery
77 alone does not verify any of these behaviors. See
78 [AI Gateway bindings](/providers/neon/queryaigateway) and the
79 [Effect model layer](/providers/neon/makelanguagemodellayer).
80
81 ## Diagnose access failures
82
83 | Symptom | Check | Action |
84 | --- | --- | --- |
85 | `403` with `ai gateway not enabled for account` | Correct organization, paid-plan access, and account enablement | Resolve access in the Console or with Neon Support before retrying; redeploying the Function does not enable the account |
86 | Model access denied | Approval for that model, not just gateway access | Select an accessible model or request foundation-model access |
87 | Authentication failure | Credential scope and target branch | Use an `ai_gateway:invoke` service credential or the same-branch Function's injected credential, not a management API key |
88 | `429` / `REQUEST_LIMIT_EXCEEDED` | Token rate, daily cap, and credit balance | Inspect rate-limit headers and Billing; distinguish transient rate limiting from a quota requiring operator action |
89 | Incompatible request or model | Chat Completions versus Responses/Anthropic API | Match the model to the adapter or use the appropriate bound SDK URL |
90
91 Preserve request IDs and sanitized errors when contacting
92 [Neon Support](https://neon.com/docs/introduction/support). Exclude bearer tokens,
93 private prompts, and response data from public reports. Neon's
94 [troubleshooting guide](https://neon.com/docs/ai-gateway/troubleshooting) covers
95 current service limits.
96
97 PrivateLink and IP Allow do not cover AI Gateway traffic. A private database
98 connection does not make this model endpoint private.
99
100 ## Related guides
101
102 - [Setup](/neon/setup) — management credentials and feature prerequisites.
103 - [Private networking](/neon/guides/private-networking) — database-only network controls.
104 - [State and recovery](/neon/guides/state-recovery) — retain managed credential state.