@devries-cameron20 / t3code Mirror · read-only
Fork 0

The best way to control Claude Code, Codex and more from any computer or phone

Mirrored from https://github.com/Camo67/t3code.git

Synced Nov 21, 04:45 UTC

main
mdx 308 lines 9.23 KB
Raw
1 ---
2 title: Private database (Workers VPC)
3 description: Connect Hyperdrive to a database inside a private network — such as Aurora in an AWS VPC — through a Cloudflare Tunnel and a Workers VPC Service, with no public endpoint.
4 ---
5
6 [Hyperdrive](/cloudflare/data/hyperdrive) can reach a database that has no
7 public endpoint. A `cloudflared` connector runs inside the private network
8 and dials out to Cloudflare. A TCP VPC Service names the database host
9 behind it, and Hyperdrive egresses through that service.
10
11 This guide connects a Worker to Aurora PostgreSQL in private AWS subnets.
12 The same three Cloudflare resources work for any private network that can
13 run `cloudflared`. The finished project is in
14 [`examples/cloudflare-hyperdrive-aws-vpc`](https://github.com/alchemy-run/alchemy/tree/main/examples/cloudflare-hyperdrive-aws-vpc).
15
16 ## Add the providers
17
18 The stack needs AWS, Cloudflare, and `Alchemy.Random` for the database
19 password.
20
21 ```typescript
22 // alchemy.run.ts
23 import * as Alchemy from "alchemy";
24 import * as AWS from "@alchemy.run/aws";
25 import * as Cloudflare from "@alchemy.run/cloudflare";
26 import * as Effect from "effect/Effect";
27 import * as Layer from "effect/Layer";
28
29 export default Alchemy.Stack(
30 "CloudflareHyperdriveAwsVpc",
31 {
32 providers: Layer.mergeAll(AWS.providers(), Cloudflare.providers(), Alchemy.RandomProvider()),
33 state: Alchemy.localState(),
34 },
35 Effect.gen(function* () {
36 return {};
37 }),
38 );
39 ```
40
41 ## Create the network
42
43 `AWS.EC2.Network` builds a VPC with public and private subnets. The
44 connector runs in the public subnets so it can dial out; Aurora goes in
45 the private ones.
46
47 ```typescript
48 // src/Database.ts
49 import * as AWS from "@alchemy.run/aws";
50 import * as Effect from "effect/Effect";
51
52 export const Network = AWS.EC2.Network("Network", {
53 cidrBlock: "10.63.0.0/16",
54 availabilityZones: 2,
55 });
56
57 export const ConnectorSecurityGroup = Effect.gen(function* () {
58 const network = yield* Network;
59 return yield* AWS.EC2.SecurityGroup("ConnectorSecurityGroup", {
60 vpcId: network.vpcId,
61 description: "cloudflared connector; no inbound connections",
62 });
63 });
64 ```
65
66 The connector's security group has no inbound rules. The tunnel is an
67 outbound connection.
68
69 ## Create the database
70
71 Aurora only admits the connector's security group. The password is
72 generated once, kept in state, and written to the cluster's admin secret.
73
74 ```typescript
75 // src/Database.ts
76 import * as Alchemy from "alchemy";
77 import * as Output from "alchemy/Output";
78 import * as Redacted from "effect/Redacted";
79
80 export const DB_USER = "app";
81 export const DB_NAME = "app";
82
83 export const DbPassword = Alchemy.Random("DbPassword", { bytes: 24 });
84
85 export const Database = Effect.gen(function* () {
86 const network = yield* Network;
87 const connector = yield* ConnectorSecurityGroup;
88 const password = yield* DbPassword;
89 const dbSecurityGroup = yield* AWS.EC2.SecurityGroup("DatabaseSecurityGroup", {
90 vpcId: network.vpcId,
91 description: "Postgres from the cloudflared connector only",
92 ingress: [
93 { ipProtocol: "tcp", fromPort: 5432, toPort: 5432, referencedGroupId: connector.groupId },
94 ],
95 });
96 const secret = yield* AWS.SecretsManager.Secret("DatabaseSecret", {
97 secretString: Output.map(password.text, (password) =>
98 Redacted.make(JSON.stringify({ username: DB_USER, password: Redacted.value(password) })),
99 ),
100 });
101 return yield* AWS.RDS.Aurora("Database", {
102 databaseName: DB_NAME,
103 engine: "aurora-postgresql",
104 engineVersion: "17.5",
105 subnetIds: network.privateSubnetIds,
106 securityGroupIds: [dbSecurityGroup.groupId],
107 secret: { resource: secret },
108 instance: { dbInstanceClass: "db.serverless", publiclyAccessible: false },
109 });
110 });
111 ```
112
113 ## Create the tunnel
114
115 A Cloudflare Tunnel carries Hyperdrive's traffic into the VPC. It needs no
116 public hostnames.
117
118 ```typescript
119 // src/Connector.ts
120 import * as AWS from "@alchemy.run/aws";
121 import * as Cloudflare from "@alchemy.run/cloudflare";
122 import * as Effect from "effect/Effect";
123 import { ConnectorSecurityGroup, Network } from "./Database.ts";
124
125 export const Tunnel = Cloudflare.Tunnel.Tunnel("Tunnel", {
126 ingress: [{ service: "http_status:404" }],
127 });
128 ```
129
130 ## Run the connector in the VPC
131
132 `cloudflared` runs on ECS Fargate. The tunnel token goes into Secrets
133 Manager and reaches the container as `TUNNEL_TOKEN`.
134
135 ```typescript
136 // src/Connector.ts
137 export const Connector = Effect.gen(function* () {
138 const network = yield* Network;
139 const securityGroup = yield* ConnectorSecurityGroup;
140 const tunnel = yield* Tunnel;
141 const token = yield* AWS.SecretsManager.Secret("TunnelToken", {
142 secretString: tunnel.token,
143 });
144 const cluster = yield* AWS.ECS.Cluster("Cluster", {});
145 return yield* AWS.ECS.Service("Cloudflared", {
146 cluster,
147 image: "cloudflare/cloudflared:latest",
148 command: ["tunnel", "--no-autoupdate", "run"],
149 secrets: { TUNNEL_TOKEN: token.secretArn },
150 vpcId: network.vpcId,
151 subnets: network.publicSubnetIds,
152 securityGroups: [securityGroup.groupId],
153 assignPublicIp: true,
154 });
155 });
156 ```
157
158 ## Expose the database as a VPC Service
159
160 `Cloudflare.Connectivity.DirectoryService` with `type: "tcp"` is a TCP VPC
161 Service. It points at the Aurora writer endpoint through the tunnel.
162
163 ```typescript
164 // src/Hyperdrive.ts
165 import * as Cloudflare from "@alchemy.run/cloudflare";
166 import * as Output from "alchemy/Output";
167 import * as Effect from "effect/Effect";
168 import { Connector, Tunnel } from "./Connector.ts";
169 import { DB_NAME, DB_USER, Database, DbPassword } from "./Database.ts";
170
171 export const AuroraService = Effect.gen(function* () {
172 const tunnel = yield* Tunnel;
173 const connector = yield* Connector;
174 const db = yield* Database;
175 const hostname = Output.map(
176 Output.all(db.cluster.endpoint, db.writer.dbInstanceArn, connector.serviceArn),
177 ([endpoint]) => endpoint!,
178 );
179 return yield* Cloudflare.Connectivity.DirectoryService("AuroraService", {
180 type: "tcp",
181 tcpPort: 5432,
182 appProtocol: "postgresql",
183 host: {
184 hostname,
185 resolverNetwork: { tunnelId: tunnel.tunnelId },
186 },
187 tlsSettings: { certVerificationMode: "disabled" },
188 });
189 });
190 ```
191
192 `cloudflared` resolves the hostname with the VPC's DNS, so the private
193 endpoint works as-is.
194
195 Hyperdrive connects to the database when it is created, so the whole path
196 must be live first. The hostname waits on the writer instance and the
197 connector service, so neither the VPC Service nor Hyperdrive is created
198 until Aurora is accepting connections and `cloudflared` is running.
199
200 Workers VPC verifies the origin certificate by default and only trusts
201 public CAs and Cloudflare Origin CA. Aurora's certificate comes from the
202 Amazon RDS CA, so verification is disabled. The connection is still
203 encrypted with TLS. A database with a publicly trusted certificate can keep
204 the default `verify_full`.
205
206 ## Point Hyperdrive at the service
207
208 A Hyperdrive `origin` with `serviceId` egresses through the VPC Service.
209 The host and port come from the service, so the origin only needs the
210 database credentials.
211
212 ```typescript
213 // src/Hyperdrive.ts
214 export const Hyperdrive = Effect.gen(function* () {
215 const service = yield* AuroraService;
216 const password = yield* DbPassword;
217 return yield* Cloudflare.Hyperdrive.Connection("Hyperdrive", {
218 origin: {
219 scheme: "postgres",
220 serviceId: service.serviceId,
221 database: DB_NAME,
222 user: DB_USER,
223 password: password.text,
224 },
225 });
226 });
227 ```
228
229 A VPC origin can't be combined with `mtls`. TLS is configured on the VPC
230 Service instead.
231
232 ## Query from a Worker
233
234 The Worker sees an ordinary Hyperdrive binding.
235
236 ```typescript
237 // src/worker.ts
238 import type { Hyperdrive } from "@cloudflare/workers-types";
239 import { Client } from "pg";
240
241 export default {
242 async fetch(_request: Request, env: { DB: Hyperdrive }): Promise<Response> {
243 const client = new Client({ connectionString: env.DB.connectionString });
244 try {
245 await client.connect();
246 const result = await client.query("select version()");
247 return Response.json(result.rows[0]);
248 } finally {
249 await client.end().catch(() => {});
250 }
251 },
252 };
253 ```
254
255 An Effect Worker binds it with `Cloudflare.Hyperdrive.Connect(Hyperdrive)`,
256 as shown in the [Hyperdrive guide](/cloudflare/data/hyperdrive#bind-hyperdrive-to-your-worker).
257
258 ## Wire up the stack
259
260 ```diff lang="typescript"
261 // alchemy.run.ts
262 +import { Connector } from "./src/Connector.ts";
263 +import { Hyperdrive } from "./src/Hyperdrive.ts";
264
265 export default Alchemy.Stack(
266 "CloudflareHyperdriveAwsVpc",
267 { /* ... */ },
268 Effect.gen(function* () {
269 - return {};
270 + yield* Connector;
271 + const hd = yield* Hyperdrive;
272 + const worker = yield* Cloudflare.Worker("Api", {
273 + main: "./src/worker.ts",
274 + compatibility: { flags: ["nodejs_compat"] },
275 + env: { DB: hd },
276 + });
277 + return { url: worker.url };
278 }),
279 );
280 ```
281
282 ## Deploy
283
284 ```sh
285 bun alchemy deploy
286 ```
287
288 Open the Worker URL. It returns the Aurora Postgres version, read through
289 Hyperdrive, the VPC Service, and the tunnel.
290
291 ## Local development
292
293 `alchemy dev` can't reach a private network. Set `dev` on the Connection to
294 a database your machine can reach:
295
296 ```typescript
297 yield* Cloudflare.Hyperdrive.Connection("Hyperdrive", {
298 origin: { /* ... */ },
299 dev: {
300 scheme: "postgres",
301 host: "localhost",
302 port: 5432,
303 database: "app",
304 user: "postgres",
305 password: Redacted.make("postgres"),
306 },
307 });
308 ```