@devries-cameron20 / t3code Mirror · read-only
Fork 0

The best way to control Claude Code, Codex and more from any computer or phone

Mirrored from https://github.com/Camo67/t3code.git

Synced Nov 21, 07:15 UTC

main
md 373 lines 16.8 KB
Rendered Raw
1 ---
2 title: Benchmarking Cloudflare Containers vs AWS MicroVMs
3 date: 2026-07-01
4 excerpt: We benchmarked AWS Lambda MicroVMs against Cloudflare Containers — 700 isolated cold boots, timed to first successful request. Every MicroVM boots in 2 to 4.5 seconds regardless of what's inside, because the workload's startup is paid once at image build and snapshotted. Opencode on Cloudflare Containers takes 10.5 seconds at the median, and the worst boots push a minute.
5 ---
6
7 We benchmarked cold starts for AWS Lambda MicroVMs and
8 Cloudflare Containers: boot a fresh instance, wait until it
9 answers a real request, record the time, tear it down, repeat.
10 700 boots across 16 variants.
11
12 The result, if latency is what matters to you: **use AWS
13 MicroVMs for your sandboxes even if everything else you run is
14 on Cloudflare**. In this benchmark, booting a MicroVM *from a
15 Cloudflare Worker* — cross-cloud, over the public AWS API — was
16 faster and far more predictable than Cloudflare's own
17 containers on Cloudflare's own platform. And with Alchemy's
18 bindings, wiring a MicroVM into a Worker or Durable Object is a
19 few lines of code — we'll show it below.
20
21 We ran [opencode](https://opencode.ai) as one of the workloads
22 because it's a popular coding agent that runs as a local
23 process, weighs about 100 MB, and has real startup latency —
24 exactly the kind of thing you'd host in a per-user sandbox. A
25 boot counts as ready when the server answers its health check
26 *and* creates a real session. Here's how the distributions
27 compare, opencode and hello world on both platforms:
28
29 ![Time-to-usable-service distributions — the hello-world container peaks at 2 seconds, the MicroVM variants are tall narrow spikes at 3–4 seconds, and the opencode container is a wide hump at 10 seconds](/blog/microvms/opencode-density.png)
30
31 And the same data as individual boots — this view is what the
32 density curves smooth over, the container outliers at 20, 40,
33 nearly 60 seconds:
34
35 ![Every boot — the container variants scatter out to 20, 40, nearly 60 seconds; the MicroVM variants never leave their cluster](/blog/microvms/opencode.png)
36
37 | | p50 | p95 | max |
38 | --- | --- | --- | --- |
39 | Cloudflare Container — opencode (via Worker) | 10.5s | 15.6s | 58.6s |
40 | AWS MicroVM — opencode (via Lambda) | 3.4s | 4.1s | 4.5s |
41 | AWS MicroVM — opencode (via Worker) | 3.6s | 4.0s | 4.2s |
42 | Cloudflare Container — hello world (via Worker) | 1.7s | 23.9s | 47.3s |
43 | AWS MicroVM — hello world (via Lambda) | 2.7s | 3.2s | 3.7s |
44 | AWS MicroVM — hello world (via Worker) | 3.1s | 3.8s | 5.4s |
45
46 What's remarkable in that table is AWS's consistency. Look at
47 the two workloads on MicroVMs: hello world is 2.7 seconds at
48 the median, opencode is 3.4. A 100 MB coding agent with real
49 startup work costs 0.7 seconds more than a 15-line server —
50 because on AWS, *everything* goes through the same path. When
51 AWS builds a MicroVM image, it doesn't just build the
52 Dockerfile — it **runs the entrypoint, waits for the
53 application to initialize, and takes a Firecracker snapshot of
54 the running memory and disk**
55 ([launch post](https://aws.amazon.com/blogs/aws/run-isolated-sandboxes-with-full-lifecycle-control-aws-lambda-introduces-microvms/)).
56 Every `RunMicrovm` resumes from that snapshot, so the workload's
57 startup is paid once, at build time. The flip side is that a
58 lighter image doesn't boot faster — it goes through the same
59 snapshot restore. Boot time on AWS is a property of the
60 platform, not the workload: 2 to 4.5 seconds, whatever's inside.
61
62 Cloudflare is the opposite: boot time is a property of the
63 workload, plus a tail you can't control. For small, light
64 containers that edge is real — the hello-world container is
65 ready in 1.7 seconds at the median, ahead of anything on AWS.
66 But a container cold start boots the image from scratch and
67 runs the entrypoint, so every single container pays opencode's
68 full startup: the fastest opencode container in the run took
69 4.4 seconds and two-thirds took over 10. And the long tail has
70 nothing to do with the workload — the hello-world container
71 shows the same outliers in the scatter plot as the opencode
72 one, with about one boot in ten over 10 seconds and the worst
73 near 50. Yes, a Cloudflare container can be faster; it can also
74 be 30–40 seconds slower, and you can't predict which you'll get.
75
76 For anything a person actually watches, AWS's consistency is
77 the better trade. You never want a user staring at a spinner
78 for 10–60 seconds when they could wait 2–4.5, every time. A
79 predictable 3 seconds is something you can design an experience
80 around; an unpredictable 1-to-60 is not. Not one of the 49
81 successful opencode MicroVM boots exceeded 4.5 seconds.
82
83 The "via Worker" MicroVM rows are the proof of the claim at the
84 top: those MicroVMs were booted **from a Cloudflare Worker**,
85 and they beat Cloudflare's own containers whenever the workload
86 is heavy enough that startup matters. Crossing clouds costs
87 nothing measurable — how that wiring works is covered below.
88
89 The entire opencode image is this:
90
91 ```dockerfile
92 FROM public.ecr.aws/lambda/microvms:al2023-minimal
93 RUN curl -fsSL https://opencode.ai/install | bash
94 WORKDIR /workspace
95 ENTRYPOINT ["opencode", "serve", "--hostname", "0.0.0.0", "--port", "8080"]
96 ```
97
98 The snapshot behavior isn't configured — it's what building a
99 MicroVM image means.
100
101 ## The same shape without the workload
102
103 Strip the workload away and the platforms keep the same
104 characters. These are the hello-world variants — a 15-line
105 `Bun.serve` on both platforms:
106
107 ![Every hello-world boot in the run — MicroVMs cluster between 2 and 4 seconds, containers scatter out past 45 seconds](/blog/microvms/every-boot.png)
108
109 Cloudflare Containers are fast at the median — around 2
110 seconds — but the samples scatter out to 20, 40, nearly 50
111 seconds. AWS MicroVMs cluster between 2 and 4 seconds. The
112 distribution view makes the trade explicit:
113
114 ![CDF of time to usable service — container curves cross the MicroVM curves before the 90th percentile](/blog/microvms/cdf.png)
115
116 The container curves race ahead early and then go flat. Flat is
117 the problem: 73–90% of boots are done by 3 seconds, and then the
118 curve stalls — the remaining 10–27% of your users are waiting
119 10, 20, 40+ seconds. Every point in that flat stretch is a
120 person staring at a spinner.
121
122 The MicroVM curves are the opposite shape: they start later but
123 go straight up. A vertical line means every boot takes the same
124 amount of time — the median user and the unluckiest user get
125 nearly the same experience. That's the property you can build a
126 product on: you know what to promise, and the promise holds for
127 the last user as well as the first.
128
129 | | p50 | p95 | max |
130 | --- | --- | --- | --- |
131 | Cloudflare Container (plain Bun image) | 1.7s | 23.9s | 47.3s |
132 | Cloudflare Container (Effect image) | 2.3s | 24.0s | 46.7s |
133 | Cloudflare Container (public echo image) | 2.6s | 24.4s | 26.1s |
134 | AWS MicroVM (plain Bun image) | 2.7s | 3.2s | 3.7s |
135 | AWS MicroVM (Effect image, Bun) | 2.9s | 4.4s | 4.8s |
136 | AWS MicroVM (Effect image, Node) | 3.2s | 3.7s | 16.6s |
137 | AWS MicroVM (booted from a Cloudflare Worker) | 2.7s | 3.7s | 4.1s |
138
139 (The full 16-variant matrix — a Python image, the Node
140 baselines, the remaining Worker-driven rows — lives in the
141 [full report](https://github.com/alchemy-run/alchemy/tree/main/benchmark/container).)
142
143 Which shape you want depends on what you're building. If a slow
144 boot retries invisibly in the background, take the faster
145 median. If a person is watching — an agent sandbox spinning up,
146 a preview environment, a REPL — the tail is the experience, and
147 you can't fix it with a spinner.
148
149 ## What we timed
150
151 One metric: **time to usable service**. Start the clock, ask the
152 platform for a new instance, and stop the clock at the first
153 successful application-level response. For the hello-world
154 variants that's an RPC round-trip into the running program; for
155 opencode it's the health check answering `{"healthy":true}`
156 plus a `POST /session` returning a real session — a write
157 through the application, proving the resumed process actually
158 works, not merely that something is listening on the port.
159 Whatever the platform does in between (pull the image, allocate,
160 boot the kernel, start or restore the runtime, open the port) is
161 inside the measurement, because it's inside the user's wait.
162
163 Every sample is a genuine cold start: each boot uses a fresh
164 instance and the instance is terminated before the next round.
165 One untimed warm-up boot runs first, so the one-time cost of
166 distributing a freshly pushed image isn't mixed into the numbers.
167
168 ## The images under test
169
170 The benchmark is an ordinary Alchemy app
171 ([`benchmark/container`](https://github.com/alchemy-run/alchemy/tree/main/benchmark/container)).
172 Each variant is an image resource, and the variants are chosen
173 to isolate one question each.
174
175 The plain-Bun baseline answers "how fast can this platform
176 possibly boot a JS server?" It's a Dockerfile and a
177 15-line `Bun.serve` script:
178
179 ```dockerfile
180 FROM public.ecr.aws/lambda/microvms:al2023-minimal
181 RUN curl -fsSL https://bun.sh/install | bash
182 WORKDIR /app
183 COPY server.js .
184 ENTRYPOINT ["bun", "/app/server.js"]
185 ```
186
187 The Effect image answers "what does Alchemy's abstraction cost
188 on top of that?" Instead of a Dockerfile and a server script,
189 you write an Effect program and Alchemy bundles it into the
190 image — the class is the infrastructure declaration *and* the
191 typed client:
192
193 ```typescript
194 export class EffectfulBun extends AWS.Lambda.MicrovmImage<
195 EffectfulBun,
196 {
197 hello: (message: string) => Effect.Effect<string>;
198 }
199 >()("MicrovmEffectfulBun") {}
200
201 export default EffectfulBun.make(
202 { main: import.meta.filename, runtime: "bun" },
203 Effect.gen(function* () {
204 return {
205 fetch: Effect.succeed(HttpServerResponse.text("ok")),
206 hello: (message) => Effect.succeed(`hello, ${message}!`),
207 };
208 }),
209 );
210 ```
211
212 Deploying that builds a MicroVM image whose entrypoint is the
213 bundled program. Anything that binds `EffectfulBun` gets
214 `hello` as a typed method — the RPC plumbing, auth headers, and
215 serialization are generated.
216
217 The gap between the two rows in the table is the entire cost of
218 that abstraction: **0.2 seconds** at the median (2.9s vs 2.7s),
219 and it buys you typed RPC, bindings, and no Dockerfile. We ran
220 the same pair on Node (the MicroVM base image ships with it) and
221 the gap there is a rounding error.
222
223 The Cloudflare side mirrors this exactly — a plain `oven/bun`
224 container, the same Effect program as a
225 `Cloudflare.Container`, and a public echo image pulled from
226 Docker Hub as a third reference point. Same story on that side
227 too: the Effect image costs nothing measurable; the tail is a
228 property of the platform, not the image.
229
230 ## Booting a MicroVM from a Cloudflare Worker
231
232 This is the part that makes "use AWS MicroVMs from Cloudflare"
233 practical rather than theoretical. Here's the entire Worker
234 from the benchmark — the binding calls are the same ones a
235 Lambda would make:
236
237 ```typescript
238 export default Cloudflare.Worker(
239 "MicrovmBenchWorker",
240 { main: import.meta.filename },
241 Effect.gen(function* () {
242 const run = yield* AWS.Lambda.RunMicrovm(EffectfulBun);
243 const auth = yield* AWS.Lambda.CreateAuthToken(EffectfulBun);
244 const terminate = yield* AWS.Lambda.TerminateMicrovm(EffectfulBun);
245
246 return {
247 fetch: Effect.gen(function* () {
248 const vm = yield* run({});
249 const { authToken } = yield* auth({
250 microvmIdentifier: vm.microvmId,
251 expirationInMinutes: 5,
252 allowedPorts: [{ port: 8080 }],
253 });
254 const sandbox = yield* AWS.Lambda.connectMicrovm(EffectfulBun, {
255 endpoint: vm.endpoint,
256 authToken,
257 });
258 const greeting = yield* sandbox.hello("bench");
259 yield* terminate({ microvmIdentifier: vm.microvmId });
260 return HttpServerResponse.text(greeting);
261 }),
262 };
263 }),
264 );
265 ```
266
267 There's no AWS account configuration in that file. The binding
268 detects what it's attached to. On a Lambda host,
269 `AWS.Lambda.RunMicrovm(EffectfulBun)` adds `lambda:RunMicrovm`
270 on that image's ARN to the function's execution role, and the
271 runtime uses the assumed IAM Role credentials already available
272 in the Lambda Function's environment.
273
274 On a Worker host there is no execution role, so the binding
275 builds the bridge itself. At deploy time it creates, once per
276 Worker:
277
278 - an IAM **User** whose only permission is `sts:AssumeRole`,
279 - an **AccessKey** for that user, bound into the Worker as a
280 secret,
281 - an IAM **Role** that trusts only that user — and this is the
282 role every MicroVM permission accumulates on.
283
284 At runtime the Worker uses the access key for exactly one
285 operation: `AssumeRole`. Everything else — `RunMicrovm`,
286 `CreateAuthToken`, `TerminateMicrovm` — is signed with the
287 short-lived session credentials that come back. Those are cached
288 and refreshed near expiry, shared across every binding on the
289 Worker, so the STS call happens once per isolate rather than
290 once per request.
291
292 Ideally the User wouldn't exist at all — OIDC federation from
293 the Worker straight into the Role would remove the long-lived
294 key entirely, and that's where we want to take this. But the
295 shape is already the right one: the static credential can do
296 nothing except mint short-lived ones, and the permissions live
297 on a role scoped to the specific images the Worker binds.
298
299 The benchmark says the bridge is free: Worker → MicroVM lands
300 within noise of Lambda → MicroVM on every variant. An
301 `AssumeRole` amortized across boots doesn't show up.
302
303 ## How the benchmark runs
304
305 A few methodology details matter more than usual here, because
306 getting them wrong changed our conclusions the first time.
307
308 **MicroVM boots run one at a time.** The MicroVM control plane
309 is rate-limited per account —
310 [`RunMicrovm` allows 5 TPS with a burst of 5](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html#microvms-quotas).
311 An early version of this benchmark launched 5 at once, sat
312 exactly on that ceiling, and measured API throttling mixed into
313 boot time — it looked like MicroVMs had a 1–2 second tail they
314 don't have. Each MicroVM sample is now a fully isolated
315 launch → ready → terminate with nothing else in flight.
316 Containers have no comparable per-boot limit, so they're
317 measured under concurrent load (10 rounds × 10 simultaneous
318 boots).
319
320 This is also why we didn't compare 100 simultaneous MicroVM
321 launches: the 5 TPS is a soft limit, raised by request. It's
322 the standard AWS pattern of putting friction on access to a
323 resource rather than a hard capacity ceiling — they'll bump it
324 when you ask, so it says nothing about how fast the platform
325 boots VMs, only how fast a fresh account is allowed to ask.
326
327 **The driver measures inside the host.** Each host (the Lambda
328 orchestrator, the Worker) exposes a `/boot` route that starts
329 the clock, launches the instance, polls until a real request
330 succeeds, and reports `readyMs`. The external driver only
331 orchestrates; network latency to the driver never contaminates
332 the measurement. On the container side the clock wraps the
333 entire Durable Object call — instance allocation, container
334 start, and the readiness probe — so nothing that happens before
335 the probe can leak out of the measurement.
336
337 **Resources are matched or tilted against the winner.** The
338 opencode container runs on Cloudflare's `standard-1` instance
339 type (4 GiB) against the MicroVM's 1 GiB, so the gap isn't
340 resource starvation — it's the snapshot.
341
342 **Everything is written down.** Every boot lands as a row in a
343 CSV, and the summary table and plots are generated from it. The
344 app, the driver, and the data are in the repo — change a
345 variant, run `bun bench`, and see for yourself.
346
347 ## Where this lands
348
349 Cloudflare Containers win the median for tiny images. AWS
350 MicroVMs win everything else: the worst case by an order of
351 magnitude, and the median too once the workload is heavy enough
352 that startup matters. For anything where a person watches the
353 boot — an agent sandbox, a preview environment, a REPL — the
354 consistency is the feature. A user who always waits 3 seconds
355 is happy; a user who sometimes waits 45 is gone.
356
357 One real advantage on Cloudflare's side that latency numbers
358 don't capture: the Durable Object sits in front of *all* of the
359 container's network traffic. Your JavaScript can intercept
360 every HTTP request the container makes — effectively a
361 man-in-the-middle proxy you control. That enables patterns
362 MicroVMs can't do natively: keep API credentials in the Worker
363 and inject them into outbound requests, so the agent running
364 inside the container never sees a key it could leak. If your
365 sandbox runs untrusted or agent-generated code against
366 credentialed APIs, that interception layer is a serious reason
367 to choose Cloudflare Containers despite the boot-time tail.
368
369 Both platforms sit behind the same abstraction: the same Effect
370 program deploys as a Cloudflare Container or an AWS MicroVM,
371 the same binding call works from a Lambda or a Worker, and the
372 cross-cloud IAM wiring is generated from the code that uses it.
373 Picking the latency shape you want is a couple of lines.