The best way to control Claude Code, Codex and more from any computer or phone
| 1 | --- |
| 2 | title: Hillclimbing distilled |
| 3 | date: 2026-10-24T16:00:00Z |
| 4 | excerpt: We asked an agent for a benchmark dashboard, then kept pushing it to drive every number on it lower. The result is 123 commits that cut distilled's bundles by a third, its added latency by four fifths and its import memory by three quarters. Distilled now adds 5 to 109 times less latency per call than the official AWS, Cloudflare and GCP SDKs, and stays ahead with every response validated. |
| 5 | --- |
| 6 | |
| 7 | import StatGrid from "../../../components/blog/StatGrid.astro"; |
| 8 | import VendorGrid from "../../../components/blog/VendorGrid.astro"; |
| 9 | import BarChart from "../../../components/blog/BarChart.astro"; |
| 10 | import BenchChart from "../../../components/blog/bench/BenchChart.astro"; |
| 11 | import StubWalkthrough from "../../../components/blog/StubWalkthrough.astro"; |
| 12 | import DedupeFigure from "../../../components/blog/DedupeFigure.astro"; |
| 13 | import BarrelFigure from "../../../components/blog/BarrelFigure.astro"; |
| 14 | |
| 15 | Every Alchemy resource talks to its cloud through |
| 16 | [distilled](https://distilled.cloud), our Effect-native SDKs covering |
| 17 | 88 providers. Bindings use it too: under the hood, a binding makes HTTP |
| 18 | calls to the cloud's API, so every Worker or Lambda with a binding |
| 19 | bundles distilled. |
| 20 | |
| 21 | That puts distilled's size in every deploy and every bundle, and until |
| 22 | recently it was heavy: one API call bundled to 57 KB, and importing all |
| 23 | of AWS took 227 MB of memory. |
| 24 | |
| 25 | So we hillclimbed it. We built a benchmark for every SDK, put it on a |
| 26 | dashboard, and pointed an agent at it with one job: make every number |
| 27 | go down. It ran the benchmark after every change, kept what helped and |
| 28 | dropped what didn't. Over 123 commits and 84 benchmark runs, a |
| 29 | single call's bundle shrank by a third and |
| 30 | importing all of AWS went from 227 MB to 13.9 MB. |
| 31 | |
| 32 | <StatGrid |
| 33 | title="Distilled today vs distilled before this work" |
| 34 | stats={[ |
| 35 | { value: "−33%", label: "bundle size", detail: "57.1 → 38.3 KB gzip" }, |
| 36 | { value: "−79%", label: "added latency per call", detail: "19.5 → 4.0 µs" }, |
| 37 | { value: "−74%", label: "import memory", detail: "2.15 → 0.57 MB" }, |
| 38 | { value: "16×", label: "less memory to import all of AWS", detail: "227 → 13.9 MB" }, |
| 39 | ]} |
| 40 | /> |
| 41 | |
| 42 | These are medians over 98 "programs": one credentials layer plus one |
| 43 | operation, for every SDK and every wire protocol it uses. Here is how |
| 44 | that compares with the official SDKs today: |
| 45 | |
| 46 | <VendorGrid /> |
| 47 | |
| 48 | ## How we kept score |
| 49 | |
| 50 | We can't make AWS or Cloudflare faster, only the SDK in front of them, |
| 51 | so real API calls would only add noise. Effect helps here: every |
| 52 | distilled request goes through the `HttpClient` service, so the |
| 53 | benchmark swaps in a fake one that answers instantly: |
| 54 | |
| 55 | ```ts |
| 56 | const MockHttp = Layer.succeed( |
| 57 | HttpClient.HttpClient, |
| 58 | HttpClient.make((request) => |
| 59 | Effect.succeed( |
| 60 | HttpClientResponse.fromWeb(request, new Response(canned.body, canned)), |
| 61 | ), |
| 62 | ), |
| 63 | ); |
| 64 | ``` |
| 65 | |
| 66 | That leaves only distilled's own cost: bundle size, added latency per |
| 67 | call, import memory and a few more. Each one is measured twice. The |
| 68 | second time turns on **strict mode**, where distilled checks every |
| 69 | response against the API spec before your code sees it, so a missing |
| 70 | or mistyped field fails right away as a typed error. Strict mode costs |
| 71 | time and bytes, so it gets its own dotted line on the charts. |
| 72 | |
| 73 | Of all those numbers, memory mattered most to us. It matters for your |
| 74 | Workers and Lambdas, but it also limits how we build Alchemy itself. |
| 75 | Our [resource factory](/blog/2026-07-02-cloudflare-resource-factory) |
| 76 | runs many agents at once, and each one type-checks, imports every |
| 77 | provider and runs live tests. On our machines memory runs out before |
| 78 | CPU does, so halving the heap lets us run about twice as many agents. |
| 79 | |
| 80 | Every run lands on a dashboard that lives in the distilled repo, one |
| 81 | dot per commit, so each chart reads like a loss curve. The benchmarks |
| 82 | now run on every distilled PR too, so a regression shows up before it |
| 83 | merges. |
| 84 | |
| 85 | The whole project came down to one dashboard and one chat thread. With |
| 86 | every change on a chart, we could follow the work without reading a |
| 87 | single diff. Our job became checking in every 30 minutes or so: is it |
| 88 | still on track, and are the changes extreme enough? They often |
| 89 | weren't. The rest of this post climbs from the fixes any reviewer |
| 90 | would approve to the ones that only happened because we kept telling |
| 91 | the agent to get weirder. |
| 92 | |
| 93 | How we said it mattered. The louder and stranger our messages got, |
| 94 | the bolder the ideas the agent came back with. Each section below |
| 95 | opens with a real message from the thread, typos included. |
| 96 | |
| 97 | <BenchChart chart="memory" /> |
| 98 | |
| 99 | ## The sane fixes |
| 100 | |
| 101 | > hillclimb optimizing bundle size and memory footprint; be ambitious, |
| 102 | > dont be afraid to boil the ocean |
| 103 | |
| 104 | This part you can get by just telling an agent to optimize. It follows |
| 105 | the worst number on the dashboard back to its cause, fixes it, runs the |
| 106 | benchmark again, and repeats. That loop never stopped. It kept running |
| 107 | in the main thread the whole time, even while other agents went off to |
| 108 | try much stranger ideas, which we'll get to. |
| 109 | |
| 110 | ### Small imports with big bills |
| 111 | |
| 112 | The single worst line was in a tiny file: |
| 113 | |
| 114 | ```ts |
| 115 | import { Effect, Predicate } from "effect"; |
| 116 | ``` |
| 117 | |
| 118 | In a bundler that's fine. In Node, importing from the `effect` package |
| 119 | root loads every Effect module, and that line cost 3.3 MB and 36 ms on |
| 120 | every import of every SDK. Importing `effect/Effect` and |
| 121 | `effect/Predicate` directly fixed it. That's how the Effect team |
| 122 | recommends importing anyway, and the Effect language service has a |
| 123 | rule for it (`importFromBarrel`). We just don't run it on generated |
| 124 | code. The same pattern hid in smaller |
| 125 | places: `effect/Config` for a few environment variables, `Schedule` |
| 126 | for retries, `Stream.flatMap` for pagination. Each now has a small |
| 127 | replacement in core, and your own `ConfigProvider` or `Schedule` still |
| 128 | works. |
| 129 | |
| 130 | <BarrelFigure /> |
| 131 | |
| 132 | ### One `/*@__PURE__*/` does not cover its arguments |
| 133 | |
| 134 | A bundle with only EC2's `describeRegions` still kept 1,695 schemas. |
| 135 | The generator marked each top-level call `/*@__PURE__*/`, but that |
| 136 | marker only covers the outer call. The calls in its arguments still |
| 137 | count as side effects, so the bundler keeps everything they reference: |
| 138 | |
| 139 | ```ts |
| 140 | // Foo is never used, so the bundler tries to drop it. |
| 141 | const Foo = /*@__PURE__*/ S.Struct({ // dropped: marked pure |
| 142 | items: S.Array(Big.pipe(S.optional)), // unused, but kept: not marked |
| 143 | }); |
| 144 | ``` |
| 145 | |
| 146 | The generator now marks every nested call too, so the whole thing |
| 147 | goes: |
| 148 | |
| 149 | ```ts |
| 150 | const Foo = /*@__PURE__*/ S.Struct({ |
| 151 | items: /*@__PURE__*/ S.Array(/*@__PURE__*/ Big.pipe(S.optional)), |
| 152 | }); |
| 153 | ``` |
| 154 | |
| 155 | The EC2 bundle dropped from 137 KB to 75 KB. |
| 156 | |
| 157 | ### A tree-shaking bug in rolldown |
| 158 | |
| 159 | Every program that made an HTTP call kept about 15 KB gzip of Effect |
| 160 | Schema, even ones that never validated anything, because of one rarely used class in |
| 161 | Effect's HTTP client. Rolldown 1.2.5 and 1.2.6 failed to tree-shake |
| 162 | it, and 1.2.7 fixed that. That one upgrade is the cliff in this chart: |
| 163 | |
| 164 | <BenchChart chart="bundle" /> |
| 165 | |
| 166 | At this point the agent said the numbers looked good and suggested |
| 167 | stopping. Every agent does this. It finds the safe wins, reports them, |
| 168 | and waits. |
| 169 | |
| 170 | ## Stranger, but defensible |
| 171 | |
| 172 | > Okay can you keep improving, we want all of these wayyyy wayyyy lower. |
| 173 | > Do insane stuff, hillclimb the f\*\*\* out of this |
| 174 | |
| 175 | The sane fixes left the numbers better, but still far from where we |
| 176 | wanted them. So every time the agent reported progress, we asked for |
| 177 | something bigger and stranger, and fed it hunches we couldn't prove |
| 178 | yet: that the specs were full of duplicate shapes, and that schemas |
| 179 | were most of the cost. The agent measured each hunch first, then |
| 180 | turned the ones that held up into code. |
| 181 | |
| 182 | ### One em dash doubles a module's memory |
| 183 | |
| 184 | V8 keeps every module's source text in memory. If one character is |
| 185 | outside Latin-1, it stores the whole file at two bytes per character. |
| 186 | Escaping three characters in S3's module cut its import memory from |
| 187 | 2,240 KB to 1,570 KB. Shipped code is now Latin-1 only, and a test |
| 188 | enforces it. |
| 189 | |
| 190 | ### The same schema, thousands of times |
| 191 | |
| 192 | API specs are repetitive. Stripe alone has 12,529 named shapes, and |
| 193 | most of them are copies: dozens of enums that are all just "a string", |
| 194 | empty response objects, and the same struct redeclared on every |
| 195 | endpoint. Each copy became its own object at import. |
| 196 | |
| 197 | The generator now keeps one of each. Comparing shapes field by field |
| 198 | breaks down when shapes refer to themselves, so it uses the algorithm |
| 199 | compilers use to shrink state machines, |
| 200 | [partition refinement](https://en.wikipedia.org/wiki/DFA_minimization#Moore's_algorithm): |
| 201 | assume everything of the same kind is the same, then keep splitting |
| 202 | groups until every member of a group points at the same groups. A |
| 203 | split can make the shapes that point at it split in the next round, |
| 204 | so changes ripple up one level at a time. Shapes that only point at |
| 205 | themselves never split, so recursion just works. |
| 206 | Stripe went from 12,529 shapes to 2,720, and its import memory from |
| 207 | 9.9 MB to 0.86 MB. |
| 208 | |
| 209 | <DedupeFigure /> |
| 210 | |
| 211 | ### Faster calls |
| 212 | |
| 213 | Every call used to run as a generator, with one `yield*` per step: |
| 214 | |
| 215 | ```ts |
| 216 | Effect.gen(function* () { |
| 217 | const protocol = yield* Protocol; |
| 218 | const client = yield* HttpClient.HttpClient; |
| 219 | const request = yield* protocol.encode({ input, inputAst, config }); |
| 220 | const response = yield* client.execute(request); |
| 221 | return yield* protocol.decode({ response, outputAst, errors, config }); |
| 222 | }); |
| 223 | ``` |
| 224 | |
| 225 | Each `yield*` is a trip through Effect's runtime, and encode and decode |
| 226 | were generators too. Now, on the first call of an operation, the |
| 227 | protocol compiles a plan for its schemas: which fields go in the path, |
| 228 | the query and the body. Every later call builds the request from that |
| 229 | plan with plain JavaScript, sends it, and decodes the same way. The |
| 230 | only effect left in the middle is the HTTP request. |
| 231 | |
| 232 | This is added latency per call across the whole project. The strict |
| 233 | line took longer to come down, for reasons the next section explains. |
| 234 | |
| 235 | <BenchChart chart="speed" /> |
| 236 | |
| 237 | ### Faster, smaller AWS signing |
| 238 | |
| 239 | AWS also signs every request, and signing was the single most |
| 240 | expensive part of an AWS call. SigV4 used to take six async WebCrypto |
| 241 | steps, each one a promise round trip. Switching to synchronous hashing |
| 242 | with `node:crypto` alone took S3 `listBuckets` from 352 µs to 149 µs |
| 243 | of added latency, DynamoDB `listTables` from 218 µs to 103 µs and STS |
| 244 | `getCallerIdentity` from 282 µs to 125 µs. Preparing keys once and |
| 245 | building the canonical request without parsing URLs took the rest. |
| 246 | S3 `listBuckets` now adds 19.4 µs. A side-by-side check of 33,798 cases |
| 247 | across 24 AWS services gave identical requests and results. |
| 248 | |
| 249 | Synchronous hashing has a trade-off: it blocks the event loop while it |
| 250 | runs. For a request, that's a few microseconds of hashing, less than |
| 251 | one async round trip, which also costs a trip through the event loop |
| 252 | and evicts whatever was in the CPU cache. A huge request body is where |
| 253 | blocking would start to matter. The hashing goes through a swappable |
| 254 | `Crypto` service, so runtimes without Node use WebCrypto, and you can |
| 255 | plug in your own. |
| 256 | |
| 257 | We also pulled the signer out into its own small package, |
| 258 | `@distilled.cloud/aws4signer`. Cloudflare R2 speaks the S3 API and |
| 259 | needs SigV4 signing too, so code that signs R2 requests no longer |
| 260 | pulls in the rest of the AWS SDK. |
| 261 | |
| 262 | ### Where that left us |
| 263 | |
| 264 | Before the big rewrites started, the numbers already looked like a |
| 265 | finished project. Compared with where we began, added latency per |
| 266 | call was down from 17.3 µs to 10.7 µs, and import memory from 2.10 MB |
| 267 | to 0.84 MB. Most people would stop here. |
| 268 | |
| 269 | The bundle barely moved, though: 57.0 KB to 54.0 KB. Every program |
| 270 | still carried Effect Schema, whether it validated anything or not. To |
| 271 | fix that, we had to stop tuning distilled and start replacing parts of |
| 272 | it. |
| 273 | |
| 274 | ## Experiments, and our own schema library |
| 275 | |
| 276 | > experiments should be radical shifts boiling the ocean; ambitious |
| 277 | > sh\*t; agents are phenomenal and can do extreme things |
| 278 | |
| 279 | The biggest wins came from ideas that might not work, and trying one |
| 280 | can take a long time. If the one agent doing the work stops to try it, |
| 281 | everything else stops too. So we told it to keep the main thread on |
| 282 | the slowest number and hand radical ideas to new agents. That split |
| 283 | the work into three roles that run at the same time: |
| 284 | |
| 285 | 1. **The main thread hillclimbs.** It runs the benchmark after every |
| 286 | change, picks the worst number, and fixes it with small, safe |
| 287 | commits. Along the way it writes down the wild ideas it doesn't |
| 288 | have time for. |
| 289 | 2. **Subagents chase the wild ideas.** Each one gets its own worktree |
| 290 | and one idea: lazy error classes, bundling each SDK into a few |
| 291 | files, rewriting the AWS call path, Node-only service stubs. It |
| 292 | builds the idea for real and benchmarks it as an experiment |
| 293 | against the current head. The dashboard keeps experiments off the |
| 294 | timeline and shows each one next to its baseline. |
| 295 | 3. **The main thread merges what worked.** An experiment that beats |
| 296 | its baseline without regressing anything else gets merged. One that |
| 297 | doesn't stays behind as a data point, so nobody tries it again. |
| 298 | |
| 299 | An experiment is just a benchmark run with a name: |
| 300 | |
| 301 | ```sh |
| 302 | pnpm bench -- --experiment "schema-free generated code" |
| 303 | ``` |
| 304 | |
| 305 | It stays off the commit timeline and shows up on every chart as a flat |
| 306 | dashed line, so we could see at a glance whether an idea beat the |
| 307 | branch: |
| 308 | |
| 309 | <BenchChart chart="experiments" /> |
| 310 | |
| 311 | While a subagent worked on a big bet, the main thread kept landing |
| 312 | small wins. 38 of our 84 benchmark runs were experiments. Some ideas |
| 313 | took three or four experiments to try different implementations, and |
| 314 | we kept the best one. The biggest experiment was replacing Effect Schema, and the dashed line |
| 315 | at the bottom of the chart above is the version that won. |
| 316 | |
| 317 | ### Why Effect Schema had to go |
| 318 | |
| 319 | Every generated schema was a real Effect Schema object. That put about |
| 320 | 37 KB gzip of Schema code into every bundle, and built thousands of |
| 321 | schema objects at import. Most of that was never used. Encoding a |
| 322 | request and reading a response only need the schema's *structure*: |
| 323 | field names, which field goes in the path or a header, how a timestamp |
| 324 | is formatted, and so on. Effect Schema's parser only matters when you turn on |
| 325 | strict validation. |
| 326 | |
| 327 | We didn't set out to write a schema library. The agent tried |
| 328 | everything else first, on real shapes from EC2, S3, DynamoDB, Stripe |
| 329 | and Cloudflare Workers. Stripe, after dedupe: |
| 330 | |
| 331 | <div class="alc-ours-last"> |
| 332 | |
| 333 | | Option | Import memory | First-use memory | µs per validation | 1-op bundle gz | |
| 334 | | --- | ---: | ---: | ---: | ---: | |
| 335 | | Effect Schema (lazy) | 0.86 MB | 11.3 MB | 24 | 27.5 KB | |
| 336 | | valibot | 20 MB | 0.2 MB | 25 | 6.0 KB | |
| 337 | | zod/mini (before dedupe) | 114 MB | 0.7 MB | 40 | 9.3 KB | |
| 338 | | TypeBox (before dedupe) | 14 MB | 1.9 MB | 206 | 27.9 KB | |
| 339 | | schema-lite (ours) | 2.3 MB | 0.17 MB | 13 | 4.6 KB | |
| 340 | |
| 341 | </div> |
| 342 | |
| 343 | Every library lost on at least one axis we cared about. Effect's own |
| 344 | compilers didn't help either: the ahead-of-time compiler can't compile |
| 345 | through recursive shapes, so it generated 4.6 MB of code for EC2, and |
| 346 | the JIT mode doubled first-use memory. An earlier attempt, |
| 347 | [PR #671](https://github.com/alchemy-run/distilled/pull/671), replaced |
| 348 | schemas with descriptors that only carry wire metadata. It was small, |
| 349 | but it couldn't validate, so strict mode broke. |
| 350 | |
| 351 | ### Our problem is smaller |
| 352 | |
| 353 | A general schema library has to handle any schema a user can write: |
| 354 | transformations, refinements, defaults, branded types, async checks, |
| 355 | self-referencing structures in full generality, and detailed error |
| 356 | reports for all of it. That machinery costs bytes and memory. |
| 357 | |
| 358 | Distilled needs almost none of it. Every schema is generated from an |
| 359 | API spec and fixed at build time. They use about a dozen node kinds: |
| 360 | strings, numbers, booleans, literals, structs, arrays, records, unions |
| 361 | and optional fields. A recursive shape is just a named reference to |
| 362 | another generated shape. And there is only one question to answer: |
| 363 | does this decoded JSON match the shape the spec promised? So we wrote |
| 364 | `@distilled.cloud/core/schema-lite`: plain data, plus one small |
| 365 | validator. |
| 366 | |
| 367 | ### Same API, plain objects |
| 368 | |
| 369 | schema-lite is a small subset of Effect Schema. It exports the |
| 370 | functions generated code already calls (`Struct`, `Array`, `optional`, |
| 371 | `Union`, `suspend` and so on) with the same names and types, so the |
| 372 | switch was one import line in every generated file: |
| 373 | |
| 374 | ```diff lang="ts" |
| 375 | -import * as S from "@distilled.cloud/core/schema"; |
| 376 | +import * as S from "@distilled.cloud/core/schema-lite"; |
| 377 | ``` |
| 378 | |
| 379 | If you know Effect Schema, you can read distilled's generated code. |
| 380 | What changed is what each call returns: a plain object with the same |
| 381 | fields as an Effect Schema AST node, and nothing else. |
| 382 | |
| 383 | ```ts |
| 384 | S.Struct({ id: S.String, tags: S.optional(S.Array(S.String)) }); |
| 385 | |
| 386 | // builds this, and only this: |
| 387 | { |
| 388 | _tag: "Objects", |
| 389 | propertySignatures: [ |
| 390 | { name: "id", type: { _tag: "String" } }, |
| 391 | { |
| 392 | name: "tags", |
| 393 | type: { |
| 394 | _tag: "Union", |
| 395 | context: { isOptional: true }, |
| 396 | types: [ |
| 397 | { _tag: "Arrays", elements: [], rest: [{ _tag: "String" }] }, |
| 398 | { _tag: "Undefined" }, |
| 399 | ], |
| 400 | }, |
| 401 | }, |
| 402 | ], |
| 403 | } |
| 404 | ``` |
| 405 | |
| 406 | ### Protocols run unchanged |
| 407 | |
| 408 | Distilled's protocol code (REST and AWS's six wire protocols) already |
| 409 | worked by walking the AST: `_tag`, `propertySignatures`, |
| 410 | `annotations`. A lite node has exactly those fields, so the protocols |
| 411 | didn't need to know anything changed. AWS also needs to turn |
| 412 | timestamps and base64 blobs into wire values, and lite nodes carry |
| 413 | that in an `encoding` field with the same shape Effect uses. We |
| 414 | checked the switch by encoding 1,516 requests and decoding 426 |
| 415 | responses through both implementations. The bytes were identical. |
| 416 | |
| 417 | ### Strict mode, and still faster than the official SDKs |
| 418 | |
| 419 | Official SDKs trust the API. Providers change their APIs without |
| 420 | warning, smaller ones especially (*cough* Cloudflare *cough*). The |
| 421 | request still succeeds, so bad data flows into your app until |
| 422 | something breaks far from the cause. Strict mode turns that into a |
| 423 | controlled failure: a typed `ParseError` with the path to the bad |
| 424 | field, right at the call. It's one line: |
| 425 | |
| 426 | ```ts |
| 427 | program.pipe(Effect.provide(ResponseValidation.strict)); |
| 428 | ``` |
| 429 | |
| 430 | Under schema-lite, each node compiles once into a small checker, and |
| 431 | the compiler itself only loads if you turn strict mode on. A test |
| 432 | checks that it accepts and rejects exactly what Effect Schema does. |
| 433 | |
| 434 | Distilled also types **every error**, so each operation ships the full |
| 435 | union of errors it can fail with. We kept both, and compared them with |
| 436 | the official AWS SDK v3, `cloudflare` and `@google-cloud/storage` |
| 437 | packages, each imported the way its users would: |
| 438 | |
| 439 | <BarChart |
| 440 | title="Bundle size of one call (gzip)" |
| 441 | unit=" KB" |
| 442 | decimals={1} |
| 443 | series={[ |
| 444 | { name: "official SDK", tone: "official" }, |
| 445 | { name: "distilled", tone: "distilled" }, |
| 446 | { name: "distilled, strict", tone: "strict" }, |
| 447 | ]} |
| 448 | rows={[ |
| 449 | { label: "S3 ListBuckets", values: [157.0, 61.0, 62.3], note: "2.5× smaller" }, |
| 450 | { label: "DynamoDB", values: [144.8, 53.8, 55.2], note: "2.6× smaller" }, |
| 451 | { label: "KMS", values: [134.1, 53.2, 54.5], note: "2.5× smaller" }, |
| 452 | { label: "STS", values: [130.1, 55.2, 56.6], note: "2.3× smaller" }, |
| 453 | { label: "EC2", values: [265.2, 55.1, 56.5], note: "4.7× smaller" }, |
| 454 | { label: "Lambda", values: [141.3, 55.0, 56.4], note: "2.5× smaller" }, |
| 455 | { label: "Cloudflare Workers", values: [124.5, 55.8, 57.4], note: "2.2× smaller" }, |
| 456 | { label: "GCP Storage", values: [239.1, 39.7, 41.3], note: "5.8× smaller" }, |
| 457 | ]} |
| 458 | caption="The ratio compares the official SDK with distilled in strict mode." |
| 459 | /> |
| 460 | |
| 461 | <BarChart |
| 462 | title="Latency each SDK adds to one call (p50)" |
| 463 | unit=" µs" |
| 464 | decimals={1} |
| 465 | cap={160} |
| 466 | series={[ |
| 467 | { name: "official SDK", tone: "official" }, |
| 468 | { name: "distilled", tone: "distilled" }, |
| 469 | { name: "distilled, strict", tone: "strict" }, |
| 470 | ]} |
| 471 | rows={[ |
| 472 | { label: "S3 ListBuckets", values: [140.2, 19.4, 22.0], note: "6.4× faster" }, |
| 473 | { label: "DynamoDB", values: [87.3, 14.3, 15.5], note: "5.6× faster" }, |
| 474 | { label: "KMS", values: [83.0, 14.4, 16.4], note: "5.1× faster" }, |
| 475 | { label: "STS", values: [93.2, 16.1, 16.6], note: "5.6× faster" }, |
| 476 | { label: "EC2", values: [97.3, 16.1, 18.4], note: "5.3× faster" }, |
| 477 | { label: "Lambda", values: [79.3, 15.3, 16.1], note: "4.9× faster" }, |
| 478 | { label: "Cloudflare Workers", values: [15.5, 3.0, 1.0], note: "5.1× faster" }, |
| 479 | { label: "GCP Storage", values: [673.9, 6.2, 7.0], note: "96× faster" }, |
| 480 | ]} |
| 481 | caption="GCP's official bar is cut to fit. The ratio compares the official SDK with the slower of distilled's two modes. Differences of a microsecond or two, like Cloudflare's strict bar, are within run-to-run noise." |
| 482 | /> |
| 483 | |
| 484 | Distilled with strict mode on is still 5 to 96 times faster per call |
| 485 | than official SDKs that check nothing. |
| 486 | |
| 487 | Per call isn't the whole story for a CLI or a Lambda function. What |
| 488 | they wait for is the cold start: import the SDK, set up a client and |
| 489 | make the first call, in a fresh process. Bundled the way Alchemy |
| 490 | deploys to Lambda and Workers, distilled starts 2.0× faster than the |
| 491 | official SDK on AWS, 3.0× on Cloudflare and 1.2× on GCP. We measured |
| 492 | this with a separate local harness that isn't part of the benchmark |
| 493 | suite yet. |
| 494 | |
| 495 | Unbundled, in plain `node_modules` like a CLI, distilled is slower on |
| 496 | AWS (107.5 ms against 42.8 ms) and GCP (96.0 ms against 67.2 ms), and |
| 497 | still faster on Cloudflare (94.3 ms against 156.2 ms), because Node |
| 498 | spends about 77 ms loading Effect's ~150 separate modules, which a |
| 499 | bundle turns into one file. |
| 500 | |
| 501 | The first call on its own is about 1.5 ms slower in distilled on |
| 502 | Cloudflare. The official SDK is a thin wrapper around `fetch`, and |
| 503 | about 0.9 ms of the gap is the tracing span Effect opens for every |
| 504 | request. |
| 505 | |
| 506 | ## Meta tree shaking |
| 507 | |
| 508 | > SURELY we can't be at the limit yet! KEEP GOING, like ride this crazy |
| 509 | > wave all the way into the volcano mannnnnnnn! kawabunga! |
| 510 | |
| 511 | That is the most unhinged message we sent the whole time, and it |
| 512 | produced the most unhinged idea. Importing all of AWS still took |
| 513 | 35 MB, and the agent had told us that was close to the floor. After |
| 514 | the volcano message, it came back with this: do a bundler's tree |
| 515 | shaking at run time, inside Node, on code that was never bundled. |
| 516 | |
| 517 | ### Operations as data |
| 518 | |
| 519 | Importing a service used to build every operation in it. EC2 has |
| 520 | hundreds of operations and AWS about 19,500. Now each operation waits |
| 521 | until someone calls it. We call this meta tree shaking. In spirit, the |
| 522 | published JavaScript looks like this: |
| 523 | |
| 524 | ```js |
| 525 | let built; |
| 526 | export const getDomain = (input) => { |
| 527 | // First call: build the real operation, then remember it. |
| 528 | built ??= API.make(() => ({ |
| 529 | input: GetDomainRequest, |
| 530 | output: Domain, |
| 531 | errors: [UnknownResendError], |
| 532 | protocol: ResendProtocol, |
| 533 | })); |
| 534 | return built(input); |
| 535 | }; |
| 536 | ``` |
| 537 | |
| 538 | The real output stores the description as a short string instead of |
| 539 | code, because a string takes less memory than the code that builds |
| 540 | the same object. Error classes work the same way, so a call that |
| 541 | succeeds never builds one. |
| 542 | |
| 543 | ### Tree shaking at run time |
| 544 | |
| 545 | A bundler drops unused code before you deploy. The Alchemy CLI doesn't |
| 546 | bundle, so we do the same cut at run time instead. Every service ships |
| 547 | as two files: a tiny **stub** that Node loads, and the **full module**, |
| 548 | which stays on disk with an index at the end saying where each |
| 549 | operation's bytes are and what it needs. This is the real start of the |
| 550 | stub for Fly.io's `machines` service, exactly as it ships: |
| 551 | |
| 552 | ```js |
| 553 | import{$L}from"../_bundle/lazy.js"; |
| 554 | const{o:$o,e:$e,E:$E}=/*@__PURE__*/$L("../services.full/machines.js","ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo",55605,7533); |
| 555 | ``` |
| 556 | |
| 557 | Yes, that's 99 `o`s, one empty placeholder per operation. No human |
| 558 | would write this, and no human has to read it. The two numbers say |
| 559 | where the index is. The index looks like this: |
| 560 | |
| 561 | ```txt |
| 562 | # name bytes needs |
| 563 | 44 getMachine 19123..19407 13 sdk.js helpers, __d6 |
| 564 | 60 listMachines 25842..26325 16 sdk.js helpers, __d6 |
| 565 | 110 __d6 48375..48982 8 sdk.js helpers, __d3, __d5, __d4, __d22 |
| 566 | ``` |
| 567 | |
| 568 | That's the dependency graph a bundler builds, written down for Node. |
| 569 | The first call to a placeholder reads just that operation's bytes, then |
| 570 | follows `needs` the same way. Helpers in `sdk.js` are already loaded |
| 571 | with the stub, so they cost nothing. |
| 572 | |
| 573 | ### No `eval`, ever |
| 574 | |
| 575 | Reading part of a JavaScript file and running it sounds like it needs |
| 576 | `eval` or `new Function`. That would ruin the whole idea. Every `eval` |
| 577 | makes V8 parse and compile new code, which costs the time and memory |
| 578 | we were trying to save. It also means running bytes read from disk as |
| 579 | code, which security tools flag and many runtimes block outright. |
| 580 | |
| 581 | So the full module never stores operations as code. It stores them as |
| 582 | data. This is the real entry for `getMachine`: |
| 583 | |
| 584 | ```txt |
| 585 | {input:#1,output:#9:4,errors:[$10,$11],protocol:$12,retry:$13} |
| 586 | ``` |
| 587 | |
| 588 | `#1` is another entry in the same operation, and `#9:4` is an entry in |
| 589 | the shared holder for `Machine`. `$10` to `$13` are values already in |
| 590 | memory: the two error classes, the protocol and the retry policy. The |
| 591 | loader is a small interpreter for this format. It reads the bytes, |
| 592 | follows the references and calls the same schema-lite functions the |
| 593 | code would have called, so the result is identical. Nothing it reads |
| 594 | from disk is ever run as code, and V8 never compiles a single extra |
| 595 | line. |
| 596 | |
| 597 | Step through a real traced run that calls `getMachine`, then |
| 598 | `listMachines`, then `getMachine` again. If you want to see how |
| 599 | unhinged it really is, tick the box under the figure for the shipped |
| 600 | code: |
| 601 | |
| 602 | <StubWalkthrough /> |
| 603 | |
| 604 | Three calls read 13,629 of the file's 63,178 bytes, and 7,533 of those |
| 605 | are the index. |
| 606 | |
| 607 | Don't worry: none of this touches your code unless you ask for it. |
| 608 | Stubs load only when Node resolves with the `distilled-lazy` export |
| 609 | condition (`node --conditions=distilled-lazy`). Plain Node and every |
| 610 | bundler get the normal full module. |
| 611 | |
| 612 | ### Importing a whole provider |
| 613 | |
| 614 | Programs that import a whole provider are where meta tree shaking pays |
| 615 | off most. AWS went from 227 MB to 13.9 MB. |
| 616 | |
| 617 | <BarChart |
| 618 | title="Memory to import every service of a provider" |
| 619 | unit=" MB" |
| 620 | decimals={1} |
| 621 | series={[ |
| 622 | { name: "before", tone: "before" }, |
| 623 | { name: "after", tone: "distilled" }, |
| 624 | ]} |
| 625 | rows={[ |
| 626 | { label: "AWS (438)", values: [227, 13.9], note: "16× less" }, |
| 627 | { label: "GCP (318)", values: [111, 9.4], note: "12× less" }, |
| 628 | { label: "Azure (224)", values: [112, 8.0], note: "14× less" }, |
| 629 | { label: "Cloudflare (129)", values: [76, 3.2], note: "24× less" }, |
| 630 | ]} |
| 631 | caption="Import time fell too: AWS from 1.61 s to 0.14 s, Cloudflare from 0.63 s to 0.05 s." |
| 632 | /> |
| 633 | |
| 634 | <BenchChart chart="providerHeap" /> |
| 635 | |
| 636 | The same metric summed over every provider: 924 MB at the start of the |
| 637 | branch, 94 MB at the end. |
| 638 | |
| 639 | Most of what's left for AWS is V8 itself. Every module Node loads |
| 640 | costs about 4.8 KB, and every named export about 325 bytes, even if |
| 641 | nothing reads it. All of AWS is 438 modules and tens of thousands of |
| 642 | exports, which comes to about 10 MB before any distilled code runs. |
| 643 | The only way below that is to export fewer names, which is an API |
| 644 | change. |
| 645 | |
| 646 | ## Breaking changes |
| 647 | |
| 648 | :::caution |
| 649 | These shipped in `@distilled.cloud/*` `1.0.0-rc.16`. |
| 650 | |
| 651 | - Schema values are no longer exported. Use `op.input` and |
| 652 | `op.output`. |
| 653 | - Error classes do not validate their fields on construction. |
| 654 | `ParseError.cause` is a `SchemaMismatch` with a path. |
| 655 | - `SigV4.SigningKeyCache` is a plain store. |
| 656 | - `lib/` is bundled, minified, generated output. |
| 657 | |
| 658 | `instanceof`, `_tag`, `Effect.catchTag` and the TypeScript types for |
| 659 | errors work as before. |
| 660 | ::: |
| 661 | |
| 662 | ## Try meta tree shaking in Alchemy |
| 663 | |
| 664 | The Alchemy CLI ships with meta tree shaking turned off. To opt in, |
| 665 | pass `--fast-distilled`, or set `ALCHEMY_FAST_DISTILLED=1`: |
| 666 | |
| 667 | ```sh |
| 668 | alchemy deploy --fast-distilled |
| 669 | ``` |
| 670 | |
| 671 | The flag makes the CLI load distilled with the `distilled-lazy` |
| 672 | condition, so every provider your app imports starts as a stub. It |
| 673 | applies under Node. Under Bun it has no effect. |