The best way to control Claude Code, Codex and more from any computer or phone
Mirrored from https://github.com/Camo67/t3code.git
Synced Nov 21, 04:45 UTC
| 1 | --- |
| 2 | title: Test harness |
| 3 | description: Reference for @alchemy.run/test — every helper, hook, and option exposed by Test.make, and how to run it with alchemy-test, bun test, or Vitest. |
| 4 | --- |
| 5 | |
| 6 | `Test.make` is the same Effect-aware harness on three runners: |
| 7 | [`alchemy-test`, `bun test`, and Vitest](#runners). For the |
| 8 | end-to-end walkthrough, see |
| 9 | [Testing a Stack](/testing/testing-a-stack); for provider-lifecycle |
| 10 | testing, see [Testing Providers](/testing/testing-providers). |
| 11 | |
| 12 | ## What Test.make returns |
| 13 | |
| 14 | A single call returns a self-contained API for the file: |
| 15 | |
| 16 | ```typescript |
| 17 | const { test, beforeAll, beforeEach, afterAll, afterEach, deploy, destroy } = |
| 18 | Test.make({ |
| 19 | providers: Cloudflare.providers(), |
| 20 | state: Cloudflare.state(), |
| 21 | }); |
| 22 | ``` |
| 23 | |
| 24 | | Helper | Purpose | |
| 25 | |---|---| |
| 26 | | `test(name, effect)` | Effect-aware test. `HttpClient` and your providers Layer are in scope. | |
| 27 | | `test.skip` / `test.skipIf` / `test.only` / `test.todo` | Skip / focus / todo modifiers (same shape as `bun.test`). | |
| 28 | | `test.provider(name, fn)` | [Provider-lifecycle test](/testing/testing-providers) against a scratch in-memory stack. | |
| 29 | | `beforeAll(effect)` | Run an Effect once. Returns a lazy accessor (`yield* result`) usable inside tests. | |
| 30 | | `beforeEach(effect)` | Run an Effect before every test. | |
| 31 | | `afterAll(effect)` / `afterAll.skipIf(predicate)` | Cleanup hook with conditional teardown. | |
| 32 | | `afterEach(effect)` | Run an Effect after every test. | |
| 33 | | `deploy(Stack, opts?)` | Plan + apply a stack, resolve to its outputs. | |
| 34 | | `destroy(Stack, opts?)` | Plan + apply against an empty desired state. | |
| 35 | |
| 36 | `Test.getWhenReady` / `Test.executeWhenReady` are module-level HTTP |
| 37 | cold-start helpers, taught in [Testing a Stack](/testing/testing-a-stack). |
| 38 | |
| 39 | `expect` (and `describe`) come from the runner directly — |
| 40 | `@alchemy.run/test`, `bun:test`, or `@effect/vitest`. |
| 41 | |
| 42 | ## Test.make options |
| 43 | |
| 44 | ```typescript |
| 45 | Test.make({ |
| 46 | providers, // required |
| 47 | state, // optional |
| 48 | profile, // optional |
| 49 | stage, // optional |
| 50 | dev, // optional — run the suite against local emulators |
| 51 | adopt, // optional |
| 52 | sidecar, // optional — defaults to dev |
| 53 | }); |
| 54 | ``` |
| 55 | |
| 56 | ### `providers` (required) |
| 57 | |
| 58 | The provider Layer that resolves resource implementations. |
| 59 | Usually the same one your `Stack` uses: |
| 60 | |
| 61 | ```typescript |
| 62 | providers: Cloudflare.providers(), |
| 63 | // or merge multiple: |
| 64 | providers: Layer.mergeAll(Cloudflare.providers(), Stripe.providers()), |
| 65 | ``` |
| 66 | |
| 67 | Credentials resolve through the same `AuthProviders` registry as |
| 68 | `alchemy deploy`, so tests pick up stored profiles or |
| 69 | the env-var auth methods registered by each provider. |
| 70 | |
| 71 | ### `state` |
| 72 | |
| 73 | The state store used by top-level `deploy(Stack)` and |
| 74 | `destroy(Stack)` (not by `test.provider`). Defaults to |
| 75 | `localState()` — `.alchemy/` on disk. |
| 76 | |
| 77 | ```typescript |
| 78 | state: Cloudflare.state(), // R2-backed, survives across CI runners |
| 79 | state: localState({ path: ".alchemy-test/" }), // separate dir |
| 80 | state: undefined, // omit → defaults to localState() |
| 81 | ``` |
| 82 | |
| 83 | Persistent state lets `deploy(Stack)` skip recreating unchanged |
| 84 | resources between runs. The run-against-an-existing-stack pattern |
| 85 | built on it lives in [Testing a Stack](/testing/testing-a-stack). |
| 86 | |
| 87 | ### `profile` |
| 88 | |
| 89 | Override `ALCHEMY_PROFILE` for this file only. Useful for |
| 90 | pinning tests to a sandbox profile regardless of what's set in |
| 91 | the environment: |
| 92 | |
| 93 | ```typescript |
| 94 | Test.make({ |
| 95 | providers: AWS.providers(), |
| 96 | profile: "test-sandbox", |
| 97 | }); |
| 98 | ``` |
| 99 | |
| 100 | When omitted, the harness reads `ALCHEMY_PROFILE` from env / `.env` |
| 101 | the same way the CLI does. |
| 102 | |
| 103 | ### `stage` |
| 104 | |
| 105 | Default stage for `deploy(Stack)` / `destroy(Stack)`. Defaults to |
| 106 | `test_$USER` (e.g. `test_sam`) so two people running the same suite |
| 107 | against one account don't collide. `ALCHEMY_TEST_STAGE` replaces |
| 108 | the default for the whole run |
| 109 | ([from the command line](#setting-the-stage-from-the-command-line)), |
| 110 | e.g. one stage per [git worktree](/environments/worktrees). Override |
| 111 | per file, or per call: |
| 112 | |
| 113 | ```typescript |
| 114 | Test.make({ providers, stage: "ci-pr-42" }); |
| 115 | |
| 116 | // or per-call: |
| 117 | beforeAll(deploy(Stack, { stage: "ci-pr-42" })); |
| 118 | afterAll.skipIf(!process.env.CI)(destroy(Stack, { stage: "ci-pr-42" })); |
| 119 | ``` |
| 120 | |
| 121 | A unique stage per PR or test run lets multiple suites run in |
| 122 | parallel against the same provider account without colliding. |
| 123 | |
| 124 | ### `dev` |
| 125 | |
| 126 | Off by default — tests deploy to the real cloud. `dev: true` runs |
| 127 | the whole file in local-dev mode, the same wiring as |
| 128 | [`alchemy dev`](/environments/local-development): Cloudflare |
| 129 | Workers, Durable Objects, KV, R2, D1, Queues, and Workflows run in |
| 130 | workerd with local simulators, and AWS Lambda/ECS plus the |
| 131 | emulated AWS surface run in a local Docker emulator. No cloud |
| 132 | account is touched for emulated resources. |
| 133 | |
| 134 | ```typescript |
| 135 | Test.make({ |
| 136 | providers: Cloudflare.providers(), |
| 137 | dev: true, // deploy(Stack) boots workerd / Docker simulators |
| 138 | }); |
| 139 | ``` |
| 140 | |
| 141 | Everything else in the file is unchanged — `beforeAll(deploy(Stack))` |
| 142 | boots the stack locally and the outputs (`url`, ids) point at the |
| 143 | local instances, so the same HTTP assertions run against |
| 144 | `http://localhost:<port>`. Local resource ids are `dev:`-prefixed, |
| 145 | which doubles as proof no cloud call ran. `Alchemy.remote()` still |
| 146 | pins individual resources live, exactly as it does under |
| 147 | `alchemy dev`. |
| 148 | |
| 149 | When omitted, the flag falls back to the `ALCHEMY_DEV` env var — |
| 150 | leave it out of the file and set `ALCHEMY_DEV=1` locally to run the |
| 151 | same suite against emulators while CI runs it live. Separately, |
| 152 | `ALCHEMY_TEST_DEV=1` **overrides** the option in both directions — |
| 153 | use it to force an entire existing suite through local providers |
| 154 | without editing each `Test.make`. |
| 155 | |
| 156 | What each cloud emulates is covered in |
| 157 | [Cloudflare local development](/cloudflare/local-development) and |
| 158 | [AWS local development](/aws/local-development); the step-by-step |
| 159 | walkthrough is [Cloudflare Tutorial Part 4](/cloudflare/tutorial/part-4). |
| 160 | |
| 161 | ### `adopt` |
| 162 | |
| 163 | Engine-level adoption policy for the run, matching the CLI's |
| 164 | `--adopt` flag. When `true`, a resource with no prior state whose |
| 165 | physical counterpart already exists in the cloud is adopted via |
| 166 | `provider.read` instead of failing. Defaults to `false`. |
| 167 | |
| 168 | ### `sidecar` |
| 169 | |
| 170 | Only meaningful in dev mode, and defaults to the resolved `dev` |
| 171 | flag: dev tests run local providers behind the same RPC sidecar |
| 172 | process the real `alchemy dev` command uses. `sidecar: false` runs |
| 173 | them in-process instead — useful when debugging provider code, |
| 174 | since there's no child process between you and the breakpoint. |
| 175 | |
| 176 | ## Hooks |
| 177 | |
| 178 | ### `beforeAll(effect) → Effect.Effect<A>` |
| 179 | |
| 180 | Runs the Effect once before any test in the file. Stores the |
| 181 | result and returns a lazy accessor — `yield* accessor` inside |
| 182 | any test or other hook returns the resolved value: |
| 183 | |
| 184 | ```typescript |
| 185 | const stack = beforeAll(deploy(Stack)); |
| 186 | const seed = beforeAll(Effect.gen(function* () { |
| 187 | yield* DynamoDB.putItem({ /* ... */ }); |
| 188 | return Date.now(); |
| 189 | })); |
| 190 | |
| 191 | test( |
| 192 | "uses both", |
| 193 | Effect.gen(function* () { |
| 194 | const { url } = yield* stack; |
| 195 | const startedAt = yield* seed; |
| 196 | /* ... */ |
| 197 | }), |
| 198 | ); |
| 199 | ``` |
| 200 | |
| 201 | Default timeout is **120s**. Override with the second argument: |
| 202 | |
| 203 | ```typescript |
| 204 | beforeAll(deploy(Stack), { timeout: 300_000 }); |
| 205 | ``` |
| 206 | |
| 207 | ### `beforeEach(effect)` |
| 208 | |
| 209 | Runs the Effect before every test. No accessor returned — for |
| 210 | side-effect setup only (truncate a table, reset a feature flag, |
| 211 | …). |
| 212 | |
| 213 | ### `afterAll(effect)` and `afterAll.skipIf(predicate)` |
| 214 | |
| 215 | Cleanup hook with conditional teardown: |
| 216 | |
| 217 | ```typescript |
| 218 | afterAll(destroy(Stack)); // always destroy |
| 219 | afterAll.skipIf(!process.env.CI)(destroy(Stack)); // CI only |
| 220 | afterAll.skipIf(true)(destroy(Stack)); // never (debugging) |
| 221 | ``` |
| 222 | |
| 223 | `afterAll.skipIf(true)` short-circuits without registering a |
| 224 | hook at all — there's no risk of an `Effect` being constructed |
| 225 | and dropped. |
| 226 | |
| 227 | ### `afterEach(effect)` |
| 228 | |
| 229 | Runs after every test. Combine with `beforeEach` for |
| 230 | test-isolated fixtures. |
| 231 | |
| 232 | ## Test variants |
| 233 | |
| 234 | ```typescript |
| 235 | test.skip("not ready yet", Effect.gen(function* () { /* ... */ })); |
| 236 | |
| 237 | test.skipIf(process.env.CI)( |
| 238 | "local-only smoke test", |
| 239 | Effect.gen(function* () { /* ... */ }), |
| 240 | ); |
| 241 | |
| 242 | test.only( |
| 243 | "the one I'm debugging", |
| 244 | Effect.gen(function* () { /* ... */ }), |
| 245 | ); |
| 246 | |
| 247 | test.todo("backfill once R2 has multipart helper"); |
| 248 | ``` |
| 249 | |
| 250 | `test.provider` mirrors the same shape (semantics in |
| 251 | [Testing Providers](/testing/testing-providers)): |
| 252 | |
| 253 | ```typescript |
| 254 | test.provider.skip(name, fn); |
| 255 | test.provider.skipIf(condition)(name, fn); |
| 256 | ``` |
| 257 | |
| 258 | :::note[Vitest differences] |
| 259 | On Vitest, `test.only` is supported but `test.todo` is a stub |
| 260 | (maps to `it.todo`). `test.skip` / `test.skipIf` work identically. |
| 261 | ::: |
| 262 | |
| 263 | ## HttpClient is in scope |
| 264 | |
| 265 | `HttpClient` is wired into every `test` Effect, so you can call |
| 266 | it directly: |
| 267 | |
| 268 | ```typescript |
| 269 | import * as HttpClient from "effect/http/HttpClient"; |
| 270 | |
| 271 | test( |
| 272 | "health check", |
| 273 | Effect.gen(function* () { |
| 274 | const { url } = yield* stack; |
| 275 | const res = yield* HttpClient.get(`${url}/health`); |
| 276 | expect(res.status).toBe(200); |
| 277 | }), |
| 278 | ); |
| 279 | ``` |
| 280 | |
| 281 | The implementation comes from |
| 282 | `effect/http/FetchHttpClient` — same client the CLI |
| 283 | uses. For a full PUT/GET round-trip against a deployed stack, |
| 284 | see [Testing a Stack → Drive the live URL](/testing/testing-a-stack). |
| 285 | |
| 286 | ## Runners |
| 287 | |
| 288 | [`@alchemy.run/test`](https://www.npmjs.com/package/@alchemy.run/test) |
| 289 | holds `Test.make` for every runner. Install it once, whichever runner |
| 290 | you use: |
| 291 | |
| 292 | ```sh |
| 293 | bun add -d @alchemy.run/test |
| 294 | ``` |
| 295 | |
| 296 | Each runner has its own adapter in the package. The `Test.make` API |
| 297 | is identical on all three; only the import and the command change. |
| 298 | |
| 299 | ### alchemy-test |
| 300 | |
| 301 | `alchemy-test` is the runner Alchemy's own suite uses, and it ships |
| 302 | in the same package. It runs every test file in one bun process with |
| 303 | files and tests in flight at once, which suits slow tests that wait |
| 304 | on real clouds. |
| 305 | |
| 306 | Import the harness from `@alchemy.run/test/Alchemy` and `expect` from |
| 307 | `@alchemy.run/test`. |
| 308 | |
| 309 | ```typescript |
| 310 | import * as Test from "@alchemy.run/test/Alchemy"; |
| 311 | import { expect } from "@alchemy.run/test"; |
| 312 | ``` |
| 313 | |
| 314 | Run it with the `alchemy-test` binary. It picks up every |
| 315 | `*.test.ts` under `./test`, or the paths you pass. |
| 316 | |
| 317 | ```sh |
| 318 | bunx alchemy-test |
| 319 | bunx alchemy-test test/integ.test.ts -t "serves the url" |
| 320 | ``` |
| 321 | |
| 322 | Flags set the environment before any test file is imported: |
| 323 | `--profile <name>` sets `ALCHEMY_PROFILE`, `--stage <name>` sets |
| 324 | `ALCHEMY_TEST_STAGE`, and `--fast` sets `FAST=1`. |
| 325 | |
| 326 | ```sh |
| 327 | bunx alchemy-test --profile testing --stage test_sam_feat-x |
| 328 | ``` |
| 329 | |
| 330 | Passing tests stay quiet; a failing test prints its error and |
| 331 | captured output inline. Every test's output goes to a per-run log |
| 332 | under `.alchemy/log/test/`, whose path is printed at the end. Add |
| 333 | `--tui` for a live view in an interactive terminal. |
| 334 | |
| 335 | ### bun test |
| 336 | |
| 337 | Import the harness from `@alchemy.run/test/Bun` and `expect` from |
| 338 | `bun:test`. |
| 339 | |
| 340 | ```typescript |
| 341 | import * as Test from "@alchemy.run/test/Bun"; |
| 342 | import { expect } from "bun:test"; |
| 343 | ``` |
| 344 | |
| 345 | Run it with bun's built-in runner. Every `test(...)` becomes a |
| 346 | `bun:test` test that runs its Effect. |
| 347 | |
| 348 | ```sh |
| 349 | bun test test/integ.test.ts |
| 350 | ``` |
| 351 | |
| 352 | ### Vitest |
| 353 | |
| 354 | Install `vitest` and `@effect/vitest` next to `@alchemy.run/test`. |
| 355 | |
| 356 | ```sh |
| 357 | npm add -D @alchemy.run/test vitest @effect/vitest |
| 358 | ``` |
| 359 | |
| 360 | Import the harness from `@alchemy.run/test/Vitest` and `expect` from |
| 361 | `@effect/vitest`. |
| 362 | |
| 363 | ```typescript |
| 364 | import * as Test from "@alchemy.run/test/Vitest"; |
| 365 | import { expect } from "@effect/vitest"; |
| 366 | ``` |
| 367 | |
| 368 | Every `test(...)` runs as an `@effect/vitest` `it.live` test, with |
| 369 | the same 120s default hook timeout. |
| 370 | |
| 371 | ```sh |
| 372 | npx vitest run test/integ.test.ts |
| 373 | ``` |
| 374 | |
| 375 | ### Setting the stage from the command line |
| 376 | |
| 377 | The default stage is `test_$USER` on every runner. `ALCHEMY_TEST_STAGE` |
| 378 | replaces it for the whole run. `alchemy-test` sets it with |
| 379 | `--stage`; with `bun test` or Vitest, set the variable yourself. |
| 380 | |
| 381 | ```sh |
| 382 | bunx alchemy-test --stage ci-pr-42 |
| 383 | ALCHEMY_TEST_STAGE=ci-pr-42 bun test |
| 384 | ALCHEMY_TEST_STAGE=ci-pr-42 npx vitest run |
| 385 | ``` |
| 386 | |
| 387 | A `stage` passed to `Test.make` or to a single `deploy` still wins |
| 388 | over the variable. |
| 389 | |
| 390 | ## Where next |
| 391 | |
| 392 | - [Testing a Stack](/testing/testing-a-stack) — deploy once, drive the live URL, tear down. |
| 393 | - [Testing Providers](/testing/testing-providers) — exercise create / update / replace / delete with `test.provider`. |
| 394 | - [Testing](/testing) — the testing overview. |
| 395 | - [State Store](/state-store) — choosing between `localState()`, `Cloudflare.state()`, and friends. |
| 396 | - [Profiles](/environments/profiles) — how `ALCHEMY_PROFILE` and the `profile` factory option resolve credentials. |