Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

next
md 62 lines 6.68 KB
Rendered Raw
1 # Fake Composio backend
2
3 Test support for the CLI suites in `../operations` and `../agent-journeys`. One process answers for every host the `composio` binary talks to, so a test never reaches a real Composio, GitHub, or PostHog endpoint.
4
5 | Host the CLI calls | Pointed here by | Listener |
6 | ------------------ | -------------------------------------------------------- | -------- |
7 | Backend API | `COMPOSIO_BASE_URL` | HTTP |
8 | Dashboard | `COMPOSIO_WEB_URL` | HTTPS |
9 | Agents service | `COMPOSIO_AGENTS_BASE_URL` | HTTP |
10 | GitHub releases | `COMPOSIO_GITHUB_API_BASE_URL` and `COMPOSIO_GITHUB_URL` | HTTP |
11 | PostHog | `COMPOSIO_POSTHOG_INGEST_URL` | HTTP |
12
13 `fake.cliEnv()` returns that environment. The Dashboard listener speaks TLS because the CLI sends a user API key to the Dashboard only over `https:` (or to loopback); the certificate is generated with the host's `openssl` at start and trusted in the container through `NODE_EXTRA_CA_CERTS`.
14
15 ## What it gives a test
16
17 ```ts
18 import { startCliRig } from '@e2e-tests/cli-fake-backend/rig';
19
20 const rig = await startCliRig();
21 await rig.login();
22 const result = await rig.run('composio search "send a slack message"');
23
24 result.exitCode; // what the CLI did
25 result.requests; // what the fake received while it ran
26 rig.fake.world; // seeded state, and stand-ins for a person (linkLoginSession, connect)
27 rig.fake.faults.inject({
28 route: 'toolRouter.session.search',
29 fault: { kind: 'status', status: 500 },
30 });
31 ```
32
33 - **Journal** (`src/journal.ts`). Every request, in order, with what the test usually wants to assert already resolved: the service it was for, the route that answered, the key class (`user` for `uk_`, `project` for `ak_`, `agent` for `cak_`), whether this deployment issued the key, the org, the project by its seeded label (`consumer`, `developer`, `developer-other`), how the project was learned (`scope-headers`, `project-key`, `key-default`, `route`), the project the request itself named (`requestedProject`, which a tool-router session can override), the user the request acted as, whether it changed anything (`tool-executed`, `connection-started`), and for an execution the tool and arguments it ran with, whichever of the three execute routes ran it.
34 - **World** (`src/world.ts`). One org member, one user API key, a consumer project, two developer projects, four toolkits, five tools. Ids and timestamps are fixed, so two runs print the same bytes. A second fake started with `{ userApiKey }` is a deployment that never issued the first one's key.
35 - **Faults** (`src/faults.ts`). Per route or per `METHOD /path` pattern, for a number of requests: a status with the standard error envelope, a delay, a reset before the response, a reset in the middle of the body, a body that is not JSON, JSON of another shape, an HTML page, an empty body.
36
37 Run it by hand to poke at the CLI:
38
39 ```bash
40 bun ts/e2e-tests/cli/fake-backend/src/main.ts /tmp/fake-ca.pem
41 ```
42
43 ## Fidelity
44
45 **Verified.** Each route named after a `@composio/client` operation answers with a body that the client's own response schema for that operation decodes. `contract.test.ts` calls every such route and decodes the answer with `schemaLoaders` from `@composio/client/effect/schemas`; it also fails when a typed route has no sample. Run it with `pnpm --filter @e2e-tests/cli-fake-backend test:e2e:cli` (no Docker needed).
46
47 **Assumed, not verified.** Read these before trusting a result that depends on them.
48
49 - The client schemas are permissive (unknown fields pass, most fields are optional). A decoded response has the required fields with the right types; it is not shown to be what the real backend sends for that request.
50 - Routes the client does not type are written from the CLI source that calls them: the Dashboard procedures `/api/cli/trpc/execute` and `executeMeta` (envelope from `dashboard-tool-execution.ts`), `POST /api/v3/org/project/{id}/api_keys/create`, `GET /api/v3/tools/{slug}/get_latest_version`, `POST /api/v3/cli/analytics`, the agents service, the GitHub release API, PostHog capture.
51 - Error responses use the envelope the CLI's error extraction reads (`{ error: { message, code, slug, status, request_id, suggested_fix } }`). The slugs and codes are the ones the CLI special-cases (`ToolRouterV2_NoActiveConnection`, `ActionExecute_ConnectedAccountNotFound`, 4302); every other slug is invented.
52 - Authorization rules are a model: a user key with `x-org-id` and `x-project-id` acts on that project; a project key acts on its project; **a user key with no scope acts on the org's default developer project**, the one `auth/session/info` reports. The last rule is a guess at what the real backend does with such a request. The journal marks those requests `key-default` so a test can see when the CLI relied on it.
53 - `auth/session/info` for a user key reports the default developer project, never the consumer project.
54 - Search is keyword matching over five tools. Tool executions return canned data. Nothing here says a real toolkit would accept the arguments.
55 - **Tool arguments are not validated.** A tool runs with whatever arrives, a missing required argument included. The CLI validates against the tool's schema first, but sends the call anyway when it cannot load that schema (`validateToolInputBeforeExecute`), relying on the server. The real answer to an invalid input was never captured, so the fake does not invent one; a test that depends on the arguments asserts them from the journal.
56 - A tool-router session acts on the project it was created for, whatever project the request names. Whether the real backend rejects a request whose scope differs from its session's is unknown; the journal keeps both (`project`, `requestedProject`) so a test can require them to agree.
57 - A link request starts a connection as `INITIATED`; it turns `ACTIVE` after `world.linkCompletesAfterPolls` status reads, or never, or when its redirect URL is loaded: there is no OAuth page behind that URL, and a `GET` of it stands for the person approving. The world remembers which approvals came from the person and whether any answer has since reported them (`unreportedApprovals`), which is how a journey's person knows the agent cannot have seen one yet.
58 - Retry-After, rate-limit headers, pagination past one page, request size limits, and timeouts on the server side are not modelled.
59
60 ## Not here
61
62 Triggers, realtime (Pusher), logs, file uploads, `proxy`, MCP, and toolkit versions beyond one fixed version per toolkit have no routes. A command that needs one gets a 404 with slug `Fake_UnhandledRoute`, which the journal records with `route: null`.