@devries-cameron20 / composio Mirror · read-only / commits / f900de24c
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

Mirrored from https://github.com/Camo67/composio.git

Synced Nov 21, 08:30 UTC

fix(cli): exit 1 when `composio execute` reports a failed execution (#5002)

This PR: - Makes `composio execute` exit 1 when an execution fails (the tool reported a failure, the Dashboard or backend could not run it, a meta tool failed, or any call in `--parallel` failed); stdout keeps the same `{"successful": false, ...}` JSON and nothing extra is printed. - Exports `ReportedToolExecutionError` and catches it in `cli-main.ts` next to `ShellSetupAbortError`, the other "already reported, just exit non-zero" failure. - Keeps `execute()` in `composio run` rejecting with the tool's error message, `slug`, and failed `result` now that the spawned `composio execute` exits 1. - Adds subprocess tests against a local fake backend/Dashboard and a run-helper test, plus a CHANGELOG entry. ## Context `composio execute` raised `ReportedToolExecutionError` after printing the failed result, and `cli-main.ts` filtered it out of the error report but never set `process.exitCode`, so the process exited 0. Scripts and agents that check `$?` treated failed executions as successes. This was reproduced on production (`composio execute HACKERNEWS_GET_USER -d '{"username":"zz-no-such-user-zz-123456"}'` printed `successful: false` and exited 0 on `next` and on the released CLI). `--parallel` already raised the same error when any call failed (`One or more parallel tool executions failed.`), so it now exits 1 whenever the top-level `successful` is `false`. No docs, skills, or E2E suites assumed exit 0 for failed executions. `composio run`'s helper used to throw a generic `composio execute … failed with exit code 1` for any non-zero exit. A non-zero exit whose stdout is a `successful: false` result is now returned to `execute()`, which throws the tool's error as before. Any other non-zero exit still throws the exit-code error. Verification: - New tests in `test/src/cli-main.test.ts` spawn `src/bin.ts` against a local fake backend/Dashboard. They cover a tool failure, a Dashboard 503, a meta tool failure, and a mixed `--parallel` run, and assert exit 1, the stdout JSON, and empty stderr. Two control cases, a successful call and an all-successful `--parallel` run, exit 0. With the `cli-main.ts` change reverted, all four failure cases fail with `expected +0 to be 1`. - New tests in `run-helpers-runtime.test.ts` check that `execute()` rejects with `User not found` when the child exits 1 with a failed result (this fails without the helper change), and that other non-zero exits still reject with the exit code. - Ran CLI `pnpm run test` (2202 passed; `validate:command-tests` still has 2 exemptions), `pnpm run typecheck`, `lint:effect`, oxlint, prettier, and root `pnpm typecheck`. - Built the binary from `origin/next` and from this branch and ran both with an isolated HOME and a fake `uak_` key against a local fake server: | Command | `next` | this PR | | --- | --- | --- | | `execute HACKERNEWS_GET_USER` (tool reports failure) | 0 | 1 | | `execute HACKERNEWS_GET_ITEM` (Dashboard 503) | 0 | 1 | | `execute COMPOSIO_SEARCH_TOOLS` (meta tool failure) | 0 | 1 | | `execute --parallel` (one call fails) | 0 | 1 | | `execute --parallel` (all calls succeed) | 0 | 0 | | `execute HACKERNEWS_GET_FRONTPAGE` (success) | 0 | 0 | | `run` script catching `execute()` | 0, `caught: User not found` | 0, `caught: User not found` (same `slug` and `result`) | Stdout and stderr were byte-identical between the two binaries in every case.

Alberto Schiabel committed Oct 21, 2026 at 18:52 UTC f900de24c4fa187222876972873588b0f7d5d778
6 files changed +270 -2
ts/packages/cli/CHANGELOG.md
+7
@@ -4,6 +4,13 @@
4
5 ### Patch Changes
6
7 +- `composio execute` now exits 1 when a tool execution fails: the tool
8 + reported a failure, the Composio Dashboard or backend could not run it, or,
9 + with `--parallel`, any one of the calls failed. It used to exit 0. Meta tools
10 + behave the same. The `{"successful": false, ...}` JSON on stdout and the
11 + messages on stderr are unchanged. In `composio run`, `execute()` still
12 + rejects with the tool's error message and the failed result.
13 +
14 - `composio execute` now sends the execution itself (single, `--parallel`, and
15 meta tools) through the Composio Dashboard instead of calling the backend
16 directly. Output, exit codes, error details, connection tips, and `--json`
ts/packages/cli/src/cli-main.ts
+10
@@ -72,6 +72,7 @@ import { SetupSkillInstaller } from 'src/services/setup-skill-installer';
72 import { PendingLoginSession } from 'src/services/pending-login-session';
73 import { SetupCommandError } from 'src/services/setup-command-error';
74 import { ShellSetupAbortError } from 'src/commands/install/install.errors';
75 +import { ReportedToolExecutionError } from 'src/commands/tools/commands/tools.execute.cmd';
76 import { isUsageFailure, type UsageFailure } from 'src/commands/errors';
77 import { cliInvocationContext } from 'src/services/runtime-cli-context';
78 import { readTelemetryDebugOverride, telemetryDebugModeLayer } from 'src/services/runtime-flags';
@@ -339,6 +340,15 @@ const cliProgram = (argv: ReadonlyArray<string>) =>
340 process.exitCode = 1;
341 })
342 ),
343 + // `composio execute` already printed the failed result as JSON on stdout; the typed
344 + // failure only exists so scripts and agents see a non-zero exit code.
345 + Effect.catchIf(
346 + (error): error is ReportedToolExecutionError => error instanceof ReportedToolExecutionError,
347 + () =>
348 + Effect.sync(() => {
349 + process.exitCode = 1;
350 + })
351 + ),
352 Effect.catchIf(
353 (error): error is ShellSetupAbortError => error instanceof ShellSetupAbortError,
354 // `composio install` already printed the abort reason; the typed failure
ts/packages/cli/src/commands/tools/commands/tools.execute.cmd.ts
+5 -1
@@ -142,7 +142,11 @@ export class ToolExecutionError extends Data.TaggedError(
142 'ToolExecutionError'
143 )<ToolExecutionErrorFields> {}
144
145 -class ReportedToolExecutionError extends Data.TaggedError(
145 +/**
146 + * A failed execution whose result is already on stdout as `{ "successful": false, ... }`.
147 + * `cli-main.ts` prints nothing more for it and exits 1.
148 + */
149 +export class ReportedToolExecutionError extends Data.TaggedError(
150 'ReportedToolExecutionError'
151 )<ToolExecutionErrorFields> {}
152
ts/packages/cli/src/services/run-helpers-runtime.ts
+7 -1
@@ -439,6 +439,9 @@ const createExecutePayloadMaterializer = (params: {
439 return materializeExecutePayload;
440 };
441
442 +const isReportedFailure = (result: RunCliResult): boolean =>
443 + Predicate.isObject(result) && result.successful === false;
444 +
445 const createCliRunner = (params: {
446 readonly cliPrefix: ReadonlyArray<string>;
447 readonly helperContext: RunHelperContext;
@@ -549,7 +552,10 @@ const createCliRunner = (params: {
552 )
553 );
554 const result = maybeLoadStoredCliResult(parseJson(stdout));
552 - if (exitCode !== 0) {
555 + // A failed execution exits 1 with its `successful: false` result on stdout. That result is
556 + // the answer, so it goes back to the caller (`execute()` throws its error message) instead
557 + // of a generic exit-code error.
558 + if (exitCode !== 0 && !isReportedFailure(result)) {
559 perfDebugLog('error', requestId, { exitCode, stderr: stderr.trim() || undefined });
560 helperDebugLog('cli.error', {
561 requestId,
ts/packages/cli/test/src/cli-main.test.ts
+211
@@ -1,6 +1,8 @@
1 import fs from 'node:fs';
2 import path from 'node:path';
3 import { spawn, spawnSync } from 'node:child_process';
4 +import type { IncomingMessage, ServerResponse } from 'node:http';
5 +import { Schema } from 'effect';
6 import { describe, expect, it } from 'vitest';
7 import * as tempy from 'tempy';
8 import { withHttpServer } from 'test/__utils__/http-server';
@@ -288,3 +290,212 @@ describe('CLI backend of a stored login', () => {
290 );
291 });
292 });
293 +
294 +// A stand-in for the backend and the Dashboard: the backend calls `composio execute` makes before
295 +// it runs a tool get minimal answers, and the Dashboard's execute procedures answer per tool slug.
296 +const DashboardRequest = Schema.fromJsonString(
297 + Schema.Struct({
298 + json: Schema.Struct({
299 + tool_slug: Schema.optional(Schema.String),
300 + slug: Schema.optional(Schema.String),
301 + }),
302 + })
303 +);
304 +const decodeDashboardRequest = Schema.decodeUnknownSync(DashboardRequest);
305 +
306 +const sendJson = (res: ServerResponse, status: number, body: unknown) => {
307 + res.writeHead(status, { 'content-type': 'application/json' });
308 + res.end(JSON.stringify(body));
309 +};
310 +
311 +const dashboardReply = (response: Record<string, unknown>) => ({
312 + result: { data: { json: { ok: true, response } } },
313 +});
314 +
315 +const fakeComposio = (req: IncomingMessage, res: ServerResponse) => {
316 + const { pathname } = new URL(req.url ?? '/', 'http://127.0.0.1');
317 + const chunks: Array<Buffer> = [];
318 + req.on('data', (chunk: Buffer) => chunks.push(chunk));
319 + req.on('end', () => {
320 + if (pathname.startsWith('/api/cli/trpc/')) {
321 + const { json } = decodeDashboardRequest(Buffer.concat(chunks).toString('utf8'));
322 + const slug = json.tool_slug ?? json.slug;
323 + if (slug === 'HACKERNEWS_GET_FRONTPAGE') {
324 + return sendJson(res, 200, dashboardReply({ result_type: 'completed', data: { ok: 1 } }));
325 + }
326 + if (slug === 'HACKERNEWS_GET_USER' || slug === 'COMPOSIO_SEARCH_TOOLS') {
327 + return sendJson(
328 + res,
329 + 200,
330 + dashboardReply({ result_type: 'failed', data: {}, error: 'User not found' })
331 + );
332 + }
333 + return sendJson(res, 503, { error: { message: 'Service unavailable' } });
334 + }
335 + if (pathname.endsWith('/org/consumer/project/resolve')) {
336 + return sendJson(res, 200, {
337 + org_id: 'org_test',
338 + project_id: 'proj_test',
339 + project_nano_id: 'pr_test',
340 + project_name: 'consumer',
341 + consumer_user_id: 'consumer-user-test',
342 + });
343 + }
344 + const toolSlug = /^\/api\/v3\.1\/tools\/([A-Z_]+)$/u.exec(pathname)?.[1];
345 + if (toolSlug !== undefined) {
346 + return sendJson(res, 200, {
347 + slug: toolSlug,
348 + name: toolSlug,
349 + description: '',
350 + available_versions: ['20260101_00'],
351 + input_parameters: { type: 'object', properties: {} },
352 + output_parameters: {},
353 + tags: [],
354 + });
355 + }
356 + if (pathname.endsWith('/toolkits') || pathname.endsWith('/connected_accounts')) {
357 + return sendJson(res, 200, { items: [], next_cursor: null, total_pages: 1, current_page: 1 });
358 + }
359 + return sendJson(res, 404, { error: { message: `Not found: ${pathname}` } });
360 + });
361 +};
362 +
363 +/** Shorter than the tests' 30s timeout. */
364 +const CLI_DEADLINE_MS = 20_000;
365 +
366 +const runCliAgainst = (baseUrl: string, args: ReadonlyArray<string>) => {
367 + const configDirectory = tempy.temporaryDirectory();
368 + fs.writeFileSync(
369 + path.join(configDirectory, 'user_data.json'),
370 + JSON.stringify({
371 + api_key: 'uak_test_api_key',
372 + base_url: baseUrl,
373 + web_url: baseUrl,
374 + org_id: 'org_test',
375 + test_user_id: null,
376 + })
377 + );
378 + // A host key would replace the stored one.
379 + const {
380 + COMPOSIO_USER_API_KEY: _apiKey,
381 + COMPOSIO_ENVIRONMENT: _environment,
382 + ...host
383 + } = process.env;
384 + // Asynchronous: the fake server answers from this process's event loop.
385 + return new Promise<{ status: number | null; stdout: string; stderr: string }>(
386 + (resolve, reject) => {
387 + const child = spawn('bun', ['run', 'src/bin.ts', ...args], {
388 + cwd: process.cwd(),
389 + env: {
390 + ...host,
391 + CI: '1',
392 + NO_COLOR: '1',
393 + COMPOSIO_CACHE_DIR: configDirectory,
394 + COMPOSIO_BASE_URL: baseUrl,
395 + COMPOSIO_WEB_URL: baseUrl,
396 + COMPOSIO_CLI_TELEMETRY_DISABLED: '1',
397 + },
398 + });
399 + let stdout = '';
400 + let stderr = '';
401 + child.stdout.setEncoding('utf8').on('data', (chunk: string) => (stdout += chunk));
402 + child.stderr.setEncoding('utf8').on('data', (chunk: string) => (stderr += chunk));
403 + // Stop a hung CLI before the test times out, so the fake server can close.
404 + const deadline = setTimeout(() => {
405 + child.kill('SIGKILL');
406 + reject(new Error(`CLI did not exit within ${CLI_DEADLINE_MS}ms\nstderr:\n${stderr}`));
407 + }, CLI_DEADLINE_MS);
408 + child.once('error', error => {
409 + clearTimeout(deadline);
410 + reject(error);
411 + });
412 + child.once('close', status => {
413 + clearTimeout(deadline);
414 + resolve({ status, stdout, stderr });
415 + });
416 + }
417 + );
418 +};
419 +
420 +describe('reported tool execution failures', () => {
421 + it.each([
422 + {
423 + name: 'a tool that reports a failure',
424 + args: ['execute', 'HACKERNEWS_GET_USER', '-d', '{"username":"nobody"}'],
425 + stdout: { successful: false, data: {}, error: 'User not found' },
426 + },
427 + {
428 + name: 'a Dashboard outage',
429 + args: ['execute', 'HACKERNEWS_GET_ITEM', '-d', '{"id":1}'],
430 + stdout: { successful: false, error: expect.stringContaining('(HTTP 503)') },
431 + },
432 + {
433 + name: 'a meta tool that reports a failure',
434 + args: ['execute', 'COMPOSIO_SEARCH_TOOLS', '-d', '{"queries":[]}'],
435 + stdout: { successful: false, data: {}, error: 'User not found' },
436 + },
437 + {
438 + name: 'a parallel run with one failed call',
439 + args: [
440 + 'execute',
441 + '--parallel',
442 + 'HACKERNEWS_GET_FRONTPAGE',
443 + '-d',
444 + '{}',
445 + 'HACKERNEWS_GET_USER',
446 + '-d',
447 + '{"username":"nobody"}',
448 + ],
449 + stdout: {
450 + successful: false,
451 + parallel: true,
452 + results: [
453 + { slug: 'HACKERNEWS_GET_FRONTPAGE', successful: true },
454 + { slug: 'HACKERNEWS_GET_USER', successful: false, error: 'User not found' },
455 + ],
456 + },
457 + },
458 + ])(
459 + '[Given] $name [Then] prints only the JSON result and exits 1',
460 + ({ args, stdout }) =>
461 + withHttpServer(fakeComposio, async baseUrl => {
462 + const result = await runCliAgainst(baseUrl, args);
463 +
464 + expect(JSON.parse(result.stdout)).toMatchObject(stdout);
465 + expect(result.stderr).toBe('');
466 + expect(result.status).toBe(1);
467 + }),
468 + 30_000
469 + );
470 +
471 + it.each([
472 + {
473 + name: 'a successful call',
474 + args: ['execute', 'HACKERNEWS_GET_FRONTPAGE', '-d', '{}'],
475 + },
476 + {
477 + name: 'a parallel run where every call succeeds',
478 + args: [
479 + 'execute',
480 + '--parallel',
481 + 'HACKERNEWS_GET_FRONTPAGE',
482 + '-d',
483 + '{}',
484 + 'HACKERNEWS_GET_FRONTPAGE',
485 + '-d',
486 + '{}',
487 + ],
488 + },
489 + ])(
490 + '[Given] $name [Then] exits 0',
491 + ({ args }) =>
492 + withHttpServer(fakeComposio, async baseUrl => {
493 + const result = await runCliAgainst(baseUrl, args);
494 +
495 + expect(JSON.parse(result.stdout)).toMatchObject({ successful: true });
496 + expect(result.stderr).toBe('');
497 + expect(result.status).toBe(0);
498 + }),
499 + 30_000
500 + );
501 +});
ts/packages/cli/test/src/services/run-helpers-runtime.test.ts
+30
@@ -342,6 +342,36 @@ describe('run-helpers-runtime', () => {
342 expect(fetchMock).toHaveBeenCalledTimes(2);
343 });
344
345 + it('[Given] `composio execute` exits 1 with a failed result [Then] execute() rejects with the result error', async () => {
346 + const childScript = [
347 + 'console.log(JSON.stringify({ successful: false, data: {}, error: "User not found" }));',
348 + 'process.exit(1);',
349 + ].join('\n');
350 + await installRunHelpers({ cliPrefix: [process.execPath, '-e', childScript] });
351 +
352 + const returned = readInstalledFunction('execute')('HACKERNEWS_GET_USER', {
353 + username: 'nobody',
354 + });
355 +
356 + await expect(returned).rejects.toBeInstanceOf(Error);
357 + await expect(returned).rejects.toMatchObject({
358 + message: 'User not found',
359 + slug: 'HACKERNEWS_GET_USER',
360 + result: { successful: false, error: 'User not found' },
361 + });
362 + });
363 +
364 + it('[Given] a CLI child exits non-zero without a failed result [Then] the helper rejects with the exit code', async () => {
365 + const childScript = ['console.error("boom");', 'process.exit(2);'].join('\n');
366 + await installRunHelpers({ cliPrefix: [process.execPath, '-e', childScript] });
367 +
368 + await expect(readInstalledFunction('search')('test')).rejects.toMatchObject({
369 + message: expect.stringContaining('failed with exit code 2'),
370 + exitCode: 2,
371 + stderr: 'boom',
372 + });
373 + });
374 +
375 it('[Given] normalized false flags [Then] nested CLI children receive false explicitly', async () => {
376 vi.stubEnv('COMPOSIO_RUN_ENV_SENTINEL', 'forwarded');
377 vi.stubEnv('COMPOSIO_PERF_DEBUG', '1');