@devries-cameron20 / composio Mirror · read-only / commits / 7cf62c774
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

Mirrored from https://github.com/Camo67/composio.git

Synced Nov 21, 07:15 UTC

fix(cli): correct telemetry attribution and stale e2e images (#5181)

This PR: - derives telemetry option types and effective defaults from the original command definitions through typed `withCommandTelemetry` callbacks - keeps explicit safe-field selections beside each command, shares them across root and `dev`, and summarizes private input before event construction - removes the copied input schema, synthetic defaults, and redundant property builders while reusing shared lifecycle metadata - attributes redundant help to its displayed target and omits command input properties when help or parse errors bypass parsing - reports canonical enabled telemetry flag names without emitting raw argv, script text, credentials, bodies, or header values - rebuilds local CLI images through Docker's content cache and reuses CI prebuilds only when their revision label matches the checkout - fails a rebuild instead of falling back to a stale tag and adds image-policy checks to CLI E2E CI Validation: 2,936 CLI tests passed with one skipped. Workspace typechecking, lint, formatting, and CLI skill certification passed, including 600 selected invocation contracts and Effect diagnostics. All 16 help/parse-failure omission regressions fail against the previous builders. A negative typecheck also confirmed that a misspelled projected option fails compilation. Image-policy validation: four policy tests passed, and fresh-source Docker suites passed nine version assertions and eight run assertions. The Docker changes are unchanged since that validation. Existing event families and invoked/outcome pairs remain in use. Option properties describe parsed effective values. Only commands that declare a projection contribute option names to generic events. A new flag needs no telemetry entry unless it should be measured. Help and parse failures retain available invocation metadata while omitting unparsed input properties. VHS recordings are omitted because the change repairs internal telemetry and test infrastructure without introducing a command surface.

Alberto Schiabel committed Nov 2, 2026 at 00:33 UTC 7cf62c774a9a4f65cbd781a50c65c3123886f884
25 files changed +644 -266
.github/workflows/ts.test-e2e.yml
+4
@@ -215,6 +215,9 @@ jobs:
215 echo "available=true" >> "$GITHUB_OUTPUT"
216 echo "✅ API keys are configured"
217
218 + - name: Verify CLI image reuse policy
219 + run: pnpm --filter @e2e-tests/utils test
220 +
221 - name: Read CLI version
222 id: cli-version
223 run: |
@@ -240,6 +243,7 @@ jobs:
243 composio.e2e=true
244 composio.runtime=cli
245 composio.cli_version=${{ steps.cli-version.outputs.value }}
246 + org.opencontainers.image.revision=${{ github.sha }}
247
248 - name: Run CLI E2E Tests
249 if: steps.api-keys.outputs.available == 'true'
ts/e2e-tests/_utils/README.md
+6
@@ -194,6 +194,12 @@ The following versions are pre-defined in `const.ts`:
194 - `26.12.1`
195 - `current` (resolves to `mise.toml` version)
196
197 +### CLI image freshness
198 +
199 +Local CLI suites run `docker build` before using an image. Docker checks the current build inputs against its content cache, including uncommitted source and local build outputs. The CLI version tag alone does not establish freshness.
200 +
201 +GitHub Actions can reuse its prebuilt image when the `org.opencontainers.image.revision` label matches the checkout's commit. A missing or different label triggers a rebuild. A failed rebuild fails the test setup instead of accepting an existing tag.
202 +
203 ### CLI Version Resolution
204
205 CLI versions to test are resolved in this order:
ts/e2e-tests/_utils/package.json
+2 -1
@@ -22,7 +22,8 @@
22 "scripts": {
23 "docker:build": "bun run scripts/docker-build.ts",
24 "docker:clean": "bun run scripts/docker-clean.ts",
25 - "typecheck": "tsc --noEmit"
25 + "typecheck": "tsc --noEmit",
26 + "test": "bun test test/image-lifecycle.test.ts"
27 },
28 "devDependencies": {
29 "typescript": "catalog:"
ts/e2e-tests/_utils/scripts/docker-build.ts
+7 -1
@@ -114,7 +114,13 @@ async function buildCliImage(
114 ): Promise<boolean> {
115 const dockerfilePath = resolve(repoRoot, 'ts/e2e-tests/_utils/Dockerfile.cli');
116 const imageTag = imageTagForCliVersion(cliVersion);
117 - const labels = defaultCliLabels(cliVersion);
117 + const labels = {
118 + ...defaultCliLabels(cliVersion),
119 + 'org.opencontainers.image.revision': execFileSync('git', ['rev-parse', 'HEAD'], {
120 + cwd: repoRoot,
121 + encoding: 'utf8',
122 + }).trim(),
123 + };
124 const labelArgs = Object.entries(labels)
125 .map(([k, v]) => `--label=${k}=${v}`)
126 .join(' ');
ts/e2e-tests/_utils/src/image-lifecycle.ts
+22 -10
@@ -412,9 +412,24 @@ export async function ensureCliImage(
412 const imageTag = imageTagForCliVersion(cliVersion);
413 const nodeVersion = await getMiseVersion('node', repoRoot);
414
415 - const inspect = await exec('docker', ['image', 'inspect', imageTag], { cwd: repoRoot });
416 - if (inspect.exitCode === 0) {
417 - return imageTag;
415 + const revision = await exec('git', ['rev-parse', 'HEAD'], { cwd: repoRoot });
416 + if (revision.exitCode !== 0) throw new Error('Cannot identify the CLI checkout revision');
417 + const sourceRevision = revision.stdout.trim();
418 + // CI prebuilds this image from the checkout. Locally, always let Docker compare COPY
419 + // inputs, including dirty and ignored build outputs, against its content cache.
420 + if (process.env.GITHUB_ACTIONS === 'true') {
421 + const inspect = await exec(
422 + 'docker',
423 + [
424 + 'image',
425 + 'inspect',
426 + '--format',
427 + '{{index .Config.Labels "org.opencontainers.image.revision"}}',
428 + imageTag,
429 + ],
430 + { cwd: repoRoot }
431 + );
432 + if (inspect.exitCode === 0 && inspect.stdout.trim() === sourceRevision) return imageTag;
433 }
434
435 const buildArgs = [
@@ -427,7 +442,10 @@ export async function ensureCliImage(
442 `NODE_VERSION=${nodeVersion}`,
443 '--build-arg',
444 `NODE_VERSION_DIGEST=${nodeDigestFor(nodeVersion)}`,
430 - ...labelsToArgs(defaultCliLabels(cliVersion)),
445 + ...labelsToArgs({
446 + ...defaultCliLabels(cliVersion),
447 + 'org.opencontainers.image.revision': sourceRevision,
448 + }),
449 '-t',
450 imageTag,
451 repoRoot,
@@ -436,12 +454,6 @@ export async function ensureCliImage(
454 const built = await exec('docker', buildArgs, { cwd: repoRoot });
455 if (built.exitCode !== 0) {
456 const output = built.stderr || built.stdout;
439 - if (output.includes('already exists')) {
440 - const recheck = await exec('docker', ['image', 'inspect', imageTag], { cwd: repoRoot });
441 - if (recheck.exitCode === 0) {
442 - return imageTag;
443 - }
444 - }
457
458 const err = new Error(`Failed to build Docker image ${imageTag}`);
459 (err as Error & { cause: Error }).cause = new Error(output);
ts/e2e-tests/_utils/test/image-lifecycle.test.ts new
+91
@@ -0,0 +1,91 @@
1 +import { afterEach, beforeEach, describe, expect, test } from 'bun:test';
2 +import { mkdtempSync, readFileSync, writeFileSync, chmodSync, rmSync } from 'node:fs';
3 +import { tmpdir } from 'node:os';
4 +import { join } from 'node:path';
5 +import { ensureCliImage } from '../src/image-lifecycle';
6 +import { getRepoRoot } from '../src/config';
7 +
8 +let directory: string;
9 +let previousPath: string | undefined;
10 +let previousActions: string | undefined;
11 +
12 +beforeEach(() => {
13 + directory = mkdtempSync(join(tmpdir(), 'cli-image-reuse-'));
14 + previousPath = process.env.PATH;
15 + previousActions = process.env.GITHUB_ACTIONS;
16 + process.env.PATH = `${directory}:${previousPath}`;
17 + delete process.env.GITHUB_ACTIONS;
18 + process.env.CLI_IMAGE_TEST_DIR = directory;
19 + const docker = join(directory, 'docker');
20 + writeFileSync(
21 + docker,
22 + `#!/bin/sh
23 +if [ "$1" = image ]; then
24 + cat "$CLI_IMAGE_TEST_DIR/revision"
25 + exit 0
26 +fi
27 +printf '%s\\n' "$*" >> "$CLI_IMAGE_TEST_DIR/builds"
28 +if [ -f "$CLI_IMAGE_TEST_DIR/fail" ]; then
29 + echo 'already exists' >&2
30 + exit 1
31 +fi
32 +cat "$CLI_IMAGE_TEST_DIR/source" > "$CLI_IMAGE_TEST_DIR/built-source"
33 +`
34 + );
35 + chmodSync(docker, 0o755);
36 + writeFileSync(join(directory, 'revision'), 'old-revision');
37 + writeFileSync(join(directory, 'source'), 'first-source');
38 +});
39 +
40 +afterEach(() => {
41 + if (previousPath === undefined) delete process.env.PATH;
42 + else process.env.PATH = previousPath;
43 + if (previousActions === undefined) delete process.env.GITHUB_ACTIONS;
44 + else process.env.GITHUB_ACTIONS = previousActions;
45 + delete process.env.CLI_IMAGE_TEST_DIR;
46 + rmSync(directory, { recursive: true, force: true });
47 +});
48 +
49 +const currentRevision = () =>
50 + Bun.spawnSync(['git', 'rev-parse', 'HEAD'], {
51 + cwd: getRepoRoot(),
52 + stdout: 'pipe',
53 + stderr: 'pipe',
54 + })
55 + .stdout.toString()
56 + .trim();
57 +
58 +const options = () => ({ repoRoot: getRepoRoot() });
59 +
60 +describe('CLI image reuse', () => {
61 + test('refreshes local source even when the version and revision are unchanged', async () => {
62 + writeFileSync(join(directory, 'revision'), currentRevision());
63 + await ensureCliImage('0.0.0-development', options());
64 + writeFileSync(join(directory, 'source'), 'dirty-source');
65 + await ensureCliImage('0.0.0-development', options());
66 + expect(readFileSync(join(directory, 'built-source'), 'utf8')).toBe('dirty-source');
67 + expect(readFileSync(join(directory, 'builds'), 'utf8').trim().split('\n')).toHaveLength(2);
68 + });
69 +
70 + test('rebuilds a CI image carrying a different source revision', async () => {
71 + process.env.GITHUB_ACTIONS = 'true';
72 + await ensureCliImage('0.0.0-development', options());
73 + expect(readFileSync(join(directory, 'builds'), 'utf8')).toContain(
74 + `org.opencontainers.image.revision=${currentRevision()}`
75 + );
76 + });
77 +
78 + test('reuses the CI prebuild only for the checkout revision', async () => {
79 + process.env.GITHUB_ACTIONS = 'true';
80 + writeFileSync(join(directory, 'revision'), currentRevision());
81 + await ensureCliImage('0.0.0-development', options());
82 + expect(Bun.file(join(directory, 'builds')).size).toBe(0);
83 + });
84 +
85 + test('does not accept a stale tag after a failed rebuild', async () => {
86 + writeFileSync(join(directory, 'fail'), '');
87 + await expect(ensureCliImage('0.0.0-development', options())).rejects.toThrow(
88 + 'Failed to build Docker image'
89 + );
90 + });
91 +});
ts/e2e-tests/_utils/tsconfig.json
+1 -1
@@ -11,5 +11,5 @@
11 "declaration": true,
12 "declarationMap": true
13 },
14 - "include": ["src/**/*.ts"]
14 + "include": ["src/**/*.ts", "test/**/*.ts"]
15 }
ts/packages/cli/CHANGELOG.md
+6
@@ -49,6 +49,12 @@
49
50 ### Patch Changes
51
52 +- Command telemetry now derives option types and effective defaults from each
53 + command's parser. Commands explicitly select safe properties to measure, and
54 + shared root and `dev` definitions reuse that selection. Help and parse failures
55 + omit command input properties instead of reporting synthetic defaults, false
56 + booleans, or zero counts for input that was never parsed.
57 +
58 - `composio dev triggers listen --toolkits` now matches toolkits whose slug
59 contains an underscore, such as `one_drive`, `launch_darkly` and
60 `agent_mail`. Events carry no toolkit slug, so the listener took the first
ts/packages/cli/CONTRIBUTING.md
+4
@@ -108,6 +108,10 @@ For a selected command, registration begins its identity and runs its enabled in
108
109 Explicit help, version, and parse failures follow separate paths and skip selected-command hooks. [command-lifecycle-telemetry.ts](src/services/command-lifecycle-telemetry.ts) defines the lifecycle interface. A command declares shared policy through `withCommandMeta`, rather than adding another argv parser or lifecycle reporter.
110
111 +Commands that report option properties attach `withCommandTelemetry` to their definition. Its callback receives the command's inferred, parsed input, including defaults from `Flag.withDefault`. [proxy.cmd.ts](src/commands/proxy/proxy.cmd.ts) shows a projection that reports the method, body presence, and header count while excluding the endpoint, body, header values, and account selector from events. [command-input.ts](src/analytics/command-input.ts) builds the safe properties and canonical enabled flag names from that explicit selection. A new flag needs no analytics entry unless it should be measured. Shared definitions carry the same projection into both command trees, and subcommands declare their own projections.
112 +
113 +[events.ts](src/analytics/events.ts) constructs events from those safe properties and shared invocation metadata. It does not redeclare command input schemas or parser defaults. Help and parse failures have no parsed input, so their events omit command input properties. Parsed-input projections also summarize private inputs, such as search query length, before event construction.
114 +
115 ## Verify a command change
116
117 Commands below run from the repository root after the pinned toolchain and dependencies are installed, as described in the [repository setup](../../../CONTRIBUTING.md#development-setup).
ts/packages/cli/src/analytics/command-input.ts new
+43
@@ -0,0 +1,43 @@
1 +import { Option } from 'effect';
2 +import type { Command } from 'effect/cli';
3 +import type { CommandTelemetryInput } from './types';
4 +import { withCommandMeta } from 'src/invocation/command-identity';
5 +
6 +type Value = string | number | boolean | undefined;
7 +type Field = { readonly value: Value; readonly flags?: ReadonlyArray<string> };
8 +const value = (value: Value): Field => ({ value });
9 +const flag = (name: string, enabled: boolean): Field => ({
10 + value: enabled,
11 + flags: enabled ? [name] : [],
12 +});
13 +const present = (input: Option.Option<string> | undefined) =>
14 + input !== undefined && Option.isSome(input);
15 +const optional = (input: Option.Option<string> | undefined) =>
16 + input === undefined ? undefined : Option.getOrUndefined(input);
17 +
18 +/** One allowlist supplies both safe properties and canonical enabled flag names. */
19 +const fields = (
20 + input: Readonly<Record<string, Field>>,
21 + argument?: string
22 +): CommandTelemetryInput => ({
23 + argument,
24 + options: Object.fromEntries(
25 + Object.entries(input)
26 + .filter(([, field]) => field.value !== undefined)
27 + .map(([key, field]) => [key, field.value])
28 + ),
29 + flagNames: [...new Set(Object.values(input).flatMap(field => field.flags ?? []))].sort(),
30 +});
31 +
32 +/** Select only measured fields. Neither values nor flag names are taken from raw argv. */
33 +export const telemetry = { fields, value, flag, present, optional };
34 +
35 +/** Bind a privacy projection to the same command whose parser owns its input type. */
36 +export const withCommandTelemetry = <Name extends string, Input, ContextInput, E, R>(
37 + command: Command.Command<Name, Input, ContextInput, E, R>,
38 + project: (input: NoInfer<Input>) => CommandTelemetryInput
39 +): Command.Command<Name, Input, ContextInput, E, R> =>
40 + withCommandMeta({
41 + // Command annotations erase Input. Only this command's parsed input reaches this callback.
42 + telemetryInput: input => project(input as Input),
43 + })(command);
ts/packages/cli/src/analytics/events.ts
+32 -194
@@ -1,5 +1,5 @@
1 import { cliAnalyticsProvenance } from 'src/services/client-provenance';
2 -import { Option, Schema } from 'effect';
2 +import { Option } from 'effect';
3 import { CliError } from 'effect/cli';
4 import type { CliCommandTelemetryContext, TelemetryFamily, TrackEvent } from './types';
5 import { type CommandIdentity, commandPathOf } from 'src/invocation/command-identity';
@@ -171,16 +171,6 @@ const TOOL_VALIDATION_CODES: ReadonlySet<number> = new Set([
171 3702, // ComposioTools_ValidationError
172 ]);
173
174 -const extractFlagNames = (argv: ReadonlyArray<string>): ReadonlyArray<string> =>
175 - [
176 - ...new Set(
177 - argv
178 - .slice(2)
179 - .filter(token => token.startsWith('-'))
180 - .map(token => token.split('=')[0]!)
181 - ),
182 - ].sort();
183 -
174 const argumentShape = (args: Record<string, unknown>) => {
175 const keys = Object.keys(args).sort();
176 return {
@@ -223,205 +213,49 @@ const failureReasonProperties = (error: unknown) => {
213 return {};
214 };
215
226 -const isFlagPresent = (argv: ReadonlyArray<string>, ...flags: string[]): boolean =>
227 - argv.slice(2).some(token => {
228 - if (flags.includes(token)) return true;
229 - return flags.some(flag => token.startsWith(`${flag}=`));
230 - });
216 +const commandProperties = (context: CliCommandTelemetryContext) => ({
217 + ...context.options,
218 + source: 'cli',
219 + invocation_origin: context.invocationOrigin,
220 + cli_version: context.cliVersion,
221 + command_path: context.commandPath,
222 + duration_ms: Date.now() - context.startedAt,
223 +});
224
232 -const getFlagValue = (argv: ReadonlyArray<string>, ...flags: string[]): string | undefined => {
233 - const args = argv.slice(2);
234 - for (let index = 0; index < args.length; index += 1) {
235 - const token = args[index];
236 - if (!token) continue;
237 -
238 - for (const flag of flags) {
239 - if (token === flag) {
240 - const next = args[index + 1];
241 - return next && !next.startsWith('-') ? next : undefined;
242 - }
243 - if (token.startsWith(`${flag}=`)) {
244 - return token.slice(flag.length + 1);
245 - }
246 - }
247 - }
248 - return undefined;
249 -};
225 +const parentProperties = (context: CliCommandTelemetryContext) => ({
226 + ...commandProperties(context),
227 + parent_run_id: context.parentRunId,
228 + parent_command: context.parentRunId ? 'run' : undefined,
229 +});
230
231 const getExecuteCommandProperties = (context: CliCommandTelemetryContext) => {
232 const slug = context.argument;
233 return {
254 - source: 'cli',
255 - invocation_origin: context.invocationOrigin,
256 - parent_run_id: context.parentRunId,
257 - parent_command: context.parentRunId ? 'run' : undefined,
258 - cli_version: context.cliVersion,
259 - command_path: context.commandPath,
260 - duration_ms: Date.now() - context.startedAt,
234 + ...parentProperties(context),
235 surface: context.commandPath === 'execute' ? 'root' : 'dev',
236 tool_slug: slug,
237 tool_name: slug,
238 toolkit_slug:
239 context.toolkitSlug ??
240 (typeof slug === 'string' ? guessToolkitFromToolSlug(slug) : undefined),
267 - dry_run: isFlagPresent(context.argv, '--dry-run'),
268 - get_schema: isFlagPresent(context.argv, '--get-schema'),
269 - has_data: isFlagPresent(context.argv, '--data', '-d'),
270 - skip_connection_check: isFlagPresent(context.argv, '--skip-connection-check'),
271 - skip_tool_params_check: isFlagPresent(context.argv, '--skip-tool-params-check'),
272 - skip_checks: isFlagPresent(context.argv, '--skip-checks'),
241 };
242 };
243
276 -// Search queries are user-authored free text, so only their shape is recorded.
277 -const getSearchCommandProperties = (context: CliCommandTelemetryContext) => {
278 - const query = context.argument ?? '';
279 - return {
280 - source: 'cli',
281 - invocation_origin: context.invocationOrigin,
282 - parent_run_id: context.parentRunId,
283 - parent_command: context.parentRunId ? 'run' : undefined,
284 - cli_version: context.cliVersion,
285 - command_path: context.commandPath,
286 - duration_ms: Date.now() - context.startedAt,
287 - query_length: query.length,
288 - query_term_count: query.split(/\s+/u).filter(Boolean).length,
289 - toolkits: getFlagValue(context.argv, '--toolkits'),
290 - limit: getFlagValue(context.argv, '--limit'),
291 - };
292 -};
293 -
294 -const getLinkCommandProperties = (context: CliCommandTelemetryContext) => {
295 - return {
296 - source: 'cli',
297 - invocation_origin: context.invocationOrigin,
298 - parent_run_id: context.parentRunId,
299 - parent_command: context.parentRunId ? 'run' : undefined,
300 - cli_version: context.cliVersion,
301 - command_path: context.commandPath,
302 - duration_ms: Date.now() - context.startedAt,
303 - toolkit: context.argument,
304 - no_browser: isFlagPresent(context.argv, '--no-browser'),
305 - no_wait: isFlagPresent(context.argv, '--no-wait'),
306 - has_auth_config: isFlagPresent(context.argv, '--auth-config'),
307 - };
308 -};
244 +const getLinkCommandProperties = (context: CliCommandTelemetryContext) => ({
245 + ...parentProperties(context),
246 + toolkit: context.argument,
247 +});
248
249 const getRunCommandProperties = (context: CliCommandTelemetryContext) => ({
311 - source: 'cli',
312 - invocation_origin: context.invocationOrigin,
250 + ...commandProperties(context),
251 run_id: context.runId,
314 - cli_version: context.cliVersion,
315 - command_path: context.commandPath,
316 - duration_ms: Date.now() - context.startedAt,
317 - debug: isFlagPresent(context.argv, '--debug'),
318 - dry_run: isFlagPresent(context.argv, '--dry-run'),
319 - skip_connection_check: isFlagPresent(context.argv, '--skip-connection-check'),
320 - skip_tool_params_check: isFlagPresent(context.argv, '--skip-tool-params-check'),
321 - skip_checks: isFlagPresent(context.argv, '--skip-checks'),
322 - file_mode: isFlagPresent(context.argv, '--file', '-f'),
323 - arg_count: Math.max(0, context.argv.length - 3),
324 -});
325 -
326 -const getInstallCommandProperties = (context: CliCommandTelemetryContext) => ({
327 - source: 'cli',
328 - invocation_origin: context.invocationOrigin,
329 - cli_version: context.cliVersion,
330 - command_path: context.commandPath,
331 - duration_ms: Date.now() - context.startedAt,
332 - completions: isFlagPresent(context.argv, '--completions'),
333 - no_completions: isFlagPresent(context.argv, '--no-completions'),
252 });
253
254 const getSetupCommandProperties = (context: CliCommandTelemetryContext) => ({
337 - source: 'cli',
338 - invocation_origin: context.invocationOrigin,
339 - cli_version: context.cliVersion,
340 - command_path: context.commandPath,
341 - duration_ms: Date.now() - context.startedAt,
342 - operation: isFlagPresent(context.argv, '--uninstall') ? 'uninstall' : 'setup',
343 - target: getFlagValue(context.argv, '--target') ?? 'auto',
344 - yes: isFlagPresent(context.argv, '--yes', '-y'),
345 - if_present: isFlagPresent(context.argv, '--if-present'),
255 + ...commandProperties(context),
256 stdout_is_tty: context.stdoutIsTTY,
257 });
258
349 -const getLoginCommandProperties = (context: CliCommandTelemetryContext) => ({
350 - source: 'cli',
351 - invocation_origin: context.invocationOrigin,
352 - cli_version: context.cliVersion,
353 - command_path: context.commandPath,
354 - duration_ms: Date.now() - context.startedAt,
355 - no_browser: isFlagPresent(context.argv, '--no-browser'),
356 - no_wait: isFlagPresent(context.argv, '--no-wait'),
357 - has_key: isFlagPresent(context.argv, '--key'),
358 - skip_org_picker: isFlagPresent(context.argv, '--yes', '-y'),
359 -});
360 -
361 -const getLogoutCommandProperties = (context: CliCommandTelemetryContext) => ({
362 - source: 'cli',
363 - invocation_origin: context.invocationOrigin,
364 - cli_version: context.cliVersion,
365 - command_path: context.commandPath,
366 - duration_ms: Date.now() - context.startedAt,
367 -});
368 -
369 -const getProxyCommandProperties = (context: CliCommandTelemetryContext) => ({
370 - source: 'cli',
371 - invocation_origin: context.invocationOrigin,
372 - cli_version: context.cliVersion,
373 - command_path: context.commandPath,
374 - duration_ms: Date.now() - context.startedAt,
375 - has_endpoint: typeof context.argument === 'string',
376 - toolkit: getFlagValue(context.argv, '--toolkit', '-t'),
377 - method: getFlagValue(context.argv, '--method', '-X') ?? 'GET',
378 - has_data: isFlagPresent(context.argv, '--data', '-d'),
379 - header_count: context.argv.slice(2).filter(token => token === '--header' || token === '-H')
380 - .length,
381 - skip_connection_check: isFlagPresent(context.argv, '--skip-connection-check'),
382 -});
383 -
384 -const firstOf = (values: ReadonlyArray<string>) => values[0];
385 -
386 -/**
387 - * How each family that describes an input argument finds it in the input `effect/cli` parsed for
388 - * the command (see `CliCommandTelemetryContext.argument`).
389 - */
390 -const FAMILY_ARGUMENTS: Partial<
391 - Record<TelemetryFamily, (input: unknown) => Option.Option<string>>
392 -> = {
393 - execute: input =>
394 - Option.map(
395 - Schema.decodeUnknownOption(Schema.Struct({ slug: Schema.String }))(input),
396 - ({ slug }) => slug
397 - ),
398 - search: input =>
399 - Option.flatMap(
400 - Schema.decodeUnknownOption(Schema.Struct({ query: Schema.Array(Schema.String) }))(input),
401 - ({ query }) => Option.fromNullishOr(firstOf(query))
402 - ),
403 - link: input =>
404 - Option.flatMap(
405 - Schema.decodeUnknownOption(Schema.Struct({ toolkit: Schema.Option(Schema.String) }))(input),
406 - ({ toolkit }) => toolkit
407 - ),
408 - proxy: input =>
409 - Option.map(
410 - Schema.decodeUnknownOption(Schema.Struct({ endpoint: Schema.String }))(input),
411 - ({ endpoint }) => endpoint
412 - ),
413 -};
414 -
415 -const familyArgument = (
416 - family: TelemetryFamily,
417 - input: Option.Option<unknown>
418 -): string | undefined =>
419 - Option.getOrUndefined(
420 - Option.flatMap(input, parsed =>
421 - Option.flatMap(Option.fromNullishOr(FAMILY_ARGUMENTS[family]), argument => argument(parsed))
422 - )
423 - );
424 -
259 /**
260 * Builds the telemetry context of the command `identity` names, with the input `effect/cli`
261 * parsed for it (none for help, version, and parse failures).
@@ -447,8 +281,12 @@ export const createCliCommandTelemetryContext = (params: {
281 parentRunId: invocation.parentRunId,
282 commandPath: commandPathOf(identity),
283 family,
450 - argument: familyArgument(family, params.input),
451 - flagNames: extractFlagNames(argv),
284 + ...Option.getOrElse(
285 + Option.flatMap(params.input, input =>
286 + Option.map(Option.fromNullishOr(identity.meta.telemetryInput), project => project(input))
287 + ),
288 + () => ({ options: {}, flagNames: [] })
289 + ),
290 stdoutIsTTY: terminal.stdoutIsTTY,
291 stderrIsTTY: terminal.stderrIsTTY,
292 startedAt: params.startedAt,
@@ -514,7 +352,7 @@ const SPECIAL_LIFECYCLE_FAMILIES: Record<
352 invokedEventName: CLI_ANALYTICS_EVENTS.CLI_SEARCH_INVOKED,
353 succeededEventName: CLI_ANALYTICS_EVENTS.CLI_SEARCH_SUCCEEDED,
354 failedEventName: CLI_ANALYTICS_EVENTS.CLI_SEARCH_FAILED,
517 - getProperties: getSearchCommandProperties,
355 + getProperties: parentProperties,
356 },
357 link: {
358 invokedEventName: CLI_ANALYTICS_EVENTS.CLI_LINK_INVOKED,
@@ -526,19 +364,19 @@ const SPECIAL_LIFECYCLE_FAMILIES: Record<
364 invokedEventName: CLI_ANALYTICS_EVENTS.CLI_LOGIN_INVOKED,
365 succeededEventName: CLI_ANALYTICS_EVENTS.CLI_LOGIN_SUCCEEDED,
366 failedEventName: CLI_ANALYTICS_EVENTS.CLI_LOGIN_FAILED,
529 - getProperties: getLoginCommandProperties,
367 + getProperties: commandProperties,
368 },
369 logout: {
370 invokedEventName: CLI_ANALYTICS_EVENTS.CLI_LOGOUT_INVOKED,
371 succeededEventName: CLI_ANALYTICS_EVENTS.CLI_LOGOUT_SUCCEEDED,
372 failedEventName: CLI_ANALYTICS_EVENTS.CLI_LOGOUT_FAILED,
535 - getProperties: getLogoutCommandProperties,
373 + getProperties: commandProperties,
374 },
375 proxy: {
376 invokedEventName: CLI_ANALYTICS_EVENTS.CLI_PROXY_INVOKED,
377 succeededEventName: CLI_ANALYTICS_EVENTS.CLI_PROXY_SUCCEEDED,
378 failedEventName: CLI_ANALYTICS_EVENTS.CLI_PROXY_FAILED,
541 - getProperties: getProxyCommandProperties,
379 + getProperties: commandProperties,
380 },
381 run: {
382 invokedEventName: CLI_ANALYTICS_EVENTS.CLI_RUN_INVOKED,
@@ -550,7 +388,7 @@ const SPECIAL_LIFECYCLE_FAMILIES: Record<
388 invokedEventName: CLI_ANALYTICS_EVENTS.CLI_INSTALL_INVOKED,
389 succeededEventName: CLI_ANALYTICS_EVENTS.CLI_INSTALL_SUCCEEDED,
390 failedEventName: CLI_ANALYTICS_EVENTS.CLI_INSTALL_FAILED,
553 - getProperties: getInstallCommandProperties,
391 + getProperties: commandProperties,
392 },
393 setup: {
394 invokedEventName: CLI_ANALYTICS_EVENTS.CLI_SETUP_INVOKED,
ts/packages/cli/src/analytics/types.ts
+13 -3
@@ -28,6 +28,13 @@ export type TelemetryFamily =
28 | 'install'
29 | 'setup';
30
31 +/** Explicit projection of parser-owned input; never the full command input object. */
32 +export type CommandTelemetryInput = {
33 + readonly argument?: string;
34 + readonly options: Readonly<Record<string, string | number | boolean | undefined>>;
35 + readonly flagNames: ReadonlyArray<string>;
36 +};
37 +
38 export type CliCommandTelemetryContext = {
39 readonly argv: ReadonlyArray<string>;
40 readonly cliVersion: string;
@@ -38,12 +45,15 @@ export type CliCommandTelemetryContext = {
45 readonly commandPath: string;
46 readonly family: TelemetryFamily;
47 /**
41 - * The input argument the family's events describe, from the input `effect/cli` parsed: the tool
42 - * slug of `execute`, the first query of `search`, the toolkit of `link`, the endpoint of
43 - * `proxy`. Unset for other families and when nothing was parsed (help, version, parse errors).
48 + * The safe argument the family's events describe: the tool slug of `execute` or the toolkit
49 + * of `link`, including shared definitions registered under `dev`. Unset when nothing was
50 + * parsed (help, version, parse errors) or the command declares no argument projection.
51 */
52 readonly argument?: string;
53 + /** Canonical configured telemetry options, never raw argv tokens. Empty without parsed input. */
54 readonly flagNames: ReadonlyArray<string>;
55 + /** Safe option properties projected from parsed input. No credentials, bodies, or script source. */
56 + readonly options: Readonly<Record<string, string | number | boolean | undefined>>;
57 readonly stdoutIsTTY: boolean;
58 readonly stderrIsTTY: boolean;
59 readonly startedAt: number;
ts/packages/cli/src/commands/connected-accounts/link/connected-accounts.link.cmd.ts
+15 -1
@@ -1,3 +1,4 @@
1 +import { telemetry, withCommandTelemetry } from 'src/analytics/command-input';
2 import { Argument, Command, Flag } from 'effect/cli';
3 import { withCommandMeta } from 'src/invocation/command-identity';
4 import { runConnectedAccountsLink } from './connected-accounts.link.handler';
@@ -48,7 +49,7 @@ const linkExamples = [
49 },
50 ];
51
51 -export const connectedAccountsCmd$Link = Command.make(
52 +const connectedAccountsCmd$LinkBase = Command.make(
53 'link',
54 { toolkit, authConfig, noWait, noBrowser, alias, list },
55 runConnectedAccountsLink
@@ -62,3 +63,16 @@ export const connectedAccountsCmd$Link = Command.make(
63 Command.withExamples(linkExamples),
64 withCommandMeta({ requiresLogin: true })
65 );
66 +
67 +export const connectedAccountsCmd$Link = withCommandTelemetry(
68 + connectedAccountsCmd$LinkBase,
69 + input =>
70 + telemetry.fields(
71 + {
72 + no_browser: telemetry.flag('--no-browser', input.noBrowser),
73 + no_wait: telemetry.flag('--no-wait', input.noWait),
74 + has_auth_config: telemetry.flag('--auth-config', telemetry.present(input.authConfig)),
75 + },
76 + telemetry.optional(input.toolkit)
77 + )
78 +);
ts/packages/cli/src/commands/install/install.cmd.ts
+9 -1
@@ -1,3 +1,4 @@
1 +import { telemetry, withCommandTelemetry } from 'src/analytics/command-input';
2 import { Command, Flag } from 'effect/cli';
3 import { SHELLS } from 'src/services/shell-config';
4 import { runInstall } from './install.handler';
@@ -34,7 +35,7 @@ const shellOpt = Flag.Literals('shell', SHELLS).pipe(
35 * COMPOSIO_BIN_DIR=~/.local/bin composio install
36 * ```
37 */
37 -export const installCmd = Command.make(
38 +const installCmdBase = Command.make(
39 'install',
40 { completions: completionsOpt, noCompletions: noCompletionsOpt, shell: shellOpt },
41 runInstall
@@ -43,3 +44,10 @@ export const installCmd = Command.make(
44 'Set up shell integration (PATH and completions). Set COMPOSIO_BIN_DIR to choose the directory added to PATH; it otherwise defaults to ~/.local/bin when that holds this executable, and to the directory of the running binary.'
45 )
46 );
47 +
48 +export const installCmd = withCommandTelemetry(installCmdBase, input =>
49 + telemetry.fields({
50 + completions: telemetry.flag('--completions', input.completions),
51 + no_completions: telemetry.flag('--no-completions', input.noCompletions),
52 + })
53 +);
ts/packages/cli/src/commands/login/login.cmd.ts
+17 -1
@@ -1,3 +1,4 @@
1 +import { telemetry, withCommandTelemetry } from 'src/analytics/command-input';
2 import { Command, Flag } from 'effect/cli';
3 import { runLogin } from './login.handler';
4
@@ -77,7 +78,7 @@ const agentOpt = Flag.Boolean('agent').pipe(
78 * composio login -y
79 * ```
80 */
80 -export const loginCmd = Command.make(
81 +const loginCmdBase = Command.make(
82 'login',
83 {
84 noBrowser,
@@ -92,3 +93,18 @@ export const loginCmd = Command.make(
93 },
94 runLogin
95 ).pipe(Command.withDescription('Log in to the Composio SDK.'));
96 +
97 +export const loginCmd = withCommandTelemetry(loginCmdBase, input =>
98 + telemetry.fields({
99 + no_browser: telemetry.flag('--no-browser', input.noBrowser),
100 + no_wait: telemetry.flag('--no-wait', input.noWait),
101 + has_key: {
102 + value: telemetry.present(input.key) || telemetry.present(input.userApiKey),
103 + flags: [
104 + ...(telemetry.present(input.key) ? ['--key'] : []),
105 + ...(telemetry.present(input.userApiKey) ? ['--user-api-key'] : []),
106 + ],
107 + },
108 + skip_org_picker: telemetry.flag('--yes', input.yes),
109 + })
110 +);
ts/packages/cli/src/commands/proxy/proxy.cmd.ts
+16 -1
@@ -1,3 +1,4 @@
1 +import { telemetry, withCommandTelemetry } from 'src/analytics/command-input';
2 import { withCommandMeta } from 'src/invocation/command-identity';
3 import { Argument, Command, Flag } from 'effect/cli';
4 import { runProxy } from './proxy.handler';
@@ -43,7 +44,7 @@ const skipConnectionCheck = Flag.Boolean('skip-connection-check').pipe(
44 )
45 );
46
46 -export const proxyCmd = Command.make(
47 +const proxyCmdBase = Command.make(
48 'proxy',
49 {
50 endpoint,
@@ -74,3 +75,17 @@ export const proxyCmd = Command.make(
75 ]),
76 withCommandMeta({ requiresLogin: true })
77 );
78 +
79 +export const proxyCmd = withCommandTelemetry(proxyCmdBase, input =>
80 + telemetry.fields({
81 + toolkit: telemetry.value(input.toolkit),
82 + method: telemetry.value(input.method),
83 + has_endpoint: telemetry.value(true),
84 + has_data: telemetry.flag('--data', telemetry.present(input.data)),
85 + header_count: {
86 + value: input.headers?.length ?? 0,
87 + flags: input.headers?.length ? ['--header'] : [],
88 + },
89 + skip_connection_check: telemetry.flag('--skip-connection-check', input.skipConnectionCheck),
90 + })
91 +);
ts/packages/cli/src/commands/root-help.ts
+9 -12
@@ -89,18 +89,15 @@ export const buildHelpCommand = (commands: ReadonlyArray<Command.Command.Any>) =
89 */
90 export const Help = GlobalFlag.Action({
91 flag: GlobalFlag.Help.flag,
92 - run: (value, context) =>
93 - beginAtCommandPath(context.command, context.commandPath).pipe(
94 - Effect.andThen(
95 - GlobalFlag.Help.run(value, {
96 - ...context,
97 - commandPath:
98 - context.commandPath[1] === 'help'
99 - ? [context.commandPath[0], ...context.commandPath.slice(2)]
100 - : context.commandPath,
101 - })
102 - )
103 - ),
92 + run: (value, context) => {
93 + const commandPath =
94 + context.commandPath[1] === 'help'
95 + ? [context.commandPath[0], ...context.commandPath.slice(2)]
96 + : context.commandPath;
97 + return beginAtCommandPath(context.command, commandPath).pipe(
98 + Effect.andThen(GlobalFlag.Help.run(value, { ...context, commandPath }))
99 + );
100 + },
101 });
102
103 /** `--version`: begins the command it was given on (no handler runs), then prints the version. */
ts/packages/cli/src/commands/run/run.cmd.ts
+14 -1
@@ -1,3 +1,4 @@
1 +import { telemetry, withCommandTelemetry } from 'src/analytics/command-input';
2 import { Argument, Command, Flag } from 'effect/cli';
3 import { CLI_DEBUG_FLAG_NAMES } from 'src/services/runtime-flags';
4 import { runScript } from './run.handler';
@@ -66,7 +67,7 @@ const args = Argument.String('arg').pipe(
67 Argument.withDescription('Inline code followed by arguments, or just arguments when using --file')
68 );
69
69 -export const runCmd = Command.make(
70 +const runCmdBase = Command.make(
71 'run',
72 {
73 ...runFlags,
@@ -115,3 +116,15 @@ export const runCmd = Command.make(
116 },
117 ])
118 );
119 +
120 +export const runCmd = withCommandTelemetry(runCmdBase, input =>
121 + telemetry.fields({
122 + debug: telemetry.flag('--debug', input.debug),
123 + dry_run: telemetry.flag('--dry-run', input.dryRun),
124 + skip_connection_check: telemetry.flag('--skip-connection-check', input.skipConnectionCheck),
125 + skip_tool_params_check: telemetry.flag('--skip-tool-params-check', input.skipToolParamsCheck),
126 + skip_checks: telemetry.flag('--skip-checks', input.skipChecks),
127 + file_mode: telemetry.flag('--file', telemetry.present(input.file)),
128 + arg_count: telemetry.value(input.args?.length ?? 0),
129 + })
130 +);
ts/packages/cli/src/commands/setup.cmd.ts
+12 -1
@@ -1,3 +1,4 @@
1 +import { telemetry, withCommandTelemetry } from 'src/analytics/command-input';
2 import { setupSkillCmd } from './setup-skill.cmd';
3 import { Command, Flag } from 'effect/cli';
4 import { Effect, Predicate } from 'effect';
@@ -336,7 +337,17 @@ const setupBaseCmd = Command.make(
337 )
338 );
339
339 -export const setupCmd = setupBaseCmd.pipe(
340 +export const setupCmd = withCommandTelemetry(setupBaseCmd, input =>
341 + telemetry.fields({
342 + operation: {
343 + value: input.uninstall ? 'uninstall' : 'setup',
344 + flags: input.uninstall ? ['--uninstall'] : [],
345 + },
346 + target: telemetry.value(input.target),
347 + yes: telemetry.flag('--yes', input.yes),
348 + if_present: telemetry.flag('--if-present', input.ifPresent),
349 + })
350 +).pipe(
351 Command.withDescription('Install or uninstall Composio plugins for supported agent hosts.'),
352 Command.withExamples([
353 {
ts/packages/cli/src/commands/tools/execute/tools.execute.cmd.ts
+38 -31
@@ -1,3 +1,4 @@
1 +import { telemetry, withCommandTelemetry } from 'src/analytics/command-input';
2 import { Argument, Command, Flag } from 'effect/cli';
3 import { withCommandMeta } from 'src/invocation/command-identity';
4 import { runToolsExecuteCommand } from './tools.execute.handler';
@@ -84,23 +85,34 @@ const executeExamples = [
85 },
86 ];
87
87 -export const rootToolsCmd$Execute = Command.make(
88 - 'execute',
89 - {
90 - slug,
91 - additionalSlugs,
92 - parallel,
93 - data,
94 - file,
95 - account: accountOption,
96 - getSchema,
97 - dryRun,
98 - skipConnectionCheck,
99 - skipToolParamsCheck,
100 - skipChecks,
101 - },
102 - runToolsExecuteCommand
103 -).pipe(
88 +const executeFlags = {
89 + slug,
90 + additionalSlugs,
91 + parallel,
92 + data,
93 + file,
94 + account: accountOption,
95 + getSchema,
96 + dryRun,
97 + skipConnectionCheck,
98 + skipToolParamsCheck,
99 + skipChecks,
100 +};
101 +
102 +const executeTelemetry = (input: Command.Command.Config.Infer<typeof executeFlags>) =>
103 + telemetry.fields(
104 + {
105 + dry_run: telemetry.flag('--dry-run', input.dryRun),
106 + get_schema: telemetry.flag('--get-schema', input.getSchema),
107 + has_data: telemetry.flag('--data', telemetry.present(input.data)),
108 + skip_connection_check: telemetry.flag('--skip-connection-check', input.skipConnectionCheck),
109 + skip_tool_params_check: telemetry.flag('--skip-tool-params-check', input.skipToolParamsCheck),
110 + skip_checks: telemetry.flag('--skip-checks', input.skipChecks),
111 + },
112 + input.slug
113 + );
114 +
115 +const rootToolsCmd$ExecuteBase = Command.make('execute', executeFlags, runToolsExecuteCommand).pipe(
116 Command.withDescription(
117 'Execute a tool by slug. Validates inputs against cached schemas and checks connections\nautomatically — just try it and it will tell you what to fix.'
118 ),
@@ -111,21 +123,9 @@ export const rootToolsCmd$Execute = Command.make(
123 withCommandMeta({ requiresLogin: true })
124 );
125
114 -export const devToolsCmd$Execute = Command.make(
126 +const devToolsCmd$ExecuteBase = Command.make(
127 'playground-execute',
116 - {
117 - slug,
118 - additionalSlugs,
119 - parallel,
120 - data,
121 - file,
122 - account: accountOption,
123 - getSchema,
124 - dryRun,
125 - skipConnectionCheck,
126 - skipToolParamsCheck,
127 - skipChecks,
128 - },
128 + executeFlags,
129 runToolsExecuteCommand
130 ).pipe(
131 Command.withDescription(
@@ -142,3 +142,10 @@ export const devToolsCmd$Execute = Command.make(
142 ),
143 withCommandMeta({ requiresLogin: true })
144 );
145 +
146 +export const rootToolsCmd$Execute = withCommandTelemetry(
147 + rootToolsCmd$ExecuteBase,
148 + executeTelemetry
149 +);
150 +
151 +export const devToolsCmd$Execute = withCommandTelemetry(devToolsCmd$ExecuteBase, executeTelemetry);
ts/packages/cli/src/commands/tools/search/tools.search.cmd.ts
+15 -1
@@ -1,3 +1,4 @@
1 +import { telemetry, withCommandTelemetry } from 'src/analytics/command-input';
2 import { Argument, Command, Flag } from 'effect/cli';
3 import { withCommandMeta } from 'src/invocation/command-identity';
4 import { runToolsSearch } from './tools.search.handler';
@@ -29,7 +30,7 @@ const human = Flag.Boolean('human').pipe(
30 Flag.withDescription('Show formatted human-readable search output')
31 );
32
32 -export const toolsCmd$Search = Command.make(
33 +const toolsCmd$SearchBase = Command.make(
34 'search',
35 { query, toolkits, limit, json, human },
36 runToolsSearch
@@ -71,3 +72,16 @@ export const toolsCmd$Search = Command.make(
72 ]),
73 withCommandMeta({ requiresLogin: true })
74 );
75 +
76 +export const toolsCmd$Search = withCommandTelemetry(toolsCmd$SearchBase, input => {
77 + const query = input.query[0] ?? '';
78 + return telemetry.fields({
79 + toolkits: {
80 + value: telemetry.optional(input.toolkits),
81 + flags: telemetry.present(input.toolkits) ? ['--toolkits'] : [],
82 + },
83 + limit: telemetry.value(input.limit === undefined ? undefined : String(input.limit)),
84 + query_length: telemetry.value(query.length),
85 + query_term_count: telemetry.value(query.split(/\s+/u).filter(Boolean).length),
86 + });
87 +});
ts/packages/cli/src/invocation/command-identity.ts
+5 -1
@@ -11,7 +11,7 @@
11 */
12 import { Array as Arr, Context, Effect, Option, Predicate, Record as EffectRecord } from 'effect';
13 import { Command } from 'effect/cli';
14 -import type { TelemetryFamily } from 'src/analytics/types';
14 +import type { CommandTelemetryInput, TelemetryFamily } from 'src/analytics/types';
15
16 /**
17 * The hooks the invocation runs for the selected command (`src/invocation/invocation.ts`), in
@@ -48,6 +48,8 @@ export interface CommandMetaShape {
48 * Not inherited: a subcommand reports its own family, the generic `command` by default.
49 */
50 readonly telemetryFamily?: TelemetryFamily;
51 + /** Command-specific privacy projection. Not inherited: child input has a different type. */
52 + readonly telemetryInput?: (input: unknown) => CommandTelemetryInput;
53 /**
54 * Whether the command needs a logged-in user. Inherited: a group's subcommands need one too.
55 * The selected command checks it right after it began (`src/commands/registration.ts`), so
@@ -76,6 +78,7 @@ export const CommandMeta = Context.Reference<CommandMetaShape>('commands/Command
78 /** `CommandMetaShape` with every field decided for one registration. */
79 export interface ResolvedCommandMeta {
80 readonly telemetryFamily: TelemetryFamily;
81 + readonly telemetryInput?: (input: unknown) => CommandTelemetryInput;
82 readonly requiresLogin: boolean;
83 readonly hooks: Readonly<Record<InvocationHookName, HookPolicy>>;
84 /** The shared flags the command cannot honor, declared on itself or on a group above it. */
@@ -101,6 +104,7 @@ export const resolveCommandMeta = (
104 own: CommandMetaShape
105 ): ResolvedCommandMeta => ({
106 telemetryFamily: own.telemetryFamily ?? 'command',
107 + telemetryInput: own.telemetryInput,
108 requiresLogin: own.requiresLogin ?? parent?.requiresLogin ?? false,
109 hooks: { ...(parent === undefined ? EVERY_HOOK : inheritedHooks(parent.hooks)), ...own.hooks },
110 unsupportedSharedFlags: SHARED_FLAG_NAMES.filter(flag => {
ts/packages/cli/test/__utils__/telemetry-context.ts
+9 -1
@@ -1,3 +1,5 @@
1 +import { buildRootCommand } from 'src/commands';
2 +import { identityAt } from 'src/invocation/command-identity';
3 import { Option } from 'effect';
4 import { createCliCommandTelemetryContext } from 'src/analytics/events';
5 import type { TelemetryFamily } from 'src/analytics/types';
@@ -33,7 +35,13 @@ export const commandTelemetryContext = (
35 },
36 identity: {
37 path: options.path,
36 - meta: resolveCommandMeta(undefined, { telemetryFamily: options.family ?? 'command' }),
38 + meta: resolveCommandMeta(undefined, {
39 + telemetryFamily: options.family ?? 'command',
40 + telemetryInput: identityAt(
41 + buildRootCommand({ isDevModeEnabled: true, isExperimentalFeatureEnabled: () => true }),
42 + ['composio', options.family ?? options.path[0] ?? '']
43 + ).meta.telemetryInput,
44 + }),
45 },
46 input: options.input === undefined ? Option.none() : Option.some(options.input),
47 startedAt: Date.now(),
ts/packages/cli/test/src/analytics.events.test.ts
+50 -3
@@ -211,12 +211,60 @@ describe('CLI analytics execute failure events', () => {
211 });
212 });
213
214 +describe('parsed telemetry options', () => {
215 + it('records canonical option names without retaining flag values', () => {
216 + const secret = '--customer-private';
217 + const context = commandContext(['execute', 'GMAIL_SEND_EMAIL', '-d', secret], {
218 + path: ['execute'],
219 + family: 'execute',
220 + input: { slug: 'GMAIL_SEND_EMAIL', data: Option.some(secret), dryRun: true },
221 + });
222 + expect(context.flagNames).toEqual(['--data', '--dry-run']);
223 + const event = getPrimaryLifecycleInvokedEvent(context);
224 + expect(event?.properties).toMatchObject({ has_data: true, dry_run: true });
225 + expect(JSON.stringify(event)).not.toContain(secret);
226 + expect(JSON.stringify(context.options)).not.toContain(secret);
227 + });
228 + it('uses parsed proxy values and counts repeated headers without collecting their content', () => {
229 + const context = commandContext(
230 + ['proxy', '/path', '-t', 'gmail', '-H', 'Authorization: secret'],
231 + {
232 + path: ['proxy'],
233 + family: 'proxy',
234 + input: {
235 + endpoint: '/path',
236 + toolkit: 'gmail',
237 + method: 'POST',
238 + headers: ['a: b', 'c: d'],
239 + data: Option.none(),
240 + },
241 + }
242 + );
243 + const event = getPrimaryLifecycleInvokedEvent(context);
244 + expect(event?.properties).toMatchObject({
245 + toolkit: 'gmail',
246 + method: 'POST',
247 + header_count: 2,
248 + has_data: false,
249 + });
250 + expect(JSON.stringify(event)).not.toContain('secret');
251 + });
252 + it('honors a parsed false boolean even when its flag is present', () => {
253 + const context = commandContext(['run', '--dry-run=false', 'code'], {
254 + path: ['run'],
255 + family: 'run',
256 + input: { args: ['code'], dryRun: false },
257 + });
258 + expect(getPrimaryLifecycleInvokedEvent(context)?.properties).toMatchObject({ dry_run: false });
259 + });
260 +});
261 +
262 describe('CLI analytics setup and install lifecycle events', () => {
263 it('tracks setup as a dedicated lifecycle with safe option properties', () => {
264 const context = commandContext(['setup', '--target', 'codex', '--uninstall', '--yes'], {
265 path: ['setup'],
266 family: 'setup',
219 - input: {},
267 + input: { target: 'codex', uninstall: true, yes: true, ifPresent: false },
268 });
269
270 const invoked = getPrimaryLifecycleInvokedEvent(context);
@@ -240,7 +288,7 @@ describe('CLI analytics setup and install lifecycle events', () => {
288 const context = commandContext(['install', '--completions'], {
289 path: ['install'],
290 family: 'install',
243 - input: {},
291 + input: { completions: true, noCompletions: false },
292 });
293
294 expect(getPrimaryLifecycleInvokedEvent(context)).toMatchObject({
@@ -555,7 +603,6 @@ describe('CLI analytics journey taxonomy', () => {
603 return commandContext(argv, {
604 path: [name],
605 family: name === 'version' ? 'command' : (name as TelemetryFamily),
558 - input: {},
606 invocation,
607 });
608 };
ts/packages/cli/test/src/invocation/command-identity.test.ts
+204 -1
@@ -1,7 +1,11 @@
1 import { describe, expect, it, layer } from '@effect/vitest';
2 import { Cause, Data, Effect, Option, Ref } from 'effect';
3 import type { Command } from 'effect/cli';
4 -import { getPrimaryLifecycleFailedEvent } from 'src/analytics/events';
4 +import {
5 + getPrimaryLifecycleFailedEvent,
6 + getPrimaryLifecycleInvokedEvent,
7 + getPrimaryLifecycleSucceededEvent,
8 +} from 'src/analytics/events';
9 import type { TelemetryFamily } from 'src/analytics/types';
10 import { buildRootCommand, runWithConfig } from 'src/commands';
11 import {
@@ -208,6 +212,205 @@ describe('lifecycle telemetry from the command identity', () => {
212 })
213 );
214
215 + it.effect('uses the displayed command identity for redundant help', () =>
216 + Effect.gen(function* () {
217 + const events = yield* lifecycleOf(['help', 'execute', '--help']);
218 + expect(events.map(({ event, path, family }) => ({ event, path, family }))).toEqual([
219 + { event: 'invoked', path: 'execute', family: 'execute' },
220 + { event: 'succeeded', path: 'execute', family: 'execute' },
221 + ]);
222 + })
223 + );
224 +
225 + it.effect('does not collect a rejected flag name', () =>
226 + Effect.gen(function* () {
227 + const events = yield* lifecycleOf(['--customer-secret']);
228 + expect(events.map(({ call }) => call.context.flagNames)).toEqual([[], []]);
229 + })
230 + );
231 +
232 + it.effect('does not interpret run script operands as CLI options', () =>
233 + Effect.gen(function* () {
234 + // The missing source stops before execution. Parsing and lifecycle telemetry still run.
235 + const events = yield* lifecycleOf([
236 + 'run',
237 + '--file',
238 + '/missing/cli-telemetry-test.ts',
239 + '--',
240 + '--dry-run',
241 + ]);
242 + const invoked = events[0]!;
243 + expect(invoked.path).toBe('run');
244 + expect(getPrimaryLifecycleInvokedEvent(invoked.call.context)?.properties).toMatchObject({
245 + dry_run: false,
246 + file_mode: true,
247 + arg_count: 1,
248 + });
249 + expect(invoked.call.context.flagNames).toEqual(['--file']);
250 + })
251 + );
252 +
253 + it.effect.each([
254 + { args: ['search', 'send email'], properties: { limit: '10' } },
255 + { args: ['search', 'send email', '--limit', '7'], properties: { limit: '7' } },
256 + {
257 + args: ['proxy', '/private-path', '--toolkit', 'gmail'],
258 + properties: { method: 'GET', toolkit: 'gmail', header_count: 0 },
259 + },
260 + {
261 + args: ['proxy', '/private-path', '--toolkit', 'gmail', '-X', 'POST'],
262 + properties: { method: 'POST', toolkit: 'gmail', header_count: 0 },
263 + },
264 + { args: ['setup', '--if-present'], properties: { target: 'auto', if_present: true } },
265 + ])('uses effective parser values for `$args`', ({ args, properties }) =>
266 + Effect.gen(function* () {
267 + const events = yield* lifecycleOf(args);
268 + expect(events).toHaveLength(2);
269 + events.forEach(({ call }) => {
270 + expect(getPrimaryLifecycleInvokedEvent(call.context)?.properties).toMatchObject(properties);
271 + });
272 + })
273 + );
274 +
275 + it.effect.each(
276 + [
277 + {
278 + command: 'execute',
279 + invalid: [],
280 + fields: [
281 + 'dry_run',
282 + 'get_schema',
283 + 'has_data',
284 + 'skip_connection_check',
285 + 'skip_tool_params_check',
286 + 'skip_checks',
287 + ],
288 + },
289 + {
290 + command: 'search',
291 + invalid: ['--limit', 'invalid'],
292 + fields: ['limit', 'toolkits', 'query_length', 'query_term_count'],
293 + },
294 + {
295 + command: 'link',
296 + invalid: ['--no-wait=invalid'],
297 + fields: ['no_browser', 'no_wait', 'has_auth_config'],
298 + },
299 + {
300 + command: 'login',
301 + invalid: ['--no-wait=invalid'],
302 + fields: ['no_browser', 'no_wait', 'has_key', 'skip_org_picker'],
303 + },
304 + {
305 + command: 'install',
306 + invalid: ['--completions=invalid'],
307 + fields: ['completions', 'no_completions'],
308 + },
309 + {
310 + command: 'setup',
311 + invalid: ['--target', 'invalid'],
312 + fields: ['operation', 'target', 'yes', 'if_present'],
313 + },
314 + {
315 + command: 'proxy',
316 + invalid: [],
317 + fields: [
318 + 'method',
319 + 'toolkit',
320 + 'has_endpoint',
321 + 'has_data',
322 + 'header_count',
323 + 'skip_connection_check',
324 + ],
325 + },
326 + {
327 + command: 'run',
328 + invalid: ['--file'],
329 + fields: [
330 + 'debug',
331 + 'dry_run',
332 + 'skip_connection_check',
333 + 'skip_tool_params_check',
334 + 'skip_checks',
335 + 'file_mode',
336 + 'arg_count',
337 + ],
338 + },
339 + ].flatMap(({ command, invalid, fields }) => [
340 + { command, fields, args: [command, '--help'], outcome: 'succeeded' },
341 + { command, fields, args: [command, ...invalid], outcome: 'failed' },
342 + ])
343 + )('omits unparsed input for `$args` ($outcome)', ({ command, fields, args, outcome }) =>
344 + Effect.gen(function* () {
345 + const events = yield* lifecycleOf(args);
346 + expect(events.map(({ event }) => event)).toEqual(['invoked', outcome]);
347 + events.forEach(({ call }) => {
348 + const event =
349 + call.event === 'failed'
350 + ? getPrimaryLifecycleFailedEvent(call.context, Cause.squash(call.cause))
351 + : call.event === 'succeeded'
352 + ? getPrimaryLifecycleSucceededEvent(call.context)
353 + : getPrimaryLifecycleInvokedEvent(call.context);
354 + expect(event?.properties).toMatchObject({ source: 'cli', command_path: command });
355 + fields.forEach(field => expect(event?.properties).not.toHaveProperty(field));
356 + expect(call.context.options).toEqual({});
357 + expect(call.context.flagNames).toEqual([]);
358 + if (call.event === 'failed') {
359 + expect(event?.properties).toHaveProperty('failure_reason_code', 'parse_error');
360 + }
361 + });
362 + })
363 + );
364 +
365 + it.effect('keeps the projection when a shared command is registered under dev', () =>
366 + Effect.gen(function* () {
367 + const events = yield* lifecycleOf([
368 + 'dev',
369 + 'connected-accounts',
370 + 'link',
371 + 'gmail',
372 + '--no-browser',
373 + ]);
374 + expect(events).toHaveLength(2);
375 + events.forEach(({ call }) => {
376 + expect(call.context.family).toBe('command');
377 + expect(call.context.flagNames).toEqual(['--no-browser']);
378 + expect(getPrimaryLifecycleInvokedEvent(call.context)?.properties).toMatchObject({
379 + flag_names: ['--no-browser'],
380 + });
381 + });
382 + })
383 + );
384 +
385 + it.effect('projects only measured fields from parsed proxy input', () =>
386 + Effect.gen(function* () {
387 + const events = yield* lifecycleOf([
388 + 'proxy',
389 + '/private-path',
390 + '--toolkit',
391 + 'gmail',
392 + '--account',
393 + 'private-account',
394 + '-H',
395 + 'Authorization: private-header',
396 + '-d',
397 + 'private-body',
398 + ]);
399 + const context = events[0]!.call.context;
400 + expect(context.options).toEqual({
401 + toolkit: 'gmail',
402 + method: 'GET',
403 + has_endpoint: true,
404 + has_data: true,
405 + header_count: 1,
406 + skip_connection_check: false,
407 + });
408 + expect(context.argument).toBeUndefined();
409 + expect(context.flagNames).toEqual(['--data', '--header']);
410 + expect(JSON.stringify(getPrimaryLifecycleInvokedEvent(context))).not.toContain('private-');
411 + })
412 + );
413 +
414 it.effect('reads the execute tool slug from the parsed input, and none for --help', () =>
415 Effect.gen(function* () {
416 const [executed] = yield* lifecycleOf(['execute', 'GMAIL_SEND_EMAIL']);