@devries-cameron20 / composio-1 / commits / 6a4cb542a
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

fix(core): resolve $ref/$defs so file flags behind a reference aren't dropped (#3566)

## Problem Tools whose parameter schemas express their file fields through a `$ref`/`$defs` indirection lose auto file handling: with `dangerouslyAllowAutoUploadDownloadFiles` enabled, a `file_uploadable` input behind a `$ref` is never staged (the local path is forwarded as-is and the backend rejects it), and the `format: "path"` hint never reaches the model. `GMAIL_GET_ATTACHMENT` is the canonical shape — its parameters nest the file flag two `$ref` hops deep: ``` data → $ref → #/$defs/GetAttachmentResponse → file → $ref → #/$defs/FileDownloadable (file_downloadable: true) ``` This is the TypeScript-SDK counterpart of #3506 (the Python side is handled separately in #3545). Two coupled causes defeat reference resolution: 1. **`$defs` is dropped on parse.** `ParametersSchema` (`src/types/tool.types.ts`) is a strict `z.object` that never declared `$defs`/`definitions`, so `ToolSchema.parse` stripped the root definition block off every tool. Each nested `$ref` was then dangling and unresolvable for *all* downstream consumers — providers that pass `inputParameters` straight to the model (e.g. `OpenAIProvider`) and the file modifier alike. (`JSONSchemaPropertySchema` already accepts both keywords; the root did not.) 2. **The file modifier never dereferences.** `transformProperties`, `schemaHasFileProperty`, and the upload/download hydration recurse `properties`/`anyOf`/`oneOf`/`allOf`/`items` but skip `$ref`, so a flag reachable only through a reference is invisible to the walk. ## Solution - Add `$defs`/`definitions` to `ParametersSchema` so the root definition block survives parsing and `$ref` pointers stay resolvable. - Dereference `inputParameters`/`outputParameters` in `FileToolModifier` (node runtime) before walking, via the existing `dereferenceJsonSchema` helper in `'sentinel'` mode — the same mode `MastraProvider` already uses. Tools that ship a `$ref` without a matching `$defs` target (#3307) degrade to a permissive object instead of throwing, exactly as today. No new dependencies; `dereferenceJsonSchema` already exists and is exercised by the Mastra provider. The edge (`workerd`) modifier is unchanged — its upload/download paths already throw as unsupported. ## Testing - `pnpm typecheck` — clean. - `pnpm test` (core) — 926 passing, including new cases: - `$ref`-indirected `file_uploadable` is marked `format: "path"` by `modifyToolSchema`. - `fileUploadModifier` stages a file whose flag is reachable only via `$ref`/`$defs`. - a `$ref` with no matching `$defs` target degrades gracefully (no throw). - root `$defs` survives `ToolSchema.parse` and nested `$ref` pointers remain intact. - Provider suites unaffected: `mastra` (51), `openai` (29), `vercel` (10), `anthropic` (19) all green. --------- Co-authored-by: jkomyno <alberto@composio.dev>

Serhii Zghama committed Jun 22, 2026 at 22:32 UTC 6a4cb542aed5f54da7da025157d938f8d25bdef4
5 files changed +242 -7
.changeset/file-modifier-dereference-ref.md new
+5
@@ -0,0 +1,5 @@
1 +---
2 +'@composio/core': patch
3 +---
4 +
5 +Preserve root `$defs` / `definitions` blocks on tool parameter schemas and dereference them in the Node file modifier, so auto file upload/download detection works when `file_uploadable` or `file_downloadable` is hidden behind an internal `$ref`.
ts/packages/core/src/types/tool.types.ts
+18
@@ -99,6 +99,24 @@ const ParametersSchema = z.object({
99 nullable: z.boolean().optional(),
100 description: z.string().optional(),
101 additionalProperties: z.boolean().default(false).optional(),
102 + // Definition blocks targeted by `$ref` pointers. The Composio API ships
103 + // these at the parameters root (e.g. `data` → `$ref` → `#/$defs/...`), but
104 + // because `z.object` strips unknown keys they were being dropped on parse —
105 + // leaving every nested `$ref` dangling and unresolvable downstream
106 + // (providers, file modifiers). `JSONSchemaPropertySchema` already accepts
107 + // both keywords; the parameters root must too.
108 + $defs: z
109 + .record(
110 + z.string(),
111 + z.lazy(() => JSONSchemaPropertySchema)
112 + )
113 + .optional(),
114 + definitions: z
115 + .record(
116 + z.string(),
117 + z.lazy(() => JSONSchemaPropertySchema)
118 + )
119 + .optional(),
120 });
121
122 /**
ts/packages/core/src/utils/modifiers/FileToolModifier.node.ts
+29 -7
@@ -24,6 +24,22 @@ import {
24 transformProperties,
25 schemaHasFileProperty,
26 } from './FileToolModifier.utils.neutral';
27 +import { dereferenceJsonSchema } from '../jsonSchema';
28 +
29 +/**
30 + * Inlines internal `$ref` pointers so the file walkers below can see the
31 + * `file_uploadable` / `file_downloadable` flags that live behind a
32 + * `$ref`/`$defs` indirection (e.g. `GMAIL_GET_ATTACHMENT`). The walkers only
33 + * recurse `properties`/`anyOf`/`oneOf`/`allOf`/`items` and never dereference,
34 + * so without this a flagged field reachable only through a `$ref` is silently
35 + * missed — the file is never staged/downloaded. `'sentinel'` mode keeps tools
36 + * whose schema omits the `$defs` target working (see
37 + * https://github.com/ComposioHQ/composio/issues/3307) instead of throwing.
38 + */
39 +const resolveFileSchema = (
40 + schema: JSONSchemaProperty | undefined
41 +): JSONSchemaProperty | undefined =>
42 + schema ? dereferenceJsonSchema(schema, { onUnresolved: 'sentinel' }) : schema;
43
44 const getSchemaVariants = (schema: JSONSchemaProperty | undefined): JSONSchemaProperty[] => [
45 ...(schema?.anyOf ?? []),
@@ -370,12 +386,14 @@ export class FileToolModifier {
386 return schema;
387 }
388
373 - const properties = transformProperties(schema.inputParameters.properties);
389 + const inputParameters = resolveFileSchema(schema.inputParameters)!;
390 + const properties = transformProperties(inputParameters.properties!);
391
392 return {
393 ...schema,
394 inputParameters: {
378 - ...schema.inputParameters,
395 + ...inputParameters,
396 + type: 'object',
397 properties,
398 },
399 };
@@ -396,7 +414,7 @@ export class FileToolModifier {
414
415 // Recursively transform the arguments tree without mutating the caller’s copy
416 try {
399 - const newArgs = await hydrateFiles(args, tool.inputParameters, {
417 + const newArgs = await hydrateFiles(args, resolveFileSchema(tool.inputParameters), {
418 toolSlug,
419 toolkitSlug,
420 client: this.client,
@@ -429,10 +447,14 @@ export class FileToolModifier {
447 const { result, toolSlug } = options;
448
449 // Walk result.data without mutating the original, using output schema for guidance
432 - const dataWithDownloads = await hydrateDownloads(result.data, tool.outputParameters, {
433 - toolSlug,
434 - fileDownloadDir: this.fileUploadPathOptions.fileDownloadDir,
435 - });
450 + const dataWithDownloads = await hydrateDownloads(
451 + result.data,
452 + resolveFileSchema(tool.outputParameters),
453 + {
454 + toolSlug,
455 + fileDownloadDir: this.fileUploadPathOptions.fileDownloadDir,
456 + }
457 + );
458
459 return { ...result, data: dataWithDownloads as typeof result.data };
460 }
ts/packages/core/test/tools/fileModifiers.test.ts
+120
@@ -1713,6 +1713,126 @@ describe('FileToolModifier', () => {
1713 expect(modifiedResult.data.fileOutput).toBeNull();
1714 });
1715 });
1716 +
1717 + // The schema walkers recurse properties/anyOf/oneOf/allOf/items but never
1718 + // dereference `$ref`. Composio toolkits express file flags through a
1719 + // `$ref`/`$defs` indirection (e.g. GMAIL_GET_ATTACHMENT), so without
1720 + // dereferencing the flag is invisible and the file is silently skipped.
1721 + // See https://github.com/ComposioHQ/composio/issues/3506.
1722 + describe('$ref / $defs indirection', () => {
1723 + it('marks a file_uploadable behind a $ref/$defs as format: "path" in modifyToolSchema', async () => {
1724 + const schema: Tool = {
1725 + slug: 'test-tool',
1726 + name: 'Test Tool',
1727 + description: 'A test tool',
1728 + tags: ['test'],
1729 + version: '20251201_01',
1730 + availableVersions: ['20251201_01'],
1731 + inputParameters: {
1732 + type: 'object',
1733 + properties: {
1734 + attachment: { $ref: '#/$defs/Attachment' },
1735 + legacyAttachment: { $ref: '#/definitions/LegacyAttachment' },
1736 + text: { type: 'string' },
1737 + },
1738 + $defs: {
1739 + Attachment: {
1740 + type: 'string',
1741 + file_uploadable: true,
1742 + description: 'Local path to attach',
1743 + },
1744 + },
1745 + definitions: {
1746 + LegacyAttachment: {
1747 + type: 'string',
1748 + file_uploadable: true,
1749 + description: 'Legacy local path to attach',
1750 + },
1751 + },
1752 + },
1753 + };
1754 +
1755 + const result = await fileToolModifier.modifyToolSchema(schema);
1756 +
1757 + expect(result.inputParameters?.properties?.attachment).toHaveProperty('format', 'path');
1758 + expect(result.inputParameters?.properties?.attachment).toHaveProperty(
1759 + 'file_uploadable',
1760 + true
1761 + );
1762 + expect(result.inputParameters?.properties?.legacyAttachment).toHaveProperty('format', 'path');
1763 + expect(result.inputParameters?.properties?.legacyAttachment).toHaveProperty(
1764 + 'file_uploadable',
1765 + true
1766 + );
1767 + expect(result.inputParameters?.properties?.text).not.toHaveProperty('format');
1768 + });
1769 +
1770 + it('uploads a file whose file_uploadable flag is reachable only via $ref/$defs', async () => {
1771 + const mockFileData = {
1772 + name: 'doc.pdf',
1773 + mimetype: 'application/pdf',
1774 + s3key: 'uploads/doc.pdf',
1775 + };
1776 + vi.mocked(fileUtils.getFileDataAfterUploadingToS3).mockResolvedValue(mockFileData);
1777 +
1778 + const toolWithRef: Tool = {
1779 + slug: 'test-tool',
1780 + name: 'Test Tool',
1781 + description: 'A test tool',
1782 + tags: ['test'],
1783 + version: '20251201_01',
1784 + availableVersions: ['20251201_01'],
1785 + inputParameters: {
1786 + type: 'object',
1787 + properties: {
1788 + attachment: { $ref: '#/$defs/Attachment' },
1789 + },
1790 + $defs: {
1791 + Attachment: { type: 'string', file_uploadable: true },
1792 + },
1793 + },
1794 + };
1795 +
1796 + const result = await fileToolModifier.fileUploadModifier(toolWithRef, {
1797 + toolSlug: 'test-tool',
1798 + toolkitSlug: 'test-toolkit',
1799 + params: {
1800 + arguments: { attachment: '/path/to/doc.pdf' },
1801 + userId: 'test-user',
1802 + },
1803 + });
1804 +
1805 + expect(fileUtils.getFileDataAfterUploadingToS3).toHaveBeenCalledWith('/path/to/doc.pdf', {
1806 + toolSlug: 'test-tool',
1807 + toolkitSlug: 'test-toolkit',
1808 + client: mockClient,
1809 + });
1810 + expect(result.arguments?.attachment).toEqual(mockFileData);
1811 + });
1812 +
1813 + it('degrades gracefully when a $ref has no matching $defs target', async () => {
1814 + // Some toolkits ship a `$ref` without ever declaring `$defs`
1815 + // (https://github.com/ComposioHQ/composio/issues/3307). The modifier
1816 + // must not throw; the unresolved branch simply carries no file flag.
1817 + const schema: Tool = {
1818 + slug: 'test-tool',
1819 + name: 'Test Tool',
1820 + description: 'A test tool',
1821 + tags: ['test'],
1822 + version: '20251201_01',
1823 + availableVersions: ['20251201_01'],
1824 + inputParameters: {
1825 + type: 'object',
1826 + properties: {
1827 + attachment: { $ref: '#/$defs/Missing' },
1828 + },
1829 + },
1830 + };
1831 +
1832 + const result = await fileToolModifier.modifyToolSchema(schema);
1833 + expect(result.inputParameters?.properties?.attachment).not.toHaveProperty('format');
1834 + });
1835 + });
1836 });
1837
1838 describe('Tools with dangerouslyAllowAutoUploadDownloadFiles', () => {
ts/packages/core/test/tools/tools.test.ts
+70
@@ -573,6 +573,76 @@ describe('Tools', () => {
573 });
574 });
575
576 + // `ParametersSchema` is a strict `z.object`, so any key it doesn't declare is
577 + // dropped on parse. It previously omitted `$defs`/`definitions`, so a tool
578 + // whose parameters root carries those blocks (the `$ref` targets) lost them —
579 + // leaving every nested `$ref` dangling and unresolvable for providers and the
580 + // file modifier. See https://github.com/ComposioHQ/composio/issues/3506.
581 + describe('$ref / $defs preservation on tool parameters', () => {
582 + const rawToolWithDefs = {
583 + slug: 'GMAIL_GET_ATTACHMENT',
584 + name: 'Get Attachment',
585 + description: 'Fetch a Gmail attachment',
586 + input_parameters: {
587 + type: 'object',
588 + properties: {
589 + message_id: { type: 'string' },
590 + },
591 + },
592 + output_parameters: {
593 + type: 'object',
594 + properties: {
595 + data: { $ref: '#/$defs/GetAttachmentResponse' },
596 + legacy_data: { $ref: '#/definitions/LegacyAttachmentResponse' },
597 + },
598 + $defs: {
599 + GetAttachmentResponse: {
600 + type: 'object',
601 + properties: { file: { $ref: '#/$defs/FileDownloadable' } },
602 + },
603 + FileDownloadable: {
604 + type: 'object',
605 + file_downloadable: true,
606 + properties: { s3url: { type: 'string' } },
607 + },
608 + },
609 + definitions: {
610 + LegacyAttachmentResponse: {
611 + type: 'object',
612 + properties: { file: { $ref: '#/definitions/LegacyFileDownloadable' } },
613 + },
614 + LegacyFileDownloadable: {
615 + type: 'object',
616 + file_downloadable: true,
617 + properties: { s3url: { type: 'string' } },
618 + },
619 + },
620 + },
621 + toolkit: { logo: 'https://example.com/gmail.png', slug: 'gmail', name: 'Gmail' },
622 + version: '20260515_00',
623 + };
624 +
625 + it('keeps the root $defs block so nested $ref pointers stay resolvable', async () => {
626 + mockClient.tools.retrieve.mockResolvedValueOnce(rawToolWithDefs);
627 +
628 + const tool = await context.tools.getRawComposioToolBySlug('GMAIL_GET_ATTACHMENT');
629 +
630 + expect(tool.outputParameters?.$defs).toBeDefined();
631 + expect(tool.outputParameters?.$defs?.FileDownloadable).toMatchObject({
632 + file_downloadable: true,
633 + });
634 + expect(tool.outputParameters?.definitions?.LegacyFileDownloadable).toMatchObject({
635 + file_downloadable: true,
636 + });
637 + expect(tool.outputParameters?.properties?.data).toEqual({
638 + $ref: '#/$defs/GetAttachmentResponse',
639 + });
640 + expect(tool.outputParameters?.properties?.legacy_data).toEqual({
641 + $ref: '#/definitions/LegacyAttachmentResponse',
642 + });
643 + });
644 + });
645 +
646 describe('get', () => {
647 it('should get a single tool by slug and wrap it with provider as a collection', async () => {
648 const userId = 'test-user';