Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

next
md 272 lines 10.4 KB
Rendered Raw
1 # Auto Upload and Download Files
2
3 Composio SDK includes an automatic file handling system that manages file uploads and downloads when executing tools. This guide explains how the system works, how to configure it, and how to handle files manually when auto-handling is disabled.
4
5 ## Overview
6
7 The file handling system in Composio SDK is:
8
9 - **Opt-in**: Disabled by default. Set `dangerouslyAllowAutoUploadDownloadFiles: true` to enable.
10 - **Automatic** (when enabled): Handles file uploads and downloads transparently.
11 - **Allowlisted**: Even when enabled, local upload paths must resolve inside one of the directories listed in `fileUploadDirs` (default `[~/.composio/temp]`). See [Restricting upload paths with `fileUploadDirs`](#restricting-upload-paths-with-fileuploaddirs) below.
12 - **Configurable**: Manual handling via `composio.files.upload()` / `composio.files.download()` is always available and is **not** subject to `fileUploadDirs`.
13 - **Sensitive path protection (Node)**: Local string paths used for upload are checked by default against a denylist of path segments and file names (see below).
14
15 ## Security: sensitive local paths
16
17 Auto-upload reads local files and sends them to Composio storage. That is powerful for agents, but it also means a model or caller could pass a path that points at **secrets on disk** (API keys, SSH private keys, cloud CLI config). To reduce accidental exfiltration:
18
19 - **`sensitiveFileUploadProtection`** (default `true`) — Before reading a file, the SDK resolves the path and checks it against a built-in list of path *components* (for example `.ssh`, `.aws`, `.claude`, `.kube`) and patterns for credential-like basenames (for example `.env`, default SSH private key names, a file named `credentials`). If the path matches, upload fails with `ComposioSensitiveFilePathBlockedError` unless you disable protection.
20 - **`fileUploadPathDenySegments`** — Optional extra path component names (single directory or file name) merged with the built-in list, anywhere in the resolved path.
21 - **`beforeFileUpload`** — Not a constructor option. Pass it as part of the **third argument** to `composio.tools.execute` (with `beforeExecute` / `afterExecute`). The hook receives `{ path, toolSlug, toolkitSlug }` for each `file_uploadable` value. Return a string path to upload instead, `false` to abort (`ComposioFileUploadAbortedError`), or throw.
22
23 **Not path-checked the same way:** HTTP/HTTPS URLs and `File` objects do not go through the local path denylist logic in the same manner as string paths that refer to the local filesystem.
24
25 **Disabling protection:** `sensitiveFileUploadProtection: false` turns off the denylist checks. Only use this if you understand the risk and have another control (for example a strict allowlist in `beforeFileUpload`).
26
27 ```typescript
28 import { Composio } from '@composio/core';
29
30 const composio = new Composio({
31 apiKey: process.env.COMPOSIO_API_KEY!,
32 sensitiveFileUploadProtection: true,
33 fileUploadPathDenySegments: ['my-team-secrets'],
34 });
35
36 await composio.tools.execute(
37 'YOUR_TOOL',
38 { userId: 'u', arguments: { file: '/safe/path/doc.pdf' }, dangerouslySkipVersionCheck: true },
39 {
40 beforeFileUpload: async ({ path }) => path,
41 }
42 );
43 ```
44
45 ## How It Works
46
47 ### File Upload
48
49 When a tool's input parameter is marked with `file_uploadable: true`, the SDK will:
50
51 1. Automatically detect file paths in the input
52 2. Upload the file to Composio's secure storage
53 3. Replace the file path with file metadata in the tool execution
54
55 Example tool schema with file upload:
56
57 ```typescript
58 const toolSchema = {
59 inputParameters: {
60 type: 'object',
61 properties: {
62 file: {
63 type: 'string',
64 file_uploadable: true
65 }
66 }
67 }
68 }
69 ```
70
71 When executing a tool with file upload (auto-handling enabled):
72
73 ```typescript
74 // Requires `dangerouslyAllowAutoUploadDownloadFiles: true` AND a path inside `fileUploadDirs`
75 const result = await composio.tools.execute('your-tool', {
76 arguments: {
77 file: '/path/to/local/file.txt' // Local file path
78 // or
79 file: 'https://example.com/file.txt' // Remote URL
80 }
81 });
82
83 // The SDK automatically:
84 // 1. Reads the file content
85 // 2. Uploads it to S3
86 // 3. Replaces the path with metadata:
87 // {
88 // name: string; // The original filename
89 // mimetype: string; // The file's mime type
90 // s3key: string; // The S3 key for the uploaded file
91 // }
92 ```
93
94 ### File Download
95
96 When a tool's output contains an S3 URL and mimetype, the SDK will:
97
98 1. Automatically detect file URLs in the response
99 2. Download the file to a local temporary directory
100 3. Replace the S3 URL with local file information
101
102 Example tool response with file download:
103
104 ```typescript
105 // Original response from tool
106 {
107 data: {
108 file: {
109 s3url: 'https://s3.example.com/file.txt',
110 mimetype: 'text/plain'
111 }
112 }
113 }
114
115 // After auto-download
116 {
117 data: {
118 file: {
119 uri: '/path/to/local/file.txt',
120 file_downloaded: true,
121 s3url: 'https://s3.example.com/file.txt',
122 mimeType: 'text/plain'
123 }
124 }
125 }
126 ```
127
128 ## Default: auto file handling is off
129
130 Automatic file upload/download during tool execution is **disabled** unless you pass `dangerouslyAllowAutoUploadDownloadFiles: true` to the `Composio` constructor.
131
132 ## Restricting upload paths with `fileUploadDirs`
133
134 When auto-handling is on, the SDK only reads local files from directories you have explicitly allowlisted via `fileUploadDirs`. Anything outside the list is rejected with `ComposioFileUploadPathNotAllowedError` before the file is read.
135
136 | `fileUploadDirs` value | Effective allowlist |
137 | ---------------------- | ------------------- |
138 | omitted / `undefined` | `[~/.composio/temp]` (default staging dir) |
139 | `false` | `[]` — **all** local paths rejected (URLs and `File`/`Blob` still work) |
140 | `[]` | `[]` — same as `false`; explicit "no local paths" |
141 | `[<dir1>, <dir2>, …]` | exactly those directories. `~` is expanded; entries are `realpath`-resolved; comparison is on a path-component boundary (`/tmp/foo` allows `/tmp/foo/bar` but **not** `/tmp/foo-bar`) |
142
143 URLs (`http://…` / `https://…`) and JavaScript `File` / `Blob` objects are not subject to the allowlist — only local string paths are. The denylist (`sensitiveFileUploadProtection`) still runs in addition to the allowlist; both must accept the path.
144
145 ```typescript
146 import { Composio } from '@composio/core';
147
148 // Widen the allowlist to the directories your code actually reads from.
149 const composio = new Composio({
150 apiKey: process.env.COMPOSIO_API_KEY!,
151 dangerouslyAllowAutoUploadDownloadFiles: true,
152 // Replaces the default `[~/.composio/temp]`. Include it explicitly if you
153 // still want the default staging dir to keep working.
154 fileUploadDirs: ['/srv/agent/uploads', '~/.composio/temp'],
155 });
156
157 // Reject every local path; only URLs and File/Blob objects work.
158 const composioStrict = new Composio({
159 apiKey: process.env.COMPOSIO_API_KEY!,
160 dangerouslyAllowAutoUploadDownloadFiles: true,
161 fileUploadDirs: false,
162 });
163 ```
164
165 If you want to upload from arbitrary paths without widening the allowlist, use the manual `composio.files.upload(...)` API below — it bypasses `fileUploadDirs` by design (the caller is expected to control the path).
166
167 ## Manual File Handling
168
169 When auto file handling is not enabled, you'll need to handle the file operations yourself using the `composio.files` API:
170
171 ### Manual Upload
172
173 When auto-upload is disabled, you need to handle file uploads manually:
174
175 ```typescript
176 // Upload file manually
177 const fileData = await composio.files.upload({
178 filePath: '/path/to/file.txt', // Local path or URL
179 toolSlug: 'your-tool', // Tool slug
180 toolkitSlug: 'your-toolkit' // Toolkit slug
181 });
182
183 // Execute tool with file data
184 const result = await composio.tools.execute('your-tool', {
185 arguments: {
186 file: fileData // Contains name, mimetype, and s3key
187 }
188 });
189 ```
190
191 ### Manual Download
192
193 When auto-download is disabled, you'll need to handle file downloads manually:
194
195 ```typescript
196 // Execute tool
197 const result = await composio.tools.execute('your-tool', {
198 arguments: {
199 // your arguments
200 }
201 });
202
203 // Download file manually if response contains S3 URL
204 if (result.data.file?.s3url) {
205 const downloadedFile = await composio.files.download({
206 s3Url: result.data.file.s3url,
207 toolSlug: 'your-tool',
208 mimeType: result.data.file.mimetype || 'application/txt'
209 });
210
211 // downloadedFile contains:
212 // {
213 // name: string; // Generated filename (toolSlug_timestamp.ext)
214 // mimeType: string; // The file's mime type
215 // s3Url: string; // The original S3 URL
216 // filePath: string; // Local path to the downloaded file
217 // }
218 }
219 ```
220
221 ## File Storage Location
222
223 Downloaded files are stored in:
224
225 - **Default**: `~/.composio/files/` (user's home directory).
226 - **Custom**: pass `fileDownloadDir: '/some/abs/path'` to the `Composio` constructor.
227 - Files are named using the pattern: `{toolSlug}_{timestamp}.{extension}` (server-synthesized; the S3 URL never affects the local filename).
228 - The directory is created automatically when needed.
229 - File extensions are derived from the MIME type.
230
231 ## Error Handling
232
233 The SDK includes specific error types for file operations:
234
235 ```typescript
236 import {
237 ComposioFileUploadError,
238 ComposioSensitiveFilePathBlockedError,
239 ComposioFileUploadAbortedError,
240 ComposioFileUploadPathNotAllowedError,
241 } from '@composio/core';
242
243 try {
244 await composio.tools.execute('your-tool', {
245 userId: 'u',
246 arguments: {
247 file: '/path/to/file.txt',
248 },
249 dangerouslySkipVersionCheck: true,
250 });
251 } catch (error) {
252 if (error instanceof ComposioFileUploadPathNotAllowedError) {
253 console.error('Path is outside fileUploadDirs:', error.message);
254 } else if (error instanceof ComposioSensitiveFilePathBlockedError) {
255 console.error('Blocked sensitive path:', error.message);
256 } else if (error instanceof ComposioFileUploadAbortedError) {
257 console.error('Upload aborted by beforeFileUpload:', error.message);
258 } else if (error instanceof ComposioFileUploadError) {
259 console.error('File upload failed:', error.message);
260 // Possible fixes may be included in error.possibleFixes
261 }
262 throw error;
263 }
264 ```
265
266 ## Additional Features
267
268 - **URL Support**: The file upload system supports both local file paths and URLs
269 - **MIME Type Detection**: Automatically detects MIME types from files and URLs
270 - **Automatic Directory Creation**: Creates necessary directories for file storage
271 - **Error Recovery**: Provides graceful error handling with helpful error messages
272 - **MD5 Checksum**: Generates MD5 checksums for file integrity verification