@devries-cameron20 / composio Mirror · read-only / commits / e895406ec
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

Mirrored from https://github.com/Camo67/composio.git

Synced Nov 21, 06:00 UTC

docs: add a guide to avoiding and recovering from rate limits

Adds a Production readiness guide that shows how to tell a Composio 429 from a provider quota error, retry with Retry-After and jittered backoff, keep retried writes safe, and cap concurrency per connected account, with Python, TypeScript, and API examples. Links it from the rate limits reference, which now points provider quota fixes on Composio-managed apps to your own OAuth app. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Jeevesh Jain committed Nov 11, 2026 at 16:48 UTC e895406ec4ee111f0988ac56c4379df05b92d08a
4 files changed +399
docs/content/docs/poc-to-prod/avoid-rate-limits.mdx new
+393
@@ -0,0 +1,393 @@
1 +---
2 +title: Avoid and recover from rate limits
3 +description: Tell Composio limits from provider quotas, retry with backoff, and shape tool traffic so you hit fewer limits
4 +keywords: [rate limits, 429, retry, Retry-After, exponential backoff, jitter, provider quota, concurrency, idempotency, managed auth]
5 +---
6 +
7 +A rate limit can come from Composio or from the app a tool calls. The two look different in your code and need different fixes. This guide shows how to tell them apart, retry the ones that clear on their own, and send traffic in a shape that hits fewer limits in the first place.
8 +
9 +For the limits themselves, the response headers, and the 429 body, see [Rate limits](/reference/rate-limits).
10 +
11 +## Know which limit you hit
12 +
13 +| Limit | How it reaches your code | Retry? |
14 +| --- | --- | --- |
15 +| Composio API rate limit | The SDK call throws on an HTTP `429`, with a `Retry-After` header | Yes, after `Retry-After` |
16 +| Composio plan usage | Check your Composio usage and billing | No. Waiting a few seconds does not reset it |
17 +| Provider quota, short window | The call succeeds, but the tool result fails with the provider's error, such as a `429` or `rateLimitExceeded` | Yes, with backoff |
18 +| Provider quota, daily or monthly | The tool result fails with the provider's error, which names a daily or monthly quota | No. Wait for the reset or get more quota |
19 +
20 +Only the first row raises an exception in the SDK. For a provider limit, the request to Composio succeeds and the provider's rejection comes back inside the tool result, so check both places.
21 +
22 +<Tabs groupId="language" items={['Python', 'TypeScript', 'API']} persist>
23 +<Tab value="Python">
24 +```python
25 +from composio import Composio
26 +from composio_client import RateLimitError
27 +
28 +composio = Composio()
29 +session = composio.create(user_id="user_123")
30 +
31 +try:
32 + result = session.execute("GMAIL_FETCH_EMAILS", arguments={"max_results": 10})
33 +except RateLimitError as error:
34 + # Composio's organization limit. The request was not run.
35 + print("Composio 429, retry after", error.response.headers.get("retry-after"))
36 +else:
37 + if result.result_type == "failed":
38 + # The tool ran and failed. A provider quota error lands here.
39 + print("Tool failed:", result.error)
40 +```
41 +</Tab>
42 +<Tab value="TypeScript">
43 +```typescript
44 +import { Composio } from '@composio/core';
45 +const composio = new Composio({ apiKey: 'your_api_key' });
46 +// ---cut---
47 +const session = await composio.create('user_123');
48 +
49 +try {
50 + const result = await session.execute('GMAIL_FETCH_EMAILS', { max_results: 10 });
51 + if (result.resultType === 'failed') {
52 + // The tool ran and failed. A provider quota error lands here.
53 + console.log('Tool failed:', result.error);
54 + }
55 +} catch (error) {
56 + // session.execute() rethrows the API client's error, which carries the HTTP status.
57 + if (error instanceof Error && 'status' in error && error.status === 429) {
58 + console.log("Composio 429: Composio's organization limit");
59 + }
60 + throw error;
61 +}
62 +```
63 +</Tab>
64 +<Tab value="API">
65 +```bash
66 +# SESSION_ID is the session_id returned when you created the session.
67 +# -D - prints the response headers, including X-RateLimit-Remaining and, on a 429, Retry-After.
68 +curl -sS -D - -X POST https://backend.composio.dev/api/v3.1/tool_router/session/$SESSION_ID/execute \
69 + -H "x-api-key: $COMPOSIO_API_KEY" \
70 + -H "Content-Type: application/json" \
71 + -d '{
72 + "tool_slug": "GMAIL_FETCH_EMAILS",
73 + "arguments": { "max_results": 10 }
74 + }'
75 +```
76 +</Tab>
77 +</Tabs>
78 +
79 +A provider error is free text from the provider. Match it against the names your providers use, such as `RESOURCE_EXHAUSTED`, `rateLimitExceeded`, `userRateLimitExceeded`, or `ThrottleExceeded`, and read the provider's documentation for which ones are daily quotas. The [provider quota errors](/reference/rate-limits#provider-quota-errors) table lists a few.
80 +
81 +If you call `composio.tools.execute()` in TypeScript, a Composio 429 throws `ComposioToolExecutionRateLimitError`, and a 429 while fetching the tool's schema throws `ComposioToolFetchRateLimitError`. Both have a `retryAfterSeconds` field when the response gave one, and `isComposioRateLimitError()` matches either. See [What the SDK already does](#what-the-sdk-already-does).
82 +
83 +## Retry the limits that clear
84 +
85 +The SDKs send every tool execution exactly once. They do not retry `session.execute()`, `tools.execute()`, or proxy calls, because a retry after a timeout could repeat a side effect such as sending an email twice. See [Retry failed executions deliberately](/docs/production-readiness#retry-failed-executions-deliberately). Retrying an execution is your decision, so make it carefully:
86 +
87 +1. Honor `Retry-After` on a Composio 429. It tells you when the window resets.
88 +2. For a short provider limit, back off exponentially with random jitter, so many workers don't retry at the same moment. Cap the delay and the number of attempts.
89 +3. Don't retry an exhausted daily or monthly quota. Each retry spends a request and the quota stays empty.
90 +4. Retry a write only when repeating it is safe.
91 +
92 +```mermaid
93 +flowchart TD
94 + A["Execute the tool"] --> B{"SDK threw a 429?"}
95 + B -- yes --> C["Composio organization limit"]
96 + C --> D["Wait Retry-After seconds, or back off if it is missing"]
97 + B -- no --> E{"Result failed with a provider rate limit?"}
98 + E -- no --> F["Return the result"]
99 + E -- yes --> G{"Daily or monthly quota?"}
100 + G -- yes --> H["Stop and surface the error"]
101 + G -- no --> I["Back off with jitter"]
102 + D --> J{"Safe to repeat and attempts left?"}
103 + I --> J
104 + J -- yes --> A
105 + J -- no --> H
106 +```
107 +
108 +<Tabs groupId="language" items={['Python', 'TypeScript', 'API']} persist>
109 +<Tab value="Python">
110 +```python
111 +import random
112 +import time
113 +
114 +from composio import Composio
115 +from composio_client import RateLimitError
116 +
117 +composio = Composio()
118 +session = composio.create(user_id="user_123")
119 +
120 +MAX_ATTEMPTS = 5
121 +BASE_DELAY_SECONDS = 1.0
122 +MAX_DELAY_SECONDS = 30.0
123 +
124 +# Adjust these to the error text your providers return.
125 +PROVIDER_LIMIT_MARKERS = ("429", "rate limit", "ratelimitexceeded", "resource_exhausted", "throttle")
126 +EXHAUSTED_QUOTA_MARKERS = ("daily", "per day", "dailylimitexceeded", "monthly")
127 +
128 +
129 +def backoff_seconds(attempt: int) -> float:
130 + """Exponential backoff with full jitter: a random wait up to the capped exponential delay."""
131 + return random.uniform(0, min(MAX_DELAY_SECONDS, BASE_DELAY_SECONDS * 2**attempt))
132 +
133 +
134 +def retry_after_seconds(error: RateLimitError, attempt: int) -> float:
135 + value = error.response.headers.get("retry-after", "")
136 + try:
137 + return max(0.0, float(value))
138 + except ValueError:
139 + # Missing, or an HTTP date: fall back to backoff.
140 + return backoff_seconds(attempt)
141 +
142 +
143 +def execute_with_retry(tool_slug: str, arguments: dict, *, safe_to_repeat: bool):
144 + for attempt in range(MAX_ATTEMPTS):
145 + last_attempt = attempt == MAX_ATTEMPTS - 1
146 + try:
147 + result = session.execute(tool_slug, arguments=arguments)
148 + except RateLimitError as error:
149 + # Composio's organization limit.
150 + if last_attempt or not safe_to_repeat:
151 + raise
152 + time.sleep(retry_after_seconds(error, attempt))
153 + continue
154 +
155 + error_text = (result.error or "").lower()
156 + provider_limited = result.result_type == "failed" and any(
157 + marker in error_text for marker in PROVIDER_LIMIT_MARKERS
158 + )
159 + if not provider_limited:
160 + return result
161 + if any(marker in error_text for marker in EXHAUSTED_QUOTA_MARKERS):
162 + return result # The quota is gone until it resets. Don't spend more requests.
163 + if last_attempt or not safe_to_repeat:
164 + return result
165 + time.sleep(backoff_seconds(attempt))
166 + raise RuntimeError("unreachable")
167 +
168 +
169 +emails = execute_with_retry(
170 + "GMAIL_FETCH_EMAILS", {"max_results": 10}, safe_to_repeat=True
171 +)
172 +```
173 +</Tab>
174 +<Tab value="TypeScript">
175 +```typescript
176 +import { Composio, isComposioRateLimitError } from '@composio/core';
177 +const composio = new Composio({ apiKey: 'your_api_key' });
178 +// ---cut---
179 +const session = await composio.create('user_123');
180 +
181 +const MAX_ATTEMPTS = 5;
182 +const BASE_DELAY_MS = 1_000;
183 +const MAX_DELAY_MS = 30_000;
184 +
185 +// Adjust these to the error text your providers return.
186 +const PROVIDER_LIMIT_MARKERS = ['429', 'rate limit', 'ratelimitexceeded', 'resource_exhausted', 'throttle'];
187 +const EXHAUSTED_QUOTA_MARKERS = ['daily', 'per day', 'dailylimitexceeded', 'monthly'];
188 +
189 +const sleep = (ms: number) => new Promise(resolve => setTimeout(resolve, ms));
190 +
191 +/** Exponential backoff with full jitter: a random wait up to the capped exponential delay. */
192 +function backoffMs(attempt: number): number {
193 + return Math.random() * Math.min(MAX_DELAY_MS, BASE_DELAY_MS * 2 ** attempt);
194 +}
195 +
196 +/** How long to wait after a Composio 429, or null when the error is something else. */
197 +function composioRateLimitWaitMs(error: unknown, attempt: number): number | null {
198 + // Direct tool execution throws a typed rate-limit error.
199 + if (isComposioRateLimitError(error)) {
200 + return error.retryAfterSeconds !== undefined ? error.retryAfterSeconds * 1_000 : backoffMs(attempt);
201 + }
202 + // session.execute() rethrows the API client's error, with the status and headers.
203 + if (error instanceof Error && 'status' in error && error.status === 429) {
204 + const header =
205 + 'headers' in error && error.headers instanceof Headers ? error.headers.get('retry-after') : null;
206 + const seconds = header === null ? NaN : Number(header);
207 + return Number.isFinite(seconds) && seconds >= 0 ? seconds * 1_000 : backoffMs(attempt);
208 + }
209 + return null;
210 +}
211 +
212 +async function executeWithRetry(
213 + toolSlug: string,
214 + args: Record<string, unknown>,
215 + { safeToRepeat }: { safeToRepeat: boolean }
216 +) {
217 + for (let attempt = 0; ; attempt++) {
218 + const lastAttempt = attempt === MAX_ATTEMPTS - 1;
219 + let result;
220 + try {
221 + result = await session.execute(toolSlug, args);
222 + } catch (error) {
223 + const waitMs = composioRateLimitWaitMs(error, attempt);
224 + if (waitMs === null || lastAttempt || !safeToRepeat) throw error;
225 + await sleep(waitMs);
226 + continue;
227 + }
228 +
229 + const errorText = (result.error ?? '').toLowerCase();
230 + const providerLimited =
231 + result.resultType === 'failed' && PROVIDER_LIMIT_MARKERS.some(m => errorText.includes(m));
232 + if (!providerLimited) return result;
233 + // An exhausted quota stays empty until it resets. Don't spend more requests.
234 + if (EXHAUSTED_QUOTA_MARKERS.some(m => errorText.includes(m))) return result;
235 + if (lastAttempt || !safeToRepeat) return result;
236 + await sleep(backoffMs(attempt));
237 + }
238 +}
239 +
240 +const emails = await executeWithRetry('GMAIL_FETCH_EMAILS', { max_results: 10 }, { safeToRepeat: true });
241 +```
242 +</Tab>
243 +<Tab value="API">
244 +```bash
245 +# Retry the same execute request after the number of seconds in Retry-After.
246 +RESPONSE_HEADERS=$(mktemp)
247 +for attempt in 1 2 3 4 5; do
248 + STATUS=$(curl -sS -o response.json -D "$RESPONSE_HEADERS" -w '%{http_code}' \
249 + -X POST https://backend.composio.dev/api/v3.1/tool_router/session/$SESSION_ID/execute \
250 + -H "x-api-key: $COMPOSIO_API_KEY" \
251 + -H "Content-Type: application/json" \
252 + -d '{"tool_slug": "GMAIL_FETCH_EMAILS", "arguments": {"max_results": 10}}')
253 + [ "$STATUS" != "429" ] && break
254 + WAIT=$(grep -i '^retry-after:' "$RESPONSE_HEADERS" | tr -dc '0-9')
255 + sleep "${WAIT:-$((2 ** attempt))}"
256 +done
257 +cat response.json
258 +```
259 +</Tab>
260 +</Tabs>
261 +
262 +### Make retried writes safe
263 +
264 +A read such as `GMAIL_FETCH_EMAILS` is safe to repeat. A write such as `GMAIL_SEND_EMAIL` is not. If the first attempt reached the provider, a retry sends the email again.
265 +
266 +Before you retry a write, check whether the first attempt landed. Look for the record it would have created, such as the sent message, the issue, or the calendar event, through a read tool or the provider's API. You can also check the call in [Logs](/reference/api-reference/logs). Resend only when the record is missing. If the provider's API accepts an idempotency key or a client-generated ID, pass one so a repeat is ignored.
267 +
268 +## What the SDK already does
269 +
270 +The SDKs handle part of this for you. Don't add a second layer of retries on top of these.
271 +
272 +| Behavior | Python | TypeScript |
273 +| --- | --- | --- |
274 +| Retries reads (fetching tools, listing toolkits, reading a session) on timeouts, `408`, `409`, `429`, and `5xx` | Yes | Yes |
275 +| Retries tool executions and proxy calls | No | No |
276 +| Reuses a tool's schema across executions | `tools.execute()` fetches each tool's schema once per `Tools` instance | `tools.execute()` reuses a schema for 5 minutes on `latest` and 1 hour on a pinned version, up to 500 entries per `Tools` instance. Failed fetches aren't cached |
277 +| Typed error for a Composio 429 | `composio_client.RateLimitError`, with the response headers on `error.response.headers` | `ComposioToolFetchRateLimitError` and `ComposioToolExecutionRateLimitError` from `tools.execute()`, with `retryAfterSeconds`. `isComposioRateLimitError()` matches both. `session.execute()` rethrows the API client's error with `status` `429` |
278 +| Connection polling | `wait_for_connection()` waits 1 second between polls at first, growing to 5 seconds, and keeps polling through a 429 | `waitForConnection()` behaves the same way |
279 +
280 +When a poll inside `wait_for_connection()` or `waitForConnection()` is rate limited, the method waits as long as the response asks and polls again. The client's own retries are not bound by your timeout, so a rate-limited wait can run past it. See [Waiting for connection establishment](/docs/tools-direct/authenticating-tools#waiting-for-connection-establishment).
281 +
282 +## Shape your traffic
283 +
284 +Most rate limits are easier to avoid than to recover from.
285 +
286 +### Cap concurrency per connected account
287 +
288 +Providers often meter each user or account separately. Gmail, for example, has a per-user quota on top of the per-project one. See Google's [Gmail API usage limits](https://developers.google.com/workspace/gmail/api/reference/quota). Ten agents working for one user all draw from that user's quota, even when your Composio limit has room.
289 +
290 +Queue tool calls per user ID or connected account, and run only a few at a time for each one.
291 +
292 +<Tabs groupId="language" items={['Python', 'TypeScript', 'API']} persist>
293 +<Tab value="Python">
294 +```python
295 +import threading
296 +from collections import defaultdict
297 +
298 +from composio import Composio
299 +
300 +composio = Composio()
301 +
302 +MAX_CALLS_PER_USER = 2
303 +_user_slots: defaultdict[str, threading.BoundedSemaphore] = defaultdict(
304 + lambda: threading.BoundedSemaphore(MAX_CALLS_PER_USER)
305 +)
306 +_slots_lock = threading.Lock()
307 +
308 +
309 +def execute_for_user(user_id: str, tool_slug: str, arguments: dict):
310 + with _slots_lock:
311 + slots = _user_slots[user_id]
312 + with slots: # Blocks while this user already has MAX_CALLS_PER_USER calls running.
313 + session = composio.create(user_id=user_id)
314 + return session.execute(tool_slug, arguments=arguments)
315 +```
316 +</Tab>
317 +<Tab value="TypeScript">
318 +```typescript
319 +import { Composio } from '@composio/core';
320 +const composio = new Composio({ apiKey: 'your_api_key' });
321 +// ---cut---
322 +const MAX_CALLS_PER_USER = 2;
323 +const running = new Map<string, number>();
324 +const waiting = new Map<string, Array<() => void>>();
325 +
326 +async function acquire(userId: string): Promise<void> {
327 + if ((running.get(userId) ?? 0) < MAX_CALLS_PER_USER) {
328 + running.set(userId, (running.get(userId) ?? 0) + 1);
329 + return;
330 + }
331 + // Wait for a running call to hand over its slot.
332 + await new Promise<void>(resolve => {
333 + const queue = waiting.get(userId) ?? [];
334 + queue.push(resolve);
335 + waiting.set(userId, queue);
336 + });
337 +}
338 +
339 +function release(userId: string): void {
340 + const next = waiting.get(userId)?.shift();
341 + if (next) {
342 + next(); // Hand the slot straight to the next queued call.
343 + } else {
344 + running.set(userId, (running.get(userId) ?? 1) - 1);
345 + }
346 +}
347 +
348 +async function executeForUser(userId: string, toolSlug: string, args: Record<string, unknown>) {
349 + await acquire(userId);
350 + try {
351 + const session = await composio.create(userId);
352 + return await session.execute(toolSlug, args);
353 + } finally {
354 + release(userId);
355 + }
356 +}
357 +```
358 +</Tab>
359 +<Tab value="API">
360 +```bash
361 +# Run calls for one user one after another rather than in parallel.
362 +for tool_slug in GMAIL_FETCH_EMAILS GMAIL_LIST_LABELS; do
363 + curl -sS -X POST https://backend.composio.dev/api/v3.1/tool_router/session/$SESSION_ID/execute \
364 + -H "x-api-key: $COMPOSIO_API_KEY" \
365 + -H "Content-Type: application/json" \
366 + -d "{\"tool_slug\": \"$tool_slug\", \"arguments\": {}}"
367 +done
368 +```
369 +</Tab>
370 +</Tabs>
371 +
372 +These examples create a session per call to stay short. In a real agent, create the session once per user and reuse it, as the next section describes.
373 +
374 +### Spend fewer calls
375 +
376 +- **Page in small steps and stop early.** Ask a list tool for a small page and stop paginating once you have what the task needs. Listing a whole mailbox or repository to find a few items burns quota on pages you never read.
377 +- **Use triggers instead of polling.** Calling a tool in a loop to check for new messages spends quota on empty answers. A [trigger](/docs/triggers) sends you an event when something changes.
378 +- **Reuse sessions and cache static data.** Persist a session ID for multi-turn work instead of creating a session on every turn. Keep tool definitions and other data that rarely changes in your process.
379 +- **Watch `X-RateLimit-Remaining`.** It shows the requests left in the current Composio window. Slow down background work before it reaches zero, so interactive requests still have room.
380 +
381 +## When you outgrow a Composio-managed app
382 +
383 +With Composio-managed auth, some provider quotas are shared across every connection that uses Composio's OAuth app. If you keep hitting a provider quota that backoff and traffic shaping don't fix, move to your own OAuth app. Your connections then draw from your own provider quota, and you can request a higher one from the provider.
384 +
385 +See [Managed vs custom auth](/docs/authentication/custom-app-vs-managed-app). Existing connections stay on the managed app, so your users reconnect under the new auth config.
386 +
387 +## Related
388 +
389 +<Cards>
390 + <Card title="Rate limits" href="/reference/rate-limits" description="Composio's limits, headers, 429 response, and provider quota errors." />
391 + <Card title="Production readiness" href="/docs/production-readiness" description="Retry, logging, and testing decisions before launch." />
392 + <Card title="Triggers" href="/docs/triggers" description="Receive events instead of polling with tool calls." />
393 +</Cards>
docs/content/docs/poc-to-prod/meta.json
+1
@@ -3,6 +3,7 @@
3 "pages": [
4 "../authentication/custom-app-vs-managed-app",
5 "[Rate limits](/reference/rate-limits)",
6 + "avoid-rate-limits",
7 "stream-logs-to-a-siem"
8 ]
9 }
docs/content/reference/rate-limits.mdx
+4
@@ -53,6 +53,8 @@ When you exceed the rate limit, you'll receive a `429 Too Many Requests` respons
53
54 3. **Cache what doesn't change.** Keep tool definitions and other static data client-side so you don't spend requests re-fetching them.
55
56 +For retry code in Python and TypeScript, telling Composio limits from provider quotas, and capping concurrency per connected account, see [Avoid and recover from rate limits](/docs/poc-to-prod/avoid-rate-limits).
57 +
58 ## Provider quota errors
59
60 Providers can limit an OAuth app, developer token, workspace, account, or individual API operation. With managed auth, some of those quotas are shared across connections using Composio's provider credentials.
@@ -79,6 +81,8 @@ If you need help tracing a limit, include the UTC timestamp, tool slug, Composio
81
82 If you regularly hit Composio's organization API limit, upgrade your plan or [talk to us](https://composio.dev/contact?utm_source=docs&cta_placement=docs-rate-limits) about custom limits. For a provider quota, use the provider recovery steps above.
83
84 +If you keep hitting a provider quota on a Composio-managed app, the fix is to move to your own OAuth app, so your connections draw from your own provider quota. See [Managed vs custom auth](/docs/authentication/custom-app-vs-managed-app).
85 +
86 <Cards>
87 <Card title="Errors" href="/reference/errors">
88 Understanding API error responses
docs/tests/static/product-navigation.test.ts
+1
@@ -202,6 +202,7 @@ describe('Docs product navigation', () => {
202 expect(readinessUrls).toEqual([
203 '/docs/authentication/custom-app-vs-managed-app',
204 '/reference/rate-limits',
205 + '/docs/poc-to-prod/avoid-rate-limits',
206 '/docs/poc-to-prod/stream-logs-to-a-siem',
207 ]);
208 for (const url of readinessUrls) {