@devries-cameron20 / composio Mirror · read-only
Fork 0

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

Mirrored from https://github.com/Camo67/composio.git

Synced Nov 21, 03:37 UTC

next
md 101 lines 4 KB
Rendered Raw
1 # LLM Guardrails
2
3 ## Decision
4
5 Inject invisible guardrail instructions into every `.md` response so AI code generators use the correct Composio SDK patterns. Guardrails are scoped per-page via frontmatter — different pages can get different instruction sets.
6
7 ## Problem
8
9 LLMs that crawl or receive Composio docs generate code using the old direct tool execution pattern (`composio.tools.get()`, `composio.provider.handle_tool_calls()`) instead of the current session-based pattern (`composio.create()`, `session.tools()`).
10
11 ## Architecture
12
13 ```
14 lib/llm-guardrails/
15 ├── index.ts ← getGuardrails(type), module-private GuardrailType
16 ├── session.ts ← SESSION_GUARDRAILS constant (default)
17 └── direct-execution.ts ← DIRECT_EXECUTION_GUARDRAILS constant
18 ```
19
20 ### How it flows
21
22 1. `getLLMText()` in `lib/source.ts` reads `page.data.llmGuardrails` from frontmatter
23 2. Passes it to `getGuardrails(type)` which returns the right guardrail string
24 3. Appends it to the markdown output
25
26 ### Scoping via frontmatter
27
28 Pages control which guardrails they get:
29
30 | Frontmatter value | What gets appended |
31 |---|---|
32 | *(omitted — default)* | `SESSION_GUARDRAILS` — full "use composio.create() + session.tools()" instructions, ALWAYS/NEVER rules, deprecated patterns, verification checklist |
33 | `llmGuardrails: "direct-execution"` | `DIRECT_EXECUTION_GUARDRAILS` — acknowledges direct execution is valid, but strongly recommends sessions unless user explicitly asks for it |
34 | `llmGuardrails: "none"` | Nothing |
35
36 The Zod schema in `source.config.ts` validates the field at build time — typos cause build failures.
37
38 ### Tagged pages (12 total)
39
40 `tools-direct/` (8):
41 - executing-tools, fetching-tools, authenticating-tools, custom-tools
42 - toolkit-versioning
43 - modify-tool-behavior/schema-modifiers, before-execution-modifiers, after-execution-modifiers
44
45 `auth-configuration/` (4):
46 - connected-accounts, custom-auth-configs, custom-auth-params, programmatic-auth-configs
47
48 ### Endpoint behavior
49
50 | Endpoint | Guardrails behavior |
51 |---|---|
52 | `/:path*.md` (per-page) | Appended per page based on frontmatter |
53 | `llms.txt` (index) | One-liner note at top pointing to session pattern |
54 | `llms-full.txt` (all pages) | `SESSION_GUARDRAILS` once at the top, per-page guardrails disabled (`includeGuardrails: false`) to avoid 1148x repetition |
55 | Copy Page button | Fetches `.md` endpoint, so guardrails are included automatically |
56
57 ### getLLMText options
58
59 ```typescript
60 getLLMText(page, {
61 includeFooter?: boolean; // default true — nav links at bottom
62 includeGuardrails?: boolean; // default true — guardrail block
63 })
64 ```
65
66 `llms-full.txt` passes both as `false` and handles them at the route level instead.
67
68 ## Why this approach
69
70 | Alternative | Why not |
71 |---|---|
72 | Per-page component/accordion | Requires adding to every page, new pages miss it |
73 | URL-path matching in code | Implicit — moving a file breaks it silently |
74 | Separate "Copy Prompt" button (Clerk-style) | Clashes with existing "Copy Page" button |
75 | Guardrails in frontmatter content | Visible to humans on the site |
76
77 Frontmatter tag + pipeline injection = zero visual impact, zero per-page maintenance, explicit opt-in for different behavior, build-time validation.
78
79 ## Adding a new guardrail set
80
81 1. Create `lib/llm-guardrails/your-set.ts` exporting a string constant
82 2. Add the enum value to `llmGuardrails` in `source.config.ts`
83 3. Add a case to `getGuardrails()` in `lib/llm-guardrails/index.ts`
84 4. Tag pages with `llmGuardrails: "your-set"` in frontmatter
85
86 ## Session guardrails content
87
88 The default guardrails include:
89 - Correct pattern code (Python + TypeScript)
90 - Provider packages table (9 providers)
91 - ALWAYS rules (4)
92 - NEVER rules (4)
93 - Deprecated patterns with markers
94 - Verification checklist (5 items)
95
96 ## Direct execution guardrails content
97
98 The softer guardrails include:
99 - Note that this page documents the low-level API
100 - Recommended session pattern code (Python + TypeScript)
101 - "When to use direct execution" — only when user explicitly asks for it