The best way to control Claude Code, Codex and more from any computer or phone

main
md 208 lines 12.7 KB
Rendered Raw
1 # Composer context references
2
3 > For maintainers. Using T3 Code? See [docs/user](../user/).
4
5 Inline context references let a user message point at a typed payload from an exact position in
6 its prose: an image, a file, a terminal excerpt, a picked page element, a preview annotation, a
7 review comment, a file mention, or a skill. This document covers the wire contract and the pure
8 codecs. Editor, rendering, and clipboard behaviour land in later PRs and get their own sections
9 here as they arrive.
10
11 ## Two linked concepts
12
13 - A **context record** is the payload. It lives in `message.context.records`, keyed by a
14 `contextId`. Records never contain bytes: image and file records bind to an existing
15 `ChatAttachment` by id.
16 - A **context reference** is one occurrence in the document. It is a Markdown link in
17 `message.text` that carries only the kind and the `contextId`. Two references can point at one
18 record. A reference's label is display text and never identity.
19
20 [`composerContext.ts`][contract] defines version 1 of the record union. Every record has
21 `version`, `contextId`, `kind`, and `label`, plus kind-specific fields with bounded lengths. The
22 union is open: a kind this build does not know decodes to `UnknownContextRecord` with its
23 `payload` preserved, and known kinds are excluded from that member so a malformed image record
24 fails its own schema rather than sliding through unchecked. `OrchestrationMessageContext` wraps
25 the records with `ForwardCompatibleArray`, so one undecodable record is dropped instead of failing
26 the whole message. The field is optional on the V2 conversation message and the commands that
27 start or queue a turn; the orchestrator carries it through untouched.
28
29 ## Identity namespaces
30
31 - `ComposerContextId`: durable payload identity. Branded. Values match `[a-z0-9_-]+` and do not
32 require a `ctx_` prefix.
33 - `ComposerContextReferenceId` (`ref_…`): one document occurrence. Branded. Lives in editor state,
34 not on the wire and is regenerated when canonical Markdown is reparsed.
35 - `ChatAttachmentId`: the existing server-owned attachment resource. A record's `attachmentId` is
36 a binding, not the chip's identity, so upload normalization can rename the resource without
37 rewriting references.
38
39 Clients mint context and reference ids. Shared code does not, because the Effect lint plugin
40 rejects direct `crypto.randomUUID()` there.
41
42 ## Canonical reference syntax
43
44 [`composerContextReferences.ts`][shared] owns the grammar:
45
46 ```text
47 [label](t3-context://v1/<kind>/<contextId>)
48 ![label](t3-context://v1/image/<contextId>)
49 ```
50
51 The parser accepts exactly the `t3-context:` scheme, the `v1` host, one kind segment matching
52 `[a-z][a-z0-9-]{0,39}`, and one id segment matching `[a-z0-9_-]{1,128}` case-insensitively. Query
53 strings, fragments, credentials, and extra segments are rejected. Labels are sanitized to survive
54 a Markdown link (no brackets or line breaks, at most 200 characters, never empty). Links that fail
55 to parse are ordinary text. `collectComposerInlineTokens` already rejects URI schemes for file
56 links, so a context link is never mistaken for a mention.
57
58 ## Provider projection
59
60 `projectComposerContextForProvider({ text, records })` builds what the provider reads:
61
62 1. Every reference becomes an in-place marker: `[Image: shot.png; ref=ctx_1]`.
63 2. A trailing `<t3_context version="1">` envelope holds one `<context kind id>` entry per unique
64 referenced id, in first-reference order. Records that are never referenced are not emitted.
65 A referenced id with no record becomes `<context … unavailable="true"/>`. Mention and skill
66 records produce a marker but no entry. Unknown kinds emit their payload as JSON.
67 3. Captured text is data: any `<` that would open or close `t3_context` or `context` is escaped,
68 so a terminal line or PR comment cannot forge a record.
69 4. Assistant citation links are expanded too (`expandAssistantCitationsForProvider`).
70
71 Attachment bytes travel on the existing attachment channel; the envelope only carries metadata.
72 Text without references or citations is returned unchanged.
73
74 ## Legacy messages
75
76 Messages sent before this feature carry trailing `<terminal_context>`, `<element_context>`, and
77 `<preview_annotation>` blocks, `<review_comment>` blocks, and U+FFFC terminal placeholders.
78 [`composerContextLegacy.ts`][legacy] upgrades them in memory:
79
80 - Review blocks become references in place. Blocks that trailed the original text are appended
81 last, matching the old send order.
82 - Trailing blocks peel off the end in reverse send order (preview, element, terminal).
83 - Placeholders bind to terminal entries in order; entries without a placeholder are appended.
84 - Ids are deterministic (`legacy_<kind>_<n>`) so re-running the upgrade is idempotent.
85
86 Event history is never rewritten. Existing web parsers in `apps/web/src/lib/` stay until the
87 transcript renderer moves to records.
88
89 [contract]: ../../packages/contracts/src/composerContext.ts
90 [shared]: ../../packages/shared/src/composerContextReferences.ts
91 [legacy]: ../../packages/shared/src/composerContextLegacy.ts
92
93 ## Editor model (web and desktop)
94
95 `ComposerContextReferenceExtension` in
96 [`ComposerPromptEditorTiptap.tsx`](../../apps/web/src/components/ComposerPromptEditorTiptap.tsx) is
97 the inline Tiptap node for every context kind. It stores `kind`, `contextId`, `label`, and a
98 per-occurrence `referenceId`, and its text content is the canonical link. The prompt string carries
99 payload identity and position, so rebuilding the editor restores equivalent chips; it does not
100 preserve the editor-local occurrence ids. Old drafts that used the U+FFFC ordinal placeholder
101 migrate on hydration: placeholders bind to the terminal contexts in array order, then any context
102 the prompt does not mention is prepended as a link.
103
104 Records stay in the draft store's typed arrays for now. The editor builds a `Map` keyed by
105 `contextId` from them (`composerContextRecordsFromDraft`) and provides it through
106 `ComposerContextRecordsContext`. `ComposerContextReferenceChip` looks the record up and renders
107 the kind's chip; an unknown kind or a missing record renders the unresolved chip instead of
108 vanishing. Removing a chip removes only that occurrence; the composer's change handler compares
109 the referenced ids against the draft array and drops records no chip points at.
110
111 Version 1 deliberately keeps kind presentation explicit in each client instead of exposing a
112 runtime handler registry. The contract and codecs are shared; web/desktop render rich chips and
113 mobile renders the readable label. Add a registry only when a third-party or runtime-defined kind
114 must provide behaviour that cannot ship with the client. Likewise, a durable occurrence id belongs
115 in the canonical reference syntax only if a future feature needs to address one occurrence across
116 serialization boundaries.
117
118 Terminal context now follows the same send path as every other context record: the persisted
119 message keeps its canonical link and structured record, while the provider projection replaces
120 the link with a readable marker and includes the excerpt once in the context envelope. The legacy
121 trailing `<terminal_context>` form is parsed only when reading messages sent by older clients.
122
123 ## Sending and reading messages
124
125 The composer sends `message.text` as canonical prose with reference links and
126 `message.context.records` built from the draft (`buildMessageContext` in
127 `apps/web/src/lib/composerContextRecords.ts`). Expired terminal excerpts are dropped from both.
128 The server projects provider text at turn start (`projectComposerContextForProvider`), so the persisted
129 message stays readable and the provider receives markers plus one envelope.
130
131 Review comments and preview annotations enter the draft through store mutators. A mounted composer
132 registers a context insertion handler so panel-originated references land at its current or
133 last-known caret; when no composer is mounted, the store appends them. Terminal excerpts and
134 attachments use the same caret-first behavior. Removing a chip in the editor removes the record;
135 removing a preview screenshot thumbnail removes its annotation and chip.
136
137 The transcript resolves a message with `resolveUserMessageContext`: structured context is used as
138 is, older messages are upgraded in memory. `ChatMarkdown` renders `t3-context://` links through
139 `renderContextReference`, which the timeline maps to chips through the web context-presentation
140 registry. The registry declares compact, details, and expanded capabilities for every known kind,
141 rejects duplicate surface handlers, and provides the unresolved fallback. Terminal excerpts,
142 elements, review comments, and preview annotations open structured details popovers; images and
143 videos use the shared media modal. Mobile renders context links as their labels.
144
145 Pull-request summaries currently travel as review-comment records with optional typed
146 `pullRequest` metadata. The metadata is a snapshot of the number, title, URL, branches, state, and
147 draft flag at attachment time. Web and desktop render the compact `#number` label and derive its
148 status tone from that snapshot. Hover shows the snapshot details; activation resolves the URL
149 against the current environment and opens the pull request in the thread's right panel. Records
150 written before the metadata was added retain their legacy details and neutral pull-request tone.
151
152 ## Attachments
153
154 Image and file records use the draft attachment's local id as `contextId` and carry an
155 `attachmentId` binding. The composer sends the upload's pending id (or the local id on the
156 data-URL path, via the optional `id` on `UploadChatImageAttachment`); the server's `Normalizer`
157 rewrites every image and file record to the persisted id it assigns, so the stored message binds
158 records to real resources. Optimistic rows bind to local ids and are replaced by the server copy.
159
160 In the composer, attaching a file or image inserts a chip at the caret (appended when the editor
161 cannot take input). Files exist only as chips: a file whose last chip is deleted is removed and
162 its upload released, and drafts that predate references get a chip appended on hydration. Images
163 keep the thumbnail shelf as their inventory; deleting a chip leaves the image, and removing a
164 thumbnail that is still referenced asks for confirmation before removing both. Old drafts do not
165 gain image chips.
166
167 In the transcript an image chip opens the gallery preview, a video chip opens the media preview,
168 and another file chip opens or downloads the file. The gallery still shows every image; file rows
169 remain only for files no chip references.
170
171 ## Clipboard
172
173 Every copy path writes the canonical Markdown as `text/plain` and, when the selection holds
174 chips, a structured fragment under `web application/x-t3-context-fragment+json`
175 (`ComposerContextClipboardFragment`: version, source environment/thread/message, records; no
176 bytes, no URLs). Composer copy and cut add it through a Lexical command listener; transcript
177 selection copy adds it from an `onCopyCapture` on the user message body while chips re-emit their
178 links through `data-markdown-copy`; the whole-message button writes both through `ClipboardItem`
179 and falls back to plain text.
180
181 On paste the composer decodes the fragment before the plain text. Records the draft does not
182 already hold are imported: terminal excerpts and review comments as they are, preview
183 annotations rebuilt from their record, and images or files re-fetched through the source
184 environment's asset URL and attached under a fresh local id, with the pasted link rewritten to
185 that id. A pasted binary reads as an unresolved chip until its bytes arrive; a fragment from
186 another environment leaves binaries unresolved. Rendered chips and sent messages are never
187 mutated by a paste.
188
189 Pasting across threads, projects, or environments uses the same path: the client mints an asset
190 URL from the source environment, downloads the bytes, and attaches them here. There is no
191 server-side clone; if the source is unreachable or the attachment is gone, a toast says so and
192 the chip stays unresolved.
193
194 ## Ids, persistence, and the stash
195
196 Producers keep their own id grammars; `toComposerContextId` folds anything outside
197 `[a-z0-9_-]` into a slug plus a hash, deterministically, at the reference and record boundary.
198 A preview annotation's context id is derived from `annotation-<id>` so it stays distinct from its
199 screenshot image, whose attachment id is the annotation id; the record links the two through
200 `screenshotContextId`.
201
202 `projection_thread_messages.context_json` persists records, so a restart or projection reload
203 keeps chips resolvable. Prompt stash entries carry `records` for terminal excerpts, review
204 comments, and preview annotations; stashing moves them out of the draft and restoring imports
205 them back through the same importer the paste path uses.
206
207 Context produced by other panels reaches the caret through `setContextInsertionHandler`: a
208 mounted composer registers an inserter for its draft and the store falls back to appending.