The best way to control Claude Code, Codex and more from any computer or phone
| 1 | # Voice input |
| 2 | |
| 3 | Transcription edits a composer draft. It does not submit an agent turn. Audio is |
| 4 | temporary client input, and only normal message submission sends the resulting |
| 5 | text to the agent. Mobile defaults to on-device transcription. Sending audio off |
| 6 | the device requires an explicit grant naming one environment and, for cloud |
| 7 | transcription, one provider. |
| 8 | |
| 9 | The [shared controller](../../packages/client-runtime/src/voice-input/controller.ts) |
| 10 | owns the operation while the client supplies capture and transcription. The |
| 11 | transcriber is bound to the recording when it starts. Capture can begin while |
| 12 | the locale and destination are still being resolved. Draft ownership, |
| 13 | text, and revision are captured before recording and checked before insertion, so |
| 14 | a late transcript cannot overwrite a draft that was edited or replaced. A |
| 15 | transcriber may expose a streaming session, in which case audio leaves during |
| 16 | capture and `finish` replaces the file-based `transcribe`. |
| 17 | |
| 18 | ## Where audio may go |
| 19 | |
| 20 | On mobile, on-device transcription needs no audio grant. Environment and cloud |
| 21 | transcription share a recorder and upload only finished recordings. The client |
| 22 | keeps the grant in device-bound secure storage, outside preference backups, and |
| 23 | rechecks it after native decoding and before upload. |
| 24 | |
| 25 | The macOS desktop app adds a `device` destination, Apple Speech. It exists only in |
| 26 | the web grant schema, not in `VoiceTranscriptionDestination`, because no environment |
| 27 | or other client can act on it. The desktop main process runs a |
| 28 | [Swift helper](../../native/apple-speech/AppleSpeech.swift) that the macOS release |
| 29 | runners compile; the Speech framework exists only in the macOS SDK. |
| 30 | |
| 31 | The [web grant](../../apps/web/src/speech/voiceConsent.ts) lives outside the preferences snapshot so another |
| 32 | window saving unrelated settings cannot resurrect a revoked grant. Each new choice |
| 33 | gets a distinct grant, even if the destination was previously selected. The |
| 34 | [web hook](../../apps/web/src/speech/useEnvironmentSpeechInput.ts) binds the |
| 35 | controller to that grant and the selected environment's connection. It requires |
| 36 | `voiceTranscription` to use the environment's model or `cloudVoiceTranscription` |
| 37 | to forward audio to the selected provider. Changing the grant or losing the |
| 38 | connection, permission, or required capability discards both the active recording |
| 39 | and any recording retained for retry. It never chooses a fallback destination. |
| 40 | The settings page's transcription test uses the same hook. The microphone loopback |
| 41 | test captures separately for local playback and sends nothing. The transport |
| 42 | refuses redirects so credentials and audio cannot follow one to another host. |
| 43 | |
| 44 | Transcript cleanup requires its own grant, off by default. |
| 45 | It sends text, never audio, to the text generation model of the draft's |
| 46 | own environment, because that is where its credentials live. |
| 47 | |
| 48 | A cloud destination binds both the forwarding environment and the named provider. |
| 49 | It uses a separate, batch-only endpoint: changing a server key cannot upgrade |
| 50 | an environment-local grant, and a cloud recording never uses the streaming |
| 51 | socket or falls back to another provider or the environment's native model. |
| 52 | |
| 53 | OpenRouter routes to many upstreams, so its grant also names the exact model and |
| 54 | the model travels with every upload as a query parameter. The server accepts only |
| 55 | ids present in OpenRouter's public ZDR-eligible transcription catalog, |
| 56 | which it caches for ten minutes after a successful read; |
| 57 | router aliases such as `openrouter/auto` or `:nitro` fail schema validation before |
| 58 | that. Requests set `provider.zdr` and carry no `models` fallback list. OpenRouter |
| 59 | applies the account's safety-retention policy when choosing an endpoint. The catalog |
| 60 | holds no credential or consent state, which is why it may be cached while key |
| 61 | presence is read fresh on every request. |
| 62 | |
| 63 | ## Server side |
| 64 | |
| 65 | The [speech service](../../apps/server/src/speech/SpeechService.ts) runs |
| 66 | transcribe-cpp in a child process so a hung native call can be killed without |
| 67 | taking the server down; see [native.ts](../../apps/server/src/speech/native.ts) |
| 68 | for the packaging constraints (Node SEA, Electron ASAR). One native request runs |
| 69 | at a time. Batch uploads carry their options in the body framing defined in |
| 70 | [shared/speech.ts](../../packages/shared/src/speech.ts); streaming sends them in |
| 71 | the socket's first command. Model files and the selected model, hardware |
| 72 | placement, and unload timeout are environment state in server settings. |
| 73 | A model file from another app is installed as a symlink only after its SHA-256 |
| 74 | matches a catalog entry, so the store never loads a file the catalog does not pin |
| 75 | and removing a model never deletes a file T3 Code did not download. |
| 76 | |
| 77 | The Silero silence detector is optional. If its download fails, recordings use |
| 78 | raw audio during the retry cooldown so a detector outage cannot block dictation. |
| 79 | Explicitly installing a speech model also retries the detector without waiting. |
| 80 | |
| 81 | ## Cancellation and resource ownership |
| 82 | |
| 83 | Cancellation invalidates a result immediately, but resources stay owned until the |
| 84 | underlying work settles. Apple's native transcription call cannot be interrupted |
| 85 | once started. Releasing the session or deleting its recording when the abort signal |
| 86 | fires would race that work. The [transcription contract](../../packages/client-runtime/src/voice-input/transcription.ts) |
| 87 | therefore requires implementations to settle only after their work has stopped; |
| 88 | the [Apple binding](../../apps/mobile/src/native/voiceTranscription.ios.ts) checks |
| 89 | cancellation between native calls and discards late results. On the server, |
| 90 | interrupting a batch request kills the worker process instead; the next request |
| 91 | reloads the model. |