The best way to control Claude Code, Codex and more from any computer or phone
Mirrored from https://github.com/Camo67/t3code.git
Synced Nov 21, 03:37 UTC
| 1 | # Thread titles |
| 2 | |
| 3 | Generates thread titles for real thread shapes with the server's Codex title generator, then sets each new title against the old one for a blind review. |
| 4 | |
| 5 | ## Run |
| 6 | |
| 7 | ```sh |
| 8 | cd evals |
| 9 | pnpm thread-titles --model <codex model> --out runs/titles |
| 10 | pnpm thread-titles --model <codex model> --out runs/titles-new --baseline runs/titles/results.json |
| 11 | ``` |
| 12 | |
| 13 | - It uses your local Codex login, like the server does. |
| 14 | - `--case <id>` limits the run to one case. Repeat the flag for multiple cases. |
| 15 | - `--initial` titles from the first message only, as a new thread does. |
| 16 | - A run writes `results.json` (titles and latency), `review.json` (titles A and B in random order, with a rubric), and `answer-key.json`. Score `review.json` before you open `answer-key.json`. |
| 17 | |
| 18 | ## Add a case |
| 19 | |
| 20 | Add an entry to `cases.ts` with an `id`, a `source`, the thread `messages`, the `previousTitle`, an optional `projectTitle`, optional `instructions` (the Thread title instructions setting), and a `rubric` that says what a good title names. |