The best way to control Claude Code, Codex and more from any computer or phone

main
md 20 lines 1014 Bytes
Rendered Raw
1 # Thread titles
2
3 Generates thread titles for real thread shapes with the server's Codex title generator, then sets each new title against the old one for a blind review.
4
5 ## Run
6
7 ```sh
8 cd evals
9 pnpm thread-titles --model <codex model> --out runs/titles
10 pnpm thread-titles --model <codex model> --out runs/titles-new --baseline runs/titles/results.json
11 ```
12
13 - It uses your local Codex login, like the server does.
14 - `--case <id>` limits the run to one case. Repeat the flag for multiple cases.
15 - `--initial` titles from the first message only, as a new thread does.
16 - A run writes `results.json` (titles and latency), `review.json` (titles A and B in random order, with a rubric), and `answer-key.json`. Score `review.json` before you open `answer-key.json`.
17
18 ## Add a case
19
20 Add an entry to `cases.ts` with an `id`, a `source`, the thread `messages`, the `previousTitle`, an optional `projectTitle`, optional `instructions` (the Thread title instructions setting), and a `rubric` that says what a good title names.