The best way to control Claude Code, Codex and more from any computer or phone

main
md 1014 Bytes

Thread titles

Generates thread titles for real thread shapes with the server's Codex title generator, then sets each new title against the old one for a blind review.

Run

cd evals
pnpm thread-titles --model <codex model> --out runs/titles
pnpm thread-titles --model <codex model> --out runs/titles-new --baseline runs/titles/results.json
  • It uses your local Codex login, like the server does.
  • --case <id> limits the run to one case. Repeat the flag for multiple cases.
  • --initial titles from the first message only, as a new thread does.
  • A run writes results.json (titles and latency), review.json (titles A and B in random order, with a rubric), and answer-key.json. Score review.json before you open answer-key.json.

Add a case

Add an entry to cases.ts with an id, a source, the thread messages, the previousTitle, an optional projectTitle, optional instructions (the Thread title instructions setting), and a rubric that says what a good title names.