| 1 | # Model Selection Decision Matrix |
| 2 | |
| 3 | Pricing source: [GitHub Copilot Models and Pricing](https://docs.github.com/en/copilot/reference/copilot-billing/models-and-pricing), fetched 2026-06-06. Review prices every two months. |
| 4 | |
| 5 | Scheduled reminder: `.github/workflows/copilot-pricing-review.yml` checks this cadence every two months and opens/updates a review issue; pricing changes still require a normal PR. |
| 6 | |
| 7 | ## Current Configuration |
| 8 | |
| 9 | | Task | Model | Cost/Run | Quality Req | Notes | |
| 10 | |------|-------|----------|-------------|-------| |
| 11 | | Weekly Analysis | Claude Sonnet 4 | ~$0.35 | quality_score ≥ 60 | Primary, full context | |
| 12 | | Reskill | Claude Sonnet 4 | ~$0.10 | N/A (advisory) | Lower token count | |
| 13 | | Copilot Failure Diagnosis | No AI | $0.00 | N/A | Copilot failures fail closed or produce publish-ineligible diagnostics; no GitHub Models/OpenAI fallback | |
| 14 | | Budget Mode | Copilot GPT-5.4 mini | ~$0.08 | quality_score ≥ 50 | Truncated context, still through Copilot | |
| 15 | | Minimal Mode | GPT-5 mini | ~$0.05 | quality_score ≥ 40 | Top 30 repos only | |
| 16 | | Scoring | Local (no AI) | $0.00 | N/A | Heuristic-based | |
| 17 | | Pre-flight | Local (no AI) | $0.00 | N/A | Token counting only | |
| 18 | |
| 19 | ## Decision Criteria |
| 20 | |
| 21 | 1. Monthly budget remaining > 50%: use Claude Sonnet 4 |
| 22 | 2. Monthly budget 20-50%: switch Copilot model to GPT-5.4 mini |
| 23 | 3. Monthly budget < 20%: switch to GPT-5 mini |
| 24 | 4. Monthly budget exhausted: diagnostic no-AI mode only (raw stats are publish-ineligible; no AI fallback) |
| 25 | |
| 26 | ## Quality Thresholds |
| 27 | |
| 28 | - Below quality_score 40: reject and retry with better model |
| 29 | - Below quality_score 50: acceptable for budget mode only |
| 30 | - Above quality_score 60: production quality |
| 31 | |
| 32 | ## Evolution Plan |
| 33 | |
| 34 | - Review model pricing every two months and quality data monthly |
| 35 | - Adjust thresholds if model pricing changes |
| 36 | - Consider direct Anthropic API if caching becomes critical |