| 1 | # Ralph Circuit Breaker — Model Rate Limit Fallback |
| 2 | |
| 3 | > Classic circuit breaker pattern (Hystrix / Polly / Resilience4j) applied to Copilot model selection. |
| 4 | > When the preferred model hits rate limits, Ralph automatically degrades to free-tier models, then self-heals. |
| 5 | |
| 6 | ## Problem |
| 7 | |
| 8 | When running multiple Ralph instances across repos, Copilot model rate limits cause cascading failures. |
| 9 | All Ralphs fail simultaneously when the preferred model (e.g., `claude-sonnet-4.6`) hits quota. |
| 10 | |
| 11 | Premium models burn quota fast: |
| 12 | | Model | Multiplier | Risk | |
| 13 | |-------|-----------|------| |
| 14 | | `claude-sonnet-4.6` | 1x | Moderate with many Ralphs | |
| 15 | | `claude-opus-4.6` | 10x | High | |
| 16 | | `gpt-5.4` | 50x | Very high | |
| 17 | | `gpt-5.4-mini` | **0x** | **Free — unlimited** | |
| 18 | | `gpt-5-mini` | **0x** | **Free — unlimited** | |
| 19 | | `gpt-4.1` | **0x** | **Free — unlimited** | |
| 20 | |
| 21 | ## Circuit Breaker States |
| 22 | |
| 23 | ``` |
| 24 | ┌─────────┐ rate limit error ┌────────┐ |
| 25 | │ CLOSED │ ───────────────────► │ OPEN │ |
| 26 | │ (normal)│ │(fallback)│ |
| 27 | └────┬────┘ ◄──────────────── └────┬────┘ |
| 28 | │ 2 consecutive │ |
| 29 | │ successes │ cooldown expires |
| 30 | │ ▼ |
| 31 | │ ┌──────────┐ |
| 32 | └───── success ◄──────── │HALF-OPEN │ |
| 33 | (close) │ (testing) │ |
| 34 | └──────────┘ |
| 35 | ``` |
| 36 | |
| 37 | ### CLOSED (normal operation) |
| 38 | - Use preferred model from config |
| 39 | - Every successful response confirms circuit stays closed |
| 40 | - On rate limit error → transition to OPEN |
| 41 | |
| 42 | ### OPEN (rate limited — fallback active) |
| 43 | - Fall back through the free-tier model chain: |
| 44 | 1. `gpt-5.4-mini` |
| 45 | 2. `gpt-5-mini` |
| 46 | 3. `gpt-4.1` |
| 47 | - Start cooldown timer (default: 10 minutes) |
| 48 | - When cooldown expires → transition to HALF-OPEN |
| 49 | |
| 50 | ### HALF-OPEN (testing recovery) |
| 51 | - Try preferred model again |
| 52 | - If 2 consecutive successes → transition to CLOSED |
| 53 | - If rate limit error → back to OPEN, reset cooldown |
| 54 | |
| 55 | ## State File: `.squad/ralph-circuit-breaker.json` |
| 56 | |
| 57 | ```json |
| 58 | { |
| 59 | "state": "closed", |
| 60 | "preferredModel": "claude-sonnet-4.6", |
| 61 | "fallbackChain": ["gpt-5.4-mini", "gpt-5-mini", "gpt-4.1"], |
| 62 | "currentFallbackIndex": 0, |
| 63 | "cooldownMinutes": 10, |
| 64 | "openedAt": null, |
| 65 | "halfOpenSuccesses": 0, |
| 66 | "consecutiveFailures": 0, |
| 67 | "metrics": { |
| 68 | "totalFallbacks": 0, |
| 69 | "totalRecoveries": 0, |
| 70 | "lastFallbackAt": null, |
| 71 | "lastRecoveryAt": null |
| 72 | } |
| 73 | } |
| 74 | ``` |
| 75 | |
| 76 | ## PowerShell Functions |
| 77 | |
| 78 | Paste these into your `ralph-watch.ps1` or source them from a shared module. |
| 79 | |
| 80 | ### `Get-CircuitBreakerState` |
| 81 | |
| 82 | ```powershell |
| 83 | function Get-CircuitBreakerState { |
| 84 | param([string]$StateFile = ".squad/ralph-circuit-breaker.json") |
| 85 | |
| 86 | if (-not (Test-Path $StateFile)) { |
| 87 | $default = @{ |
| 88 | state = "closed" |
| 89 | preferredModel = "claude-sonnet-4.6" |
| 90 | fallbackChain = @("gpt-5.4-mini", "gpt-5-mini", "gpt-4.1") |
| 91 | currentFallbackIndex = 0 |
| 92 | cooldownMinutes = 10 |
| 93 | openedAt = $null |
| 94 | halfOpenSuccesses = 0 |
| 95 | consecutiveFailures = 0 |
| 96 | metrics = @{ |
| 97 | totalFallbacks = 0 |
| 98 | totalRecoveries = 0 |
| 99 | lastFallbackAt = $null |
| 100 | lastRecoveryAt = $null |
| 101 | } |
| 102 | } |
| 103 | $default | ConvertTo-Json -Depth 3 | Set-Content $StateFile |
| 104 | return $default |
| 105 | } |
| 106 | |
| 107 | return (Get-Content $StateFile -Raw | ConvertFrom-Json) |
| 108 | } |
| 109 | ``` |
| 110 | |
| 111 | ### `Save-CircuitBreakerState` |
| 112 | |
| 113 | ```powershell |
| 114 | function Save-CircuitBreakerState { |
| 115 | param( |
| 116 | [object]$State, |
| 117 | [string]$StateFile = ".squad/ralph-circuit-breaker.json" |
| 118 | ) |
| 119 | |
| 120 | $State | ConvertTo-Json -Depth 3 | Set-Content $StateFile |
| 121 | } |
| 122 | ``` |
| 123 | |
| 124 | ### `Get-CurrentModel` |
| 125 | |
| 126 | Returns the model Ralph should use right now, based on circuit state. |
| 127 | |
| 128 | ```powershell |
| 129 | function Get-CurrentModel { |
| 130 | param([string]$StateFile = ".squad/ralph-circuit-breaker.json") |
| 131 | |
| 132 | $cb = Get-CircuitBreakerState -StateFile $StateFile |
| 133 | |
| 134 | switch ($cb.state) { |
| 135 | "closed" { |
| 136 | return $cb.preferredModel |
| 137 | } |
| 138 | "open" { |
| 139 | # Check if cooldown has expired |
| 140 | if ($cb.openedAt) { |
| 141 | $opened = [DateTime]::Parse($cb.openedAt) |
| 142 | $elapsed = (Get-Date) - $opened |
| 143 | if ($elapsed.TotalMinutes -ge $cb.cooldownMinutes) { |
| 144 | # Transition to half-open |
| 145 | $cb.state = "half-open" |
| 146 | $cb.halfOpenSuccesses = 0 |
| 147 | Save-CircuitBreakerState -State $cb -StateFile $StateFile |
| 148 | Write-Host " [circuit-breaker] Cooldown expired. Testing preferred model..." -ForegroundColor Yellow |
| 149 | return $cb.preferredModel |
| 150 | } |
| 151 | } |
| 152 | # Still in cooldown — use fallback |
| 153 | $idx = [Math]::Min($cb.currentFallbackIndex, $cb.fallbackChain.Count - 1) |
| 154 | return $cb.fallbackChain[$idx] |
| 155 | } |
| 156 | "half-open" { |
| 157 | return $cb.preferredModel |
| 158 | } |
| 159 | default { |
| 160 | return $cb.preferredModel |
| 161 | } |
| 162 | } |
| 163 | } |
| 164 | ``` |
| 165 | |
| 166 | ### `Update-CircuitBreakerOnSuccess` |
| 167 | |
| 168 | Call after every successful model response. |
| 169 | |
| 170 | ```powershell |
| 171 | function Update-CircuitBreakerOnSuccess { |
| 172 | param([string]$StateFile = ".squad/ralph-circuit-breaker.json") |
| 173 | |
| 174 | $cb = Get-CircuitBreakerState -StateFile $StateFile |
| 175 | $cb.consecutiveFailures = 0 |
| 176 | |
| 177 | if ($cb.state -eq "half-open") { |
| 178 | $cb.halfOpenSuccesses++ |
| 179 | if ($cb.halfOpenSuccesses -ge 2) { |
| 180 | # Recovery! Close the circuit |
| 181 | $cb.state = "closed" |
| 182 | $cb.openedAt = $null |
| 183 | $cb.halfOpenSuccesses = 0 |
| 184 | $cb.currentFallbackIndex = 0 |
| 185 | $cb.metrics.totalRecoveries++ |
| 186 | $cb.metrics.lastRecoveryAt = (Get-Date).ToString("o") |
| 187 | Save-CircuitBreakerState -State $cb -StateFile $StateFile |
| 188 | Write-Host " [circuit-breaker] RECOVERED — back to preferred model ($($cb.preferredModel))" -ForegroundColor Green |
| 189 | return |
| 190 | } |
| 191 | Save-CircuitBreakerState -State $cb -StateFile $StateFile |
| 192 | Write-Host " [circuit-breaker] Half-open success $($cb.halfOpenSuccesses)/2" -ForegroundColor Yellow |
| 193 | return |
| 194 | } |
| 195 | |
| 196 | # closed state — nothing to do |
| 197 | } |
| 198 | ``` |
| 199 | |
| 200 | ### `Update-CircuitBreakerOnRateLimit` |
| 201 | |
| 202 | Call when a model response indicates rate limiting (HTTP 429 or error message containing "rate limit"). |
| 203 | |
| 204 | ```powershell |
| 205 | function Update-CircuitBreakerOnRateLimit { |
| 206 | param([string]$StateFile = ".squad/ralph-circuit-breaker.json") |
| 207 | |
| 208 | $cb = Get-CircuitBreakerState -StateFile $StateFile |
| 209 | $cb.consecutiveFailures++ |
| 210 | |
| 211 | if ($cb.state -eq "closed" -or $cb.state -eq "half-open") { |
| 212 | # Open the circuit |
| 213 | $cb.state = "open" |
| 214 | $cb.openedAt = (Get-Date).ToString("o") |
| 215 | $cb.halfOpenSuccesses = 0 |
| 216 | $cb.currentFallbackIndex = 0 |
| 217 | $cb.metrics.totalFallbacks++ |
| 218 | $cb.metrics.lastFallbackAt = (Get-Date).ToString("o") |
| 219 | Save-CircuitBreakerState -State $cb -StateFile $StateFile |
| 220 | |
| 221 | $fallbackModel = $cb.fallbackChain[0] |
| 222 | Write-Host " [circuit-breaker] RATE LIMITED — falling back to $fallbackModel (cooldown: $($cb.cooldownMinutes)m)" -ForegroundColor Red |
| 223 | return |
| 224 | } |
| 225 | |
| 226 | if ($cb.state -eq "open") { |
| 227 | # Already open — try next fallback in chain if current one also fails |
| 228 | if ($cb.currentFallbackIndex -lt ($cb.fallbackChain.Count - 1)) { |
| 229 | $cb.currentFallbackIndex++ |
| 230 | $nextModel = $cb.fallbackChain[$cb.currentFallbackIndex] |
| 231 | Write-Host " [circuit-breaker] Fallback also limited — trying $nextModel" -ForegroundColor Red |
| 232 | } |
| 233 | # Reset cooldown timer |
| 234 | $cb.openedAt = (Get-Date).ToString("o") |
| 235 | Save-CircuitBreakerState -State $cb -StateFile $StateFile |
| 236 | } |
| 237 | } |
| 238 | ``` |
| 239 | |
| 240 | ## Integration with ralph-watch.ps1 |
| 241 | |
| 242 | In your Ralph polling loop, wrap the model selection: |
| 243 | |
| 244 | ```powershell |
| 245 | # At the top of your polling loop |
| 246 | $model = Get-CurrentModel |
| 247 | |
| 248 | # When invoking copilot CLI |
| 249 | $result = copilot-cli --model $model ... |
| 250 | |
| 251 | # After the call |
| 252 | if ($result -match "rate.?limit" -or $LASTEXITCODE -eq 429) { |
| 253 | Update-CircuitBreakerOnRateLimit |
| 254 | } else { |
| 255 | Update-CircuitBreakerOnSuccess |
| 256 | } |
| 257 | ``` |
| 258 | |
| 259 | ### Full integration example |
| 260 | |
| 261 | ```powershell |
| 262 | # Source the circuit breaker functions |
| 263 | . .squad-templates/ralph-circuit-breaker-functions.ps1 |
| 264 | |
| 265 | while ($true) { |
| 266 | $model = Get-CurrentModel |
| 267 | Write-Host "Polling with model: $model" |
| 268 | |
| 269 | try { |
| 270 | # Your existing Ralph logic here, but pass $model |
| 271 | $response = Invoke-RalphCycle -Model $model |
| 272 | |
| 273 | # Success path |
| 274 | Update-CircuitBreakerOnSuccess |
| 275 | } |
| 276 | catch { |
| 277 | if ($_.Exception.Message -match "rate.?limit|429|quota|Too Many Requests") { |
| 278 | Update-CircuitBreakerOnRateLimit |
| 279 | # Retry immediately with fallback model |
| 280 | continue |
| 281 | } |
| 282 | # Other errors — handle normally |
| 283 | throw |
| 284 | } |
| 285 | |
| 286 | Start-Sleep -Seconds $pollInterval |
| 287 | } |
| 288 | ``` |
| 289 | |
| 290 | ## Configuration |
| 291 | |
| 292 | Override defaults by editing `.squad/ralph-circuit-breaker.json`: |
| 293 | |
| 294 | | Field | Default | Description | |
| 295 | |-------|---------|-------------| |
| 296 | | `preferredModel` | `claude-sonnet-4.6` | Model to use when circuit is closed | |
| 297 | | `fallbackChain` | `["gpt-5.4-mini", "gpt-5-mini", "gpt-4.1"]` | Ordered fallback models (all free-tier) | |
| 298 | | `cooldownMinutes` | `10` | How long to wait before testing recovery | |
| 299 | |
| 300 | ## Metrics |
| 301 | |
| 302 | The state file tracks operational metrics: |
| 303 | |
| 304 | - **totalFallbacks** — How many times the circuit opened |
| 305 | - **totalRecoveries** — How many times it recovered to preferred model |
| 306 | - **lastFallbackAt** — ISO timestamp of last rate limit event |
| 307 | - **lastRecoveryAt** — ISO timestamp of last successful recovery |
| 308 | |
| 309 | Query metrics with: |
| 310 | ```powershell |
| 311 | $cb = Get-Content .squad/ralph-circuit-breaker.json | ConvertFrom-Json |
| 312 | Write-Host "Fallbacks: $($cb.metrics.totalFallbacks) | Recoveries: $($cb.metrics.totalRecoveries)" |
| 313 | ``` |