| 1 | # SquadScope Operator Guide |
| 2 | |
| 3 | This guide covers everything needed to set up, operate, monitor, and troubleshoot SquadScope in production. |
| 4 | |
| 5 | ## Prerequisites |
| 6 | |
| 7 | Before starting, ensure you have: |
| 8 | |
| 9 | 1. **A GitHub repository** — Fork or create a copy of SquadScope |
| 10 | 2. **GitHub Pages enabled** — In repo settings, set source to "GitHub Actions" |
| 11 | 3. **GitHub Copilot subscription** — Required for AI-powered analysis (primary path) |
| 12 | - Copilot Pro ($20/month) or Copilot Team subscription |
| 13 | - Account with Copilot API access |
| 14 | 4. **Fine-grained Personal Access Token (PAT)** |
| 15 | - Scope: `github_pat_...` (not `ghp_...`) |
| 16 | - Permission: Account → Copilot Requests (at least) |
| 17 | - No expiration recommended (or set far future) |
| 18 | 5. **Basic CLI tools** — `gh` CLI installed (`brew install gh` on macOS, `apt install gh` on Linux) |
| 19 | |
| 20 | ## Initial Setup |
| 21 | |
| 22 | ### Step 1: Clone the repository |
| 23 | |
| 24 | ```bash |
| 25 | git clone --recurse-submodules https://github.com/YOUR_USERNAME/SquadScope.git |
| 26 | cd SquadScope |
| 27 | ``` |
| 28 | |
| 29 | If you forgot `--recurse-submodules`, initialize them now: |
| 30 | |
| 31 | ```bash |
| 32 | git submodule update --init --recursive |
| 33 | ``` |
| 34 | ### Step 2: Verify Hugo version locally |
| 35 | |
| 36 | Ensure your Hugo binary is v0.146.0 or newer: |
| 37 | |
| 38 | ```bash |
| 39 | hugo version |
| 40 | ``` |
| 41 | |
| 42 | If needed, download the extended version: |
| 43 | - macOS: `brew install hugo` |
| 44 | - Linux: Download from [hugo releases](https://github.com/gohugoio/hugo/releases) |
| 45 | - Windows: Download from releases or use Chocolatey |
| 46 | |
| 47 | ### Step 3: Configure GitHub Copilot secret |
| 48 | |
| 49 | Create a fine-grained PAT on GitHub: |
| 50 | |
| 51 | 1. Navigate to **Settings → Developer settings → Personal access tokens → Fine-grained tokens** |
| 52 | 2. Click **Generate new token** |
| 53 | 3. Configure: |
| 54 | - **Token name:** `SquadScope-Copilot-Token` |
| 55 | - **Expiration:** No expiration (or set to 1 year) |
| 56 | - **Repository access:** Select your SquadScope repository |
| 57 | - **Account permissions:** Scroll to **Copilot Requests** and select **Read and write** |
| 58 | - Click **Generate token** |
| 59 | 4. Copy the token (you won't see it again) |
| 60 | |
| 61 | Add the token as a repository secret: |
| 62 | |
| 63 | ```bash |
| 64 | # Using GitHub CLI |
| 65 | gh secret set COPILOT_GH_TOKEN --body YOUR_PAT_HERE -R YOUR_USERNAME/SquadScope |
| 66 | |
| 67 | # Or manually in repo settings: |
| 68 | # Settings → Secrets and variables → Actions → New repository secret |
| 69 | # Name: COPILOT_GH_TOKEN |
| 70 | # Value: github_pat_... |
| 71 | ``` |
| 72 | |
| 73 | Verify the secret exists: |
| 74 | |
| 75 | ```bash |
| 76 | gh secret list -R YOUR_USERNAME/SquadScope |
| 77 | ``` |
| 78 | |
| 79 | ### Step 4: Configure optional webhook notifications |
| 80 | |
| 81 | To notify a team channel whenever a weekly summary is published, add a repository secret named `WEBHOOK_URL` (a secret, not a variable, because webhook URLs are credentials that should be masked and protected): |
| 82 | |
| 83 | ```bash |
| 84 | gh secret set WEBHOOK_URL --body "https://example.com/webhook" -R YOUR_USERNAME/SquadScope |
| 85 | ``` |
| 86 | |
| 87 | You can also add it in the GitHub UI under **Settings → Secrets and variables → Actions → Secrets**. |
| 88 | |
| 89 | Supported endpoints: |
| 90 | |
| 91 | - **Discord:** Create a webhook in the target channel's **Edit Channel → Integrations → Webhooks** settings, then paste that webhook URL into `WEBHOOK_URL`. |
| 92 | - **Slack:** Create an **Incoming Webhook** app for the target channel, then paste that webhook URL into `WEBHOOK_URL`. |
| 93 | - **Custom endpoint:** Any endpoint that accepts an HTTP `POST` with a JSON body containing `content` and `username` fields. |
| 94 | |
| 95 | If `WEBHOOK_URL` is unset, the workflow skips the webhook step automatically. |
| 96 | |
| 97 | ### Step 5: Configure optional Podcaster handoff |
| 98 | |
| 99 | To ask the separate Podcaster service to generate an episode after a normal weekly article is published and deployed, configure: |
| 100 | |
| 101 | - Actions variable `PODCASTER_ENDPOINT`, for example `https://<function-app-name>.azurewebsites.net/api/generate` or local testing URL `http://localhost:7071/api/generate` |
| 102 | - Actions secret `PODCASTER_API_KEY` |
| 103 | |
| 104 | The workflow sends `week`, `article_url`, `article_path`, `article_sha256` when available, `publish_run_id`, `publish_mode`, and source artifact references after the normal article deploy succeeds. Normal runs and **audited force-replace corrections** — a `force-replace` policy manifest that carries an operator `audit.actor` and `audit.reason` (the same signal that gates promotion) — call Podcaster so a material content correction refreshes the episode. Plain (non-audited) `restore` replays are deliberately excluded and are a clean skip (a notice, exit 0 — not a failure), so replays never spam the Podcaster and never fail the sync job. Dry-run, candidate-only, no-AI, and failed runs also do not call Podcaster. If either endpoint value is missing, the handoff is skipped. The API key is sent only as the `x-podcaster-api-key` header and must not be printed, logged, or committed. Handoff failure is non-critical and does not roll back or block article publication. |
| 105 | |
| 106 | ### Step 6: Enable GitHub Pages |
| 107 | |
| 108 | 1. Navigate to repo **Settings → Pages** |
| 109 | 2. Set **Source** to "GitHub Actions" |
| 110 | 3. (Optional) Configure custom domain if desired |
| 111 | 4. Save |
| 112 | |
| 113 | ### Step 7: Test local build |
| 114 | |
| 115 | Ensure the Hugo build works locally: |
| 116 | |
| 117 | ```bash |
| 118 | hugo server |
| 119 | ``` |
| 120 | |
| 121 | Visit `http://localhost:1313` and verify the site loads. Press `Ctrl+C` to stop. |
| 122 | |
| 123 | For production build: |
| 124 | |
| 125 | ```bash |
| 126 | hugo --minify |
| 127 | ``` |
| 128 | |
| 129 | This generates the `public/` directory with optimized static assets. |
| 130 | |
| 131 | ## Running the Pipeline |
| 132 | |
| 133 | ### Option A: Automatic scheduling (default) |
| 134 | |
| 135 | The pipeline runs automatically on Sundays at **11:53 UTC** (`53 11 * * 0`) via `.github/workflows/crawl-and-publish.yml`. |
| 136 | |
| 137 | GitHub-hosted scheduled workflows are **best effort**, not an exact-time SLA, so a cron match does not guarantee the job will start at 11:53. Go to your repo's **Actions** tab to monitor runs. |
| 138 | |
| 139 | #### Schedule latency and mitigation ladder |
| 140 | |
| 141 | SquadScope treats GitHub Actions `schedule` on shared runners as a convenience default, not as a punctual trigger. This repository has already observed multi-hour delays on the previous Monday `06:53 UTC` slot: |
| 142 | |
| 143 | - 2026-05-25: started at **11:55 UTC** |
| 144 | - 2026-06-01: started at **14:37 UTC** |
| 145 | - 2026-06-08: started at **12:17 UTC** |
| 146 | |
| 147 | Supported operating model, in order: |
| 148 | |
| 149 | 1. **Default:** keep the current Sunday `53 11 * * 0` cron as the low-effort baseline. |
| 150 | 2. **Manual fallback:** run `gh workflow run crawl-and-publish.yml -R YOUR_USERNAME/SquadScope` (or use the Actions UI) when an operator needs to start the pipeline immediately. |
| 151 | 3. **Recommended mitigation for punctual launches:** keep the workflow as-is and have an external scheduler call the existing `workflow_dispatch` trigger. Example CLI target for a cron job or scheduler host: |
| 152 | |
| 153 | ```bash |
| 154 | gh workflow run crawl-and-publish.yml -R YOUR_USERNAME/SquadScope |
| 155 | ``` |
| 156 | |
| 157 | Example GitHub API dispatch: |
| 158 | |
| 159 | ```bash |
| 160 | curl -L \ |
| 161 | -X POST \ |
| 162 | -H "Accept: application/vnd.github+json" \ |
| 163 | -H "Authorization: Bearer $GITHUB_TOKEN" \ |
| 164 | https://api.github.com/repos/YOUR_USERNAME/SquadScope/actions/workflows/crawl-and-publish.yml/dispatches \ |
| 165 | -d '{"ref":"main","inputs":{"run_mode":"normal","source_refresh_policy":"reuse-same-day"}}' |
| 166 | ``` |
| 167 | |
| 168 | 4. **Optional future escalation:** move to self-hosted runners only if you need tighter operational control; that is not required for the default setup. |
| 169 | |
| 170 | ### Option B: Manual trigger |
| 171 | |
| 172 | Run the workflow manually: |
| 173 | |
| 174 | ```bash |
| 175 | gh workflow run crawl-and-publish.yml -R YOUR_USERNAME/SquadScope |
| 176 | ``` |
| 177 | |
| 178 | Or through the GitHub UI: |
| 179 | 1. Go to **Actions → Crawl and Publish** |
| 180 | 2. Click **Run workflow** |
| 181 | 3. Confirm |
| 182 | |
| 183 | The workflow takes ~2-3 minutes depending on GitHub API response times. |
| 184 | |
| 185 | #### Manual rerun modes |
| 186 | |
| 187 | Manual runs default to `run_mode=normal` and `source_refresh_policy=reuse-same-day`. Normal mode is fail-closed: it may publish only after the existing analysis and freshness gates pass, and same-day successful source artifacts are reused instead of scraping again. Missing, failed, stale, wrong-week, or wrong-window sources are refreshed. |
| 188 | |
| 189 | ##### Rerun mode reference |
| 190 | |
| 191 | All rerun modes are validated before any publishing side effects: |
| 192 | |
| 193 | | Mode | Crawl | Promotion | Intent | Use case | |
| 194 | |------|-------|-----------|--------|----------| |
| 195 | | `normal` (default) | ✓ Fresh | ✓ Guarded gates | Produce fresh analysis, publish if gates pass | Standard weekly run | |
| 196 | | `dry-run` | ✓ Fresh | ✗ Never | Build candidates only for inspection | Test analysis quality, verify gates, debug analysis | |
| 197 | | `candidate-only` | ✓ Fresh | ✗ Manifest blocks | Run crawl/analysis but hold for manual approval | Staged analysis, manual promotion workflow | |
| 198 | | `restore` | ✗ Hydrate | ✓ Guarded gates | Regenerate a prior week from one source-bound immutable raw run | Restore/audit trail, regenerate HTML/feeds | |
| 199 | | `force-replace` | ✓ Fresh | ✓ Guarded gates | Explicit replacement run, gates still enforce | Planned content refresh, operator override | |
| 200 | |
| 201 | ##### Source refresh policies |
| 202 | |
| 203 | Control how same-day artifacts are handled during reruns: |
| 204 | |
| 205 | | Policy | Behavior | Use case | |
| 206 | |--------|----------|----------| |
| 207 | | `reuse-same-day` (default) | Reuse eligible same-day raw artifacts; refresh missing/stale/failed sources | Safe rerun without redundant API calls | |
| 208 | | `refresh-missing-stale` | Like reuse-same-day but also refresh sources with missing or stale status | Partial refresh, correct specific source issues | |
| 209 | | `force-refresh` | Refresh all sources regardless of prior status | Force all new data, ignore cache | |
| 210 | |
| 211 | Same-day artifact reuse is safe by design: |
| 212 | - Only successfully crawled artifacts are eligible for reuse |
| 213 | - Missing, failed, stale (>24 hours old), or wrong-week sources are always refreshed |
| 214 | - Source status (reused/refreshed/missing/failed/stale) is recorded in the publish manifest for audit trail |
| 215 | |
| 216 | Invalid combinations fail immediately with clear error messages: |
| 217 | - `rebuild_week` without `run_mode=restore` |
| 218 | - `run_mode=restore` without both `rebuild_week` and `source_run_id` |
| 219 | - `run_mode=restore` with `source_refresh_policy=force-refresh` |
| 220 | - `publish_release=true` with `dry-run` or `candidate-only` |
| 221 | |
| 222 | ##### Durable raw evidence and restore |
| 223 | |
| 224 | Each publishing crawl writes the current week's raw payloads to the existing |
| 225 | `publish` branch under: |
| 226 | |
| 227 | ```text |
| 228 | data/raw-store/<week>/<source_run_id>/ |
| 229 | ``` |
| 230 | |
| 231 | The run directory is immutable: a repeated write to the same week/run path fails |
| 232 | instead of replacing files. Its `manifest.json` records the source workflow run, |
| 233 | `raw-data` artifact ID/name, source head SHA, original paths, sizes, and SHA-256 |
| 234 | hashes. |
| 235 | |
| 236 | Restore must identify that exact source run: |
| 237 | |
| 238 | ```bash |
| 239 | gh workflow run crawl-and-publish.yml \ |
| 240 | -R YOUR_USERNAME/SquadScope \ |
| 241 | -f run_mode=restore \ |
| 242 | -f rebuild_week=2026-W23 \ |
| 243 | -f source_run_id=26753498571 |
| 244 | ``` |
| 245 | |
| 246 | The workflow verifies the immutable manifest identity and every stored hash before |
| 247 | copying any file into `data/raw/`. The selected manifest is authoritative for the |
| 248 | week, so same-week raw files left by the checkout or artifact overlay but absent |
| 249 | from that source run are removed before analysis. The publish eligibility manifest |
| 250 | records the verified source run/artifact provenance and rechecks restored input |
| 251 | hashes before promotion. |
| 252 | |
| 253 | The GitHub Actions `raw-data` artifact has **90-day retention** and remains a |
| 254 | transport mechanism for jobs, same-day reuse, and emergency recovery only. It is not |
| 255 | the durable raw store. The durable copy is the immutable week/run directory on |
| 256 | `publish`; it is intentionally not synced into `main`. |
| 257 | |
| 258 | ### Option C: Run individual stages locally |
| 259 | |
| 260 | For debugging or testing, run stages separately: |
| 261 | |
| 262 | #### Crawl |
| 263 | |
| 264 | ```bash |
| 265 | python3 scripts/crawl.py --as-of 2026-05-18 |
| 266 | ``` |
| 267 | |
| 268 | Output: `data/raw/2026-W20.json`, `data/snapshots/2026-W20-stars.json` |
| 269 | |
| 270 | #### Analyze (Fallback — if Copilot unavailable) |
| 271 | |
| 272 | ```bash |
| 273 | python3 scripts/analyze_fallback.py \ |
| 274 | --raw-json data/raw/2026-W20.json \ |
| 275 | --output data/analyzed/2026-W20-summary.md \ |
| 276 | --current-datetime 2026-05-18T16:00:00Z |
| 277 | ``` |
| 278 | |
| 279 | Output: `data/analyzed/2026-W20-summary.md` |
| 280 | |
| 281 | #### Quality gate |
| 282 | |
| 283 | ```bash |
| 284 | python3 scripts/analysis_gate.py \ |
| 285 | --analysis-file data/analyzed/2026-W20-summary.md \ |
| 286 | --raw-json data/raw/2026-W20.json \ |
| 287 | --current-datetime 2026-05-18T16:00:00Z |
| 288 | ``` |
| 289 | |
| 290 | If the gate fails, it will exit with a non-zero code and log the reason (e.g., quality_score < 60). |
| 291 | |
| 292 | #### Generate |
| 293 | |
| 294 | ```bash |
| 295 | python3 scripts/generate_content.py data/analyzed/2026-W20-summary.md |
| 296 | ``` |
| 297 | |
| 298 | Output: `content/weekly/2026/W20.md` |
| 299 | |
| 300 | #### Build and deploy |
| 301 | |
| 302 | ```bash |
| 303 | hugo --minify |
| 304 | ``` |
| 305 | |
| 306 | Output: `public/` directory ready for GitHub Pages. |
| 307 | |
| 308 | ### Option D: Analysis-only rerun (reuse crawl artifacts, no re-crawl) |
| 309 | |
| 310 | When a report is wrong because of an **analysis/rendering bug** (not a data |
| 311 | problem) — for example the report claims "No industry press data was available" |
| 312 | even though the press crawl succeeded — you can regenerate the week's analysis |
| 313 | **without re-crawling GitHub and without re-crawling press**. This reuses the |
| 314 | existing immutable crawl artifacts: |
| 315 | |
| 316 | - `data/raw/<WEEK>.json` — GitHub raw payload |
| 317 | - `data/raw/<WEEK>-external-news.json` — press/RSS crawl output |
| 318 | - `data/analyzed/<WEEK>-correlations.json` — press↔repo correlations |
| 319 | - `data/analyzed/<WEEK>-press-context.md` — rendered press context |
| 320 | |
| 321 | **Step 1 (optional) — regenerate press context from existing crawl artifacts** |
| 322 | (press-only; no network calls). Reruns correlation + rendering only: |
| 323 | |
| 324 | ```bash |
| 325 | python3 scripts/correlate.py \ |
| 326 | --raw data/raw/2026-W30.json \ |
| 327 | --techcrunch data/raw/2026-W30-external-news.json \ |
| 328 | --output data/analyzed/2026-W30-correlations.json |
| 329 | |
| 330 | python3 scripts/render_press_context.py --week 2026-W30 \ |
| 331 | > data/analyzed/2026-W30-press-context.md |
| 332 | ``` |
| 333 | |
| 334 | **Step 2 — regenerate the analysis prompt/report from existing artifacts** |
| 335 | (no GitHub crawl, no press crawl). Pass the existing raw JSON and press context: |
| 336 | |
| 337 | ```bash |
| 338 | python3 scripts/analyze_fallback.py \ |
| 339 | --raw-json data/raw/2026-W30.json \ |
| 340 | --output data/analyzed/2026-W30-summary.md \ |
| 341 | --current-datetime 2026-07-27T12:00:00Z \ |
| 342 | --press-context data/analyzed/2026-W30-press-context.md \ |
| 343 | --print-prompt # inspect the prompt; drop this flag + wire Copilot CLI to emit the report |
| 344 | ``` |
| 345 | |
| 346 | The rendered prompt includes a `## Press Context` block only when the press |
| 347 | context contains real press data — even when a Step-1 synthesis narrative is |
| 348 | supplied via `--synthesis-input` (the narrative condenses *historical* context |
| 349 | but never replaces real press data). A press-less week still renders the |
| 350 | non-empty `NO_PRESS_SENTINEL` marker (defined in |
| 351 | `scripts/render_press_context.py`), which `analyze_fallback.py` treats as absent; |
| 352 | the block is suppressed and the `press_correlations` component is recorded as |
| 353 | `included: false`. Confirm whether real press reached the prompt with: |
| 354 | |
| 355 | ```bash |
| 356 | python3 scripts/analyze_fallback.py ... --print-prompt | grep -c "## Press Context" |
| 357 | ``` |
| 358 | |
| 359 | A non-zero count means the "Where Industry Meets Code" and "Press & Industry" |
| 360 | sections will be written from real press data. A zero count on a press-less week |
| 361 | is expected: the sentinel was correctly suppressed. To distinguish real press |
| 362 | from the marker, use `--preflight-report-json` to audit the |
| 363 | `press_correlations` component (`included: true` for real press, `false` for |
| 364 | no press), and consult the `NO_PRESS_SENTINEL` symbol in |
| 365 | `scripts/render_press_context.py` as the authoritative marker definition. |
| 366 | |
| 367 | ## Understanding Source Artifacts and Reuse |
| 368 | |
| 369 | ### Source artifact tracking |
| 370 | |
| 371 | When SquadScope crawls, it records detailed information about each source artifact: |
| 372 | |
| 373 | - **Status:** One of `reused`, `refreshed`, `missing`, `failed`, or `stale` |
| 374 | - **Artifact checksum:** SHA256 of successful crawls for integrity verification |
| 375 | - **Timestamp:** When the artifact was produced or reused |
| 376 | - **Code checksum:** Hash of the crawler code that produced it (detects version drift) |
| 377 | |
| 378 | This metadata is stored in the **publish manifest** (`data/candidates/YYYY-WNN/RUN_ID/publish-manifest.json`) for every run, creating an auditable trail of: |
| 379 | - Which sources were fetched vs. reused |
| 380 | - Why sources were refreshed (missing, failed, stale, code drift) |
| 381 | - Provenance of every analysis artifact |
| 382 | |
| 383 | ### Examining source status |
| 384 | |
| 385 | After a run completes, check the publish manifest to see which sources were reused or refreshed: |
| 386 | |
| 387 | ```bash |
| 388 | # Find the latest manifest for week 2026-W21 |
| 389 | find data/candidates/2026-W21 -name publish-manifest.json | sort -V | tail -1 | xargs cat | jq '.source_artifacts' |
| 390 | ``` |
| 391 | |
| 392 | Output shows: |
| 393 | ```json |
| 394 | { |
| 395 | "source_artifacts": [ |
| 396 | { |
| 397 | "role": "raw_github", |
| 398 | "path": "data/candidates/2026-W21/github-crawl.json", |
| 399 | "exists": true, |
| 400 | "size_bytes": 45823, |
| 401 | "sha256": "686085ace216e10d36837a91471e28a334b2fc3d93cc1085b8d5d0e7616891bf", |
| 402 | "same_day_reuse": { |
| 403 | "status": "reused", |
| 404 | "source": "default" |
| 405 | }, |
| 406 | "freshness": { |
| 407 | "status": "fresh", |
| 408 | "reasons": [] |
| 409 | }, |
| 410 | "provenance": { |
| 411 | "generated_at": "2026-05-20T10:15:00Z", |
| 412 | "sha256": "686085ace216e10d36837a91471e28a334b2fc3d93cc1085b8d5d0e7616891bf" |
| 413 | } |
| 414 | }, |
| 415 | { |
| 416 | "role": "external_news", |
| 417 | "path": "data/candidates/2026-W21/news-articles.json", |
| 418 | "exists": true, |
| 419 | "size_bytes": 28941, |
| 420 | "sha256": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1", |
| 421 | "same_day_reuse": { |
| 422 | "status": "not_reused", |
| 423 | "source": "refresh_policy" |
| 424 | }, |
| 425 | "freshness": { |
| 426 | "status": "stale", |
| 427 | "reasons": ["source_refresh_policy=refresh-missing-stale"] |
| 428 | }, |
| 429 | "provenance": { |
| 430 | "generated_at": "2026-05-21T08:30:00Z", |
| 431 | "sha256": "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1" |
| 432 | } |
| 433 | } |
| 434 | ] |
| 435 | } |
| 436 | ``` |
| 437 | |
| 438 | ### Safe rerun scenario |
| 439 | |
| 440 | **Scenario:** You rerun Monday's analysis on Tuesday morning (same week) to fix a quality gate failure. |
| 441 | |
| 442 | **Expected behavior with `source_refresh_policy=reuse-same-day` (default):** |
| 443 | 1. Monday's successful GitHub crawl is reused (1 API call saved) |
| 444 | 2. Any failed or missing sources from Monday are refreshed |
| 445 | 3. Analysis gates run on fresh analysis only |
| 446 | 4. If gates pass, publish replaces Monday's article |
| 447 | 5. If gates fail, the publish manifest blocks the promotion and preserves Monday's good article |
| 448 | |
| 449 | **This is safe because:** |
| 450 | - Only Monday's *successful* artifacts are reused |
| 451 | - Any source that failed on Monday is fetched fresh |
| 452 | - The manifest explicitly records what was reused |
| 453 | - Quality gates prevent bad analysis from being published |
| 454 | - Good prior analysis is preserved if the retry fails |
| 455 | |
| 456 | ### When to use each policy |
| 457 | |
| 458 | **Use `reuse-same-day` (default):** |
| 459 | - Standard reruns within the same day |
| 460 | - Debugging analysis issues |
| 461 | - Retrying quality gates after minor fixes |
| 462 | |
| 463 | **Use `refresh-missing-stale`:** |
| 464 | - Some sources failed and you've fixed the crawler |
| 465 | - You want to update stale sources without fully refreshing |
| 466 | |
| 467 | **Use `force-refresh`:** |
| 468 | - You suspect source data is corrupted or needs manual validation |
| 469 | - You're testing source updates |
| 470 | - Policy: Always use explicit intent for full refresh |
| 471 | |
| 472 | ## Safe Restore from Backup |
| 473 | |
| 474 | ### Understanding backups |
| 475 | |
| 476 | Before any weekly article or analysis is replaced in the `publish` branch, an immutable backup is created at: |
| 477 | |
| 478 | ``` |
| 479 | data/backups/YYYY-WNN/RUN_ID/[analysis|content]/manifest.json |
| 480 | ``` |
| 481 | |
| 482 | Backups include: |
| 483 | - The exact prior content being replaced (e.g., `content/weekly/2026/W21.md`) |
| 484 | - The prior analysis file (e.g., `data/analyzed/2026-W21-summary.md`) |
| 485 | - SHA256 checksums of all backed-up files |
| 486 | - Timestamp and run context |
| 487 | |
| 488 | ### When backups are created |
| 489 | |
| 490 | A backup is automatically created when: |
| 491 | 1. A new analysis is about to replace a prior week's analysis, OR |
| 492 | 2. A new content page is about to replace a prior week's HTML page |
| 493 | |
| 494 | Backups are immutable—they cannot be modified or deleted by subsequent runs. |
| 495 | |
| 496 | ### Viewing available backups |
| 497 | |
| 498 | ```bash |
| 499 | # List all available backups for week 2026-W21 |
| 500 | find data/backups/2026-W21 -name manifest.json | sort -V |
| 501 | |
| 502 | # Inspect a backup manifest |
| 503 | cat data/backups/2026-W21/RUN_ID/content/manifest.json | jq . |
| 504 | ``` |
| 505 | |
| 506 | Backup manifest shows: |
| 507 | ```json |
| 508 | { |
| 509 | "schema_version": "publish_backup_v1", |
| 510 | "week": "2026-W21", |
| 511 | "run_id": 12345678, |
| 512 | "timestamp": "2026-05-20T10:15:00Z", |
| 513 | "backed_up_artifacts": [ |
| 514 | { |
| 515 | "path": "data/analyzed/2026-W21-summary.md", |
| 516 | "sha256": "abc123...", |
| 517 | "exists_before_replacement": true |
| 518 | }, |
| 519 | { |
| 520 | "path": "content/weekly/2026/W21.md", |
| 521 | "sha256": "def456...", |
| 522 | "exists_before_replacement": true |
| 523 | } |
| 524 | ] |
| 525 | } |
| 526 | ``` |
| 527 | |
| 528 | ### Restore a prior week from backup |
| 529 | |
| 530 | To restore a prior week (e.g., restore 2026-W21 to a known-good state): |
| 531 | |
| 532 | 1. **Identify the backup manifest** you want to restore: |
| 533 | ```bash |
| 534 | # List backups for week 2026-W21, sorted by timestamp |
| 535 | find data/backups/2026-W21 -name manifest.json | sort -V |
| 536 | ``` |
| 537 | |
| 538 | 2. **Trigger the restore workflow:** |
| 539 | ```bash |
| 540 | gh workflow run restore-publish-backup.yml \ |
| 541 | -R YOUR_USERNAME/SquadScope \ |
| 542 | -f "backup_manifest=data/backups/2026-W21/RUN_ID/content/manifest.json" |
| 543 | ``` |
| 544 | |
| 545 | Or through the UI: |
| 546 | - Go to **Actions → Restore publish backup** |
| 547 | - Click **Run workflow** |
| 548 | - Paste the backup manifest path (e.g., `data/backups/2026-W21/12345678/content/manifest.json`) |
| 549 | - Click **Run workflow** |
| 550 | |
| 551 | 3. **Restore will:** |
| 552 | - Check out the `publish` branch |
| 553 | - Validate the backup manifest integrity |
| 554 | - Restore all backed-up files to their pre-replacement state |
| 555 | - Commit with message: `restore: publish backup {manifest_path}` |
| 556 | - Force-push to `publish` with lease safety guards |
| 557 | |
| 558 | 4. **Monitor the restore:** |
| 559 | ```bash |
| 560 | gh run view --log -R YOUR_USERNAME/SquadScope |
| 561 | ``` |
| 562 | |
| 563 | ### Restore operation guarantees |
| 564 | |
| 565 | - **Immutable:** Backup manifests are never modified after creation |
| 566 | - **Atomic:** Restore applies all backed-up files or fails with no partial changes |
| 567 | - **Lease-guarded:** Force-push uses `--force-with-lease` to detect concurrent modifications |
| 568 | - **Audited:** Restore commit message includes the backup manifest path for traceability |
| 569 | - **Non-destructive:** Restoring does not delete new backups created since the restore date |
| 570 | |
| 571 | **After a restore:** |
| 572 | - The `publish` branch is reverted to the state before that run |
| 573 | - Previous good analysis remains published |
| 574 | - The restore itself appears in git history for audit trail |
| 575 | |
| 576 | ## No-AI Fallback Policy |
| 577 | |
| 578 | ### Why no-AI is not a replacement strategy |
| 579 | |
| 580 | SquadScope includes a no-AI fallback analysis script (`scripts/analyze_fallback.py`) that can generate a basic summary using heuristics when Copilot is unavailable. However, **no-AI output is explicitly not a replacement for Copilot analysis** and has specific constraints: |
| 581 | |
| 582 | **No-AI fallback is used only when:** |
| 583 | 1. Copilot CLI fails with a non-auth error (after retries), OR |
| 584 | 2. Copilot encounters a non-recoverable error (e.g., context too large), OR |
| 585 | 3. Copilot is completely inaccessible |
| 586 | |
| 587 | **No-AI output characteristics:** |
| 588 | - Lower quality_score (typically 40–50 vs. 60+ for Copilot) |
| 589 | - Simple heuristic categorization (no editorial synthesis) |
| 590 | - May have incomplete signal/noise/gaps sections |
| 591 | - Preserved as a "rejected candidate" artifact |
| 592 | - **Does not publish without explicit operator override** |
| 593 | |
| 594 | ### Publish manifest promotion policy |
| 595 | |
| 596 | When Copilot fails and no-AI fallback is generated: |
| 597 | 1. The no-AI candidate is created and stored in `data/candidates/YYYY-WNN/RUN_ID/` |
| 598 | 2. The publish manifest records `analysis_source: "no-ai"` and `quality_validation: "failed"` |
| 599 | 3. **The promotion guard blocks publication** regardless of whether gates pass |
| 600 | 4. The prior week's good analysis remains published (safe default) |
| 601 | |
| 602 | To inspect a rejected no-AI candidate: |
| 603 | |
| 604 | ```bash |
| 605 | # List rejected candidates for week 2026-W21 |
| 606 | find data/candidates/2026-W21 -name 'candidate-no-ai-attempt-*.md' |
| 607 | |
| 608 | # Review the no-AI candidate and its gate report |
| 609 | cat data/candidates/2026-W21/RUN_ID/diagnostics/candidate-no-ai-attempt-0.md |
| 610 | cat data/candidates/2026-W21/RUN_ID/diagnostics/gate-no-ai-attempt-0.json | jq '.validation_failures' |
| 611 | ``` |
| 612 | |
| 613 | ### Copilot access failures |
| 614 | |
| 615 | If Copilot CLI fails with an authentication or access error: |
| 616 | - The workflow **fails immediately without attempting no-AI fallback** |
| 617 | - An issue is created (or updated) to notify the operator to renew `COPILOT_GH_TOKEN` |
| 618 | - The failure report is available at `data/candidates/YYYY-WNN/RUN_ID/diagnostics/copilot-cli-failure-*.json` |
| 619 | |
| 620 | This ensures that **transient Copilot issues do not silently degrade to no-AI analysis.** |
| 621 | |
| 622 | ### Copilot retries |
| 623 | |
| 624 | If Copilot produces output that doesn't pass the quality gate, the workflow automatically retries up to 3 times: |
| 625 | - Each retry includes focused diagnostics from the prior gate failure |
| 626 | - If all retries fail, no-AI fallback is attempted as a last resort |
| 627 | - Each retry and its diagnostics are recorded for audit trail |
| 628 | |
| 629 | ## Rejected Candidate Diagnostics |
| 630 | |
| 631 | When analysis fails to pass quality gates, detailed diagnostics are recorded for investigation: |
| 632 | |
| 633 | ### Candidate directory structure |
| 634 | |
| 635 | ``` |
| 636 | data/candidates/YYYY-WNN/RUN_ID/ |
| 637 | ├── YYYY-WNN-summary.md # Candidate analysis (if produced) |
| 638 | ├── YYYY-WNN-content.md # Generated HTML candidate (if produced) |
| 639 | ├── publish-manifest.json # Eligibility and provenance |
| 640 | └── diagnostics/ |
| 641 | ├── analysis-preflight.json # Deterministic input manifest, context budget, evidence inventory |
| 642 | ├── analysis-preflight.md # Preflight diagnostic report |
| 643 | ├── copilot-cli-attempt-N.log # Raw Copilot CLI stderr/stdout |
| 644 | ├── gate-copilot-cli-attempt-N.json # Quality gate failure details |
| 645 | ├── candidate-copilot-cli-attempt-N.md # Candidate snapshot |
| 646 | ├── candidate-no-ai-attempt-0.md # No-AI fallback (if used) |
| 647 | └── gate-no-ai-attempt-0.json # No-AI gate report |
| 648 | ``` |
| 649 | |
| 650 | ### Examining a failed gate report |
| 651 | |
| 652 | ```bash |
| 653 | # View gate failure for attempt 0 |
| 654 | cat data/candidates/2026-W21/RUN_ID/diagnostics/gate-copilot-cli-attempt-0.json | jq '{ |
| 655 | passed: .passed, |
| 656 | gates: .gates, |
| 657 | errors_before_repair: .errors_before_repair, |
| 658 | repair_actions: .repair_actions, |
| 659 | failure_class: .failure_class, |
| 660 | failure_summary: .failure_summary |
| 661 | }' |
| 662 | ``` |
| 663 | |
| 664 | Gate report output: |
| 665 | ```json |
| 666 | { |
| 667 | "passed": false, |
| 668 | "gates": { |
| 669 | "structural_schema": { |
| 670 | "passed": false, |
| 671 | "errors": [ |
| 672 | "Signal section is empty or malformed", |
| 673 | "Signal section must contain at least 3 significant claims" |
| 674 | ] |
| 675 | }, |
| 676 | "editorial_quality": { |
| 677 | "passed": false, |
| 678 | "errors": [ |
| 679 | "Noise section has fewer than 3 spurious claims" |
| 680 | ] |
| 681 | }, |
| 682 | "ai_provenance": { |
| 683 | "passed": true, |
| 684 | "errors": [] |
| 685 | }, |
| 686 | "evidence_citation": { |
| 687 | "passed": true, |
| 688 | "errors": [] |
| 689 | } |
| 690 | }, |
| 691 | "errors_before_repair": [ |
| 692 | "Signal section is empty or malformed", |
| 693 | "Signal section must contain at least 3 significant claims", |
| 694 | "Noise section has fewer than 3 spurious claims" |
| 695 | ], |
| 696 | "repair_actions": [ |
| 697 | "Expanded Signal section with 3 significant claims from trending repositories", |
| 698 | "Added 3 spurious/false claims to Noise section" |
| 699 | ], |
| 700 | "errors_after_repair": [], |
| 701 | "failure_class": "passed", |
| 702 | "failure_summary": { |
| 703 | "failure_class": "passed", |
| 704 | "failure_categories": [], |
| 705 | "error_count": 0, |
| 706 | "retryable": false |
| 707 | } |
| 708 | } |
| 709 | ``` |
| 710 | |
| 711 | `analysis-preflight.json` is the source of truth for prompt inputs. It includes byte/token/checksum metadata for each prompt component and evidence inventories (`raw_new_repos`, `raw_trending_repos`, `prompt_new_repos`, `prompt_trending_repos`) so operators can verify whether a final repo link was present in current crawl evidence or only in compacted prompt context. |
| 712 | |
| 713 | ### Quality gate specifics |
| 714 | |
| 715 | The publish manifest records: |
| 716 | - `quality_score`: 0–100, where 60+ is publishable |
| 717 | - `quality_source`: "copilot-cli", "no-ai", etc. |
| 718 | - `validation_status`: "passed" or "failed" |
| 719 | - `validation_failures`: Array of specific failures with repair suggestions |
| 720 | |
| 721 | Failed gates block promotion but don't prevent the candidate from being stored for audit trail. |
| 722 | |
| 723 | ## Map/Reduce Analysis Status (Dry-Run Only) |
| 724 | |
| 725 | ### Why map/reduce remains experimental |
| 726 | |
| 727 | SquadScope's analysis pipeline is currently single-pass for production use. Map/reduce analysis—dividing evidence into smaller chunks, analyzing each independently, then combining results—is **only available as a dry-run experimental feature** and cannot publish. This decision was made after evidence from live runs and is documented in `docs/PRD-matrix-crawl-map-reduce-analysis.md`. |
| 728 | |
| 729 | ### Current limitations preventing map/reduce publication |
| 730 | |
| 731 | 1. **Analysis specification mismatch:** Map/reduce mapper outputs would create intermediate artifacts not conforming to `docs/analysis-spec.md` |
| 732 | 2. **Citation preservation:** Combining mapper outputs risks losing original citations and creating false attribution |
| 733 | 3. **Claim deduplication:** Reducer must reliably dedupe claims across mappers; no production-grade deduplication exists yet |
| 734 | 4. **Token accounting:** Final combined analysis may exceed token budgets; mechanism for bounded reduction is unproven |
| 735 | 5. **Quality gate compliance:** Existing gates expect single-pass analysis structure; map/reduce will need new gates |
| 736 | |
| 737 | ### QA gates required before map/reduce can publish (#258) |
| 738 | |
| 739 | Before map/reduce analysis can be enabled for production promotion, all of these QA gates must pass: |
| 740 | |
| 741 | - [ ] **Mapper-reducer contract testing:** Mappers and reducers in sandboxed runs must produce deterministic, validated outputs |
| 742 | - [ ] **Citation preservation testing:** Full analysis -> map/reduce roundtrip must preserve or improve citation count/accuracy |
| 743 | - [ ] **Claim deduplication testing:** Reducer must reliably identify and merge duplicate claims across mappers |
| 744 | - [ ] **Token budget compliance:** End-to-end analysis must stay within token limits; no truncation or quality regression |
| 745 | - [ ] **Spec compliance testing:** Generated analysis must pass all existing `analysis_gate.py` checks without modification |
| 746 | - [ ] **Human editorial review:** Blind A/B comparison of single-pass vs. map/reduce outputs from 4+ weeks of real data |
| 747 | - [ ] **Fallback behavior:** Ensure Copilot retries and no-AI fallback work correctly with map/reduce logic |
| 748 | |
| 749 | ### Testing map/reduce in dry-run mode |
| 750 | |
| 751 | To test map/reduce without risk of publication: |
| 752 | |
| 753 | ```bash |
| 754 | gh workflow run crawl-and-publish.yml \ |
| 755 | -R YOUR_USERNAME/SquadScope \ |
| 756 | -f "run_mode=dry-run" |
| 757 | ``` |
| 758 | |
| 759 | Dry-run mode ensures: |
| 760 | - Analysis is generated but never promoted |
| 761 | - No HTML is published |
| 762 | - Candidates are stored for inspection |
| 763 | - You can review results before any live traffic sees them |
| 764 | |
| 765 | ### Expected timeline |
| 766 | |
| 767 | Map/reduce publication will be enabled in a future phase after: |
| 768 | - All QA gates (#258) are implemented and passing |
| 769 | - Human review confirms output quality meets or exceeds single-pass |
| 770 | - Operator documentation is completed |
| 771 | - Rollback procedures are tested |
| 772 | |
| 773 | ## Monitoring the Cron Schedule |
| 774 | |
| 775 | ### View recent runs |
| 776 | |
| 777 | ```bash |
| 778 | gh run list -R YOUR_USERNAME/SquadScope --workflow crawl-and-publish.yml --limit 10 |
| 779 | ``` |
| 780 | |
| 781 | ### Check a specific run |
| 782 | |
| 783 | ```bash |
| 784 | gh run view RUN_ID -R YOUR_USERNAME/SquadScope |
| 785 | ``` |
| 786 | |
| 787 | ### Stream live logs |
| 788 | |
| 789 | ```bash |
| 790 | gh run view RUN_ID -R YOUR_USERNAME/SquadScope --log |
| 791 | ``` |
| 792 | |
| 793 | ### Monitor in the UI |
| 794 | |
| 795 | Navigate to **Actions → Crawl and Publish** in your repo. Green checkmarks = success, red X = failure. |
| 796 | |
| 797 | ## Troubleshooting Common Failures |
| 798 | |
| 799 | ### ❌ Copilot auth failure |
| 800 | |
| 801 | **Error message:** `COPILOT_GITHUB_TOKEN not configured` or `401 Unauthorized` |
| 802 | |
| 803 | **Cause:** The `COPILOT_GH_TOKEN` secret is missing, expired, or has insufficient permissions. |
| 804 | |
| 805 | **Fix:** |
| 806 | 1. Verify the secret exists: `gh secret list -R YOUR_USERNAME/SquadScope` |
| 807 | 2. Regenerate the PAT if expired (Settings → Developer settings → Personal access tokens) |
| 808 | 3. Confirm it has **Account → Copilot Requests** permission |
| 809 | 4. Update the secret: `gh secret set COPILOT_GH_TOKEN --body YOUR_NEW_PAT -R YOUR_USERNAME/SquadScope` |
| 810 | 5. Re-run the workflow |
| 811 | |
| 812 | **Fallback:** There is no GitHub Models/OpenAI fallback for weekly analysis. Token/auth failures fail immediately and create or update an issue assigned to `@jmservera` to renew `COPILOT_GH_TOKEN`. |
| 813 | |
| 814 | ### ❌ GitHub API rate limits exceeded |
| 815 | |
| 816 | **Error message:** `API rate limit exceeded (60/60)` or `secondary rate limit` |
| 817 | |
| 818 | **Cause:** Too many API calls in a short time window. GitHub's search endpoint is particularly strict. |
| 819 | |
| 820 | **Fix:** |
| 821 | 1. The crawler has built-in backoff logic. Wait 15 minutes and retry. |
| 822 | 2. For immediate relief, configure a `GITHUB_TOKEN` (built-in) in the workflow to increase rate limits from 60/hour to 5,000/hour. |
| 823 | 3. If using a fine-grained PAT, ensure it has minimal required permissions (reduces quota consumption). |
| 824 | |
| 825 | **Avoid:** Running crawl jobs back-to-back in rapid succession. |
| 826 | |
| 827 | ### ❌ Copilot quota exhausted |
| 828 | |
| 829 | **Error message:** `Quota exhausted` or `429 Too Many Requests` |
| 830 | |
| 831 | **Cause:** You've used up your Copilot API requests for the billing cycle (typically thousands per month for Copilot Pro). |
| 832 | |
| 833 | **Fix:** |
| 834 | 1. Check your Copilot usage: https://github.com/settings/copilot |
| 835 | 2. If you hit the limit, wait for the next billing cycle or upgrade your plan |
| 836 | 3. The workflow retries transient Copilot failures, but no GitHub Models/OpenAI fallback is configured for weekly analysis. |
| 837 | |
| 838 | **Mitigation:** Copilot Pro includes generous quota. For automated pipelines, consider Copilot Team (more quota, better for organizations). |
| 839 | |
| 840 | ### ❌ Hugo build fails |
| 841 | |
| 842 | **Error message:** `error: /content/weekly/... failed to parse date` or `theme not found` |
| 843 | |
| 844 | **Cause:** |
| 845 | - Hugo version mismatch (too old) |
| 846 | - Theme submodules not initialized |
| 847 | - Corrupted frontmatter in generated content |
| 848 | |
| 849 | **Fix:** |
| 850 | 1. Check Hugo version: `hugo version` (must be v0.146.0+) |
| 851 | 2. Verify submodules: `git submodule update --init --recursive` |
| 852 | 3. Check generated markdown in `content/weekly/` for valid YAML frontmatter |
| 853 | 4. Run locally: `hugo server` to see detailed error messages |
| 854 | |
| 855 | ### ❌ GitHub Pages doesn't update |
| 856 | |
| 857 | **Error message:** Site shows old content or doesn't deploy |
| 858 | |
| 859 | **Cause:** |
| 860 | - Pages source not set to "GitHub Actions" |
| 861 | - Deployment workflow didn't complete successfully |
| 862 | - Cache not cleared |
| 863 | |
| 864 | **Fix:** |
| 865 | 1. Verify Pages source: Repo **Settings → Pages → Source = "GitHub Actions"** |
| 866 | 2. Check the **Actions** tab for failed deploy jobs |
| 867 | 3. Clear browser cache (`Ctrl+Shift+Delete`) and refresh |
| 868 | 4. Force redeploy: Push a dummy commit to `main` or manually trigger `deploy-site.yml` workflow |
| 869 | |
| 870 | ### ❌ Quality gate blocks publish |
| 871 | |
| 872 | **Error message:** `quality_score < 60` or `Missing required sections (Signal, Noise, Gaps)` |
| 873 | |
| 874 | **Cause:** The AI analysis didn't meet quality thresholds. This is a **feature, not a bug** — the gate prevents low-quality content from publishing. |
| 875 | |
| 876 | **Fix:** |
| 877 | 1. Check the analysis file: `cat data/analyzed/YYYY-WNN-summary.md` |
| 878 | 2. Review the quality_score in the YAML frontmatter |
| 879 | 3. If analysis is genuinely poor, this may indicate a problem with the raw data (crawler issue) or Copilot API issues |
| 880 | 4. **Do NOT bypass the gate.** Instead: |
| 881 | - Investigate why analysis quality was low |
| 882 | - Retry the workflow (Copilot may have had transient issues) |
| 883 | - If consistent, open an issue with the analyzed output for review |
| 884 | |
| 885 | ### ❌ Reskill workflow doesn't trigger |
| 886 | |
| 887 | **Error message:** Reskill job skipped in the logs |
| 888 | |
| 889 | **Cause:** Reskill only runs every 5th pipeline execution. If you've only run 1-4 times, it won't trigger yet. |
| 890 | |
| 891 | **Fix:** |
| 892 | 1. Run the workflow 4 more times to trigger a reskill (or wait for 4 more weeks) |
| 893 | 2. Check `.squad/run-counter.txt` to see how many runs have executed |
| 894 | 3. To manually test reskill logic, run: `python3 scripts/reskill.py --current-week YYYY-WNN --current-datetime 2026-05-18T16:00:00Z` (or wait for automatic 5th run trigger in `crawl-and-publish.yml`) |
| 895 | |
| 896 | ## The Reskill Cycle |
| 897 | |
| 898 | Every 5th pipeline run, SquadScope enters a "reskill" phase where it reviews its own behavior and proposes improvements. |
| 899 | |
| 900 | ### What happens |
| 901 | |
| 902 | 1. **Copilot reads:** |
| 903 | - `.squad/agents/*/history.md` — Learnings from all agents |
| 904 | - `.squad/decisions.md` — Architectural decisions made |
| 905 | - `data/analyzed/` — Recent analysis outputs (quality trend) |
| 906 | - `.squad/run-counter.txt` — Run history |
| 907 | |
| 908 | 2. **Analysis:** |
| 909 | - What patterns are working well? |
| 910 | - What should change in prompts or logic? |
| 911 | - Are there quality drift signals? |
| 912 | |
| 913 | 3. **Output:** |
| 914 | - Recommendations written to `.squad/reskill/YYYY-WNN.md` |
| 915 | - Optional: PR proposed with prompt refinements (not auto-merged) |
| 916 | |
| 917 | ### What to expect |
| 918 | |
| 919 | - **First reskill (run 5):** Observations and baseline recommendations |
| 920 | - **Subsequent reskills:** Trend analysis ("quality is improving" vs. "signal/noise classification drifting") |
| 921 | - **Prompts may be refined:** If reskill suggests prompt changes, review the PR and decide whether to merge |
| 922 | |
| 923 | ### No manual action required |
| 924 | |
| 925 | Reskill runs automatically. You can review the outputs in `.squad/reskill/` directory, but the system works fine without human intervention. |
| 926 | |
| 927 | ## Updating Configuration |
| 928 | |
| 929 | ### Change the crawl schedule |
| 930 | |
| 931 | Edit `.github/workflows/crawl-and-publish.yml`: |
| 932 | |
| 933 | ```yaml |
| 934 | on: |
| 935 | schedule: |
| 936 | - cron: '53 11 * * 0' # Sunday 11:53 UTC; still best-effort on GitHub-hosted runners |
| 937 | ``` |
| 938 | |
| 939 | Times are in UTC. Use crontab.guru to generate your schedule, but assume GitHub-hosted `schedule` can start late and use the `workflow_dispatch` mitigation ladder above if exact timing matters. |
| 940 | |
| 941 | ### Change the reskill interval |
| 942 | |
| 943 | Edit `.github/workflows/crawl-and-publish.yml` or `.squad/run-counter.txt`: |
| 944 | |
| 945 | ```yaml |
| 946 | # In workflow, modify the modulo check: |
| 947 | if [ $((COUNTER % 5)) -eq 0 ]; then # Change 5 to desired interval |
| 948 | ``` |
| 949 | |
| 950 | ### Update the analyzer prompts |
| 951 | |
| 952 | Prompts live in `prompts/` directory. Edit and commit changes. They take effect on the next workflow run. |
| 953 | |
| 954 | ### Customize quality gate thresholds |
| 955 | |
| 956 | Edit `scripts/analysis_gate.py`: |
| 957 | |
| 958 | ```python |
| 959 | MIN_QUALITY_SCORE = 60 # Change threshold here |
| 960 | MIN_WORD_COUNT = 200 |
| 961 | ``` |
| 962 | |
| 963 | ## Monitoring Site Health |
| 964 | |
| 965 | ### Check RSS feed generation |
| 966 | |
| 967 | ```bash |
| 968 | curl https://YOUR_SITE/index.xml |
| 969 | ``` |
| 970 | |
| 971 | Should return valid XML with recent article entries. |
| 972 | |
| 973 | ### Verify Hugo build output |
| 974 | |
| 975 | ```bash |
| 976 | # After running locally: |
| 977 | ls public/ |
| 978 | hugo --minify && wc -l public/*.html |
| 979 | ``` |
| 980 | |
| 981 | Should show HTML files for each content page. |
| 982 | |
| 983 | ### Monitor GitHub API usage |
| 984 | |
| 985 | Check your GitHub API rate limits: |
| 986 | |
| 987 | ```bash |
| 988 | gh api rate_limit |
| 989 | ``` |
| 990 | |
| 991 | Output shows `limit`, `remaining`, and `reset` time. If you're consistently hitting limits, consider: |
| 992 | - Running less frequently |
| 993 | - Optimizing the crawler queries |
| 994 | - Using a GitHub App instead of PAT (higher limits) |
| 995 | |
| 996 | ## Disaster Recovery |
| 997 | |
| 998 | ### Revert a bad analysis |
| 999 | |
| 1000 | If analysis was published but is clearly incorrect: |
| 1001 | |
| 1002 | 1. Manually edit `data/analyzed/YYYY-WNN-summary.md` to correct it |
| 1003 | 2. Commit and push |
| 1004 | 3. Re-run `generate` and `deploy` stages (or full pipeline) |
| 1005 | 4. The week's content will be regenerated with corrected data |
| 1006 | |
| 1007 | ### Reset the run counter |
| 1008 | |
| 1009 | If you want reskill to trigger immediately: |
| 1010 | |
| 1011 | ```bash |
| 1012 | echo "5" > .squad/run-counter.txt |
| 1013 | git add .squad/run-counter.txt |
| 1014 | git commit -m "Reset run counter to trigger reskill" |
| 1015 | git push |
| 1016 | ``` |
| 1017 | |
| 1018 | Next run will trigger reskill logic. |
| 1019 | |
| 1020 | ### Restore from a previous week |
| 1021 | |
| 1022 | Content is immutable by design. If you need to restore: |
| 1023 | |
| 1024 | 1. Check git history: `git log --oneline -- data/analyzed/YYYY-WNN-summary.md` |
| 1025 | 2. Revert if necessary: `git revert COMMIT_HASH` |
| 1026 | 3. Regenerate and deploy |
| 1027 | |
| 1028 | ## Manual Pipeline Tools |
| 1029 | |
| 1030 | These scripts are not part of the automated weekly pipeline but are available |
| 1031 | for manual operator use. |
| 1032 | |
| 1033 | ### Hindsight Validation |
| 1034 | |
| 1035 | Validates predictions from N weeks ago against actual outcomes. Run periodically |
| 1036 | to track prediction accuracy: |
| 1037 | |
| 1038 | ```bash |
| 1039 | python scripts/hindsight_validation.py --topic ai-ml --weeks-ago 4 |
| 1040 | ``` |
| 1041 | |
| 1042 | Output: a scorecard summary comparing predicted outcomes with real data. |
| 1043 | |
| 1044 | ### Topic Learning Initialization |
| 1045 | |
| 1046 | Bootstraps per-topic learning state directories with seeded wisdom. Use when |
| 1047 | adding a new topic channel or resetting learning state: |
| 1048 | |
| 1049 | ```bash |
| 1050 | python scripts/init_topic_learning.py --topic my-new-topic |
| 1051 | ``` |
| 1052 | |
| 1053 | ### Momentum Tracker |
| 1054 | |
| 1055 | Checks press-correlated repos at week +2/+4 and classifies growth as |
| 1056 | "sustained" or "faded". Already referenced in site methodology content: |
| 1057 | |
| 1058 | ```bash |
| 1059 | python scripts/momentum_tracker.py --topic ai-ml --week 2026-W21 --lag 4 |
| 1060 | ``` |
| 1061 | |
| 1062 | ### Budget Alerts |
| 1063 | |
| 1064 | Evaluates run cost and monthly cumulative spend against thresholds. Wired into |
| 1065 | the `crawl-and-publish` workflow automatically, but can also be run manually: |
| 1066 | |
| 1067 | ```bash |
| 1068 | python scripts/budget_alerts.py --metrics data/metrics/token-usage.jsonl |
| 1069 | ``` |
| 1070 | |
| 1071 | ## Getting Help |
| 1072 | |
| 1073 | - **Logs:** Check **Actions** tab in GitHub UI for detailed workflow logs |
| 1074 | - **Common issues:** See troubleshooting section above |
| 1075 | - **Architecture questions:** See `.squad/decisions.md` for design rationale |
| 1076 | - **PRD:** See `docs/PRD.md` for feature requirements and future phases |
| 1077 | - **Issues:** Open an issue in the repository with logs and error messages |
| 1078 | |
| 1079 | ## What's Next? |
| 1080 | |
| 1081 | Once SquadScope is running smoothly: |
| 1082 | |
| 1083 | 1. **Monitor quality:** Review weekly analyses for patterns and trends |
| 1084 | 2. **Iterate prompts:** Based on reskill recommendations, refine analysis quality |
| 1085 | 3. **Extend sources:** Add additional data sources (HackerNews, Reddit, etc.) via MCP tools |
| 1086 | 4. **Add notifications:** Configure GitHub Releases, Discord/Slack webhooks, or custom webhook integrations |
| 1087 | 5. **Topic channels:** Explore multi-topic feature (see `docs/PRD-topic-channels.md`) |
| 1088 | |
| 1089 | SquadScope is designed to improve itself. Trust the system, monitor the trends, and enjoy curated tech news delivered every week. |