| 1 | # SquadScope / Claracle Architecture |
| 2 | |
| 3 | ## Overview |
| 4 | |
| 5 | SquadScope, publicly branded as **Claracle**, is an AI-powered GitHub trend observatory. It crawls GitHub and selected external sources each week, uses Copilot CLI to analyze signal versus noise, generates Hugo content, deploys a public static site at **www.claracle.com**, periodically reskills its AI squad state, and hands published articles to the Podcaster system for episode generation. |
| 6 | |
| 7 | ## Tech Stack |
| 8 | |
| 9 | - **Hugo** static site generator with the **PaperMod** theme |
| 10 | - **Pagefind** for client-side search |
| 11 | - **Python** scripts for crawl, analysis prep, content generation, publishing, and handoff automation |
| 12 | - **GitHub Actions** for orchestration |
| 13 | - **GitHub Pages** for hosting |
| 14 | - **GitHub Copilot CLI** for AI analysis (two-step: `weekly-synthesis` → `weekly-analysis`, both gpt-5.5) and reskill work |
| 15 | - **No OpenAI / GitHub Models fallback** for weekly analysis |
| 16 | |
| 17 | ## Directory Structure |
| 18 | |
| 19 | - `scripts/` — Python pipeline automation for crawl, analyze, generate, publish, reskill, and handoff |
| 20 | - `content/` — Hugo content, including `weekly/`, `monthly/`, and `yearly/` |
| 21 | - `data/` — pipeline artifacts, including `raw/`, `analyzed/`, `snapshots/`, `metrics/`, and `cache/` |
| 22 | - `config/` — cross-workflow shared config such as `podcast.json` and `external_news_sources.json` |
| 23 | - `layouts/`, `assets/`, `static/` — Hugo theme and site customizations |
| 24 | - `infra/` — reserved for Infrastructure as Code if/when introduced; not present today |
| 25 | - `tests/` — unit and integration tests |
| 26 | - `docs/` — operator guides, pipeline specs, design notes, and audits |
| 27 | - `.squad/` — AI team state, history, reskill outputs, and conventions |
| 28 | |
| 29 | ## Data Flow |
| 30 | |
| 31 | 1. **Crawl** |
| 32 | GitHub API data is written to `data/raw/YYYY-WNN.json`, and external RSS/news enrichment is written to `data/raw/YYYY-WNN-external-news.json`. |
| 33 | 2. **Analyze** |
| 34 | Two-step Copilot CLI analysis using dedicated agents (both gpt-5.5): |
| 35 | - *Weekly Synthesis* agent produces a compact industry narrative from press/historical context |
| 36 | - *Weekly Analysis* agent consumes the synthesis plus raw data to generate `data/analyzed/YYYY-WNN-summary.md` |
| 37 | - Additional outputs: `data/analyzed/YYYY-WNN-correlations.json`, `data/analyzed/YYYY-WNN-press-context.md` |
| 38 | 3. **Generate** |
| 39 | The analyzed summary is transformed into Hugo content at `content/weekly/YYYY/WNN.md`. |
| 40 | 4. **Deploy** |
| 41 | Hugo builds the site and GitHub Pages serves the generated output. |
| 42 | 5. **Reskill** |
| 43 | Every 5th successful crawl run writes retrospective squad learning to `.squad/reskill/YYYY-WNN.md`. |
| 44 | 6. **Podcaster Handoff** |
| 45 | After publish, the article and podcast config are POSTed to the Podcaster API. |
| 46 | |
| 47 | ## Shared Interfaces (with SquadScope-Podcaster) |
| 48 | |
| 49 | - `config/podcast.json` is the source of truth for podcast editorial direction |
| 50 | - `podcast_config`: hosts, voices, styles, show name, and URL |
| 51 | - `script_directions`: `opening_cues`, `closing_cues`, and `episode_style` |
| 52 | - `music_mix`: track, voice guardrail, and intro/outro mix parameters |
| 53 | - Handoff payload fields: |
| 54 | - `week` |
| 55 | - `article_url` |
| 56 | - `article_content` |
| 57 | - `article_title` |
| 58 | - `article_sha256` |
| 59 | - `source_artifacts` |
| 60 | - `podcast_config` |
| 61 | - `script_directions` |
| 62 | - `breaking_news` — optional last-moment news text to include in the episode (omitted when not provided) |
| 63 | - Transport: HTTP `POST` with `x-podcaster-api-key` header |
| 64 | - Handoff implementation: `scripts/podcaster_handoff.py` |
| 65 | |
| 66 | ## Environment Variables / Secrets |
| 67 | |
| 68 | - `PODCASTER_ENDPOINT` — Podcaster API URL |
| 69 | - `PODCASTER_API_KEY` — Podcaster authentication key |
| 70 | - `GITHUB_TOKEN` — GitHub crawl and workflow access |
| 71 | - Copilot token/permission via GitHub Actions for Copilot CLI analysis |
| 72 | |
| 73 | ## Key Commands |
| 74 | |
| 75 | - Weekly pipeline: `.github/workflows/crawl-and-publish.yml` |
| 76 | - Local development: `hugo server` |
| 77 | - Build: `hugo --minify` |
| 78 | - Crawl: `python3 scripts/crawl.py --as-of YYYY-MM-DD` |
| 79 | - Generate: `python3 scripts/generate_content.py` |
| 80 | - Handoff: `python3 scripts/podcaster_handoff.py` |