docs: add architecture.md, AGENTS.md, and copilot-instructions for AI-friendly repo (#449)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Juan Manuel Servera committed
Jun 13, 2026 at 16:02 UTC
85ab286f28fd2d9a1104e376c2375d7d7214ada2
4 files changed
+166
.github/copilot-instructions.md
new
+36
@@ -0,0 +1,36 @@
1
+# Copilot Instructions for SquadScope
2
+
3
+This repository uses the **Squad agent** as the default for all AI-assisted work.
4
+
5
+## Default Agent
6
+
7
+Always use `--agent squad` when running Copilot CLI on this repository.
8
+
9
+## Repository Context
10
+
11
+- **Project:** SquadScope (public brand: Claracle) — AI-powered GitHub trend analysis
12
+- **Architecture:** See `architecture.md` in repo root
13
+- **Squad team:** See `.squad/team.md` for current roster
14
+
15
+## Key Conventions
16
+
17
+- All pipeline scripts are in `scripts/` (Python)
18
+- Content is Hugo markdown in `content/`
19
+- Config shared with Podcaster lives in `config/podcast.json`
20
+- Changes to `config/podcast.json` MUST be coordinated with SquadScope-Podcaster repo
21
+- Never commit secrets; use GitHub environment secrets
22
+- PRs required for `main` branch (branch protection enabled)
23
+- CI must be correct, not just green — verify rendered output for site changes
24
+
25
+## Cross-Repo Impact
26
+
27
+Changes to these files affect the Podcaster repo:
28
+
29
+- `config/podcast.json` — Podcaster reads this config for episode generation
30
+- `scripts/podcaster_handoff.py` — defines the handoff payload contract
31
+
32
+## Testing
33
+
34
+- Run `pytest tests/` for unit tests
35
+- Hugo build: `hugo --minify` must succeed
36
+- Handoff smoke: `.github/workflows/podcaster-handoff-smoke.yml`
AGENTS.md
new
+28
@@ -0,0 +1,28 @@
1
+# AI Agents
2
+
3
+This repository is managed by an AI Squad team. The default agent for all work is **Squad**.
4
+
5
+## Usage
6
+
7
+```bash
8
+# Default: use Squad agent for all tasks
9
+copilot --agent squad -p "your task here" --allow-all-tools
10
+
11
+# For automated issue work (Ralph loop)
12
+squad triage --execute --interval 20 --copilot-flags "--allow-all-tools"
13
+```
14
+
15
+## Team Roster
16
+
17
+See `.squad/team.md` for the full team composition. Key members:
18
+
19
+- **Leela** — Lead (architecture, code review, triage)
20
+- **Bender** — Crawler (data pipeline, GitHub API)
21
+- **Farnsworth** — Analyst (content generation, AI analysis)
22
+- **Amy** — Frontend (Hugo templates, CSS, UX)
23
+- **Fry** — Tester (pytest, quality gates)
24
+
25
+## Related Repositories
26
+
27
+- [SquadScope-Podcaster](https://github.com/jmservera/SquadScope-Podcaster) — Podcast generation engine (consumes `config/podcast.json`)
28
+- [SquadScope-Coordinator](https://github.com/jmservera/SquadScope-Coordinator) — Orchestration layer
architecture.md
new
+76
@@ -0,0 +1,76 @@
1
+# SquadScope / Claracle Architecture
2
+
3
+## Overview
4
+
5
+SquadScope, publicly branded as **Claracle**, is an AI-powered GitHub trend observatory. It crawls GitHub and selected external sources each week, uses Copilot CLI to analyze signal versus noise, generates Hugo content, deploys a public static site at **www.claracle.com**, periodically reskills its AI squad state, and hands published articles to the Podcaster system for episode generation.
6
+
7
+## Tech Stack
8
+
9
+- **Hugo** static site generator with the **PaperMod** theme
10
+- **Pagefind** for client-side search
11
+- **Python** scripts for crawl, analysis prep, content generation, publishing, and handoff automation
12
+- **GitHub Actions** for orchestration
13
+- **GitHub Pages** for hosting
14
+- **GitHub Copilot CLI** for AI analysis and reskill work
15
+- **No OpenAI / GitHub Models fallback** for weekly analysis
16
+
17
+## Directory Structure
18
+
19
+- `scripts/` — Python pipeline automation for crawl, analyze, generate, publish, reskill, and handoff
20
+- `content/` — Hugo content, including `weekly/`, `monthly/`, and `yearly/`
21
+- `data/` — pipeline artifacts, including `raw/`, `analyzed/`, `snapshots/`, `metrics/`, and `cache/`
22
+- `config/` — cross-workflow shared config such as `podcast.json` and `external_news_sources.json`
23
+- `layouts/`, `assets/`, `static/` — Hugo theme and site customizations
24
+- `infra/` — reserved for Infrastructure as Code if/when introduced; not present today
25
+- `tests/` — unit and integration tests
26
+- `docs/` — operator guides, pipeline specs, design notes, and audits
27
+- `.squad/` — AI team state, history, reskill outputs, and conventions
28
+
29
+## Data Flow
30
+
31
+1. **Crawl**
32
+ GitHub API data is written to `data/raw/YYYY-WNN.json`, and external RSS/news enrichment is written to `data/raw/YYYY-WNN-external-news.json`.
33
+2. **Analyze**
34
+ Copilot CLI consumes raw artifacts and produces `data/analyzed/YYYY-WNN-summary.md`.
35
+3. **Generate**
36
+ The analyzed summary is transformed into Hugo content at `content/weekly/YYYY/WNN.md`.
37
+4. **Deploy**
38
+ Hugo builds the site and GitHub Pages serves the generated output.
39
+5. **Reskill**
40
+ Every 5th successful crawl run writes retrospective squad learning to `.squad/reskill/YYYY-WNN.md`.
41
+6. **Podcaster Handoff**
42
+ After publish, the article and podcast config are POSTed to the Podcaster API.
43
+
44
+## Shared Interfaces (with SquadScope-Podcaster)
45
+
46
+- `config/podcast.json` is the source of truth for podcast editorial direction
47
+ - `podcast_config`: hosts, voices, styles, show name, and URL
48
+ - `script_directions`: `opening_cues`, `closing_cues`, and `episode_style`
49
+ - `music_mix`: track, voice guardrail, and intro/outro mix parameters
50
+- Handoff payload fields:
51
+ - `week`
52
+ - `article_url`
53
+ - `article_content`
54
+ - `article_title`
55
+ - `article_sha256`
56
+ - `source_artifacts`
57
+ - `podcast_config`
58
+ - `script_directions`
59
+- Transport: HTTP `POST` with `x-podcaster-api-key` header
60
+- Handoff implementation: `scripts/podcaster_handoff.py`
61
+
62
+## Environment Variables / Secrets
63
+
64
+- `PODCASTER_ENDPOINT` — Podcaster API URL
65
+- `PODCASTER_API_KEY` — Podcaster authentication key
66
+- `GITHUB_TOKEN` — GitHub crawl and workflow access
67
+- Copilot token/permission via GitHub Actions for Copilot CLI analysis
68
+
69
+## Key Commands
70
+
71
+- Weekly pipeline: `.github/workflows/crawl-and-publish.yml`
72
+- Local development: `hugo server`
73
+- Build: `hugo --minify`
74
+- Crawl: `python3 scripts/crawl.py --as-of YYYY-MM-DD`
75
+- Generate: `python3 scripts/generate_content.py`
76
+- Handoff: `python3 scripts/podcaster_handoff.py`
docs/cleanup-candidates.md
new
+26
@@ -0,0 +1,26 @@
1
+# Cleanup Candidates
2
+
3
+This is a conservative cleanup pass focused on `scripts/`, `docs/`, `data/`, and tracked root-level files. I checked script references against workflows and repository docs; items below are candidates, not automatic deletions.
4
+
5
+## Findings
6
+
7
+| Path | Reason | Recommendation |
8
+| --- | --- | --- |
9
+| `scripts/calibrate_hype_risk.py` | No workflow, doc, test, or script references were found. It appears to be an unwired experiment rather than an active pipeline component. | **Archive or delete** unless there is a near-term plan to wire it into the learning loop. |
10
+| `scripts/momentum_tracker.py` | No workflow or docs references were found, and it does not appear to be called by the weekly pipeline. It looks like a standalone learning-loop utility that never got integrated. | **Archive or document/wire up**; if the momentum loop is deferred, move it out of the active script surface. |
11
+| `scripts/budget_alerts.py` | Only test coverage references were found; no workflow or operator documentation calls it. Current cost tracking appears to stop at ledger generation. | **Wire into CI or archive** to reduce dormant maintenance surface. |
12
+| `scripts/hindsight_validation.py` | Only test references were found; no workflow or docs integration was found for scheduled hindsight validation. | **Archive or add an owning workflow/doc** if hindsight validation is still part of the roadmap. |
13
+| `scripts/init_topic_learning.py` | Only test references were found, with no workflow or operator-guide usage. Topic learning state currently looks pre-seeded in-repo rather than initialized through this command. | **Archive or document** as a manual bootstrap tool. |
14
+| `data/raw/2026-W21-techcrunch.json` | Legacy raw artifact format. The current pipeline writes `YYYY-WNN-external-news.json` and only keeps `*-techcrunch.json` as a compatibility fallback. | **Archive or delete** once the legacy fallback is removed. |
15
+| `data/raw/2026-W22-techcrunch.json` | Same issue as above: legacy TechCrunch-only artifact retained after the canonical external-news merge format was introduced. | **Archive or delete** once fallback support is no longer needed. |
16
+| `docs/rollout-checklist.md` | The optional verification section still says a Copilot outage should fall back to GitHub Models, but the current repo explicitly documents that there is **no** GitHub Models/OpenAI fallback for weekly analysis. | **Update** to match the current fail-closed Copilot-only pipeline. |
17
+| `docs/processed/qa-report.md` | This report says the deploy workflow does not build Pagefind and that reskill is still a placeholder. Both statements are stale relative to the current workflow. | **Archive or mark obsolete** so readers do not treat it as current operational truth. |
18
+| `docs/processed/learning-audit.md` | The audit says there is no reskill job, no `.squad/skills/` directory, and no run-counter support. Those claims no longer match the repository shape or workflow wiring. | **Archive or add a superseded banner**. |
19
+| `docs/processed/PRD-techcrunch-integration-2026-05-cross-source-correlation.md` | This PRD still describes `data/raw/YYYY-WNN-techcrunch.json` as the new primary artifact, but the implemented pipeline now uses `YYYY-WNN-external-news.json`. | **Archive or annotate as superseded** by the merged external-news design. |
20
+| `docs/growth/distribution-strategy.md` | The document still points to `https://jmservera.github.io/SquadScope/` and related RSS URLs instead of the current Claracle domain. | **Update** branding and canonical URLs to `https://www.claracle.com/`. |
21
+| `README.md` | The root README still advertises the old GitHub Pages URLs rather than the public Claracle domain. | **Update** to align external-facing docs with the live brand/domain. |
22
+
23
+## Notes
24
+
25
+- **Unused config files:** none clearly identified. `config/podcast.json` and `config/external_news_sources.json` are both referenced by workflows/scripts.
26
+- **Empty or stub files:** no actionable tracked candidates found in the requested areas. `.gitkeep` placeholders and `scripts/__init__.py` appear intentional.