| 1 | # PRD: Topic-Specific News Channels for SquadScope |
| 2 | |
| 3 | **Author:** Leela (Lead/Architect) |
| 4 | **Date:** 2026-05-18 |
| 5 | **Status:** Draft |
| 6 | **Type:** Feature PRD |
| 7 | **Depends on:** docs/analysis-spec.md, docs/pipeline-validation.md, docs/learning-audit.md, .squad/decisions.md |
| 8 | |
| 9 | --- |
| 10 | |
| 11 | ## Executive Summary |
| 12 | |
| 13 | SquadScope currently crawls all of GitHub looking for "what's interesting this week." This produces broad but shallow coverage — jack of all topics, master of none. This PRD defines how to generalize SquadScope into a **topic-channel system** where each deployment is configured for a specific domain (e.g., `ai-ml`, `rust`, `security`), producing focused, expert-level weekly digests with calibrated learning per topic. |
| 14 | |
| 15 | The approach is **feature first, not a separate platform.** v1 delivers a single configurable topic per instance (one fork/config per topic). Multi-topic single-instance is deferred to v2. |
| 16 | |
| 17 | --- |
| 18 | |
| 19 | ## Problem Statement |
| 20 | |
| 21 | ### Why topic-specific is better than general |
| 22 | |
| 23 | 1. **Signal quality degrades with breadth.** A general GitHub crawl returns repos spanning AI, systems programming, web frameworks, security tools, and student homework. No single editorial voice can meaningfully assess "is this Rust crate important?" and "is this ML paper implementation notable?" in the same breath. |
| 24 | |
| 25 | 2. **Learning cannot calibrate across domains.** The learning audit (docs/learning-audit.md) identified that wisdom must flow back into analysis (G7). But shared wisdom across domains produces uncalibrated judgments — a heuristic like "repos with >500 stars/week are always significant" is true in AI/ML but false in niche systems programming. |
| 26 | |
| 27 | 3. **GitHub topic filtering is noisy.** Simply adding `topic:rust` to a search query returns thousands of results including tutorials, homework, and abandoned projects. Topic filtering needs a multi-stage pipeline: query → score → filter → analyze. |
| 28 | |
| 29 | 4. **Readers want depth, not breadth.** A security professional subscribing to SquadScope doesn't want to scroll past 15 ML repos to find the 3 security tools that matter this week. |
| 30 | |
| 31 | 5. **Predictions need domain context.** "This will be important" means something different in each field. A prediction ledger must be per-topic to be meaningful. |
| 32 | |
| 33 | --- |
| 34 | |
| 35 | ## Goals & Non-Goals |
| 36 | |
| 37 | ### Goals |
| 38 | |
| 39 | - **G1:** Define a topic configuration format (`squadscope.topic.yml`) that controls all topic-specific behavior |
| 40 | - **G2:** Namespace all data, content, prompts, RSS, and learning state by topic |
| 41 | - **G3:** Deliver per-topic RSS feeds at `/topics/{topic}/index.xml` |
| 42 | - **G4:** Implement a scoring pipeline for GitHub results (not just keyword filters) |
| 43 | - **G5:** Isolate learning state per topic (wisdom, skills, prediction scores) |
| 44 | - **G6:** Add a prediction ledger that tracks claims vs outcomes per topic |
| 45 | - **G7:** Define topic quality criteria (minimum viable coverage thresholds) |
| 46 | - **G8:** Ship 2 example topic configs: `ai-ml` and `rust` |
| 47 | |
| 48 | ### Non-Goals |
| 49 | |
| 50 | - Multi-topic single-instance deployment (v2) |
| 51 | - Topic marketplace or discovery |
| 52 | - User-facing topic configuration UI |
| 53 | - Real-time or daily publishing cadence |
| 54 | - Cross-topic trend correlation |
| 55 | - Non-GitHub data sources per topic (future enhancement) |
| 56 | |
| 57 | --- |
| 58 | |
| 59 | ## Topic Configuration |
| 60 | |
| 61 | ### File: `squadscope.topic.yml` |
| 62 | |
| 63 | Each SquadScope instance has exactly one topic config at the repository root. Unless explicitly normalized by the scorer, `language_boost` keys should match the crawler's raw GitHub language strings exactly (for example `Python`, `Rust`, `Jupyter Notebook`). |
| 64 | |
| 65 | ```yaml |
| 66 | # squadscope.topic.yml — defines a single topic channel |
| 67 | topic: |
| 68 | id: ai-ml # URL-safe identifier |
| 69 | name: "AI & Machine Learning" # Human-readable name |
| 70 | description: "Weekly digest of significant AI/ML repositories, frameworks, and research implementations on GitHub" |
| 71 | |
| 72 | # Crawler queries — multiple queries combined for coverage |
| 73 | queries: |
| 74 | primary: |
| 75 | - "topic:machine-learning stars:>50 pushed:>{last_week}" |
| 76 | - "topic:deep-learning stars:>50 pushed:>{last_week}" |
| 77 | - "topic:artificial-intelligence stars:>30 pushed:>{last_week}" |
| 78 | - "topic:llm stars:>20 pushed:>{last_week}" |
| 79 | secondary: |
| 80 | - "topic:transformers stars:>100 pushed:>{last_week}" |
| 81 | - "topic:diffusion stars:>30 pushed:>{last_week}" |
| 82 | - "topic:rag stars:>20 pushed:>{last_week}" |
| 83 | |
| 84 | # Scoring pipeline — repos must pass these filters |
| 85 | scoring: |
| 86 | min_stars: 20 # Absolute minimum to consider |
| 87 | min_stars_gained: 10 # Minimum weekly star delta |
| 88 | max_age_days: 365 # Exclude repos older than this from "new" category |
| 89 | min_relevance_score: 40 # Repos below this score do not reach analysis |
| 90 | language_boost: # Keys match raw crawler language values |
| 91 | Python: 1.2 |
| 92 | Jupyter Notebook: 1.1 |
| 93 | Rust: 1.0 |
| 94 | topic_relevance: # Required topic overlap (at least one must match) |
| 95 | - machine-learning |
| 96 | - deep-learning |
| 97 | - artificial-intelligence |
| 98 | - neural-network |
| 99 | - llm |
| 100 | - nlp |
| 101 | - computer-vision |
| 102 | - reinforcement-learning |
| 103 | noise_topics: # Penalty topics (reduce score) |
| 104 | - tutorial |
| 105 | - course |
| 106 | - awesome-list |
| 107 | - homework |
| 108 | noise_name_patterns: # Regex patterns that reduce relevance score |
| 109 | - "^awesome-" |
| 110 | - "-tutorial$" |
| 111 | - "-course$" |
| 112 | |
| 113 | # Quality thresholds — topic must meet these to justify a channel |
| 114 | quality: |
| 115 | min_repos_per_week: 8 # Minimum repos passing filters weekly |
| 116 | max_false_positive_rate: 0.25 # Max 25% irrelevant results after scoring |
| 117 | min_signal_repos: 3 # At least 3 genuinely significant repos per issue |
| 118 | |
| 119 | # Content configuration |
| 120 | content: |
| 121 | tone: "technical, analytical" |
| 122 | audience: "ML engineers and researchers" |
| 123 | emphasis: |
| 124 | - "Novel architectures and training techniques" |
| 125 | - "Production-ready frameworks and tools" |
| 126 | - "Significant performance improvements" |
| 127 | de_emphasis: |
| 128 | - "Yet another wrapper around OpenAI API" |
| 129 | - "Awesome lists and link collections" |
| 130 | - "Course materials and tutorials" |
| 131 | |
| 132 | # Learning configuration |
| 133 | learning: |
| 134 | wisdom_file: "topics/ai-ml/wisdom.md" |
| 135 | skills_dir: "topics/ai-ml/skills/" |
| 136 | predictions_file: "topics/ai-ml/predictions.jsonl" |
| 137 | reskill_context: |
| 138 | - "What ML-specific heuristics should we update?" |
| 139 | - "Are we over/under-weighting any sub-domain?" |
| 140 | - "Which prediction categories are we worst at?" |
| 141 | ``` |
| 142 | |
| 143 | ### Second example: `rust` |
| 144 | |
| 145 | ```yaml |
| 146 | topic: |
| 147 | id: rust |
| 148 | name: "Rust Ecosystem" |
| 149 | description: "Weekly digest of significant Rust crates, tools, and ecosystem developments" |
| 150 | |
| 151 | queries: |
| 152 | primary: |
| 153 | - "language:rust stars:>30 pushed:>{last_week}" |
| 154 | - "topic:rust stars:>20 pushed:>{last_week}" |
| 155 | - "topic:rust-lang stars:>20 pushed:>{last_week}" |
| 156 | secondary: |
| 157 | - "topic:cargo stars:>50 pushed:>{last_week}" |
| 158 | - "topic:wasm language:rust stars:>30 pushed:>{last_week}" |
| 159 | |
| 160 | scoring: |
| 161 | min_stars: 15 |
| 162 | min_stars_gained: 8 |
| 163 | max_age_days: 730 |
| 164 | min_relevance_score: 40 |
| 165 | language_boost: |
| 166 | Rust: 1.5 |
| 167 | C: 1.0 |
| 168 | topic_relevance: |
| 169 | - rust |
| 170 | - rust-lang |
| 171 | - cargo |
| 172 | - wasm |
| 173 | - systems-programming |
| 174 | - embedded |
| 175 | noise_topics: |
| 176 | - tutorial |
| 177 | - learning-rust |
| 178 | - rust-exercises |
| 179 | noise_name_patterns: |
| 180 | - "^rust-by-example" |
| 181 | - "-exercises$" |
| 182 | |
| 183 | quality: |
| 184 | min_repos_per_week: 5 |
| 185 | max_false_positive_rate: 0.30 |
| 186 | min_signal_repos: 2 |
| 187 | |
| 188 | content: |
| 189 | tone: "systems-oriented, precise" |
| 190 | audience: "Rust developers and systems programmers" |
| 191 | emphasis: |
| 192 | - "Crates reaching stability milestones" |
| 193 | - "Performance and safety innovations" |
| 194 | - "Ecosystem tooling improvements" |
| 195 | de_emphasis: |
| 196 | - "Beginner tutorials" |
| 197 | - "Reimplementations of existing tools without novel approach" |
| 198 | |
| 199 | learning: |
| 200 | wisdom_file: "topics/rust/wisdom.md" |
| 201 | skills_dir: "topics/rust/skills/" |
| 202 | predictions_file: "topics/rust/predictions.jsonl" |
| 203 | reskill_context: |
| 204 | - "Are we calibrated for the Rust ecosystem's smaller scale?" |
| 205 | - "Which crate categories are we missing?" |
| 206 | ``` |
| 207 | |
| 208 | --- |
| 209 | |
| 210 | ## Pipeline Changes |
| 211 | |
| 212 | ### Overview |
| 213 | |
| 214 | ``` |
| 215 | ┌─────────────────────────────────────────────────────────────────────────┐ |
| 216 | │ Topic-Aware Pipeline (v1) │ |
| 217 | ├─────────────────────────────────────────────────────────────────────────┤ |
| 218 | │ │ |
| 219 | │ squadscope.topic.yml │ |
| 220 | │ │ │ |
| 221 | │ ▼ │ |
| 222 | │ ┌───────────┐ query+ ┌───────────┐ scored ┌──────────────┐ │ |
| 223 | │ │ Crawler │──────────►│ Scorer │──────────►│ Analyzer │ │ |
| 224 | │ └───────────┘ raw JSON └───────────┘ repos └──────────────┘ │ |
| 225 | │ │ │ |
| 226 | │ ┌───────────┼──────────┐ │ |
| 227 | │ ▼ ▼ ▼ │ |
| 228 | │ ┌─────────┐ ┌──────────┐ ┌────┐│ |
| 229 | │ │ Content │ │Prediction│ │RSS ││ |
| 230 | │ │ Page │ │ Ledger │ │Feed││ |
| 231 | │ └─────────┘ └──────────┘ └────┘│ |
| 232 | │ │ |
| 233 | │ Learning loop (per topic): │ |
| 234 | │ predictions.jsonl → validate_predictions.py → scorecard → reskill │ |
| 235 | │ → updated wisdom.md → injected into next analysis prompt │ |
| 236 | │ │ |
| 237 | └─────────────────────────────────────────────────────────────────────────┘ |
| 238 | ``` |
| 239 | |
| 240 | ### 1. Crawler Changes (`scripts/crawl.py`) |
| 241 | |
| 242 | **Current:** Hardcoded queries in `crawl.py` searching for generic trending repos. |
| 243 | |
| 244 | **Proposed:** |
| 245 | - Read `squadscope.topic.yml` at startup |
| 246 | - Build search queries from `queries.primary` and `queries.secondary` |
| 247 | - Apply `scoring.min_stars` as a pre-filter in the GitHub API query |
| 248 | - Template `{last_week}` in query strings to ISO date of 7 days ago |
| 249 | - Output to `data/raw/{topic_id}/YYYY-WNN.json` (namespaced) |
| 250 | |
| 251 | ```python |
| 252 | # Pseudocode for topic-aware crawling |
| 253 | config = load_topic_config("squadscope.topic.yml") |
| 254 | queries = config["queries"]["primary"] + config["queries"]["secondary"] |
| 255 | for q in queries: |
| 256 | q = q.replace("{last_week}", last_week_iso()) |
| 257 | results = search_github(q) |
| 258 | all_repos.extend(results) |
| 259 | |
| 260 | # Deduplicate by full_name, then classify into the existing raw payload shape |
| 261 | unique_repos = deduplicate(all_repos) |
| 262 | new_repos, trending_repos = partition_repo_sets(unique_repos) |
| 263 | payload = { |
| 264 | "week": current_iso_week(), |
| 265 | "new_repos": new_repos, |
| 266 | "trending_repos": trending_repos, |
| 267 | "signals": build_signals(new_repos, trending_repos), |
| 268 | "metadata": build_metadata(config, queries), |
| 269 | } |
| 270 | write_json(f"data/raw/{config['topic']['id']}/YYYY-WNN.json", payload) |
| 271 | ``` |
| 272 | |
| 273 | ### 2. New: Scoring Pipeline (`scripts/score_repos.py`) |
| 274 | |
| 275 | A new pipeline stage between crawl and analyze. Repos get a **relevance score** (0-100): |
| 276 | |
| 277 | | Factor | Weight | Scoring Logic | |
| 278 | |--------|--------|---------------| |
| 279 | | Topic overlap | 30% | Count of repo topics matching `scoring.topic_relevance` | |
| 280 | | Star momentum | 25% | `stars_gained / min_stars_gained` ratio (capped at 3x) | |
| 281 | | Language match | 15% | Boost from `scoring.language_boost` | |
| 282 | | Noise penalty | -20% | Repos matching `noise_topics` or `noise_name_patterns` | |
| 283 | | Recency | 10% | Days since last push (more recent = higher) | |
| 284 | |
| 285 | **Output:** `data/scored/{topic_id}/YYYY-WNN.json` — same top-level schema as raw (`week`, `new_repos`, `trending_repos`, `signals`, `metadata`), with `relevance_score` added to repo entries and scoring/filter metadata appended under `metadata`. Only repos with `relevance_score >= scoring.min_relevance_score` pass to analysis. |
| 286 | |
| 287 | ### 3. Analysis Prompt Changes (`prompts/analyze-weekly.md`) |
| 288 | |
| 289 | **Current:** Static prompt with no topic context or learned state. |
| 290 | |
| 291 | **Proposed:** Topic-aware prompt template with injection points: |
| 292 | |
| 293 | ```markdown |
| 294 | # Weekly Analysis: {{TOPIC_NAME}} |
| 295 | |
| 296 | You are analyzing GitHub repositories for the **{{TOPIC_NAME}}** channel. |
| 297 | Audience: {{AUDIENCE}} |
| 298 | Tone: {{TONE}} |
| 299 | |
| 300 | ## Emphasis |
| 301 | {{EMPHASIS_LIST}} |
| 302 | |
| 303 | ## De-emphasis |
| 304 | {{DE_EMPHASIS_LIST}} |
| 305 | |
| 306 | ## Learned Wisdom (from prior reskill cycles) |
| 307 | {{WISDOM_CONTENT}} |
| 308 | |
| 309 | ## Active Skills |
| 310 | {{SKILLS_CONTENT}} |
| 311 | |
| 312 | ## Prediction Track Record |
| 313 | {{PREDICTION_SCORECARD}} |
| 314 | |
| 315 | ## Instructions |
| 316 | Analyze the scored repositories in `data/scored/{{TOPIC_ID}}/YYYY-WNN.json`. |
| 317 | ... |
| 318 | ``` |
| 319 | |
| 320 | ### 4. Content Namespacing |
| 321 | |
| 322 | | Asset | Current Path | Topic-Aware Path | |
| 323 | |-------|-------------|-----------------| |
| 324 | | Raw crawl data | `data/raw/YYYY-WNN.json` | `data/raw/{topic_id}/YYYY-WNN.json` | |
| 325 | | Scored data | N/A (new) | `data/scored/{topic_id}/YYYY-WNN.json` | |
| 326 | | Analysis output | `data/analyzed/YYYY-WNN-summary.md` | `data/analyzed/{topic_id}/YYYY-WNN-summary.md` | |
| 327 | | Star snapshots | `data/snapshots/YYYY-WNN.json` | `data/snapshots/{topic_id}/YYYY-WNN.json` | |
| 328 | | Hugo content | `content/weekly/YYYY-WNN.md` | `content/topics/{topic_id}/YYYY-WNN.md` | |
| 329 | | RSS feed | `/index.xml` | `/topics/{topic_id}/index.xml` | |
| 330 | | Wisdom | `.squad/identity/wisdom.md` | `topics/{topic_id}/wisdom.md` | |
| 331 | | Skills | `.squad/skills/` | `topics/{topic_id}/skills/` | |
| 332 | | Predictions | N/A (new) | `topics/{topic_id}/predictions.jsonl` | |
| 333 | |
| 334 | ### 5. RSS Per Topic |
| 335 | |
| 336 | Hugo taxonomy configuration: |
| 337 | |
| 338 | ```toml |
| 339 | # hugo.toml additive changes |
| 340 | [taxonomies] |
| 341 | tag = "tags" |
| 342 | category = "categories" |
| 343 | topic = "topics" |
| 344 | |
| 345 | [outputFormats.RSS] |
| 346 | mediaType = "application/rss+xml" |
| 347 | baseName = "index" |
| 348 | |
| 349 | [params] |
| 350 | topicId = "ai-ml" # From squadscope.topic.yml |
| 351 | ``` |
| 352 | |
| 353 | Each topic gets its own RSS feed at `/topics/{topic_id}/index.xml`. The site root `/index.xml` remains as an aggregate feed (or is removed in single-topic mode). |
| 354 | |
| 355 | --- |
| 356 | |
| 357 | ## Learning System Integration |
| 358 | |
| 359 | ### Per-Topic Learning State |
| 360 | |
| 361 | Each topic maintains isolated learning state: |
| 362 | |
| 363 | ``` |
| 364 | topics/{topic_id}/ |
| 365 | ├── wisdom.md # Accumulated heuristics for this domain |
| 366 | ├── skills/ # Extracted patterns and rules |
| 367 | │ ├── SKILL-001.md |
| 368 | │ └── SKILL-002.md |
| 369 | ├── predictions.jsonl # Prediction ledger (append-only) |
| 370 | └── scorecards/ # Hindsight validation results |
| 371 | ├── 2026-W21.json |
| 372 | └── 2026-W25.json |
| 373 | ``` |
| 374 | |
| 375 | ### Why Isolation Matters |
| 376 | |
| 377 | From the learning audit: "shared wisdom across domains produces uncalibrated judgments." Examples: |
| 378 | |
| 379 | - AI/ML wisdom: "Repos with HuggingFace integrations tend to gain adoption quickly" → **meaningless for Rust** |
| 380 | - Rust wisdom: "Crates with `no_std` support indicate systems-level seriousness" → **meaningless for AI/ML** |
| 381 | - Security wisdom: "CVE-related repos spike and fade within 2 weeks" → **misleading if applied to general software** |
| 382 | |
| 383 | ### Prediction Ledger (`predictions.jsonl`) |
| 384 | |
| 385 | Each analysis produces machine-readable predictions appended to the ledger: |
| 386 | |
| 387 | ```jsonl |
| 388 | {"week":"2026-W21","repo":"owner/name","claim":"signal","confidence":0.8,"category":"framework","predicted_stars_4w":500} |
| 389 | {"week":"2026-W21","repo":"owner/name","claim":"noise","confidence":0.7,"category":"wrapper","reason":"thin wrapper around existing API"} |
| 390 | {"week":"2026-W21","repo":"owner/name","claim":"gap","confidence":0.6,"category":"missing-tooling","description":"No good Rust WASM debugger exists yet"} |
| 391 | ``` |
| 392 | |
| 393 | **Fields:** |
| 394 | - `week`: ISO week of the prediction |
| 395 | - `repo`: Full repository name (or null for gap predictions) |
| 396 | - `claim`: One of `signal`, `noise`, `gap` |
| 397 | - `confidence`: 0.0-1.0 how sure the system is |
| 398 | - `category`: Domain-specific category |
| 399 | - `predicted_stars_4w`: Expected star count in 4 weeks (for signal/noise) |
| 400 | - `reason`/`description`: Human-readable explanation |
| 401 | |
| 402 | ### Hindsight Validation (`scripts/validate_predictions.py`) |
| 403 | |
| 404 | Runs 4 weeks after predictions are made. Compares claims to outcomes: |
| 405 | |
| 406 | ```python |
| 407 | # Validation logic |
| 408 | for prediction in load_predictions(topic_id, target_week): |
| 409 | if prediction["claim"] == "signal": |
| 410 | actual_stars = get_current_stars(prediction["repo"]) |
| 411 | predicted = prediction["predicted_stars_4w"] |
| 412 | score = min(actual_stars / predicted, 2.0) # Cap at 2x |
| 413 | scorecard.append({"prediction": prediction, "actual": actual_stars, "score": score}) |
| 414 | elif prediction["claim"] == "noise": |
| 415 | # Noise repos should have plateaued or declined |
| 416 | delta = get_star_delta(prediction["repo"], weeks=4) |
| 417 | score = 1.0 if delta < prediction.get("predicted_stars_4w", 50) else 0.0 |
| 418 | scorecard.append({"prediction": prediction, "actual_delta": delta, "score": score}) |
| 419 | ``` |
| 420 | |
| 421 | **Scorecard output** feeds into reskill: "Last month we were 72% accurate on signal calls but only 45% on noise calls in ai-ml. We tend to overestimate wrapper libraries." |
| 422 | |
| 423 | ### Reskill Integration |
| 424 | |
| 425 | The reskill prompt (run every 5th cycle) now receives: |
| 426 | 1. Topic-specific wisdom from `topics/{topic_id}/wisdom.md` |
| 427 | 2. Latest scorecard from `topics/{topic_id}/scorecards/` |
| 428 | 3. Prediction accuracy trend across last 5 scorecards |
| 429 | 4. Topic config context (what we're optimizing for) |
| 430 | |
| 431 | Reskill outputs are written back to topic-specific paths, ensuring one topic's learnings never contaminate another. |
| 432 | |
| 433 | --- |
| 434 | |
| 435 | ## Content Architecture |
| 436 | |
| 437 | ### Topic Channels |
| 438 | |
| 439 | URL structure: |
| 440 | ``` |
| 441 | / → Home (links to topic channel) |
| 442 | /topics/{topic_id}/ → Topic landing page (latest + archive) |
| 443 | /topics/{topic_id}/2026-W21 → Weekly issue page |
| 444 | /topics/{topic_id}/index.xml → RSS feed for this topic |
| 445 | ``` |
| 446 | |
| 447 | ### Hugo Content Structure |
| 448 | |
| 449 | ``` |
| 450 | content/ |
| 451 | └── topics/ |
| 452 | └── ai-ml/ |
| 453 | ├── _index.md # Topic landing page |
| 454 | ├── 2026-W21.md # Weekly issue |
| 455 | ├── 2026-W22.md |
| 456 | └── ... |
| 457 | ``` |
| 458 | |
| 459 | ### Navigation |
| 460 | |
| 461 | For v1 (single-topic instance), the site homepage redirects to the topic channel. The topic archive page lists all weekly issues with summaries. |
| 462 | |
| 463 | For v2 (multi-topic), a topic selector would appear in navigation. |
| 464 | |
| 465 | --- |
| 466 | |
| 467 | ## v1 Scope: Single Configurable Topic |
| 468 | |
| 469 | ### What ships in v1 |
| 470 | |
| 471 | 1. **`squadscope.topic.yml` config format** — fully specified, validated at pipeline start |
| 472 | 2. **Topic-aware crawler** — reads queries from config, outputs to namespaced paths |
| 473 | 3. **Scoring pipeline** — `scripts/score_repos.py` with configurable weights |
| 474 | 4. **Topic-aware analysis prompt** — injects topic context, wisdom, and scorecard |
| 475 | 5. **Prediction ledger** — appended to after each analysis |
| 476 | 6. **Hindsight validation script** — runs on 4-week-old predictions |
| 477 | 7. **Per-topic learning state** — isolated wisdom, skills, scorecards |
| 478 | 8. **Topic RSS feed** — at `/topics/{topic_id}/index.xml` |
| 479 | 9. **Two example configs** — `examples/topics/ai-ml.yml` and `examples/topics/rust.yml` |
| 480 | 10. **Config validation script** — `scripts/validate_topic_config.py` |
| 481 | |
| 482 | ### What does NOT ship in v1 |
| 483 | |
| 484 | - Multi-topic in a single instance |
| 485 | - Topic discovery or marketplace |
| 486 | - Cross-topic learning transfer |
| 487 | - Dynamic query generation |
| 488 | - Topic health monitoring dashboard |
| 489 | |
| 490 | ### Deployment Model (v1) |
| 491 | |
| 492 | One SquadScope fork per topic. Each fork: |
| 493 | - Has its own `squadscope.topic.yml` |
| 494 | - Runs its own GitHub Actions schedule |
| 495 | - Produces its own GitHub Pages site |
| 496 | - Accumulates its own learning state |
| 497 | - Has its own RSS feed |
| 498 | |
| 499 | This is intentionally simple. Forks share the same codebase but diverge on configuration and learned state. |
| 500 | |
| 501 | --- |
| 502 | |
| 503 | ## v2 Vision: Multi-Topic Single Instance |
| 504 | |
| 505 | **Deferred.** Documented here for future planning only. |
| 506 | |
| 507 | ### What v2 would add |
| 508 | |
| 509 | - Single instance running multiple topics on different schedules |
| 510 | - Shared infrastructure, isolated topic state |
| 511 | - Topic health monitoring (auto-disable topics below quality thresholds) |
| 512 | - Cross-topic signals ("this repo is trending in BOTH ai-ml and rust channels") |
| 513 | - Topic marketplace (community-contributed topic configs) |
| 514 | - Unified navigation across topics |
| 515 | |
| 516 | ### Why v2 is premature now |
| 517 | |
| 518 | - Adds orchestration complexity (per-topic cron, per-topic secrets) |
| 519 | - Learning isolation is harder in shared instances (accidental cross-contamination) |
| 520 | - No user demand signal yet — need v1 adoption data first |
| 521 | - GitHub Actions concurrency constraints make multi-topic scheduling complex |
| 522 | |
| 523 | --- |
| 524 | |
| 525 | ## Implementation Plan |
| 526 | |
| 527 | ### Issues to Create |
| 528 | |
| 529 | | # | Title | Phase | Depends On | Assignee Profile | |
| 530 | |---|-------|-------|-----------|-----------------| |
| 531 | | 1 | Define `squadscope.topic.yml` schema and validator | Foundation | — | Architect | |
| 532 | | 2 | Namespace data directories by topic ID | Foundation | #1 | Crawler | |
| 533 | | 3 | Implement scoring pipeline (`scripts/score_repos.py`) | Pipeline | #1, #2 | Crawler | |
| 534 | | 4 | Make crawler read queries from topic config | Pipeline | #1, #2 | Crawler | |
| 535 | | 5 | Create topic-aware analysis prompt template | Pipeline | #1 | Analyzer | |
| 536 | | 6 | Add prediction ledger output to analysis | Pipeline | #5 | Analyzer | |
| 537 | | 7 | Implement hindsight validation script | Learning | #6 | Analyzer | |
| 538 | | 8 | Per-topic learning state directories and seeding | Learning | #1 | Architect | |
| 539 | | 9 | Wire prediction scorecard into reskill prompt | Learning | #7 | Analyzer | |
| 540 | | 10 | Hugo topic taxonomy and per-topic RSS | Content | #2 | Site | |
| 541 | | 11 | Topic landing page template | Content | #10 | Site | |
| 542 | | 12 | Example config: ai-ml | Validation | #1-#4 | Validator | |
| 543 | | 13 | Example config: rust | Validation | #1-#4 | Validator | |
| 544 | | 14 | Topic quality threshold enforcement | Quality | #3 | Crawler | |
| 545 | | 15 | End-to-end integration test with example topic | Validation | All | Validator | |
| 546 | |
| 547 | ### Dependencies |
| 548 | |
| 549 | ``` |
| 550 | #1 (schema) ─┬─► #2 (namespacing) ─┬─► #3 (scorer) ──► #4 (crawler) |
| 551 | │ │ │ |
| 552 | │ └─► #10 (Hugo) ▼ |
| 553 | │ #12, #13 (examples) |
| 554 | └─► #5 (prompt) ──► #6 (predictions) ──► #7 (validation) |
| 555 | │ |
| 556 | ▼ |
| 557 | #9 (reskill wiring) |
| 558 | ``` |
| 559 | |
| 560 | ### Estimated Effort |
| 561 | |
| 562 | - **Foundation (Issues 1-2):** 1 session |
| 563 | - **Pipeline (Issues 3-6):** 2-3 sessions |
| 564 | - **Learning (Issues 7-9):** 2 sessions |
| 565 | - **Content (Issues 10-11):** 1 session |
| 566 | - **Validation (Issues 12-15):** 1-2 sessions |
| 567 | |
| 568 | **Total:** ~7-9 work sessions |
| 569 | |
| 570 | --- |
| 571 | |
| 572 | ## Open Questions |
| 573 | |
| 574 | | # | Question | Impact | Proposed Resolution | |
| 575 | |---|----------|--------|-------------------| |
| 576 | | OQ1 | Should topic configs live in repo root or `topics/` dir? | File organization | Repo root for v1 (single topic); move to `topics/` in v2 | |
| 577 | | OQ2 | How to handle repos that span multiple topics? | Dedup in multi-topic v2 | v1: irrelevant (single topic). v2: each topic scores independently | |
| 578 | | OQ3 | What's the minimum weeks of data before learning is meaningful? | Reskill timing | Propose 4 weeks minimum before first hindsight validation runs | |
| 579 | | OQ4 | Should prediction confidence be system-generated or human-calibrated initially? | Learning accuracy | Start with fixed confidence (0.7 for signal, 0.5 for noise), calibrate after 8 weeks of scorecard data | |
| 580 | | OQ5 | Enrichment signals beyond stars — which to add first? | Prediction quality | Forks and contributor count (cheapest API calls, highest signal per learning-audit G13) | |
| 581 | | OQ6 | Should topic quality thresholds auto-disable a topic or just warn? | Reliability | Warn-only for v1 (log to workflow summary), auto-disable in v2 | |
| 582 | |
| 583 | --- |
| 584 | |
| 585 | ## Success Metrics |
| 586 | |
| 587 | ### Quantitative (measurable after 8 weeks of operation) |
| 588 | |
| 589 | | Metric | Target | Measurement | |
| 590 | |--------|--------|-------------| |
| 591 | | False positive rate | < 25% per topic | Manual audit of 20 random "signal" calls per month | |
| 592 | | Prediction accuracy (signal) | > 65% | Hindsight validation scorecard | |
| 593 | | Prediction accuracy (noise) | > 55% | Hindsight validation scorecard | |
| 594 | | Repos per weekly issue | ≥ quality.min_repos_per_week from config | Automated count | |
| 595 | | RSS subscribers per topic | > 0 within 4 weeks | Analytics (if available) | |
| 596 | | Learning improvement trend | Prediction accuracy increases by ≥ 5% over 8 weeks | Scorecard comparison | |
| 597 | |
| 598 | ### Qualitative |
| 599 | |
| 600 | - Topic experts find the digest "saves them time" vs. manual GitHub browsing |
| 601 | - Analysis tone matches configured audience expectations |
| 602 | - Signal/Noise/Gaps sections feel calibrated to the specific domain |
| 603 | - Learned wisdom in `wisdom.md` contains domain-specific (not generic) heuristics after 3 reskill cycles |
| 604 | |
| 605 | --- |
| 606 | |
| 607 | ## Relationship to Existing Work |
| 608 | |
| 609 | ### Analysis Spec (`docs/analysis-spec.md`) |
| 610 | This PRD extends the approved analyzer contract. Topic-aware raw and scored artifacts keep the existing top-level payload shape (`week`, `new_repos`, `trending_repos`, `signals`, `metadata`) while adding a topic namespace prefix and repo-level `relevance_score` data. |
| 611 | |
| 612 | ### Pipeline Validation (`docs/pipeline-validation.md`) |
| 613 | This PRD preserves the current Crawl → Analyze → Generate workflow expectations while adding one topic-aware scoring step between crawl and analyze. Existing quality gates and artifact validation remain in force. |
| 614 | |
| 615 | ### Learning Audit (`docs/learning-audit.md`) |
| 616 | This PRD directly addresses: |
| 617 | - **G7 (prompt feedback loop):** Topic-aware prompt template with `{{WISDOM_CONTENT}}` injection |
| 618 | - **G8 (hindsight validation):** `scripts/validate_predictions.py` with per-topic scorecards |
| 619 | - **G9 (prediction registry):** `predictions.jsonl` format defined |
| 620 | - **G13 (enrichment signals):** Fork/contributor data noted as OQ5, planned for scorer enrichment |
| 621 | |
| 622 | ### Decisions (`.squad/decisions.md`) |
| 623 | - Respects Decision 3 (pipeline stage contracts) — adds a scoring stage but preserves existing boundaries |
| 624 | - Respects Decision 4 (reviewer gate) — quality gate applies per-topic |
| 625 | - Extends Decision 6 (reskill) — reskill reads per-topic state instead of global state |