feat(rai): add methodology disclosure page (#212)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Juan Manuel Servera committed
May 25, 2026 at 23:30 UTC
27f507b5f3224a1d80c2ee8e21330bf98aad219d
6 files changed
+89
-3
.squad/decisions/inbox/farnsworth-methodology-disclosure.md
new
+18
@@ -0,0 +1,18 @@
1
+# Source-selection methodology disclosure
2
+
3
+- **Date:** 2026-05-25
4
+- **Owner:** Farnsworth
5
+- **Status:** Proposed for merge
6
+
7
+## Decision
8
+
9
+Source-selection biases are publicly disclosed at `/methodology/`; updates to scoring, source ingestion, crawl thresholds, or press coverage should be reflected there.
10
+
11
+## Context
12
+
13
+Nibbler's second responsible-AI sweep identified source-selection bias disclosure as a high-severity fairness and transparency gap. The methodology page gives readers a plain-English explanation of source inputs, ranking logic, and interpretation limits.
14
+
15
+## Consequences
16
+
17
+- Pipeline changes that alter source mix or scoring should include a reader-facing methodology update.
18
+- Future bias metrics can link back to `/methodology/` as the stable disclosure surface.
content/about/_index.md
+1
-1
@@ -14,7 +14,7 @@ Use the homepage to jump straight into the latest summary, scan the archive, or
14
15
SquadScope turns GitHub activity into readable weekly, monthly, and yearly trend reports for busy developers and technical leaders.
16
17
-The project favors editorial signal over dashboard noise: what mattered this week, what is merely loud, and what is missing from the conversation.
17
+The project favors editorial signal over dashboard noise: what mattered this week, what is merely loud, and what is missing from the conversation. Read the [methodology](/methodology/) for how we choose sources, rank signals, and disclose known biases.
18
19
## Pipeline transparency
20
content/methodology/_index.md
new
+44
@@ -0,0 +1,44 @@
1
++++
2
+title = 'Methodology'
3
+date = '2026-05-25T00:00:00+02:00'
4
+draft = false
5
+summary = 'How SquadScope finds, ranks, and explains GitHub trend signals.'
6
+description = 'Plain-English methodology for SquadScope source selection, scoring, ranking, and known bias limits.'
7
++++
8
+
9
+## What this page is for
10
+
11
+SquadScope is a weekly, AI-assisted read on developer and open-source momentum. We use automated crawls to find candidate projects, then generate editorial briefs from that evidence. This page explains where we look, how we rank what we find, and how readers should account for the gaps.
12
+
13
+## Where we look
14
+
15
+We start with GitHub repository search in `scripts/crawl.py`. On the general weekly crawl, we look for repositories created during the current seven-day window and repositories pushed during that same window, with both searches requiring more than 50 stars. Results are sorted by stars through the GitHub search API, deduplicated, checked for a README, and filtered to remove forks, templates, missing descriptions, homework/course material, demos, and other low-signal patterns. Topic-specific crawls can replace the default search with configured primary and secondary topic queries.
16
+
17
+We also ingest press context. Today the automated press source is **TechCrunch RSS**, crawled by `scripts/techcrunch_crawler.py` over the same default seven-day window. That crawler extracts titles, summaries, categories, GitHub links, and likely company or project names, then keeps technology/open-source relevant articles for correlation. We do not currently ingest The Verge, Wired, Ars Technica, Hacker News, Reddit, non-English outlets, or paywalled newsletters as first-class automated sources. If we add those sources, this page should change.
18
+
19
+The cadence is weekly. Unless an operator passes explicit dates, the GitHub and TechCrunch crawls look back seven days from the crawl time and write an ISO-week artifact such as `2026-W22.json`.
20
+
21
+## How we score and rank
22
+
23
+Raw GitHub results are not treated as a final ranking by themselves. `scripts/crawl.py` first records two buckets: new repositories and trending repositories. New repos are sorted by star count. Trending repos use prior weekly star snapshots when available, so a project with a strong week-over-week increase can rise above a project that is simply large. The weekly `signals` object summarizes common repo topics across the collected set; those topics help the analysis describe broad movement instead of only individual winners.
24
+
25
+For topic channels, `scripts/score_repos.py` can compute a 0-100 relevance score. The score is capped and additive: up to 25 points for total stars, up to 25 for stars gained, up to 15 for language fit, up to 25 for matching configured topics, and up to 10 for freshness. Stars and stars gained use logarithmic scales, so very large projects still get credit but do not grow without limit. Topic matching caps at three matching topics. Defaults include a 20-star minimum, 10 stars gained, a 365-day freshness window, and a minimum relevance score of 40 unless the topic config changes them.
26
+
27
+`signals.json` is not a separate source; in the current payload, `signals` is the structured summary generated from crawled repositories. `scripts/momentum_tracker.py` adds a hindsight check for press-correlated repos: after two or four weeks, a repo is marked sustained if its later stars gained are at least 20% of the initial gain; otherwise it is marked faded. The recorded decay rate is `1 - current / initial`, clamped between 0 and 1.
28
+
29
+## Biases readers should account for
30
+
31
+- **English-language source bias:** GitHub metadata and the current TechCrunch feed favor English-language projects and coverage; treat non-English ecosystem absence as under-observation, not lack of activity.
32
+- **High-star bias:** The default crawl requires more than 50 stars and sorts by stars, so important niche projects can be missed until they are already visible.
33
+- **US and Bay Area press bias:** TechCrunch coverage reflects a US startup lens; read press correlation as one narrative input, not a global technology map.
34
+- **BigCo signal-strength bias:** Large companies create more repos, launches, docs, and articles, so their activity can look more important than quieter independent work.
35
+- **Survivorship bias:** We mostly see projects that remain public and crawlable; failed experiments, abandoned private work, and deleted repos are largely invisible.
36
+- **Recency bias:** A weekly window is good at catching spikes but can miss slower ecosystem shifts; use monthly and yearly rollups for longer arcs.
37
+
38
+## What we do not do
39
+
40
+We do not sell paid placements, accept sponsor influence over selection, or reserve ranking slots for partners. Editorial articles are AI-generated from collected signals; see the site footer disclosure and [About](/about/) page for that operating model. We do not retroactively rewrite past articles after publication. Until the dedicated errata process from issue #209 lands, corrections and concerns should be raised through [GitHub issues](https://github.com/jmservera/SquadScope/issues/new).
41
+
42
+## Future work
43
+
44
+We want this methodology to become more measurable. The next improvements are adding non-English source coverage, expanding hindsight validation for past calls (see [issue #38](https://github.com/jmservera/SquadScope/issues/38)), and adding a source-diversity metric to the public cost dashboard so readers can see whose signals the pipeline tends to hear.
hugo.toml
+7
-1
@@ -65,11 +65,17 @@ rssLimit = 20
65
url = '/yearly/'
66
weight = 30
67
68
+ [[menu.main]]
69
+ identifier = 'methodology'
70
+ name = 'Methodology'
71
+ url = '/methodology/'
72
+ weight = 40
73
+
74
[[menu.main]]
75
identifier = 'about'
76
name = 'About'
77
url = '/about/'
72
- weight = 40
78
+ weight = 50
79
80
ignoreFiles = ['data/analyzed/.*\\.md$', 'data/metrics/.*\\.md$', 'data/metrics/.*\\.jsonl$']
81
layouts/methodology/list.html
new
+18
@@ -0,0 +1,18 @@
1
+{{- define "main" -}}
2
+<article class="post-single" data-pagefind-body>
3
+ {{ partial "pagefind-metadata.html" . }}
4
+
5
+ <header class="post-header">
6
+ <h1 class="post-title entry-hint-parent">{{ .Title }}</h1>
7
+ {{- with (.Description | default .Params.summary) }}
8
+ <div class="post-description">{{ . }}</div>
9
+ {{- end }}
10
+ </header>
11
+
12
+ {{- if .Content }}
13
+ <div class="post-content md-content">
14
+ {{ .Content }}
15
+ </div>
16
+ {{- end }}
17
+</article>
18
+{{- end -}}
layouts/partials/ai-disclosure.html
+1
-1
@@ -5,7 +5,7 @@
5
<p>
6
Editorial content on SquadScope is generated by AI models, with human curation of sources and editorial structure by the
7
<a href="https://github.com/jmservera/SquadScope" target="_blank" rel="noopener">SquadScope maintainer</a>.
8
- Read more about <a href="{{ "about/" | absLangURL }}">how we work</a>.
8
+ Read more about <a href="{{ "about/" | absLangURL }}">how we work</a> and <a href="{{ "methodology/" | absLangURL }}">how we choose sources</a>.
9
</p>
10
<p>
11
Spot an error? <a href="https://github.com/jmservera/SquadScope/issues" target="_blank" rel="noopener">Report it on GitHub</a>.