| 1 | # PRD: TechCrunch RSS Integration for Cross-Signal Enrichment |
| 2 | |
| 3 | **Author:** Farnsworth (Analyst), revised by Bender (Crawler) |
| 4 | **Date:** 2026-05-19 |
| 5 | **Status:** Completed — implemented and archived 2026-06-10 (see `scripts/techcrunch_crawler.py`, `tests/test_techcrunch_crawler.py`, and `data/raw/*-external-news.json`) |
| 6 | **Type:** Feature PRD (Enrichment Signal) |
| 7 | **Depends on:** `docs/processed/PRD-topic-channels.md` |
| 8 | |
| 9 | > **Archived as-is.** This PRD is preserved as a historical planning record. The current canonical paths, field names, and interfaces live in code — see `scripts/techcrunch_crawler.py`, `config/external_news_sources.json`, and `.github/workflows/crawl-and-publish.yml`. The most misleading specifics below have been corrected to match what shipped (the crawler now ingests multiple external news sources, not TechCrunch alone). |
| 10 | |
| 11 | --- |
| 12 | |
| 13 | ## Executive Summary |
| 14 | |
| 15 | SquadScope tracks GitHub repository trends weekly. This PRD proposes adding TechCrunch RSS as a **supplementary enrichment signal** — not a primary data source — to detect the *delta* between press hype and actual GitHub traction. When TechCrunch covers a technology or project that also shows unusual GitHub star activity, that correlation is newsworthy. When press coverage does NOT correlate with GitHub activity, that absence is equally informative. |
| 16 | |
| 17 | **Key constraint acknowledged upfront:** The correlation hit rate between TechCrunch articles and specific GitHub repositories is estimated at only **5–15%**. This feature is designed as a low-cost enrichment layer that adds value when correlations exist, and degrades gracefully (adds zero noise) when they don't. |
| 18 | |
| 19 | --- |
| 20 | |
| 21 | ## Problem Statement |
| 22 | |
| 23 | ### The Gap: Press Hype vs. Real Adoption |
| 24 | |
| 25 | 1. **GitHub stars measure developer interest.** A repo gaining 500 stars in a week signals genuine traction from people who build things. |
| 26 | |
| 27 | 2. **TechCrunch coverage measures press/VC interest.** An article about a startup or technology signals attention from the funding and media ecosystem. |
| 28 | |
| 29 | 3. **The delta between these signals is the insight.** Three scenarios produce editorial value: |
| 30 | - **Hype confirmed:** TechCrunch covers X, and X's GitHub repos are surging → "Real momentum, developers agree" |
| 31 | - **Hype without substance:** TechCrunch covers Y, but Y has zero or declining GitHub activity → "Marketing over engineering" |
| 32 | - **Quiet breakout:** No press coverage, but a repo is exploding on GitHub → "Under the radar" |
| 33 | |
| 34 | 4. **Currently, SquadScope only sees scenario 3.** Adding press signal enables detecting scenarios 1 and 2, making the weekly digest more insightful. |
| 35 | |
| 36 | ### Why TechCrunch Specifically |
| 37 | |
| 38 | - TechCrunch has a well-maintained RSS feed (`https://techcrunch.com/feed/`) with full article metadata |
| 39 | - It covers the startup/tech ecosystem most likely to overlap with open-source GitHub activity |
| 40 | - RSS is free, requires no API key, and is stable |
| 41 | - Other sources (HN, Reddit) can follow the same plugin pattern later |
| 42 | |
| 43 | --- |
| 44 | |
| 45 | ## Value Proposition: The Delta Model |
| 46 | |
| 47 | The value of this integration is **NOT** in summarizing TechCrunch articles (readers can read TechCrunch themselves). The value is in the **cross-reference delta**: |
| 48 | |
| 49 | ``` |
| 50 | Value = f(TechCrunch_coverage, GitHub_activity) where: |
| 51 | - Both high → "Confirmed trend" (correlation) |
| 52 | - TC high, GH low → "Hype alert" (anti-correlation) |
| 53 | - TC low, GH high → "Sleeper hit" (absence signal) |
| 54 | - Both low → No signal (filtered out) |
| 55 | ``` |
| 56 | |
| 57 | This positions SquadScope as providing analysis that neither TechCrunch nor GitHub alone can offer. |
| 58 | |
| 59 | --- |
| 60 | |
| 61 | ## Honest Assessment: Correlation Rates |
| 62 | |
| 63 | ### Expected Hit Rates |
| 64 | |
| 65 | | Correlation Type | Estimated Rate | Reasoning | |
| 66 | |-----------------|---------------|-----------| |
| 67 | | Direct match (TC mentions a specific repo) | 2–5% | Few TC articles name exact repos | |
| 68 | | Indirect match (TC covers technology X, repo uses topic X) | 10–15% | Broader topic matching catches more | |
| 69 | | No correlation found | 80–93% | Most TC articles have no GitHub signal | |
| 70 | |
| 71 | ### Why Low Rates Are Acceptable |
| 72 | |
| 73 | 1. **Low false-positive cost:** Uncorrelated articles are simply ignored — they add zero noise to the output. |
| 74 | 2. **High value per hit:** When a correlation IS found, it's genuinely interesting editorial content. |
| 75 | 3. **Asymmetric payoff:** Even 2–3 notable correlations per week would meaningfully enrich a weekly digest. |
| 76 | 4. **Trend over time:** Cross-referencing accumulated over weeks reveals patterns invisible in any single week. |
| 77 | |
| 78 | ### What This Feature Is NOT |
| 79 | |
| 80 | - NOT a TechCrunch summarizer |
| 81 | - NOT a primary data source for SquadScope |
| 82 | - NOT expected to produce signal every week |
| 83 | - NOT a replacement for GitHub-native trend detection |
| 84 | |
| 85 | --- |
| 86 | |
| 87 | ## Filtering Strategy |
| 88 | |
| 89 | ### The Problem: Volume |
| 90 | |
| 91 | TechCrunch publishes **30–50 articles per day** (210–350 per week). Without aggressive filtering, this overwhelms the pipeline with noise. The crawler must reduce this to a manageable set before any correlation attempt. |
| 92 | |
| 93 | ### Three-Stage Filtering Pipeline |
| 94 | |
| 95 | > **As-shipped note:** This three-stage design (category allowlist → keyword filter → entity extraction) was the original plan and is **not** how the crawler ultimately shipped. The implemented crawler (`scripts/techcrunch_crawler.py`) uses a single `compute_relevance_score()` heuristic and keeps articles whose `relevance_score` meets a threshold (currently `>= 0.4`); there is no RSS `<category>` allowlist or separate multi-stage pipeline. The pipeline below is preserved as historical design rationale only. |
| 96 | |
| 97 | ``` |
| 98 | Stage 1: Category Filter (RSS metadata) |
| 99 | Input: ~250 articles/week (full RSS feed) |
| 100 | Filter: Keep only categories relevant to developer tools/open-source |
| 101 | Output: ~60-80 articles/week (70% reduction) |
| 102 | Method: Allowlist of RSS <category> tags |
| 103 | |
| 104 | Stage 2: Keyword Filter (title + description) |
| 105 | Input: ~60-80 articles/week |
| 106 | Filter: Must contain technology/developer keywords |
| 107 | Output: ~20-30 articles/week (60% reduction) |
| 108 | Method: Keyword scoring (open-source, GitHub, developer, API, SDK, framework, etc.) |
| 109 | |
| 110 | Stage 3: Entity Extraction (lightweight) |
| 111 | Input: ~20-30 articles/week |
| 112 | Filter: Extract mentioned technologies, companies, project names |
| 113 | Output: ~20-30 enriched article records with entity tags |
| 114 | Method: Regex patterns + known project name dictionary |
| 115 | ``` |
| 116 | |
| 117 | ### Category Allowlist (Initial) |
| 118 | |
| 119 | ```yaml |
| 120 | allowed_categories: |
| 121 | - Apps |
| 122 | - Artificial Intelligence |
| 123 | - Cloud |
| 124 | - Developer |
| 125 | - Enterprise |
| 126 | - Hardware |
| 127 | - Open Source |
| 128 | - Robotics |
| 129 | - Security |
| 130 | - Startups |
| 131 | |
| 132 | blocked_categories: |
| 133 | - Media & Entertainment |
| 134 | - Transportation |
| 135 | - Government & Policy |
| 136 | - Crypto # Too noisy, low GitHub correlation |
| 137 | ``` |
| 138 | |
| 139 | ### Keyword Scoring |
| 140 | |
| 141 | Each article gets a relevance score (0–10) based on title + description: |
| 142 | |
| 143 | | Keyword Group | Weight | Examples | |
| 144 | |--------------|--------|----------| |
| 145 | | Direct GitHub mentions | +5 | "GitHub", "open source", "repository" | |
| 146 | | Developer tools | +3 | "API", "SDK", "framework", "library", "CLI" | |
| 147 | | Technology names | +2 | "Python", "Rust", "Kubernetes", "LLM" | |
| 148 | | Funding/startup | +1 | "raises", "Series A", "launch" | |
| 149 | |
| 150 | **Threshold:** Articles scoring ≥ 3 proceed to entity extraction. Expected pass rate: ~40% of category-filtered articles. |
| 151 | |
| 152 | --- |
| 153 | |
| 154 | ## Temporal Alignment |
| 155 | |
| 156 | ### The Problem: RSS is Real-Time, SquadScope is Weekly |
| 157 | |
| 158 | TechCrunch publishes continuously. SquadScope runs weekly (Monday 06:53 UTC). This creates a timing mismatch: |
| 159 | |
| 160 | - An article published Tuesday about Project X won't be seen until the following Monday |
| 161 | - By then, the GitHub star surge may have already peaked and fallen |
| 162 | |
| 163 | ### Solution: Weekly Batch with 7-Day Window |
| 164 | |
| 165 | ``` |
| 166 | ┌─────────────────────────────────────────────────────┐ |
| 167 | │ Monday 06:53 UTC: Crawl job runs │ |
| 168 | │ │ |
| 169 | │ 1. Fetch all RSS items from past 7 days │ |
| 170 | │ 2. Filter (3-stage pipeline above) │ |
| 171 | │ 3. Extract entities from filtered articles │ |
| 172 | │ 4. Cross-reference entities against weekly │ |
| 173 | │ GitHub trending repos (already collected) │ |
| 174 | │ 5. Output correlation data for Farnsworth │ |
| 175 | └─────────────────────────────────────────────────────┘ |
| 176 | ``` |
| 177 | |
| 178 | ### Why Weekly Batch Is Sufficient |
| 179 | |
| 180 | 1. **SquadScope is a weekly digest.** Real-time alerting is out of scope. |
| 181 | 2. **7-day accumulation helps.** A trend covered across multiple articles in a week is stronger signal. |
| 182 | 3. **GitHub stars data is also weekly.** Both signals align on the same time window. |
| 183 | 4. **Simplicity:** No state management, no incremental polling, no deduplication across runs. |
| 184 | |
| 185 | ### Freshness Guarantee |
| 186 | |
| 187 | - RSS feed items older than 7 days are discarded |
| 188 | - If the RSS feed doesn't contain 7 days of history (TechCrunch's feed typically holds 20–30 items), supplement with the feed's full available content |
| 189 | - Each item's `<pubDate>` is checked against the collection window |
| 190 | |
| 191 | --- |
| 192 | |
| 193 | ## Correlation Approach |
| 194 | |
| 195 | ### Entity-to-Repository Matching |
| 196 | |
| 197 | ``` |
| 198 | TechCrunch Entity → GitHub Signal |
| 199 | ───────────────────────────────────────────────── |
| 200 | "Anthropic" (company) → repos with topic:anthropic or org:anthropic |
| 201 | "LangChain" (project) → repo langchain-ai/langchain stars_gained |
| 202 | "Rust 2024 edition" (tech) → repos with topic:rust AND stars_gained > threshold |
| 203 | "Series B: Acme Corp" → repos owned by acme-corp org |
| 204 | ``` |
| 205 | |
| 206 | ### Matching Strategies (in priority order) |
| 207 | |
| 208 | 1. **Exact name match:** Article mentions "LangChain" → search for repos named `langchain*` |
| 209 | 2. **Organization match:** Article mentions company → search GitHub org |
| 210 | 3. **Topic match:** Article discusses technology → match against repo topics |
| 211 | 4. **Description match:** Fuzzy match article entities against repo descriptions |
| 212 | |
| 213 | ### Scoring Correlation Strength |
| 214 | |
| 215 | | Match Type | Confidence | Example | |
| 216 | |-----------|-----------|---------| |
| 217 | | Exact repo name in article | 0.9 | "...announced on their GitHub repo langchain-ai/langchain..." | |
| 218 | | Organization name + topic overlap | 0.7 | Article about Anthropic + repo topics include "claude" | |
| 219 | | Technology keyword + trending | 0.5 | Article about "Rust" + Rust repo trending | |
| 220 | | Company name only (no GitHub signal) | 0.3 | Article about startup with no public repos | |
| 221 | |
| 222 | ### Output Format |
| 223 | |
| 224 | ```json |
| 225 | { |
| 226 | "week": "2026-W21", |
| 227 | "correlations": [ |
| 228 | { |
| 229 | "article_title": "LangChain raises $25M Series A", |
| 230 | "article_url": "https://techcrunch.com/...", |
| 231 | "article_date": "2026-05-15", |
| 232 | "matched_repos": ["langchain-ai/langchain"], |
| 233 | "match_type": "exact_name", |
| 234 | "confidence": 0.9, |
| 235 | "github_signal": { |
| 236 | "stars_gained": 847, |
| 237 | "percentile": 98 |
| 238 | }, |
| 239 | "delta_type": "confirmed_trend" |
| 240 | } |
| 241 | ], |
| 242 | "unmatched_articles": 24, |
| 243 | "total_filtered_articles": 27 |
| 244 | } |
| 245 | ``` |
| 246 | |
| 247 | --- |
| 248 | |
| 249 | ## Technical Implementation |
| 250 | |
| 251 | ### Architecture: Plugin Pattern (Decision #7) |
| 252 | |
| 253 | This integration implements the `DataSource` protocol defined in Decision #7: |
| 254 | |
| 255 | ```python |
| 256 | # As shipped: scripts/techcrunch_crawler.py |
| 257 | class NewsFeedSource: |
| 258 | """RSS data source following the DataSource protocol.""" |
| 259 | |
| 260 | def __init__(self, config: NewsSourceConfig) -> None: |
| 261 | self.config = config |
| 262 | |
| 263 | def get_name(self) -> str: |
| 264 | return self.config.name |
| 265 | |
| 266 | def get_rate_limits(self) -> dict: |
| 267 | # Plain dict, not a RateLimits type. |
| 268 | return {"requests_per_minute": self.config.requests_per_minute} |
| 269 | |
| 270 | def crawl( |
| 271 | self, |
| 272 | since: datetime, |
| 273 | until: datetime, |
| 274 | feed_url: str | None = None, |
| 275 | ) -> list[dict]: |
| 276 | """Fetch, filter, and extract entities from an RSS feed.""" |
| 277 | ... |
| 278 | ``` |
| 279 | |
| 280 | `TechCrunchSource` is a thin subclass of `NewsFeedSource`; each source is |
| 281 | described by a `NewsSourceConfig` (`name`, `feed_url`, `requests_per_minute`) |
| 282 | loaded from `config/external_news_sources.json`. |
| 283 | |
| 284 | ### File Layout |
| 285 | |
| 286 | ``` |
| 287 | scripts/ |
| 288 | techcrunch_crawler.py # Multi-source RSS crawler (DataSource protocol) |
| 289 | correlate.py # Cross-reference GitHub vs external news |
| 290 | config/ |
| 291 | external_news_sources.json # Per-source feed_url + requests_per_minute |
| 292 | data/ |
| 293 | raw/ |
| 294 | {week}-external-news.json # Merged crawl output (legacy: {week}-techcrunch.json) |
| 295 | analyzed/ |
| 296 | {week}-correlations.json # Cross-reference results (output for Farnsworth) |
| 297 | ``` |
| 298 | |
| 299 | ### Dependencies |
| 300 | |
| 301 | | Dependency | Purpose | Size Impact | |
| 302 | |-----------|---------|-------------| |
| 303 | | `feedparser` | RSS parsing | ~200 KB, pure Python | |
| 304 | | `re` (stdlib) | Entity extraction patterns | None | |
| 305 | | `datetime` (stdlib) | Window filtering | None | |
| 306 | |
| 307 | No additional API keys or authentication required. RSS is public. |
| 308 | |
| 309 | ### Integration with Existing Crawl Workflow |
| 310 | |
| 311 | ```yaml |
| 312 | # In crawl-and-publish.yml (additions only) |
| 313 | - name: Crawl external news RSS |
| 314 | run: | |
| 315 | python3 scripts/techcrunch_crawler.py \ |
| 316 | --sources config/external_news_sources.json \ |
| 317 | --output "data/raw/${WEEK}-external-news.json" \ |
| 318 | --since "$SINCE" \ |
| 319 | --until "$UNTIL" |
| 320 | |
| 321 | - name: Cross-reference correlations |
| 322 | run: | |
| 323 | python3 scripts/correlate.py \ |
| 324 | --raw "$WEEK_FILE" \ |
| 325 | --techcrunch "data/raw/${WEEK}-external-news.json" \ |
| 326 | --output "data/analyzed/${WEEK}-correlations.json" |
| 327 | ``` |
| 328 | |
| 329 | ### Error Handling |
| 330 | |
| 331 | | Failure Mode | Response | Impact | |
| 332 | |-------------|----------|--------| |
| 333 | | RSS feed unreachable | Retry 3×, then skip TechCrunch for this week | None — enrichment is optional | |
| 334 | | RSS feed format changed | Log warning, skip parsing, open issue | None — graceful degradation | |
| 335 | | Zero correlations found | Normal — output empty correlations file | Expected most weeks | |
| 336 | | Malformed XML in feed | Skip malformed items, process rest | Partial data is fine | |
| 337 | |
| 338 | --- |
| 339 | |
| 340 | ## Cost Estimate |
| 341 | |
| 342 | ### Compute Cost |
| 343 | |
| 344 | | Resource | Usage | Cost | |
| 345 | |---------|-------|------| |
| 346 | | RSS fetch | 1 HTTP request/week | $0.00 | |
| 347 | | Python processing | ~5 seconds CPU | $0.00 (free Actions minutes) | |
| 348 | | Correlation script | ~2 seconds CPU | $0.00 | |
| 349 | | **Total infrastructure cost** | | **$0.00/week** | |
| 350 | |
| 351 | ### Token Cost (if Farnsworth uses correlations in analysis) |
| 352 | |
| 353 | | Component | Size | Tokens | Cost Impact | |
| 354 | |-----------|------|--------|-------------| |
| 355 | | Correlations JSON (typical week, 2–5 hits) | ~2 KB | ~570 | +$0.002/week | |
| 356 | | Correlations JSON (zero hits) | ~0.2 KB | ~57 | +$0.0002/week | |
| 357 | | Correlations JSON (exceptional week, 10+ hits) | ~5 KB | ~1,400 | +$0.004/week | |
| 358 | |
| 359 | **Annual token cost impact: $0.10–$0.21/year** (negligible relative to $16/year baseline). |
| 360 | |
| 361 | ### Development Cost |
| 362 | |
| 363 | | Task | Effort | Priority | |
| 364 | |------|--------|----------| |
| 365 | | `techcrunch.py` plugin | 2–3 hours | Medium | |
| 366 | | `correlate.py` script | 2–3 hours | Medium | |
| 367 | | Configuration + tests | 1–2 hours | Medium | |
| 368 | | Workflow integration | 1 hour | Low | |
| 369 | | **Total** | **6–9 hours** | | |
| 370 | |
| 371 | --- |
| 372 | |
| 373 | ## Success Criteria |
| 374 | |
| 375 | ### Quantitative Metrics (measured after 8 weeks of operation) |
| 376 | |
| 377 | | Metric | Target | Measurement | |
| 378 | |--------|--------|-------------| |
| 379 | | RSS fetch success rate | ≥ 95% | Weeks with successful fetch / total weeks | |
| 380 | | Filter reduction ratio | 85–95% reduction | (Raw articles - filtered) / raw articles | |
| 381 | | Correlation hit rate | ≥ 5% of filtered articles | Articles with ≥1 GitHub match / filtered articles | |
| 382 | | False positive rate | ≤ 2% | Matches marked incorrect in manual review / total matches | |
| 383 | | Zero-noise weeks | 100% | Weeks where zero-correlation produces zero output noise | |
| 384 | | Enrichment value (subjective) | ≥ 3/5 quality rating | Monthly review: "Did correlations improve the digest?" | |
| 385 | |
| 386 | ### Qualitative Success Indicators |
| 387 | |
| 388 | - At least 1 "hype vs reality" insight per month that wouldn't exist without this signal |
| 389 | - Zero instances where TechCrunch noise degrades the digest quality |
| 390 | - The feature is invisible when it has nothing useful to contribute |
| 391 | |
| 392 | ### Failure Criteria (triggers feature removal) |
| 393 | |
| 394 | - Hit rate below 2% after 8 weeks → feature adds complexity without value |
| 395 | - False positives above 10% → feature introduces noise |
| 396 | - RSS feed breaks and stays broken for 4+ consecutive weeks → dependency unreliable |
| 397 | - Farnsworth (analyst) consistently ignores correlation data in analysis → no downstream value |
| 398 | |
| 399 | --- |
| 400 | |
| 401 | ## Phased Rollout |
| 402 | |
| 403 | ### Phase 1: RSS Collection Only (Week 1–2) |
| 404 | |
| 405 | - Implement `scripts/techcrunch_crawler.py` with 3-stage filtering |
| 406 | - Output `data/raw/{week}-external-news.json` (legacy `data/raw/{week}-techcrunch.json`) |
| 407 | - No integration with analysis — just collect and validate filter quality |
| 408 | - **Exit criteria:** Filter reduces volume by ≥ 80%, entity extraction produces meaningful tags |
| 409 | |
| 410 | ### Phase 2: Correlation Script (Week 3–4) |
| 411 | |
| 412 | - Implement `correlate.py` cross-reference logic |
| 413 | - Output `data/analyzed/{week}-correlations.json` |
| 414 | - Manual review of correlation quality for 2 weeks |
| 415 | - **Exit criteria:** Hit rate ≥ 3%, false positive rate ≤ 5% |
| 416 | |
| 417 | ### Phase 3: Analysis Integration (Week 5–6) |
| 418 | |
| 419 | - Farnsworth consumes `data/analyzed/{week}-correlations.json` in analysis prompt |
| 420 | - Correlation data appears in weekly digest when relevant |
| 421 | - **Exit criteria:** At least 1 correlation adds editorial value in 2 of 4 weeks |
| 422 | |
| 423 | ### Phase 4: Steady State (Week 7+) |
| 424 | |
| 425 | - Monitor success metrics |
| 426 | - Tune keyword lists and category filters based on actual hit rates |
| 427 | - Consider adding second source (HN) if TechCrunch proves the plugin model |
| 428 | |
| 429 | --- |
| 430 | |
| 431 | ## Open Questions |
| 432 | |
| 433 | | # | Question | Impact | Proposed Resolution | |
| 434 | |---|----------|--------|---------------------| |
| 435 | | OQ1 | Does TechCrunch's RSS feed include full article text or just excerpts? | Medium — affects entity extraction quality | Spike: inspect actual feed content. If excerpts only, extraction limited to title + summary. | |
| 436 | | OQ2 | How stable is TechCrunch's RSS feed over time? | Low — RSS is a mature standard | Monitor for 4 weeks before hard dependency. Breakage triggers graceful skip. | |
| 437 | | OQ3 | Should entity extraction use AI (LLM) or stay rule-based? | Medium — cost vs quality trade-off | Start rule-based (zero cost). Upgrade to LLM extraction in Phase 4 if hit rates are too low. | |
| 438 | | OQ4 | What's the right confidence threshold for surfacing correlations? | Medium — affects noise level | Start conservative (confidence ≥ 0.7). Lower if too few results after 4 weeks. | |
| 439 | | OQ5 | Should correlations appear as a separate section in the digest or inline? | Low — editorial decision | Defer to Farnsworth. Provide data; let analyst decide presentation. | |
| 440 | | OQ6 | Can we use GitHub's topic taxonomy to improve matching? | Medium — could boost hit rate | Investigate `GET /repos/{owner}/{repo}/topics` coverage during Phase 2. | |
| 441 | |
| 442 | --- |
| 443 | |
| 444 | ## Relationship to Other PRDs |
| 445 | |
| 446 | - **`docs/processed/PRD-topic-channels.md`:** Topic channels define per-domain crawling. TechCrunch integration is orthogonal — it enriches ANY topic channel with press signal. A `rust` channel could correlate TechCrunch Rust articles with Rust repo trends. |
| 447 | - **PRD-cost-estimation.md:** TechCrunch adds negligible cost ($0.10–$0.21/year in tokens). No budget concern. |
| 448 | - **Crawler plugin architecture:** TechCrunch is the first non-GitHub `DataSource` plugin, validating the extensible crawler design. |
| 449 | |
| 450 | --- |
| 451 | |
| 452 | *Archived as-is. This PRD reflects the original plan; the integration shipped on 2026-06-10. For canonical paths, field names, and interfaces, see the code (`scripts/techcrunch_crawler.py`, `config/external_news_sources.json`, `.github/workflows/crawl-and-publish.yml`).* |