feat: narrative divergence analysis with linked repo names (#136)
* decisions: add press context dual-mode rendering decision Merge decision from inbox: Press Context Dual-Mode Rendering (2026-05-19T20:50:22+02:00) This decision documents the implementation of dual-mode rendering in render_press_context.py to serve AI prompts (full data + instructions) and reader-facing fallback (clean narrative) separately. - Added reader_mode parameter to render_press_context() - Implemented correlations truncation to top 10 in reader mode - Stripped AI instructions from reader output - Replaced divergence instruction bullets with reader narrative PR #135 merged. 16 new tests, all 498 passing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: narrative divergence analysis with linked repo names Rewrites format_divergences() reader_mode to produce flowing prose: - Groups top topics by star count (capped at 6) - Uses repo name (without org) as link text: [repo](url) - TechCrunch articles linked by title - Closes with interpretive paragraph about gaps - AI prompt mode remains unchanged Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Juan Manuel Servera committed
May 19, 2026 at 21:30 UTC
d06fc50281c04b65260285a3190b1a989214d33a
4 files changed
+184
-43
.squad/agents/farnsworth/history.md
+1
@@ -27,3 +27,4 @@
27
- **2026-05-19T15:22:00+02:00:** Topic-aware prompt template implemented (Issue #63). Key architecture decisions: (1) Used `{{#IF_TOPIC}}`/`{{#IF_NO_TOPIC}}` conditional blocks rather than Jinja2 to keep the template readable as standalone markdown and avoid adding template engine dependencies. (2) Wisdom injection is two-tier — global wisdom from `.squad/identity/wisdom.md` (existing) plus per-topic wisdom from `topics/{id}/wisdom.md` (new). (3) Render script (`scripts/render_topic_prompt.py`) is zero-dependency (stdlib only, with optional PyYAML), so it works in any CI environment without pip install. (4) Backward compatibility guaranteed: when no `squadscope.topic.yml` exists, the template collapses cleanly to general-mode analysis identical to the existing `analyze-weekly.md` behavior.
28
- **2026-05-19T20:07:19+02:00:** Fixed correlator "0 repos" bug (PR #130). Root cause: `correlate.py` loaded repos via `raw_data.get("repos")` but `crawl.py` writes them under `new_repos` and `trending_repos`. Key paths: `scripts/correlate.py:320`, `scripts/crawl.py:857-858`. Lesson: when integrating scripts in a pipeline, always verify the producer's *actual output schema* against the consumer's expected input schema — don't assume key names match. The CI skill pattern ("test the wire") would have caught this if applied at integration time.
29
- **2026-05-19T20:50:22+02:00:** Press context dual-mode rendering implemented. Three reader-facing bugs fixed: (1) correlation list truncated to top-10 in reader mode (sorted by confidence desc, hype_risk severity); (2) `### Instructions` block stripped from reader output — it is AI prompt input only; (3) `#### Divergence Instructions` replaced with a plain narrative sentence for reader display. Architecture: `render_press_context.py` gained `reader_mode` kwarg propagated to `format_correlations_list(top_n=)` and `format_divergences(reader_mode=)`. `analyze_fallback._render_press_section_no_ai` now calls `_strip_ai_instructions()` which post-processes the pre-rendered file via regex — chosen because the fallback reads a file path, not raw JSON, so re-rendering from scratch would require threading data paths through. Key paths: `scripts/render_press_context.py`, `scripts/analyze_fallback.py`. 16 new tests added; 498 total pass.
30
+- **2026-05-19T21:24:54+02:00:** Divergence reader-mode upgraded from bullet lists to narrative paragraphs. `format_divergences(reader_mode=True)` now calls `_format_unpublicized_narrative()` and `_format_uncovered_narrative()` — deterministic template-driven prose (no LLM), capped at top 6 topics (by star count) and 5 uncovered trends. Repo links use only the repo name part after `/` (e.g., `[wasm-lib](https://github.com/org/wasm-lib)`). AI-mode format (reader_mode=False) is unchanged. Key insight: for reader-facing output, the data shape matters less than telling a coherent story — aggregate by topic, link to repos by short name, conclude with interpretation. 499 tests pass.
.squad/decisions/decisions.md
+14
@@ -86,3 +86,17 @@
86
- **Date:** 2026-05-19
87
- **Decision:** All future work organized into versioned milestones (v0.5, v0.6, etc.). PRDs are decomposed into issues, assigned to milestones, then moved to docs/processed/. This enables progress tracking and versioning.
88
- **Why:** User directive — makes work easier to follow and enables versioning.
89
+
90
+## 2026-05-19: Press Context Dual-Mode Rendering
91
+
92
+- **Owner:** Farnsworth
93
+- **Date:** 2026-05-19T20:50:22+02:00
94
+- **Status:** Implemented
95
+- **Decision:** Implement dual-mode rendering in `render_press_context.py` to serve AI prompts (full data + instructions) and reader-facing fallback (clean narrative) separately via `reader_mode` parameter and post-processing.
96
+- **Why:** The press context serves two audiences. AI prompts need full data and model instructions; reader-facing pages should not expose AI directives or 100+ repo lists.
97
+- **Changes:**
98
+ - `render_press_context(reader_mode=False)` — new kwarg. When True, limits correlations to top 10, strips `### Instructions` block, and passes reader_mode to `format_divergences()`
99
+ - `format_correlations_list(top_n=None)` — new kwarg. Truncates display and appends "…and N more repos"
100
+ - `format_divergences(reader_mode=False)` — new kwarg. Replaces instruction bullets with reader-friendly narrative
101
+ - `analyze_fallback._strip_ai_instructions(content)` — new helper. Applied in no-AI path to post-process rendered content
102
+- **Consequences:** AI prompt path unchanged (full instructions + list continue to model); no-AI fallback now produces clean reader output. 16 new tests cover truncation, sorting, instruction stripping, narrative injection. All 498 tests passing. PR #135 merged.
scripts/render_press_context.py
+157
-41
@@ -98,13 +98,131 @@ def format_correlations_list(correlations: list[dict], *, top_n: int | None = No
98
return "\n".join(lines)
99
100
101
+def _repo_link(full_name: str) -> str:
102
+ """Format a repo as a markdown link using only the repo name (after the slash)."""
103
+ repo_name = full_name.split("/")[-1]
104
+ return f"[{repo_name}](https://github.com/{full_name})"
105
+
106
+
107
+def _join_links(links: list[str]) -> str:
108
+ """Join a list of markdown links into a readable phrase."""
109
+ if len(links) == 1:
110
+ return links[0]
111
+ if len(links) == 2:
112
+ return f"{links[0]} and {links[1]}"
113
+ return f"{', '.join(links[:-1])}, and {links[-1]}"
114
+
115
+
116
+def _format_unpublicized_narrative(items: list[dict]) -> str:
117
+ """Generate narrative paragraph(s) for dev activity without press coverage."""
118
+ if not items:
119
+ return ""
120
+
121
+ # Sort topics by total stars, cap at 6
122
+ sorted_items = sorted(
123
+ items,
124
+ key=lambda x: sum(r.get("stars", 0) for r in x.get("github_repos", [])),
125
+ reverse=True,
126
+ )[:6]
127
+
128
+ topic_parts: list[tuple[str, list[str]]] = []
129
+ for item in sorted_items:
130
+ topic = item.get("topic", "unknown")
131
+ repos = sorted(
132
+ item.get("github_repos", []),
133
+ key=lambda r: r.get("stars", 0),
134
+ reverse=True,
135
+ )
136
+ links = [_repo_link(r["full_name"]) for r in repos[:3] if r.get("full_name")]
137
+ if links:
138
+ topic_parts.append((topic, links))
139
+
140
+ if not topic_parts:
141
+ return ""
142
+
143
+ # First paragraph: intro + first three topics
144
+ first_batch = topic_parts[:3]
145
+ fragments = [
146
+ f"{topic} saw activity with {_join_links(links)}"
147
+ for topic, links in first_batch
148
+ ]
149
+ para1 = (
150
+ "Developer activity this week shows momentum in areas the tech press isn't covering. "
151
+ + "; ".join(fragments)
152
+ + "."
153
+ )
154
+
155
+ paragraphs = [para1]
156
+
157
+ # Second paragraph for remaining topics
158
+ if len(topic_parts) > 3:
159
+ second_batch = topic_parts[3:]
160
+ fragments2 = [
161
+ f"{topic} with {_join_links(links)}" for topic, links in second_batch
162
+ ]
163
+ paragraphs.append("Additional activity surfaced in " + ", ".join(fragments2) + ".")
164
+
165
+ paragraphs.append(
166
+ "These gaps suggest that foundational developer tooling — the infrastructure "
167
+ "that powers daily workflows — grows through community word-of-mouth rather than press cycles."
168
+ )
169
+
170
+ return "\n\n".join(paragraphs)
171
+
172
+
173
+def _format_uncovered_narrative(items: list[dict]) -> str:
174
+ """Generate a narrative paragraph for tech trends without dev activity."""
175
+ if not items:
176
+ return ""
177
+
178
+ display = items[:5]
179
+
180
+ topic_names = [item.get("topic", "unknown") for item in display]
181
+
182
+ # Collect up to two article links across all topics
183
+ article_links: list[str] = []
184
+ for item in display:
185
+ for a in item.get("techcrunch_articles", [])[:1]:
186
+ title = a.get("title", "article")
187
+ url = a.get("url", "")
188
+ if url:
189
+ article_links.append(f"[{title}]({url})")
190
+ if len(article_links) >= 2:
191
+ break
192
+
193
+ if len(topic_names) == 1:
194
+ topics_str = topic_names[0]
195
+ elif len(topic_names) == 2:
196
+ topics_str = f"{topic_names[0]} and {topic_names[1]}"
197
+ else:
198
+ topics_str = f"{', '.join(topic_names[:-1])}, and {topic_names[-1]}"
199
+
200
+ if article_links:
201
+ if len(article_links) == 1:
202
+ article_str = f"Articles like {article_links[0]} generated buzz"
203
+ else:
204
+ article_str = (
205
+ f"Articles like {article_links[0]} and {article_links[1]} generated buzz"
206
+ )
207
+ else:
208
+ article_str = "Press articles generated buzz"
209
+
210
+ return (
211
+ f"TechCrunch heavily covered {topics_str} this week, but GitHub shows minimal "
212
+ f"matching developer activity. {article_str}, yet no significant new repositories "
213
+ f"emerged in these spaces — suggesting these are still in the narrative or "
214
+ f"announcement phase rather than implementation."
215
+ )
216
+
217
+
218
def format_divergences(divergences: dict, *, reader_mode: bool = False) -> str:
219
"""Format divergences section into markdown.
220
221
Args:
222
divergences: Divergence data dict.
106
- reader_mode: When True, replaces the AI instruction block with a
107
- reader-friendly conclusion sentence.
223
+ reader_mode: When True, renders narrative paragraphs with inline repo/article
224
+ links instead of raw bullet lists. When False (AI prompt mode),
225
+ the original bullet-list format is preserved unchanged.
226
"""
227
if not divergences:
228
return ""
@@ -117,47 +235,45 @@ def format_divergences(divergences: dict, *, reader_mode: bool = False) -> str:
235
236
lines = ["\n### Divergence Analysis\n"]
237
120
- # In reader mode, cap divergence lists to keep output concise
121
- max_items = 10 if reader_mode else None
122
-
123
- if uncovered:
124
- lines.append("#### 🔍 Tech Trends Without Dev Activity")
125
- lines.append("Topics heavily covered by TechCrunch with no matching GitHub repos:\n")
126
- display_uncovered = uncovered[:max_items] if max_items else uncovered
127
- for item in display_uncovered:
128
- topic = item.get("topic", "unknown")
129
- articles = item.get("techcrunch_articles", [])
130
- article_refs = ", ".join(
131
- f"[{a.get('title', 'article')}]({a.get('url', '')})"
132
- for a in articles[:3]
133
- )
134
- lines.append(f"- **{topic}**: {article_refs}")
135
- if max_items and len(uncovered) > max_items:
136
- lines.append(f"- …and {len(uncovered) - max_items} more tech trends without dev activity")
137
- lines.append("")
138
-
139
- if unpublicized:
140
- lines.append("#### 🚀 Dev Activity Without Press Coverage")
141
- lines.append("GitHub repos/trends with no matching TechCrunch coverage:\n")
142
- display_unpub = unpublicized[:max_items] if max_items else unpublicized
143
- for item in display_unpub:
144
- topic = item.get("topic", "unknown")
145
- repos = item.get("github_repos", [])
146
- repo_refs = ", ".join(
147
- f"{r.get('full_name', '?')} (⭐{r.get('stars', 0)})"
148
- for r in repos[:3]
149
- )
150
- lines.append(f"- **{topic}**: {repo_refs}")
151
- if max_items and len(unpublicized) > max_items:
152
- lines.append(f"- …and {len(unpublicized) - max_items} more dev topics without press coverage")
153
- lines.append("")
154
-
238
if reader_mode:
156
- lines.append(
157
- "These divergences highlight gaps between what the tech industry is reporting "
158
- "and what developers are actually building."
159
- )
239
+ # Narrative mode: flowing prose with inline links, no raw data dumps
240
+ if uncovered:
241
+ lines.append("#### 🔍 Tech Trends Without Dev Activity\n")
242
+ lines.append(_format_uncovered_narrative(uncovered))
243
+ lines.append("")
244
+
245
+ if unpublicized:
246
+ lines.append("#### 🚀 Dev Activity Without Press Coverage\n")
247
+ lines.append(_format_unpublicized_narrative(unpublicized))
248
+ lines.append("")
249
else:
250
+ # AI prompt mode: full raw data for model consumption — keep unchanged
251
+ if uncovered:
252
+ lines.append("#### 🔍 Tech Trends Without Dev Activity")
253
+ lines.append("Topics heavily covered by TechCrunch with no matching GitHub repos:\n")
254
+ for item in uncovered:
255
+ topic = item.get("topic", "unknown")
256
+ articles = item.get("techcrunch_articles", [])
257
+ article_refs = ", ".join(
258
+ f"[{a.get('title', 'article')}]({a.get('url', '')})"
259
+ for a in articles[:3]
260
+ )
261
+ lines.append(f"- **{topic}**: {article_refs}")
262
+ lines.append("")
263
+
264
+ if unpublicized:
265
+ lines.append("#### 🚀 Dev Activity Without Press Coverage")
266
+ lines.append("GitHub repos/trends with no matching TechCrunch coverage:\n")
267
+ for item in unpublicized:
268
+ topic = item.get("topic", "unknown")
269
+ repos = item.get("github_repos", [])
270
+ repo_refs = ", ".join(
271
+ f"{r.get('full_name', '?')} (⭐{r.get('stars', 0)})"
272
+ for r in repos[:3]
273
+ )
274
+ lines.append(f"- **{topic}**: {repo_refs}")
275
+ lines.append("")
276
+
277
lines.append("#### Divergence Instructions")
278
lines.append("Use divergences to identify:")
279
lines.append("- 🔮 Where industry is moving but devs haven't caught up")
tests/test_render_press_context.py
+12
-2
@@ -270,8 +270,18 @@ class TestFormatDivergencesReaderMode:
270
271
def test_reader_mode_has_narrative(self):
272
result = format_divergences(self._divergences(), reader_mode=True)
273
- assert "gaps between what the tech industry is reporting" in result
274
- assert "developers are actually building" in result
273
+ # Narrative prose paragraphs, no raw bullet lists
274
+ assert "TechCrunch heavily covered" in result
275
+ assert "Developer activity this week" in result
276
+ assert "- **quantum-computing**:" not in result
277
+ assert "- **wasm-tooling**:" not in result
278
+
279
+ def test_reader_mode_has_repo_links(self):
280
+ result = format_divergences(self._divergences(), reader_mode=True)
281
+ # Repo name as link, not full_name with stars
282
+ assert "[wasm-lib](https://github.com/org/wasm-lib)" in result
283
+ # Article link preserved
284
+ assert "[Quantum Leap](https://tc.com/q)" in result
285
286
def test_reader_mode_still_shows_data(self):
287
result = format_divergences(self._divergences(), reader_mode=True)