Set crawl cron to 06:53 UTC and align schedule docs with workflow (#322)
* Initial plan * Align weekly cron documentation with workflow schedule * Update weekly crawl schedule to 06:53 UTC --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Juan Manuel Servera <8036360+jmservera@users.noreply.github.com>
Copilot committed
Jun 9, 2026 at 08:58 UTC
4370e1c84b9d93e69e7a8a5ba9b72b69d3b51a1d
6 files changed
+11
-10
.github/workflows/crawl-and-publish.yml
+2
-1
@@ -2,7 +2,8 @@ name: Crawl and publish weekly data
2
3
on:
4
schedule:
5
- - cron: '54 7 * * 1'
5
+ # Mondays at 06:53 UTC (GitHub can delay scheduled workflows during peak load)
6
+ - cron: '53 6 * * 1'
7
workflow_dispatch:
8
inputs:
9
run_mode:
README.md
+2
-2
@@ -101,7 +101,7 @@ JSON Markdown Hugo Pages Improvements
101
102
## Automated weekly pipeline
103
104
-`.github/workflows/crawl-and-publish.yml` runs the full weekly automation every Monday at 08:00 UTC:
104
+`.github/workflows/crawl-and-publish.yml` runs the full weekly automation every Monday at 06:53 UTC:
105
106
1. **Crawl:** GitHub API → `data/raw/YYYY-WNN.json`; external RSS feeds → `data/raw/YYYY-WNN-external-news.json`
107
2. **Analyze:** Copilot → `data/analyzed/YYYY-WNN-summary.md`
@@ -112,7 +112,7 @@ JSON Markdown Hugo Pages Improvements
112
113
### Schedule and manual runs
114
115
-- **Automated schedule:** Monday 08:00 UTC (`0 8 * * 1`)
115
+- **Automated schedule:** Monday 06:53 UTC (`53 6 * * 1`)
116
- **Manual trigger:** `gh workflow run crawl-and-publish.yml` or GitHub Actions UI
117
118
### Required secrets
docs/PRD-techcrunch-integration.md
+2
-2
@@ -151,7 +151,7 @@ Each article gets a relevance score (0–10) based on title + description:
151
152
### The Problem: RSS is Real-Time, SquadScope is Weekly
153
154
-TechCrunch publishes continuously. SquadScope runs weekly (Monday 08:00 UTC). This creates a timing mismatch:
154
+TechCrunch publishes continuously. SquadScope runs weekly (Monday 06:53 UTC). This creates a timing mismatch:
155
156
- An article published Tuesday about Project X won't be seen until the following Monday
157
- By then, the GitHub star surge may have already peaked and fallen
@@ -160,7 +160,7 @@ TechCrunch publishes continuously. SquadScope runs weekly (Monday 08:00 UTC). Th
160
161
```
162
┌─────────────────────────────────────────────────────┐
163
-│ Monday 08:00 UTC: Crawl job runs │
163
+│ Monday 06:53 UTC: Crawl job runs │
164
│ │
165
│ 1. Fetch all RSS items from past 7 days │
166
│ 2. Filter (3-stage pipeline above) │
docs/operator-guide.md
+2
-2
@@ -123,7 +123,7 @@ This generates the `public/` directory with optimized static assets.
123
124
### Option A: Automatic scheduling (default)
125
126
-The pipeline runs automatically every **Monday at 08:00 UTC** via `.github/workflows/crawl-and-publish.yml`.
126
+The pipeline runs automatically every **Monday at 06:53 UTC** via `.github/workflows/crawl-and-publish.yml`.
127
128
You don't need to do anything. Go to your repo's **Actions** tab to monitor runs.
129
@@ -797,7 +797,7 @@ Edit `.github/workflows/crawl-and-publish.yml`:
797
```yaml
798
on:
799
schedule:
800
- - cron: '0 8 * * 1' # Change to your desired time (UTC)
800
+ - cron: '53 6 * * 1' # Change to your desired time (UTC)
801
```
802
803
Times are in UTC. Use crontab.guru to generate your schedule.
docs/pipeline-validation.md
+1
-1
@@ -5,7 +5,7 @@ This checklist validates the automated weekly workflow in `.github/workflows/cra
5
## Trigger and scheduling
6
7
- [x] `schedule` is enabled in the workflow.
8
-- [x] Cron is `0 8 * * 1`, which runs every Monday at 08:00 UTC.
8
+- [x] Cron is `53 6 * * 1`, which runs every Monday at 06:53 UTC.
9
- [x] `workflow_dispatch` is enabled for manual runs from the Actions tab or `gh workflow run crawl-and-publish.yml`.
10
- [x] `concurrency.group` is `weekly-crawl` with `cancel-in-progress: false`, so a second run waits instead of overlapping the active run.
11
docs/rollout-checklist.md
+2
-2
@@ -68,7 +68,7 @@ Use this checklist to verify your SquadScope instance is ready for production. C
68
- [ ] Verify workflow file: `.github/workflows/crawl-and-publish.yml` has schedule trigger:
69
```yaml
70
schedule:
71
- - cron: '0 8 * * 1' # Monday 08:00 UTC
71
+ - cron: '53 6 * * 1' # Monday 06:53 UTC
72
```
73
- [ ] Confirm schedule is correct for your timezone
74
@@ -79,7 +79,7 @@ Use this checklist to verify your SquadScope instance is ready for production. C
79
80
## First Automated Run
81
82
-- [ ] **Automated run completed (wait until next Monday 08:00 UTC or manual trigger)**
82
+- [ ] **Automated run completed (wait until next Monday 06:53 UTC or manual trigger)**
83
- [ ] Check Actions tab at scheduled time
84
- [ ] Verify workflow completed with green checkmark
85
- [ ] Monitor logs for any warnings or errors