main
md 6.06 KB

siGit Code GitHub App — automatic PR reviews

The hosted GitHub App behind siGit Code's pull-request reviews. Users install the App on their github.com repos; every reviewable pull-request event hits POST /github/webhooks, and GithubPrReviewJob posts a walkthrough summary plus line-level comments on the PR, powered by Onde Cloud inference.

How it fits together

github.com ──webhook──▶ GithubWebhooksController ──enqueue──▶ GithubPrReviewJob (Solid Queue)
                          │ verifies X-Hub-Signature-256        │ fetch PR + diff (GithubAppService)
                          │ syncs github_app_installations      │ budget/annotate (GithubReviewPrompt)
                          └ <10s, never does slow work          │ inference (OndeCloudService)
                                                                │ validate lines (GithubPatchIndex)
                                                                └ POST one review back to GitHub

State lives in two tables: github_app_installations (mirror of App installations, plus the cached 1-hour installation token) and github_pr_reviews (one row per repo + PR + head SHA — the idempotency guard, and the audit trail: status, skip reason, error, comment count).

The registered App

The App already exists — do not create a second one. It lives under the getsigit org:

  • Name: siGit Code, App ID 3492899
  • Public page: https://github.com/apps/sigit-code
  • Settings: github.com/organizations/getsigit/settings/apps/sigit-code
  • Homepage URL: https://sigit.si/code
  • Webhook URL: https://sigit.si/github/webhooks
  • Permissions: Pull requests: Read and write, Contents: Read-only, Metadata: Read-only (mandatory)
  • Subscribed events: Pull request. (installation and installation_repositories are always delivered to App webhooks; there is no checkbox for them.)

If you ever register a replacement from scratch, mirror those settings, set "Where can this App be installed" to Any account, generate a webhook secret (bin/rails secret | head -c 48), and Generate a private key (downloads a .pem) for the config below.

Configuration

Preferred in production: Rails credentials, because the private key is multi-line and prod env vars live in the git user's ~/.profile, which is single-line only:

github_app:
  app_id: "3492899"
  webhook_secret: "..."
  private_key: |
    -----BEGIN RSA PRIVATE KEY-----
    ...
    -----END RSA PRIVATE KEY-----

Edit credentials locally and commit — never on the server. config/credentials.yml.enc is git-tracked, and the deploy hook's git checkout -f main overwrites the server copy, silently deleting any server-side edit on the next deploy (this happened: the App credentials were added on the server on 2026-07-05 and wiped by the v1.1.0 deploy the next day). Run EDITOR=nano bin/rails credentials:edit in a local checkout that has config/master.key, paste the block, commit the .enc, and deploy.

When pasting the key, indent every PEM line to match private_key: |. Verify after deploy without booting the app: bin/rails runner 'puts GithubAppService.configured?' on the server — and expect POST /github/webhooks to return 401 (not 503) to an unsigned probe. Note the webhook_secret here is separate from stripe.webhook_secret; they share a key name but live under different top-level parents.

ENV fallback (dev, or single-line values): GITHUB_APP_ID, GITHUB_APP_WEBHOOK_SECRET, and GITHUB_APP_PRIVATE_KEY with literal \n escapes in place of newlines. See .env.example.

Switches:

  • SIGIT_GITHUB_REVIEWS_ENABLED=false — the kill switch. Stops new enqueues and drains already-queued jobs as no-ops (the job re-checks). Reviews are on by default whenever the App credentials are configured.
  • SIGIT_GITHUB_REVIEWS_MODEL — Onde tier for reviews (default onde-large).

Production rollout

The deploy is the usual git push to the bare-repo hook (see .agents/skills/deployment/SKILL.md), but this feature is the first user of Solid Queue, so on the first deploy:

  1. Verify the queue schema loaded: the hook's db:prepare should load db/queue_schema.rb into sigitsi_production_queue (that DB exists but was empty — a known issue in the deployment skill). Check with psql sigitsi_production_queue -c '\dt' — expect solid_queue_* tables.
  2. Amend the post-receive hook before relying on bin/jobs start: the hook starts a jobs supervisor on every deploy but never stops the previous one, so supervisors stack. Add something like pkill -f 'solid_queue' || true before the bin/jobs start line.
  3. App credentials are already in production credentials (see Configuration); restart Puma after the deploy so the new code picks them up.
  4. Install the App on a test repo, open a PR, and watch:
    • the jobs log (output-jobs.log) for the review run,
    • the github_pr_reviews row (status, skip_reason, error_message),
    • the PR itself for the summary + line comments.

Operational notes

  • GitHub never retries webhook deliveries (10-second timeout, fire-once). A missed delivery is recovered by the next push (synchronize) or manually: App settings → Advanced → Recent Deliveries → Redeliver (kept 30 days).
  • Skipped, not failed: drafts, closed/superseded PRs, PRs over the size budget (> GithubReviewPrompt::MAX_FILES files or nothing reviewable) are recorded as skipped with a skip_reason; over-budget PRs get a short explanatory comment instead of a review.
  • 422 from GitHub on review creation means a comment targeted a line outside the diff. The job degrades automatically: full review → summary-only review → plain issue comment. If these show up in the logs, look at GithubPatchIndex first.
  • One review per head SHA: redelivered webhooks and rapid pushes collapse via the unique (repo_full_name, pr_number, head_sha) index; a newer push supersedes queued reviews of older SHAs.
  • Reviews are ungated in v1 — any installation gets them. Billing or account linking is future work.