docs(specs): reverse-engineered spec for the rendered read view

new file mode 100644 index 0000000..fb8925b --- /dev/null +++ b/.agents/specs/rendered-read-view.md @@ -0,0 +1,221 @@ +# Spec: Rendered README and web code-reading view + +Status: implemented on `feature/rendered-read-view`. +Scope owner: `sigit-si` (Rails web app). + +This is a reverse-engineered specification of the repository read experience: +rendered Markdown, the syntax-highlighted blob view, permalinks and line +selection, the raw endpoint, and the rendering security boundary. It documents +the contract the code fulfills so the behavior can be re-implemented or verified +independently. Requirement IDs (RM-, BV-, PL-, RAW-, SEC-, PERF-) are referenced +by the test map at the end. + +## 1. Goal and non-goals + +Goal: make the web repo-browsing experience render code and Markdown instead of +serving them as plain text, with reading affordances comparable to a standard +code host (rendered READMEs, highlighting, line numbers, permalinks, raw bytes). + +Non-goals (not built here): pull-request review UI, issues, blame, code search, +diff view. This spec covers README rendering and the blob view only. + +## 2. Surfaces and routes + +Existing routes (unchanged by this work, listed for context): + +| Route | Controller action | Purpose | +|-------|-------------------|---------| +| `GET /:username/:repository` | `repositories#show` | repo landing page, renders README | +| `GET /:username/:repository/blob/:branch/*path` | `blobs#show` | file view | +| `GET /:username/:repository/raw/:branch/*path` | `blobs#raw` | exact bytes | + +`:branch` accepts any ref string, including a full commit SHA (used by +permalinks). `*path` is the repository-relative file path. + +## 3. Components + +| Component | File | Responsibility | +|-----------|------|----------------| +| `MarkdownRenderer` | `app/services/markdown_renderer.rb` | GFM to sanitized HTML | +| `SyntaxHighlighter` | `app/services/syntax_highlighter.rb` | per-line highlighted HTML | +| `GitRepositoryService` (extended) | `app/services/git_repository_service.rb` | `blob_sha`, `blob_size`, `commit_sha` | +| `BlobsController` | `app/controllers/blobs_controller.rb` | blob view, raw, large-file and binary handling, caching | +| `RepositoriesController` | `app/controllers/repositories_controller.rb` | README render, clone URLs | +| `line_selection_controller.js` | `app/javascript/controllers/` | line anchors, range select, SHA permalink | +| `clipboard_controller.js` | `app/javascript/controllers/` | copy code and clone URLs | +| `code_copy_controller.js` | `app/javascript/controllers/` | copy buttons on rendered Markdown code blocks | + +## 4. Markdown rendering (RM) + +`MarkdownRenderer.render(text, context: nil)` returns a sanitized HTML string. +`context` is a `MarkdownRenderer::Context` carrying `username`, `repository`, +`ref`, and `dir` (the directory of the file being rendered, "" for repo root). + +- RM-1: Blank or nil input returns an empty string. +- RM-2: GitHub-Flavored Markdown is supported: tables, task lists, strikethrough, + autolinks, fenced code, footnotes, headings. +- RM-3: Fenced code blocks are syntax-highlighted server-side with Rouge and + wrapped in `<pre class="highlight code-block">`. An unknown or absent language + falls back to plain text without raising. +- RM-4: Headings get a stable id slug and an in-page anchor link + (`<a class="heading-anchor" href="#slug">`). Repeated heading text yields + unique slugs (`name`, `name-1`, `name-2`). +- RM-5: Task list items (`- [ ]`, `- [x]`) render as disabled checkbox inputs on + `li.task-list-item`. +- RM-6: When `context` is present, relative links resolve to the blob view and + relative images resolve to the raw endpoint, both against the repo at `ref`: + - link `docs/guide.md` from root to `/:user/:repo/blob/:ref/docs/guide.md` + - image `assets/logo.png` from root to `/:user/:repo/raw/:ref/assets/logo.png` +- RM-7: Relative paths resolve against `context.dir`, collapsing `.` and `..` + without escaping the repository root. A traversal that would escape the root is + left unresolved. +- RM-8: Absolute URLs (any scheme), root-relative paths (`/...`), `mailto:`, and + in-page anchors (`#...`) are passed through unchanged. + +README discovery on the repo landing page is unchanged +(`GitRepositoryService.readme_content`): the first match of `README.md`, +`README.txt`, `README`, `readme.md`. Its rendered HTML is cached (see PERF-2). + +## 5. Blob view (BV) + +`BlobsController#show` pins the response and view formats to HTML so a file +extension in the path does not trigger Rails format negotiation, then classifies +the blob. + +- BV-1: A missing blob (`blob_sha` or `blob_size` nil) renders the 404 page. +- BV-2: A previewable image (`png jpg jpeg gif webp avif bmp ico svg`) at or + below `MAX_LOAD_BYTES` is shown inline as a base64 `data:` URI. SVG is safe + here because it loads through an `<img>` tag. +- BV-3: A file larger than `MAX_LOAD_BYTES` (5 MB) is not read into memory. The + view shows a size notice and a "View raw" link only (`@too_large`). +- BV-4: A binary file (NUL byte in the first 8192 bytes, or invalid UTF-8) shows + a "Binary file" notice with size and a "View raw" link. +- BV-5: A Markdown blob (`md markdown mdown mkd`) at or below + `MAX_MARKDOWN_BYTES` (512 KB) renders as Markdown by default. `?plain=1` forces + the highlighted source view, and a toggle link switches between the two. +- BV-6: Any other text file is shown with per-line Rouge highlighting, line + numbers, and a per-line anchor target (`tr#L<n>`). +- BV-7: A text file larger than `MAX_HIGHLIGHT_BYTES` (512 KB) is shown as + escaped plain text without highlighting, with a notice (`@highlight_skipped`). +- BV-8: A file with more than `MAX_DISPLAY_LINES` (5000) lines is truncated to + the first 5000 displayed lines, with a notice and a "View raw" link + (`@truncated`). +- BV-9: `SyntaxHighlighter.lines(content, filename:)` returns one HTML fragment + per source line. It tokenizes the whole file in one pass so multi-line + constructs highlight correctly, does not emit a trailing blank line when the + file ends in a newline, counts a final line with no trailing newline, and on + any lexer error falls back to HTML-escaped plain lines. + +## 6. Permalinks and line selection (PL) + +Implemented in `line_selection_controller.js`, driven by data attributes on the +blob table and line-number anchors. + +- PL-1: Clicking a line number sets the URL hash to `#L<n>` and highlights that + line (`.line-selected`). +- PL-2: Shift-clicking a second line selects the inclusive range and sets the + hash to `#L<lo>-L<hi>`, regardless of click order. +- PL-3: On load (and on `hashchange`), a `#L<n>` or `#L<lo>-L<hi>` hash + highlights the matching lines and scrolls the first into view. +- PL-4: "Copy permalink" copies an absolute URL built from + `data-line-selection-permalink-base-value` plus the current line selection. + The base path is `repository_blob_path(owner, repo, @commit_sha, file_path)`, + where `@commit_sha = GitRepositoryService.commit_sha(disk_path, branch)`. The + link is pinned to the commit the ref points at, so it keeps resolving to the + same content after the branch moves. + +## 7. Raw endpoint (RAW) + +`BlobsController#raw` serves the file bytes for install scripts, badges, and CI. + +- RAW-1: Returns the exact, unmodified bytes of the blob. +- RAW-2: Sets `X-Content-Type-Options: nosniff`. +- RAW-3: Raster image extensions are served with their real image content type; + every other file is served as `text/plain; charset=utf-8`. SVG is served as + text/plain (not `image/svg+xml`) so it cannot execute as active content on + direct navigation. +- RAW-4: A missing file returns 404 with no body. + +## 8. Security (SEC) + +- SEC-1: All rendered Markdown HTML passes through an allow-list sanitizer + (`Rails::HTML5::SafeListSanitizer`, falling back to HTML4) before reaching a + view. Only the tags and attributes in `MarkdownRenderer::ALLOWED_TAGS` and + `ALLOWED_ATTRIBUTES` survive. +- SEC-2: `<script>`, `<style>`, `<noscript>`, `<template>`, `<svg>`, `<math>`, + `<head>`, `<title>`, `<object>`, and `<embed>` blocks are pruned with their + contents before sanitization, so disallowed code never survives even as inert + text. +- SEC-3: Inline event handlers (`on*`) and dangerous-scheme URLs (for example + `javascript:`) are removed. Redcarpet runs with `safe_links_only`; the + sanitizer is the authoritative gate. +- SEC-4: Raw user content cannot run as active content on the app origin + (RAW-2, RAW-3). +- SEC-5: Result: malicious HTML or JS embedded in a README does not execute and + its payload text does not appear in the rendered output. + +## 9. Performance (PERF) + +- PERF-1: Highlighting and Markdown rendering run server-side. +- PERF-2: Rendered output is cached in `Rails.cache` keyed by the blob SHA + (content-addressed, so the cache is shared across refs and never goes stale). + Markdown cache keys also include `ref` and `dir` because relative-link + resolution depends on them. +- PERF-3: Blob size is read with `git cat-file -s` before the content is loaded, + so the large-file thresholds (BV-3, BV-7, BV-8) avoid loading or tokenizing + oversized files. + +## 10. Convenience controls + +- CV-1: The blob view has a "Copy" button that copies the file text (joined from + the per-line code cells via `clipboard_controller.js`). +- CV-2: Rendered Markdown code blocks get a hover "Copy" button injected by + `code_copy_controller.js`. +- CV-3: The repo page has a Clone control exposing HTTPS and SSH URLs, each with + a copy button. URLs come from `request.base_url`/`request.host` so they match + the serving host. + +## 11. Constants + +Defined on `BlobsController`: + +| Constant | Value | Requirement | +|----------|-------|-------------| +| `MAX_HIGHLIGHT_BYTES` | 512 KB | BV-7 | +| `MAX_MARKDOWN_BYTES` | 512 KB | BV-5 | +| `MAX_LOAD_BYTES` | 5 MB | BV-3 | +| `MAX_DISPLAY_LINES` | 5000 | BV-8 | +| `MARKDOWN_EXTENSIONS` | `md markdown mdown mkd` | BV-5 | +| `PREVIEW_IMAGE_TYPES` | raster set plus `svg` | BV-2 | +| `RAW_IMAGE_TYPES` | raster set, no `svg` | RAW-3 | + +## 12. Test map + +RSpec (`spec/`) is the project test framework. Tests assume the Rails +environment boots and a Postgres test database is available +(`bin/rails db:test:prepare`). + +| Requirement | Test | +|-------------|------| +| RM-1, RM-2, RM-3, RM-4, RM-5, RM-6, RM-7, RM-8 | `spec/services/markdown_renderer_spec.rb` | +| BV-9 | `spec/services/syntax_highlighter_spec.rb` | +| BV-5, BV-6, PL-4, SEC-5 | `spec/requests/blob_view_spec.rb` | +| RAW-1, RAW-2, RAW-3, RAW-4 | `spec/requests/raw_blob_spec.rb` | +| SEC-1, SEC-2, SEC-3 | `spec/services/markdown_renderer_spec.rb` (sanitization group) | + +Gaps not yet covered by automated tests (manual or future work): BV-3, BV-4, +BV-7, BV-8 (the large-file and binary branches), PL-1 to PL-3 (client-side line +selection), CV-1 to CV-3 (clipboard controllers). + +## 13. Verification + +``` +bundle exec rspec spec/services spec/requests # renderer, highlighter, raw, blob view +bundle exec rails server # then open /:user/:repo with a README +``` + +Manual checks on a running server: a README renders as HTML with working code +copy buttons; a source file shows highlighting and line numbers; `#L10` +highlights and scrolls; click and shift-click select lines and update the URL; +"Copy permalink" yields a SHA-pinned URL; `/raw/...` returns exact bytes with +`X-Content-Type-Options: nosniff`.