| 1 | # Spec: Rendered README and web code-reading view |
| 2 | |
| 3 | Status: implemented on `feature/rendered-read-view`. |
| 4 | Scope owner: `sigit-si` (Rails web app). |
| 5 | |
| 6 | This is a reverse-engineered specification of the repository read experience: |
| 7 | rendered Markdown, the syntax-highlighted blob view, permalinks and line |
| 8 | selection, the raw endpoint, and the rendering security boundary. It documents |
| 9 | the contract the code fulfills so the behavior can be re-implemented or verified |
| 10 | independently. Requirement IDs (RM-, BV-, PL-, RAW-, SEC-, PERF-) are referenced |
| 11 | by the test map at the end. |
| 12 | |
| 13 | ## 1. Goal and non-goals |
| 14 | |
| 15 | Goal: make the web repo-browsing experience render code and Markdown instead of |
| 16 | serving them as plain text, with reading affordances comparable to a standard |
| 17 | code host (rendered READMEs, highlighting, line numbers, permalinks, raw bytes). |
| 18 | |
| 19 | Non-goals (not built here): pull-request review UI, issues, blame, code search, |
| 20 | diff view. This spec covers README rendering and the blob view only. |
| 21 | |
| 22 | ## 2. Surfaces and routes |
| 23 | |
| 24 | Existing routes (unchanged by this work, listed for context): |
| 25 | |
| 26 | | Route | Controller action | Purpose | |
| 27 | |-------|-------------------|---------| |
| 28 | | `GET /:username/:repository` | `repositories#show` | repo landing page, renders README | |
| 29 | | `GET /:username/:repository/blob/:branch/*path` | `blobs#show` | file view | |
| 30 | | `GET /:username/:repository/raw/:branch/*path` | `blobs#raw` | exact bytes | |
| 31 | |
| 32 | `:branch` accepts any ref string, including a full commit SHA (used by |
| 33 | permalinks). `*path` is the repository-relative file path. |
| 34 | |
| 35 | ## 3. Components |
| 36 | |
| 37 | | Component | File | Responsibility | |
| 38 | |-----------|------|----------------| |
| 39 | | `MarkdownRenderer` | `app/services/markdown_renderer.rb` | GFM to sanitized HTML | |
| 40 | | `SyntaxHighlighter` | `app/services/syntax_highlighter.rb` | per-line highlighted HTML | |
| 41 | | `GitRepositoryService` (extended) | `app/services/git_repository_service.rb` | `blob_sha`, `blob_size`, `commit_sha` | |
| 42 | | `BlobsController` | `app/controllers/blobs_controller.rb` | blob view, raw, large-file and binary handling, caching | |
| 43 | | `RepositoriesController` | `app/controllers/repositories_controller.rb` | README render, clone URLs | |
| 44 | | `line_selection_controller.js` | `app/javascript/controllers/` | line anchors, range select, SHA permalink | |
| 45 | | `clipboard_controller.js` | `app/javascript/controllers/` | copy code and clone URLs | |
| 46 | | `code_copy_controller.js` | `app/javascript/controllers/` | copy buttons on rendered Markdown code blocks | |
| 47 | |
| 48 | ## 4. Markdown rendering (RM) |
| 49 | |
| 50 | `MarkdownRenderer.render(text, context: nil)` returns a sanitized HTML string. |
| 51 | `context` is a `MarkdownRenderer::Context` carrying `username`, `repository`, |
| 52 | `ref`, and `dir` (the directory of the file being rendered, "" for repo root). |
| 53 | |
| 54 | - RM-1: Blank or nil input returns an empty string. |
| 55 | - RM-2: GitHub-Flavored Markdown is supported: tables, task lists, strikethrough, |
| 56 | autolinks, fenced code, footnotes, headings. |
| 57 | - RM-3: Fenced code blocks are syntax-highlighted server-side with Rouge and |
| 58 | wrapped in `<pre class="highlight code-block">`. An unknown or absent language |
| 59 | falls back to plain text without raising. |
| 60 | - RM-4: Headings get a stable id slug and an in-page anchor link |
| 61 | (`<a class="heading-anchor" href="#slug">`). Repeated heading text yields |
| 62 | unique slugs (`name`, `name-1`, `name-2`). |
| 63 | - RM-5: Task list items (`- [ ]`, `- [x]`) render as disabled checkbox inputs on |
| 64 | `li.task-list-item`. |
| 65 | - RM-6: When `context` is present, relative links resolve to the blob view and |
| 66 | relative images resolve to the raw endpoint, both against the repo at `ref`: |
| 67 | - link `docs/guide.md` from root to `/:user/:repo/blob/:ref/docs/guide.md` |
| 68 | - image `assets/logo.png` from root to `/:user/:repo/raw/:ref/assets/logo.png` |
| 69 | - RM-7: Relative paths resolve against `context.dir`, collapsing `.` and `..` |
| 70 | without escaping the repository root. A traversal that would escape the root is |
| 71 | left unresolved. |
| 72 | - RM-8: Absolute URLs (any scheme), root-relative paths (`/...`), `mailto:`, and |
| 73 | in-page anchors (`#...`) are passed through unchanged. |
| 74 | |
| 75 | README discovery on the repo landing page is unchanged |
| 76 | (`GitRepositoryService.readme_content`): the first match of `README.md`, |
| 77 | `README.txt`, `README`, `readme.md`. Its rendered HTML is cached (see PERF-2). |
| 78 | |
| 79 | ## 5. Blob view (BV) |
| 80 | |
| 81 | `BlobsController#show` pins the response and view formats to HTML so a file |
| 82 | extension in the path does not trigger Rails format negotiation, then classifies |
| 83 | the blob. |
| 84 | |
| 85 | - BV-1: A missing blob (`blob_sha` or `blob_size` nil) renders the 404 page. |
| 86 | - BV-2: A previewable image (`png jpg jpeg gif webp avif bmp ico svg`) at or |
| 87 | below `MAX_LOAD_BYTES` is shown inline as a base64 `data:` URI. SVG is safe |
| 88 | here because it loads through an `<img>` tag. |
| 89 | - BV-3: A file larger than `MAX_LOAD_BYTES` (5 MB) is not read into memory. The |
| 90 | view shows a size notice and a "View raw" link only (`@too_large`). |
| 91 | - BV-4: A binary file (NUL byte in the first 8192 bytes, or invalid UTF-8) shows |
| 92 | a "Binary file" notice with size and a "View raw" link. |
| 93 | - BV-5: A Markdown blob (`md markdown mdown mkd`) at or below |
| 94 | `MAX_MARKDOWN_BYTES` (512 KB) renders as Markdown by default. `?plain=1` forces |
| 95 | the highlighted source view, and a toggle link switches between the two. |
| 96 | - BV-6: Any other text file is shown with per-line Rouge highlighting, line |
| 97 | numbers, and a per-line anchor target (`tr#L<n>`). |
| 98 | - BV-7: A text file larger than `MAX_HIGHLIGHT_BYTES` (512 KB) is shown as |
| 99 | escaped plain text without highlighting, with a notice (`@highlight_skipped`). |
| 100 | - BV-8: A file with more than `MAX_DISPLAY_LINES` (5000) lines is truncated to |
| 101 | the first 5000 displayed lines, with a notice and a "View raw" link |
| 102 | (`@truncated`). |
| 103 | - BV-9: `SyntaxHighlighter.lines(content, filename:)` returns one HTML fragment |
| 104 | per source line. It tokenizes the whole file in one pass so multi-line |
| 105 | constructs highlight correctly, does not emit a trailing blank line when the |
| 106 | file ends in a newline, counts a final line with no trailing newline, and on |
| 107 | any lexer error falls back to HTML-escaped plain lines. |
| 108 | |
| 109 | ## 6. Permalinks and line selection (PL) |
| 110 | |
| 111 | Implemented in `line_selection_controller.js`, driven by data attributes on the |
| 112 | blob table and line-number anchors. |
| 113 | |
| 114 | - PL-1: Clicking a line number sets the URL hash to `#L<n>` and highlights that |
| 115 | line (`.line-selected`). |
| 116 | - PL-2: Shift-clicking a second line selects the inclusive range and sets the |
| 117 | hash to `#L<lo>-L<hi>`, regardless of click order. |
| 118 | - PL-3: On load (and on `hashchange`), a `#L<n>` or `#L<lo>-L<hi>` hash |
| 119 | highlights the matching lines and scrolls the first into view. |
| 120 | - PL-4: "Copy permalink" copies an absolute URL built from |
| 121 | `data-line-selection-permalink-base-value` plus the current line selection. |
| 122 | The base path is `repository_blob_path(owner, repo, @commit_sha, file_path)`, |
| 123 | where `@commit_sha = GitRepositoryService.commit_sha(disk_path, branch)`. The |
| 124 | link is pinned to the commit the ref points at, so it keeps resolving to the |
| 125 | same content after the branch moves. |
| 126 | |
| 127 | ## 7. Raw endpoint (RAW) |
| 128 | |
| 129 | `BlobsController#raw` serves the file bytes for install scripts, badges, and CI. |
| 130 | |
| 131 | - RAW-1: Returns the exact, unmodified bytes of the blob. |
| 132 | - RAW-2: Sets `X-Content-Type-Options: nosniff`. |
| 133 | - RAW-3: Raster image extensions are served with their real image content type; |
| 134 | every other file is served as `text/plain; charset=utf-8`. SVG is served as |
| 135 | text/plain (not `image/svg+xml`) so it cannot execute as active content on |
| 136 | direct navigation. |
| 137 | - RAW-4: A missing file returns 404 with no body. |
| 138 | |
| 139 | ## 8. Security (SEC) |
| 140 | |
| 141 | - SEC-1: All rendered Markdown HTML passes through an allow-list sanitizer |
| 142 | (`Rails::HTML5::SafeListSanitizer`, falling back to HTML4) before reaching a |
| 143 | view. Only the tags and attributes in `MarkdownRenderer::ALLOWED_TAGS` and |
| 144 | `ALLOWED_ATTRIBUTES` survive. |
| 145 | - SEC-2: `<script>`, `<style>`, `<noscript>`, `<template>`, `<svg>`, `<math>`, |
| 146 | `<head>`, `<title>`, `<object>`, and `<embed>` blocks are pruned with their |
| 147 | contents before sanitization, so disallowed code never survives even as inert |
| 148 | text. |
| 149 | - SEC-3: Inline event handlers (`on*`) and dangerous-scheme URLs (for example |
| 150 | `javascript:`) are removed. Redcarpet runs with `safe_links_only`; the |
| 151 | sanitizer is the authoritative gate. |
| 152 | - SEC-4: Raw user content cannot run as active content on the app origin |
| 153 | (RAW-2, RAW-3). |
| 154 | - SEC-5: Result: malicious HTML or JS embedded in a README does not execute and |
| 155 | its payload text does not appear in the rendered output. |
| 156 | |
| 157 | ## 9. Performance (PERF) |
| 158 | |
| 159 | - PERF-1: Highlighting and Markdown rendering run server-side. |
| 160 | - PERF-2: Rendered output is cached in `Rails.cache` keyed by the blob SHA |
| 161 | (content-addressed, so the cache is shared across refs and never goes stale). |
| 162 | Markdown cache keys also include `ref` and `dir` because relative-link |
| 163 | resolution depends on them. |
| 164 | - PERF-3: Blob size is read with `git cat-file -s` before the content is loaded, |
| 165 | so the large-file thresholds (BV-3, BV-7, BV-8) avoid loading or tokenizing |
| 166 | oversized files. |
| 167 | |
| 168 | ## 10. Convenience controls |
| 169 | |
| 170 | - CV-1: The blob view has a "Copy" button that copies the file text (joined from |
| 171 | the per-line code cells via `clipboard_controller.js`). |
| 172 | - CV-2: Rendered Markdown code blocks get a hover "Copy" button injected by |
| 173 | `code_copy_controller.js`. |
| 174 | - CV-3: The repo page has a Clone control exposing HTTPS and SSH URLs, each with |
| 175 | a copy button. URLs come from `request.base_url`/`request.host` so they match |
| 176 | the serving host. |
| 177 | |
| 178 | ## 11. Constants |
| 179 | |
| 180 | Defined on `BlobsController`: |
| 181 | |
| 182 | | Constant | Value | Requirement | |
| 183 | |----------|-------|-------------| |
| 184 | | `MAX_HIGHLIGHT_BYTES` | 512 KB | BV-7 | |
| 185 | | `MAX_MARKDOWN_BYTES` | 512 KB | BV-5 | |
| 186 | | `MAX_LOAD_BYTES` | 5 MB | BV-3 | |
| 187 | | `MAX_DISPLAY_LINES` | 5000 | BV-8 | |
| 188 | | `MARKDOWN_EXTENSIONS` | `md markdown mdown mkd` | BV-5 | |
| 189 | | `PREVIEW_IMAGE_TYPES` | raster set plus `svg` | BV-2 | |
| 190 | | `RAW_IMAGE_TYPES` | raster set, no `svg` | RAW-3 | |
| 191 | |
| 192 | ## 12. Test map |
| 193 | |
| 194 | RSpec (`spec/`) is the project test framework. Tests assume the Rails |
| 195 | environment boots and a Postgres test database is available |
| 196 | (`bin/rails db:test:prepare`). |
| 197 | |
| 198 | | Requirement | Test | |
| 199 | |-------------|------| |
| 200 | | RM-1, RM-2, RM-3, RM-4, RM-5, RM-6, RM-7, RM-8 | `spec/services/markdown_renderer_spec.rb` | |
| 201 | | BV-9 | `spec/services/syntax_highlighter_spec.rb` | |
| 202 | | BV-5, BV-6, PL-4, SEC-5 | `spec/requests/blob_view_spec.rb` | |
| 203 | | RAW-1, RAW-2, RAW-3, RAW-4 | `spec/requests/raw_blob_spec.rb` | |
| 204 | | SEC-1, SEC-2, SEC-3 | `spec/services/markdown_renderer_spec.rb` (sanitization group) | |
| 205 | |
| 206 | Gaps not yet covered by automated tests (manual or future work): BV-3, BV-4, |
| 207 | BV-7, BV-8 (the large-file and binary branches), PL-1 to PL-3 (client-side line |
| 208 | selection), CV-1 to CV-3 (clipboard controllers). |
| 209 | |
| 210 | ## 13. Verification |
| 211 | |
| 212 | ``` |
| 213 | bundle exec rspec spec/services spec/requests # renderer, highlighter, raw, blob view |
| 214 | bundle exec rails server # then open /:user/:repo with a README |
| 215 | ``` |
| 216 | |
| 217 | Manual checks on a running server: a README renders as HTML with working code |
| 218 | copy buttons; a source file shows highlighting and line numbers; `#L10` |
| 219 | highlights and scrolls; click and shift-click select lines and update the URL; |
| 220 | "Copy permalink" yields a SHA-pinned URL; `/raw/...` returns exact bytes with |
| 221 | `X-Content-Type-Options: nosniff`. |