| 1 | gitformat-diff-hunks(5) |
| 2 | ======================= |
| 3 | |
| 4 | NAME |
| 5 | ---- |
| 6 | gitformat-diff-hunks - Precomputed diff hunk store format |
| 7 | |
| 8 | SYNOPSIS |
| 9 | -------- |
| 10 | [verse] |
| 11 | $GIT_DIR/objects/info/diff-hunks |
| 12 | |
| 13 | DESCRIPTION |
| 14 | ----------- |
| 15 | |
| 16 | The diff-hunks store memoizes diff hunk coordinates so that commands |
| 17 | that need them, such as `git log --stat` and linkgit:git-blame[1], can |
| 18 | skip running the diff algorithm (and, for blame, loading the blob |
| 19 | content; the summary formats still test each pair for binariness, |
| 20 | which can load the blobs). See |
| 21 | linkgit:git-diff-hunks[1] for how the store is filled and managed and the |
| 22 | configuration that controls it. |
| 23 | |
| 24 | The store is a single file, `$GIT_DIR/objects/info/diff-hunks`, written |
| 25 | in one pass and replaced atomically, so a reader sees either the old |
| 26 | file or the complete new one. |
| 27 | |
| 28 | Entries are keyed by the object IDs of the blob pair that was diffed |
| 29 | and by the diff algorithm and ignore flags (`xdl_opts`) the pair was |
| 30 | diffed under. A blob pair fully determines the diff input, so an entry |
| 31 | is valid regardless of which commits, branches, or index states the |
| 32 | pair was encountered in, and identical diffs performed in different |
| 33 | contexts share one entry. A reader whose `xdl_opts` differ from an |
| 34 | entry does not match it and falls back to computing the diff. |
| 35 | |
| 36 | FILE FORMAT |
| 37 | ----------- |
| 38 | |
| 39 | All multi-byte integers are stored in network byte order. The file is an |
| 40 | 8-byte header, the chunk table of contents and chunk data described in |
| 41 | linkgit:gitformat-chunk[5], and a trailing checksum. |
| 42 | |
| 43 | HEADER |
| 44 | ~~~~~~ |
| 45 | |
| 46 | - 4-byte signature: `DHPF` (diff-hunks precomputed format) |
| 47 | - 1-byte version number: currently 1 |
| 48 | - 1-byte hash version: 1 for SHA-1, 2 for SHA-256. A store whose hash |
| 49 | function differs from the repository's is ignored. |
| 50 | - 1-byte number of chunks |
| 51 | - 1-byte reserved |
| 52 | |
| 53 | CHUNK LOOKUP |
| 54 | ~~~~~~~~~~~~ |
| 55 | |
| 56 | A table of contents in the format of linkgit:gitformat-chunk[5], listing |
| 57 | the offset of each chunk. Both chunks below are required; a file missing |
| 58 | either is treated as corrupt. |
| 59 | |
| 60 | CHUNK DATA |
| 61 | ~~~~~~~~~~ |
| 62 | |
| 63 | DHIX (index):: |
| 64 | A sorted sequence of fixed-size entries. Each entry is the old |
| 65 | blob object ID, the new blob object ID, a 4-byte `xdl_opts` |
| 66 | value, and a 4-byte offset into the DHDT chunk. Entries are |
| 67 | sorted by old object ID, then new object ID, then `xdl_opts`, |
| 68 | so lookups can use binary search on the full key. |
| 69 | |
| 70 | DHDT (hunk data):: |
| 71 | For each index entry, at its offset: a 4-byte hunk count followed |
| 72 | by that many 16-byte hunk records. A hunk record is four 4-byte |
| 73 | values: old start, old count, new start, new count. |
| 74 | Starts are 0-based line numbers in the old and new blob; counts |
| 75 | are numbers of lines. The hunk count is at least 1: a record with |
| 76 | no hunks would claim the blob pair equivalent, which the store |
| 77 | never records, so readers treat such a record as invalid. |
| 78 | Identical hunk blocks are stored once: |
| 79 | distinct index entries whose recorded hunks are byte-for-byte |
| 80 | equal point at the same offset. |
| 81 | |
| 82 | TRAILER |
| 83 | ~~~~~~~ |
| 84 | |
| 85 | A checksum of all preceding bytes, computed with the repository hash |
| 86 | function. |
| 87 | |
| 88 | CORRECTNESS |
| 89 | ----------- |
| 90 | |
| 91 | Serving hunks from a valid store produces the same output as recomputing |
| 92 | the diff. The diff of a blob pair is not unique: a zero context length |
| 93 | triggers xdiff's common-tail trimming, which can pick a different but |
| 94 | equally valid set of hunks than an untrimmed diff does. A pair is |
| 95 | therefore recorded only when its trimmed and untrimmed diffs are |
| 96 | identical, which is the common case. Such an entry answers any consumer |
| 97 | at any context: git-blame replays its coordinates directly (it diffs at |
| 98 | zero context), and diffstat sums its per-hunk line counts, which the |
| 99 | context length does not change. The rare pair whose two diffs differ is |
| 100 | never recorded, so every consumer computes it. |
| 101 | |
| 102 | A store that cannot be used is ignored, and the consumer falls back to |
| 103 | computing the diff. Every offset and count read from the file is |
| 104 | bounds-checked, so a store that is missing, truncated, of an unknown |
| 105 | version, or of a different object hash does not change the diff output |
| 106 | and does not produce a diagnostic; `git diff-hunks verify` is what |
| 107 | reports corruption. |
| 108 | |
| 109 | The store is not re-checksummed on the read path. The writer fsyncs the |
| 110 | file (honoring `core.fsync`) and commits it atomically, so a |
| 111 | committed store is intact, the same trust model the commit-graph and |
| 112 | multi-pack-index use. The trailing checksum is recomputed by |
| 113 | `git diff-hunks verify` to detect corruption. |
| 114 | |
| 115 | The checksum detects corruption but does not prove who wrote the file. A |
| 116 | reader trusts the coordinates in a store that passes its checks, so |
| 117 | anything able to write a checksum-valid file at the store path can |
| 118 | influence output, the same as it could by writing objects directly. |
| 119 | |
| 120 | LIMITATIONS |
| 121 | ----------- |
| 122 | |
| 123 | - Hunk counts, offsets, and line coordinates are 32-bit, capping the |
| 124 | hunk data at 4 GiB and a single entry at roughly 268 million hunks. |
| 125 | A result whose coordinates cannot be represented is not recorded. |
| 126 | |
| 127 | GIT |
| 128 | --- |
| 129 | Part of the linkgit:git[1] suite |