@matheusbsbs / matheus-bagliano / commits / 42824db113

diff-hunks: add the store format, library, and command

Blame and "git log --stat" recover hunk coordinates by diffing blob pairs, and recompute them on every run. Add a cache of those coordinates at $GIT_DIR/objects/info/diff-hunks, beside the commit-graph, so a later run can look them up instead of decompressing the blobs and running xdiff again. The store is a single chunk-format file (see gitformat-chunk(5)): an 8-byte header, a DHIX index of fixed-size entries sorted by key, a DHDT segment of hunk records, and a trailing hash checksum. An entry is keyed by the two blob object ids and the xdl_opts the pair was diffed under, so a stored result is served only where that exact key recurs, independent of path. A zero-context diff trims unchanged lines from hunk edges and can pick a different but equally valid set of hunks than an untrimmed diff, so a recording caller stores a pair only when its trimmed and untrimmed diffs are identical; such an entry answers any consumer at any context, and the rare divergent pair is always computed. Identical hunk blocks are interned once and shared across keys. The library provides a reader (repo_diff_hunks_store and _replay, gated by core.diffHunks), loaded once and cached on the object database as the commit-graph is, and a writer that accumulates entries and flushes them in one atomic pass. An absent, corrupt, or disabled store reads as all misses. A record with no hunks is invalid too: replaying it would claim the pair equivalent, which the store never asserts, so it reads as a miss. Ordinary reads are diagnostic-free. Loading parses the chunk table through read_table_of_contents_quiet(), new in chunk-format, which prints nothing on a malformed table and takes the repository's hash algorithm rather than the_hash_algo, so the file is bounds-checked under the algorithm it is keyed by. The flush closes the repository's mmapped store and forgets that loading was attempted before committing the lockfile. A warming run that also reads may hold the file it is replacing mapped, and the rename must not land on a live mapping, which Windows refuses; a read after the flush then observes the committed file. commit-graph closes its graph before committing for the same reason. Writing is off by default, enabled per run by GIT_DIFF_HUNKS_WRITE or persistently by diffHunks.write, the environment winning. A writer seeds from the existing store, so a flush merges rather than replaces. The seed's checksum is verified first: a corrupt store is discarded, not rewritten with a fresh checksum verify could no longer catch. An entry that fails the shared diff_provider_check_hunk() or names no blob is dropped with a warning, since it would only ever read as a miss. A seed that discarded or dropped anything forces the flush even when the warming run computed nothing new. The writer fsyncs through a new diff-hunks core.fsync component. "git diff-hunks" inspects and manages the file: "verify" checks the checksum, chunk table, sort order, entry bounds, and every entry's hunk sequence against that shared check, so a store whose entries could only read as misses fails verify; "clear" removes the file. Later patches wire the readers and the writer into the diff and blame paths. Signed-off-by: Michael Montalbo <mmontalbo@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Michael Montalbo committed Aug 1, 2026 at 10:41 UTC 42824db1138dbe0e64d5c49b52202f0cd30fa967
24 files changed +1467 -18
.gitignore
+1
@@ -56,6 +56,7 @@
56 /git-diagnose
57 /git-diff
58 /git-diff-files
59 +/git-diff-hunks
60 /git-diff-index
61 /git-diff-pairs
62 /git-diff-tree
Documentation/Makefile
+1
@@ -33,6 +33,7 @@ MAN5_TXT += gitattributes.adoc
33 MAN5_TXT += gitformat-bundle.adoc
34 MAN5_TXT += gitformat-chunk.adoc
35 MAN5_TXT += gitformat-commit-graph.adoc
36 +MAN5_TXT += gitformat-diff-hunks.adoc
37 MAN5_TXT += gitformat-index.adoc
38 MAN5_TXT += gitformat-loose.adoc
39 MAN5_TXT += gitformat-pack.adoc
Documentation/config.adoc
+2
@@ -472,6 +472,8 @@ include::config/credential.adoc[]
472
473 include::config/diff.adoc[]
474
475 +include::config/diff-hunks.adoc[]
476 +
477 include::config/difftool.adoc[]
478
479 include::config/extensions.adoc[]
Documentation/config/core.adoc
+9 -1
@@ -670,12 +670,13 @@ but risks losing recent work in the event of an unclean system shutdown.
670 * `pack` hardens objects added to the repo in packfile form.
671 * `pack-metadata` hardens packfile bitmaps and indexes.
672 * `commit-graph` hardens the commit-graph file.
673 +* `diff-hunks` hardens the diff-hunks store.
674 * `index` hardens the index when it is modified.
675 * `objects` is an aggregate option that is equivalent to
676 `loose-object,pack`.
677 * `reference` hardens references modified in the repo.
678 * `derived-metadata` is an aggregate option that is equivalent to
678 - `pack-metadata,commit-graph`.
679 + `pack-metadata,commit-graph,diff-hunks`.
680 * `committed` is an aggregate option that is currently equivalent to
681 `objects`. This mode sacrifices some performance to ensure that work
682 that is committed to the repository with `git commit` or similar commands
@@ -750,6 +751,13 @@ core.commitGraph::
751 to parse the graph structure of commits. Defaults to true. See
752 linkgit:git-commit-graph[1] for more information.
753
754 +core.diffHunks::
755 + If true, then Git will consult the diff-hunks store (if it
756 + exists) to skip recomputing diff hunk coordinates in commands
757 + such as `git log --stat` and linkgit:git-blame[1]. This controls
758 + only reading; writing the store is controlled by `diffHunks.write`.
759 + See linkgit:git-diff-hunks[1] for more information. Defaults to true.
760 +
761 core.useReplaceRefs::
762 If set to `false`, behave as if the `--no-replace-objects`
763 option was given on the command line. See linkgit:git[1] and
Documentation/config/diff-hunks.adoc new
+8
@@ -0,0 +1,8 @@
1 +diffHunks.write::
2 + If true, diff-producing commands (`git diff`, `git log`,
3 + `git show`, or `git diff-tree` with a `--stat`, `--numstat`, or
4 + `--shortstat` format) write the hunks
5 + they compute to the diff-hunks store, filling it as a side effect.
6 + The `GIT_DIFF_HUNKS_WRITE` environment variable overrides this for
7 + a single invocation. Reading the store is controlled separately by
8 + `core.diffHunks`. See linkgit:git-diff-hunks[1]. Defaults to false.
Documentation/git-diff-hunks.adoc new
+146
@@ -0,0 +1,146 @@
1 +git-diff-hunks(1)
2 +=================
3 +
4 +NAME
5 +----
6 +git-diff-hunks - Inspect and manage the diff-hunks store
7 +
8 +SYNOPSIS
9 +--------
10 +[synopsis]
11 +git diff-hunks verify
12 +git diff-hunks clear
13 +
14 +DESCRIPTION
15 +-----------
16 +
17 +The diff-hunks store is a cache of diff hunk coordinates, the line
18 +ranges that changed between two blobs, so that commands
19 +which need them, such as linkgit:git-blame[1] and `git log` and `git diff`
20 +with the `--stat`, `--numstat`, and `--shortstat` formats, can skip
21 +running the diff algorithm, and blame can skip loading the blob
22 +content. (The summary formats still test each pair for binariness,
23 +which can load the blobs.)
24 +
25 +The store is a single file, `$GIT_DIR/objects/info/diff-hunks`. Reading is
26 +enabled by default; writing is off by default. A `git diff`, `git log`,
27 +`git show`, or `git diff-tree` that produces one of the stat formats
28 +fills the store as a side effect, but only when writing is enabled for
29 +that run (see "WARMING THE STORE" below), so ordinary reads never
30 +modify the repository. When the store does not have the pair, holds a
31 +different object hash, the file is unreadable, or an object replacement
32 +redirects one of the blobs, the consumer falls back to computing the
33 +diff. A store only speeds up these commands; it never changes their
34 +output.
35 +
36 +`git diff-hunks` itself only inspects and manages the file. See
37 +linkgit:gitformat-diff-hunks[5] for the file format.
38 +
39 +WARMING THE STORE
40 +-----------------
41 +
42 +The store is filled by running ordinary commands with writing enabled.
43 +Turn writing on for a single invocation with the `GIT_DIFF_HUNKS_WRITE`
44 +environment variable, or persistently with the `diffHunks.write`
45 +configuration; the environment variable takes precedence. A repository
46 +owner warms the store by running the diff-producing commands they care
47 +about with writing on, for example:
48 +
49 + GIT_DIFF_HUNKS_WRITE=1 git log --all --stat >/dev/null
50 +
51 +A `--stat` walk records one entry per blob pair;
52 +linkgit:git-blame[1] replays the coordinates and the summary formats
53 +sum the counts, so a single warming walk serves both.
54 +A warming run seeds from the existing store and rewrites the file
55 +with the newly computed pairs merged in, so a later run adds to what
56 +earlier runs recorded rather than discarding it.
57 +
58 +A walk records only the pairs it diffs. `git log --all --stat` diffs
59 +each commit against its first parent, so a blame that follows a
60 +merge's second parent computes those pairs itself: blame coverage is
61 +partial on history with merges. Warming with a walk that also diffs
62 +the other parents, for example `git log --all -m --stat`, raises
63 +blame coverage at the cost of a larger store and a longer warming
64 +run.
65 +
66 +COMMANDS
67 +--------
68 +
69 +`verify`::
70 + Check the integrity of the store: the trailing hash checksum, the
71 + chunk table of contents, the sort order of the index, and the
72 + bounds of every entry. Exits with non-zero status if the store is
73 + corrupt. An absent store is valid.
74 +
75 +`clear`::
76 + Remove the store file.
77 +
78 +CORRECTNESS
79 +-----------
80 +
81 +A stored result is interchangeable with a freshly computed one because an
82 +entry is keyed by the inputs that determine the diff:
83 +
84 +* the object IDs of the old and new blob, so a result is used only for
85 + the exact contents it was computed from; and
86 +* the diff algorithm and ignore flags (`xdl_opts`) the hunks were
87 + computed under. A lookup whose `xdl_opts` differ from a stored entry
88 + misses. This is why, for example, `blame -w` and
89 + `--diff-algorithm=<algorithm>` (including a per-path
90 + `diff.<driver>.algorithm`) do not reuse entries recorded under the
91 + default settings: they change `xdl_opts`.
92 +
93 +The context length is not part of the key because only trim-stable
94 +pairs are recorded: pairs whose zero-context trimmed diff and untrimmed
95 +diff are identical, so one entry answers blame (zero context) and the
96 +summary formats (any context) alike. The rare pair where
97 +the zero-context trimming optimization picks a different but
98 +equally valid set of hunks is
99 +never recorded and is always computed.
100 +
101 +Some options shape the hunks in ways the key does not express, so a
102 +diff that uses them is excluded from the store in both directions:
103 +break detection (`-B`), `--ignore-matching-lines` (`-I`), and
104 +`--anchored`. `--ignore-blank-lines` is different: it is an ignore
105 +flag and therefore part of the key, but the summary formats exclude
106 +it anyway, because it coalesces hunks differently between the code
107 +path that emits text and the one that replays coordinates, so a
108 +served answer would not match a store-less run.
109 +linkgit:git-blame[1] additionally does not
110 +consult the store for reverse blame, ignored revisions, or paths with a
111 +textconv driver.
112 +
113 +The store carries a trailing hash checksum, but readers do not
114 +re-checksum it on every load. As with the commit-graph and
115 +multi-pack-index, the writer fsyncs the file (honoring `core.fsync`) and
116 +commits it atomically, so a committed store is intact; every offset and
117 +count is still bounds-checked as it is read. The checksum is verified by
118 +`git diff-hunks verify`, not on the read path, so structural corruption
119 +that fails a bounds check is read as an absent entry, while a record
120 +that stays within bounds but whose bytes were altered is served until
121 +`verify` detects the mismatch.
122 +
123 +CONFIGURATION
124 +-------------
125 +
126 +`core.diffHunks`::
127 + Whether commands read the store. Defaults to true. See
128 + linkgit:git-config[1].
129 +
130 +`diffHunks.write`::
131 + Whether diff-producing commands write to the store. Defaults to
132 + false. The `GIT_DIFF_HUNKS_WRITE` environment variable overrides it
133 + for a single invocation. See linkgit:git-config[1].
134 +
135 +Writing the store honors the `core.fsync` configuration through the
136 +`diff-hunks` component; see linkgit:git-config[1].
137 +
138 +SEE ALSO
139 +--------
140 +linkgit:git-blame[1],
141 +linkgit:git-log[1],
142 +linkgit:gitformat-diff-hunks[5]
143 +
144 +GIT
145 +---
146 +Part of the linkgit:git[1] suite
Documentation/gitformat-diff-hunks.adoc new
+129
@@ -0,0 +1,129 @@
1 +gitformat-diff-hunks(5)
2 +=======================
3 +
4 +NAME
5 +----
6 +gitformat-diff-hunks - Precomputed diff hunk store format
7 +
8 +SYNOPSIS
9 +--------
10 +[verse]
11 +$GIT_DIR/objects/info/diff-hunks
12 +
13 +DESCRIPTION
14 +-----------
15 +
16 +The diff-hunks store memoizes diff hunk coordinates so that commands
17 +that need them, such as `git log --stat` and linkgit:git-blame[1], can
18 +skip running the diff algorithm (and, for blame, loading the blob
19 +content; the summary formats still test each pair for binariness,
20 +which can load the blobs). See
21 +linkgit:git-diff-hunks[1] for how the store is filled and managed and the
22 +configuration that controls it.
23 +
24 +The store is a single file, `$GIT_DIR/objects/info/diff-hunks`, written
25 +in one pass and replaced atomically, so a reader sees either the old
26 +file or the complete new one.
27 +
28 +Entries are keyed by the object IDs of the blob pair that was diffed
29 +and by the diff algorithm and ignore flags (`xdl_opts`) the pair was
30 +diffed under. A blob pair fully determines the diff input, so an entry
31 +is valid regardless of which commits, branches, or index states the
32 +pair was encountered in, and identical diffs performed in different
33 +contexts share one entry. A reader whose `xdl_opts` differ from an
34 +entry does not match it and falls back to computing the diff.
35 +
36 +FILE FORMAT
37 +-----------
38 +
39 +All multi-byte integers are stored in network byte order. The file is an
40 +8-byte header, the chunk table of contents and chunk data described in
41 +linkgit:gitformat-chunk[5], and a trailing checksum.
42 +
43 +HEADER
44 +~~~~~~
45 +
46 +- 4-byte signature: `DHPF` (diff-hunks precomputed format)
47 +- 1-byte version number: currently 1
48 +- 1-byte hash version: 1 for SHA-1, 2 for SHA-256. A store whose hash
49 + function differs from the repository's is ignored.
50 +- 1-byte number of chunks
51 +- 1-byte reserved
52 +
53 +CHUNK LOOKUP
54 +~~~~~~~~~~~~
55 +
56 +A table of contents in the format of linkgit:gitformat-chunk[5], listing
57 +the offset of each chunk. Both chunks below are required; a file missing
58 +either is treated as corrupt.
59 +
60 +CHUNK DATA
61 +~~~~~~~~~~
62 +
63 +DHIX (index)::
64 + A sorted sequence of fixed-size entries. Each entry is the old
65 + blob object ID, the new blob object ID, a 4-byte `xdl_opts`
66 + value, and a 4-byte offset into the DHDT chunk. Entries are
67 + sorted by old object ID, then new object ID, then `xdl_opts`,
68 + so lookups can use binary search on the full key.
69 +
70 +DHDT (hunk data)::
71 + For each index entry, at its offset: a 4-byte hunk count followed
72 + by that many 16-byte hunk records. A hunk record is four 4-byte
73 + values: old start, old count, new start, new count.
74 + Starts are 0-based line numbers in the old and new blob; counts
75 + are numbers of lines. The hunk count is at least 1: a record with
76 + no hunks would claim the blob pair equivalent, which the store
77 + never records, so readers treat such a record as invalid.
78 + Identical hunk blocks are stored once:
79 + distinct index entries whose recorded hunks are byte-for-byte
80 + equal point at the same offset.
81 +
82 +TRAILER
83 +~~~~~~~
84 +
85 +A checksum of all preceding bytes, computed with the repository hash
86 +function.
87 +
88 +CORRECTNESS
89 +-----------
90 +
91 +Serving hunks from a valid store produces the same output as recomputing
92 +the diff. The diff of a blob pair is not unique: a zero context length
93 +triggers xdiff's common-tail trimming, which can pick a different but
94 +equally valid set of hunks than an untrimmed diff does. A pair is
95 +therefore recorded only when its trimmed and untrimmed diffs are
96 +identical, which is the common case. Such an entry answers any consumer
97 +at any context: git-blame replays its coordinates directly (it diffs at
98 +zero context), and diffstat sums its per-hunk line counts, which the
99 +context length does not change. The rare pair whose two diffs differ is
100 +never recorded, so every consumer computes it.
101 +
102 +A store that cannot be used is ignored, and the consumer falls back to
103 +computing the diff. Every offset and count read from the file is
104 +bounds-checked, so a store that is missing, truncated, of an unknown
105 +version, or of a different object hash does not change the diff output
106 +and does not produce a diagnostic; `git diff-hunks verify` is what
107 +reports corruption.
108 +
109 +The store is not re-checksummed on the read path. The writer fsyncs the
110 +file (honoring `core.fsync`) and commits it atomically, so a
111 +committed store is intact, the same trust model the commit-graph and
112 +multi-pack-index use. The trailing checksum is recomputed by
113 +`git diff-hunks verify` to detect corruption.
114 +
115 +The checksum detects corruption but does not prove who wrote the file. A
116 +reader trusts the coordinates in a store that passes its checks, so
117 +anything able to write a checksum-valid file at the store path can
118 +influence output, the same as it could by writing objects directly.
119 +
120 +LIMITATIONS
121 +-----------
122 +
123 +- Hunk counts, offsets, and line coordinates are 32-bit, capping the
124 + hunk data at 4 GiB and a single entry at roughly 268 million hunks.
125 + A result whose coordinates cannot be represented is not recorded.
126 +
127 +GIT
128 +---
129 +Part of the linkgit:git[1] suite
Documentation/meson.build
+2
@@ -41,6 +41,7 @@ manpages = {
41 'git-describe.adoc' : 1,
42 'git-diagnose.adoc' : 1,
43 'git-diff-files.adoc' : 1,
44 + 'git-diff-hunks.adoc' : 1,
45 'git-diff-index.adoc' : 1,
46 'git-diff-pairs.adoc' : 1,
47 'git-difftool.adoc' : 1,
@@ -175,6 +176,7 @@ manpages = {
176 'gitformat-bundle.adoc' : 5,
177 'gitformat-chunk.adoc' : 5,
178 'gitformat-commit-graph.adoc' : 5,
179 + 'gitformat-diff-hunks.adoc' : 5,
180 'gitformat-index.adoc' : 5,
181 'gitformat-loose.adoc' : 5,
182 'gitformat-pack.adoc' : 5,
Makefile
+2
@@ -1159,6 +1159,7 @@ LIB_OBJS += diffcore-order.o
1159 LIB_OBJS += diffcore-pickaxe.o
1160 LIB_OBJS += diffcore-rename.o
1161 LIB_OBJS += diffcore-rotate.o
1162 +LIB_OBJS += diff-hunks.o
1163 LIB_OBJS += dir-iterator.o
1164 LIB_OBJS += dir.o
1165 LIB_OBJS += editor.o
@@ -1421,6 +1422,7 @@ BUILTIN_OBJS += builtin/credential.o
1422 BUILTIN_OBJS += builtin/describe.o
1423 BUILTIN_OBJS += builtin/diagnose.o
1424 BUILTIN_OBJS += builtin/diff-files.o
1425 +BUILTIN_OBJS += builtin/diff-hunks.o
1426 BUILTIN_OBJS += builtin/diff-index.o
1427 BUILTIN_OBJS += builtin/diff-pairs.o
1428 BUILTIN_OBJS += builtin/diff-tree.o
builtin.h
+1
@@ -175,6 +175,7 @@ int cmd_credential_store(int argc, const char **argv, const char *prefix, struct
175 int cmd_describe(int argc, const char **argv, const char *prefix, struct repository *repo);
176 int cmd_diagnose(int argc, const char **argv, const char *prefix, struct repository *repo);
177 int cmd_diff_files(int argc, const char **argv, const char *prefix, struct repository *repo);
178 +int cmd_diff_hunks(int argc, const char **argv, const char *prefix, struct repository *repo);
179 int cmd_diff_index(int argc, const char **argv, const char *prefix, struct repository *repo);
180 int cmd_diff(int argc, const char **argv, const char *prefix, struct repository *repo);
181 int cmd_diff_pairs(int argc, const char **argv, const char *prefix, struct repository *repo);
builtin/diff-hunks.c new
+53
@@ -0,0 +1,53 @@
1 +#include "builtin.h"
2 +#include "config.h"
3 +#include "diff-hunks.h"
4 +#include "gettext.h"
5 +#include "parse-options.h"
6 +#include "repository.h"
7 +
8 +static const char * const diff_hunks_usage[] = {
9 + N_("git diff-hunks verify"),
10 + N_("git diff-hunks clear"),
11 + NULL
12 +};
13 +
14 +static int cmd_diff_hunks_verify(int argc, const char **argv,
15 + const char *prefix UNUSED,
16 + struct repository *r)
17 +{
18 + struct option options[] = { OPT_END() };
19 +
20 + argc = parse_options(argc, argv, NULL, options, diff_hunks_usage, 0);
21 + if (argc)
22 + usage_with_options(diff_hunks_usage, options);
23 + return diff_hunks_verify(r) ? 1 : 0;
24 +}
25 +
26 +static int cmd_diff_hunks_clear(int argc, const char **argv,
27 + const char *prefix UNUSED,
28 + struct repository *r)
29 +{
30 + struct option options[] = { OPT_END() };
31 +
32 + argc = parse_options(argc, argv, NULL, options, diff_hunks_usage, 0);
33 + if (argc)
34 + usage_with_options(diff_hunks_usage, options);
35 + return diff_hunks_clear(r) ? 1 : 0;
36 +}
37 +
38 +int cmd_diff_hunks(int argc, const char **argv, const char *prefix,
39 + struct repository *repo)
40 +{
41 + parse_opt_subcommand_fn *fn = NULL;
42 + struct option options[] = {
43 + OPT_SUBCOMMAND("verify", &fn, cmd_diff_hunks_verify),
44 + OPT_SUBCOMMAND("clear", &fn, cmd_diff_hunks_clear),
45 + OPT_END()
46 + };
47 +
48 + repo_config(repo, git_default_config, NULL);
49 +
50 + argc = parse_options(argc, argv, prefix, options, diff_hunks_usage, 0);
51 +
52 + return fn(argc, argv, prefix, repo);
53 +}
chunk-format.c
+47 -15
@@ -101,12 +101,14 @@ cleanup:
101 return result;
102 }
103
104 -int read_table_of_contents(struct chunkfile *cf,
105 - const unsigned char *mfile,
106 - size_t mfile_size,
107 - uint64_t toc_offset,
108 - int toc_length,
109 - unsigned expected_alignment)
104 +static int read_table_of_contents_1(struct chunkfile *cf,
105 + const unsigned char *mfile,
106 + size_t mfile_size,
107 + uint64_t toc_offset,
108 + int toc_length,
109 + unsigned expected_alignment,
110 + const struct git_hash_algo *algo,
111 + int quiet)
112 {
113 int i;
114 uint32_t chunk_id;
@@ -121,12 +123,14 @@ int read_table_of_contents(struct chunkfile *cf,
123 chunk_offset = get_be64(table_of_contents + 4);
124
125 if (!chunk_id) {
124 - error(_("terminating chunk id appears earlier than expected"));
126 + if (!quiet)
127 + error(_("terminating chunk id appears earlier than expected"));
128 return 1;
129 }
130 if (chunk_offset % expected_alignment != 0) {
128 - error(_("chunk id %"PRIx32" not %d-byte aligned"),
129 - chunk_id, expected_alignment);
131 + if (!quiet)
132 + error(_("chunk id %"PRIx32" not %d-byte aligned"),
133 + chunk_id, expected_alignment);
134 return 1;
135 }
136
@@ -134,16 +138,18 @@ int read_table_of_contents(struct chunkfile *cf,
138 next_chunk_offset = get_be64(table_of_contents + 4);
139
140 if (next_chunk_offset < chunk_offset ||
137 - next_chunk_offset > mfile_size - the_hash_algo->rawsz) {
138 - error(_("improper chunk offset(s) %"PRIx64" and %"PRIx64""),
139 - chunk_offset, next_chunk_offset);
141 + next_chunk_offset > mfile_size - algo->rawsz) {
142 + if (!quiet)
143 + error(_("improper chunk offset(s) %"PRIx64" and %"PRIx64""),
144 + chunk_offset, next_chunk_offset);
145 return -1;
146 }
147
148 for (i = 0; i < cf->chunks_nr; i++) {
149 if (cf->chunks[i].id == chunk_id) {
145 - error(_("duplicate chunk ID %"PRIx32" found"),
146 - chunk_id);
150 + if (!quiet)
151 + error(_("duplicate chunk ID %"PRIx32" found"),
152 + chunk_id);
153 return -1;
154 }
155 }
@@ -156,13 +162,39 @@ int read_table_of_contents(struct chunkfile *cf,
162
163 chunk_id = get_be32(table_of_contents);
164 if (chunk_id) {
159 - error(_("final chunk has non-zero id %"PRIx32""), chunk_id);
165 + if (!quiet)
166 + error(_("final chunk has non-zero id %"PRIx32""), chunk_id);
167 return -1;
168 }
169
170 return 0;
171 }
172
173 +int read_table_of_contents(struct chunkfile *cf,
174 + const unsigned char *mfile,
175 + size_t mfile_size,
176 + uint64_t toc_offset,
177 + int toc_length,
178 + unsigned expected_alignment)
179 +{
180 + return read_table_of_contents_1(cf, mfile, mfile_size, toc_offset,
181 + toc_length, expected_alignment,
182 + the_hash_algo, 0);
183 +}
184 +
185 +int read_table_of_contents_quiet(struct chunkfile *cf,
186 + const unsigned char *mfile,
187 + size_t mfile_size,
188 + uint64_t toc_offset,
189 + int toc_length,
190 + unsigned expected_alignment,
191 + const struct git_hash_algo *algo)
192 +{
193 + return read_table_of_contents_1(cf, mfile, mfile_size, toc_offset,
194 + toc_length, expected_alignment,
195 + algo, 1);
196 +}
197 +
198 struct pair_chunk_data {
199 const unsigned char **p;
200 size_t *size;
chunk-format.h
+14
@@ -39,6 +39,20 @@ int read_table_of_contents(struct chunkfile *cf,
39 int toc_length,
40 unsigned expected_alignment);
41
42 +/*
43 + * Like read_table_of_contents(), for a reader that treats a malformed
44 + * table as an absent file rather than reporting it: nothing is printed
45 + * on failure, and the trailing-checksum bound is computed with the
46 + * given hash algorithm instead of the_hash_algo.
47 + */
48 +int read_table_of_contents_quiet(struct chunkfile *cf,
49 + const unsigned char *mfile,
50 + size_t mfile_size,
51 + uint64_t toc_offset,
52 + int toc_length,
53 + unsigned expected_alignment,
54 + const struct git_hash_algo *algo);
55 +
56 #define CHUNK_NOT_FOUND (-2)
57
58 /*
command-list.txt
+2
@@ -95,6 +95,7 @@ git-describe mainporcelain
95 git-diagnose ancillaryinterrogators
96 git-diff mainporcelain info
97 git-diff-files plumbinginterrogators
98 +git-diff-hunks plumbingmanipulators
99 git-diff-index plumbinginterrogators
100 git-diff-pairs plumbinginterrogators
101 git-diff-tree plumbinginterrogators
@@ -223,6 +224,7 @@ gitfaq guide
224 gitformat-bundle developerinterfaces
225 gitformat-chunk developerinterfaces
226 gitformat-commit-graph developerinterfaces
227 +gitformat-diff-hunks developerinterfaces
228 gitformat-index developerinterfaces
229 gitformat-pack developerinterfaces
230 gitformat-signature developerinterfaces
diff-hunks.c new
+916
@@ -0,0 +1,916 @@
1 +/*
2 + * Precomputed diff hunks, keyed by diff input.
3 + *
4 + * A single store at .git/objects/info/diff-hunks maps an (old blob,
5 + * new blob, xdl_opts) key to the hunk coordinates of diffing the pair.
6 + * The key determines the diff result (only trim-stable pairs are
7 + * recorded; see diff-hunks.h), so an entry is valid in any context it
8 + * recurs in, independent of path. Reading is on by default
9 + * (core.diffHunks); writing is off by default and enabled per run or
10 + * by configuration (see diff_hunks_write_enabled), so an ordinary
11 + * command populates the store only during a warming run the
12 + * repository owner opts into.
13 + *
14 + * File layout:
15 + * Header: "DHPF"(4) + version(1) + hash_version(1)
16 + * + num_chunks(1) + reserved(1)
17 + * Table of contents (chunk-format)
18 + * DHIX chunk: sorted entries, each
19 + * old_blob_oid, new_blob_oid, xdl_opts(4), hdat_offset(4)
20 + * DHDT chunk: per entry, num_hunks(4) followed by that many 16-byte hunks
21 + * Trailing hash checksum
22 + */
23 +#include "git-compat-util.h"
24 +#include "chunk-format.h"
25 +#include "config.h"
26 +#include "csum-file.h"
27 +#include "diff-hunks.h"
28 +#include "diff-provider-internal.h"
29 +#include "gettext.h"
30 +#include "hash.h"
31 +#include "hashmap.h"
32 +#include "lockfile.h"
33 +#include "odb.h"
34 +#include "path.h"
35 +#include "repo-settings.h"
36 +#include "repository.h"
37 +#include "strbuf.h"
38 +#include "wrapper.h"
39 +
40 +#define DIFF_HUNKS_SIGNATURE 0x44485046 /* "DHPF" */
41 +/*
42 + * Bump when the on-disk format changes, or when xdiff's emitted hunk
43 + * coordinates change for a fixed (blobs, xdl_opts) key: an old store
44 + * would otherwise serve stale hunks and change command output.
45 + */
46 +#define DIFF_HUNKS_VERSION 1
47 +#define DIFF_HUNKS_HEADER_SIZE 8
48 +
49 +#define DIFF_HUNKS_CHUNKID_INDEX 0x44484958 /* "DHIX" */
50 +#define DIFF_HUNKS_CHUNKID_DATA 0x44484454 /* "DHDT" */
51 +
52 +/*
53 + * Each hunk is 16 bytes on disk:
54 + * old_start(4) old_count(4) new_start(4) new_count(4)
55 + */
56 +#define DIFF_HUNKS_HUNK_SIZE (4 * sizeof(uint32_t))
57 +
58 +/*
59 + * Result of a store lookup: num_hunks records encoded in the store's mmap,
60 + * valid until the store is freed. Read them with nth_precomputed_hunk().
61 + */
62 +struct precomputed_entry {
63 + uint32_t num_hunks;
64 + const unsigned char *hunk_data;
65 +};
66 +
67 +/* Decode a single hunk from the raw on-disk format. */
68 +static inline void decode_precomputed_hunk(const unsigned char *data,
69 + struct precomputed_hunk *h)
70 +{
71 + h->old_start = get_be32(data);
72 + h->old_count = get_be32(data + 4);
73 + h->new_start = get_be32(data + 8);
74 + h->new_count = get_be32(data + 12);
75 +}
76 +
77 +/* Decode the nth hunk of a lookup result into *h. */
78 +static inline void nth_precomputed_hunk(const struct precomputed_entry *e,
79 + uint32_t n, struct precomputed_hunk *h)
80 +{
81 + decode_precomputed_hunk(e->hunk_data + (size_t)n * DIFF_HUNKS_HUNK_SIZE, h);
82 +}
83 +
84 +/* Byte length of the (old_oid, new_oid, xdl_opts) lookup key. */
85 +static size_t store_index_key_size(const struct git_hash_algo *algo)
86 +{
87 + return 2 * algo->rawsz + sizeof(uint32_t);
88 +}
89 +
90 +/* Index entry: the lookup key followed by the 4-byte offset into DHDT. */
91 +static size_t store_index_entry_size(const struct git_hash_algo *algo)
92 +{
93 + return store_index_key_size(algo) + sizeof(uint32_t);
94 +}
95 +
96 +/*
97 + * The smallest a valid store file can be: the header, a table of contents
98 + * with one entry per chunk plus a terminating entry, and the trailing
99 + * checksum.
100 + */
101 +static size_t store_min_size(const struct git_hash_algo *algo,
102 + uint8_t num_chunks)
103 +{
104 + size_t toc_size = (num_chunks + 1) * CHUNK_TOC_ENTRY_SIZE;
105 +
106 + return DIFF_HUNKS_HEADER_SIZE + toc_size + algo->rawsz;
107 +}
108 +
109 +/*
110 + * Decode an index entry's key into pointers to the two oids and the
111 + * xdl_opts value (on-disk: old_oid, new_oid, then xdl_opts as a
112 + * big-endian uint32).
113 + */
114 +static void decode_store_index_key(const unsigned char *entry, unsigned int rawsz,
115 + const unsigned char **old_hash,
116 + const unsigned char **new_hash,
117 + uint32_t *xdl_opts)
118 +{
119 + *old_hash = entry;
120 + *new_hash = entry + rawsz;
121 + *xdl_opts = get_be32(entry + 2 * rawsz);
122 +}
123 +
124 +/* The DHDT offset stored in an index entry, in the field after its key. */
125 +static uint32_t index_entry_hdat_offset(const unsigned char *entry, size_t keysz)
126 +{
127 + return get_be32(entry + keysz);
128 +}
129 +
130 +static char *diff_hunks_store_path(struct repository *r)
131 +{
132 + return xstrfmt("%s/info/diff-hunks", repo_get_object_directory(r));
133 +}
134 +
135 +struct diff_hunks_store {
136 + const unsigned char *data;
137 + size_t data_len;
138 + const struct git_hash_algo *hash_algo;
139 + const unsigned char *index;
140 + uint32_t num_entries;
141 + const unsigned char *hdat;
142 + size_t hdat_size;
143 +};
144 +
145 +static void free_store(struct diff_hunks_store *s)
146 +{
147 + if (!s)
148 + return;
149 + if (s->data)
150 + munmap((void *)s->data, s->data_len);
151 + free(s);
152 +}
153 +
154 +/*
155 + * Open, mmap, and parse the store at fname. Returns the parsed store
156 + * or NULL on any error. The diff output is unaffected either way;
157 + * corruption is reported by verify, not treated as fatal here.
158 + */
159 +static struct diff_hunks_store *load_store_at(
160 + const struct git_hash_algo *repo_algo, const char *fname)
161 +{
162 + struct diff_hunks_store *s;
163 + struct chunkfile *cf;
164 + int fd;
165 + struct stat st;
166 + void *data;
167 + const unsigned char *p;
168 + uint8_t num_chunks;
169 + size_t index_size, entry_size, data_len;
170 +
171 + fd = git_open(fname);
172 + if (fd < 0)
173 + return NULL;
174 + if (fstat(fd, &st) || st.st_size < DIFF_HUNKS_HEADER_SIZE) {
175 + close(fd);
176 + return NULL;
177 + }
178 + data_len = xsize_t(st.st_size);
179 + data = xmmap(NULL, data_len, PROT_READ, MAP_PRIVATE, fd, 0);
180 + close(fd);
181 + p = data;
182 +
183 + num_chunks = p[6];
184 +
185 + /*
186 + * Reject a file that is not a readable store: wrong signature,
187 + * version, or object hash, or too small to hold the table of
188 + * contents that read_table_of_contents() walks (it dereferences
189 + * each entry before range-checking its offset).
190 + */
191 + if (get_be32(p) != DIFF_HUNKS_SIGNATURE ||
192 + p[4] != DIFF_HUNKS_VERSION ||
193 + p[5] != oid_version(repo_algo) ||
194 + data_len < store_min_size(repo_algo, num_chunks)) {
195 + munmap(data, data_len);
196 + return NULL;
197 + }
198 +
199 + /*
200 + * The trailing checksum is not verified here: the writer fsyncs
201 + * and commits atomically, so a committed file is intact, and
202 + * every record is bounds-checked at read (see precomputed_entry_at).
203 + * The checksum is checked separately, by diff_hunks_verify().
204 + */
205 +
206 + CALLOC_ARRAY(s, 1);
207 + s->data = data;
208 + s->data_len = data_len;
209 + s->hash_algo = repo_algo;
210 +
211 + cf = init_chunkfile(NULL);
212 + if (read_table_of_contents_quiet(cf, p, data_len,
213 + DIFF_HUNKS_HEADER_SIZE, num_chunks, 1,
214 + repo_algo) ||
215 + pair_chunk(cf, DIFF_HUNKS_CHUNKID_INDEX, &s->index, &index_size) ||
216 + pair_chunk(cf, DIFF_HUNKS_CHUNKID_DATA, &s->hdat, &s->hdat_size)) {
217 + free_chunkfile(cf);
218 + goto corrupt;
219 + }
220 + free_chunkfile(cf);
221 +
222 + entry_size = store_index_entry_size(s->hash_algo);
223 + if (index_size % entry_size)
224 + goto corrupt;
225 + s->num_entries = index_size / entry_size;
226 + return s;
227 +
228 +corrupt:
229 + free_store(s);
230 + return NULL;
231 +}
232 +
233 +static struct diff_hunks_store *diff_hunks_store_load(struct repository *r)
234 +{
235 + struct diff_hunks_store *s;
236 + char *fname;
237 +
238 + prepare_repo_settings(r);
239 + if (!r->settings.core_diff_hunks)
240 + return NULL;
241 +
242 + fname = diff_hunks_store_path(r);
243 + s = load_store_at(r->hash_algo, fname);
244 + free(fname);
245 + return s;
246 +}
247 +
248 +struct diff_hunks_store *repo_diff_hunks_store(struct repository *r)
249 +{
250 + if (!r->objects)
251 + return NULL;
252 + if (r->objects->diff_hunks_store_attempted)
253 + return r->objects->diff_hunks_store;
254 + r->objects->diff_hunks_store_attempted = 1;
255 + r->objects->diff_hunks_store = diff_hunks_store_load(r);
256 + return r->objects->diff_hunks_store;
257 +}
258 +
259 +void close_diff_hunks_store(struct object_database *o)
260 +{
261 + if (!o->diff_hunks_store)
262 + return;
263 + free_store(o->diff_hunks_store);
264 + o->diff_hunks_store = NULL;
265 +}
266 +
267 +/*
268 + * Fill *out with the hunk record at offset in the data chunk, and return
269 + * 1 if the record is in bounds, 0 otherwise. The read path does not
270 + * re-verify the checksum, and a valid checksum would not bound the count
271 + * anyway, so a read must call this and use *out only when it returns
272 + * non-zero.
273 + *
274 + * A record is a be32 hunk count followed by that many DIFF_HUNKS_HUNK_SIZE
275 + * hunks. "remaining" tracks the bytes from offset to the end of the data
276 + * chunk: it must hold the count, and after the count is consumed it must
277 + * hold every hunk. The bounds are written as subtraction and division
278 + * (never addition or multiplication) so a crafted offset or count cannot
279 + * overflow them.
280 + */
281 +static int precomputed_entry_at(const struct diff_hunks_store *s,
282 + uint32_t offset, struct precomputed_entry *out)
283 +{
284 + size_t remaining;
285 + uint32_t num_hunks;
286 +
287 + if (offset >= s->hdat_size)
288 + return 0;
289 + remaining = s->hdat_size - offset;
290 + if (remaining < sizeof(uint32_t))
291 + return 0;
292 +
293 + num_hunks = get_be32(s->hdat + offset);
294 + remaining -= sizeof(uint32_t);
295 + if (num_hunks > remaining / DIFF_HUNKS_HUNK_SIZE)
296 + return 0;
297 +
298 + out->num_hunks = num_hunks;
299 + out->hunk_data = s->hdat + offset + sizeof(uint32_t);
300 + return 1;
301 +}
302 +
303 +struct lookup_key {
304 + const struct object_id *old_oid;
305 + const struct object_id *new_oid;
306 + int xdl_opts;
307 + unsigned int rawsz;
308 +};
309 +
310 +/*
311 + * The store's total order over (old_oid, new_oid, xdl_opts), defined
312 + * once so the write-side sort (writer_entry_cmp) and the read-side
313 + * search (store_bsearch_cmp) order the keys identically.
314 + */
315 +static int cmp_store_index_key(const unsigned char *old_a, const unsigned char *new_a,
316 + uint32_t opts_a,
317 + const unsigned char *old_b, const unsigned char *new_b,
318 + uint32_t opts_b, unsigned int rawsz)
319 +{
320 + int cmp = memcmp(old_a, old_b, rawsz);
321 + if (!cmp)
322 + cmp = memcmp(new_a, new_b, rawsz);
323 + if (!cmp)
324 + cmp = (opts_a > opts_b) - (opts_a < opts_b);
325 + return cmp;
326 +}
327 +
328 +static int store_bsearch_cmp(const void *key, const void *entry_ptr)
329 +{
330 + const struct lookup_key *k = key;
331 + const unsigned char *old_hash, *new_hash;
332 + uint32_t xdl_opts;
333 +
334 + decode_store_index_key(entry_ptr, k->rawsz, &old_hash, &new_hash,
335 + &xdl_opts);
336 + return cmp_store_index_key(k->old_oid->hash, k->new_oid->hash,
337 + (uint32_t)k->xdl_opts,
338 + old_hash, new_hash, xdl_opts, k->rawsz);
339 +}
340 +
341 +static int store_get_one(struct diff_hunks_store *s, const struct lookup_key *key,
342 + struct precomputed_entry *out)
343 +{
344 + size_t entry_size = store_index_entry_size(s->hash_algo);
345 + const unsigned char *found;
346 +
347 + found = bsearch(key, s->index, s->num_entries, entry_size,
348 + store_bsearch_cmp);
349 + if (!found)
350 + return 0;
351 + return precomputed_entry_at(s,
352 + index_entry_hdat_offset(found, store_index_key_size(s->hash_algo)),
353 + out);
354 +}
355 +
356 +static int diff_hunks_store_get(struct diff_hunks_store *s,
357 + const struct object_id *old_oid,
358 + const struct object_id *new_oid,
359 + int xdl_opts,
360 + struct precomputed_entry *out)
361 +{
362 + struct lookup_key key;
363 +
364 + if (!s)
365 + return 0;
366 + /* The null OID names no blob and cannot key an entry. */
367 + if (is_null_oid(old_oid) || is_null_oid(new_oid))
368 + return 0;
369 +
370 + key.old_oid = old_oid;
371 + key.new_oid = new_oid;
372 + key.xdl_opts = xdl_opts;
373 + key.rawsz = s->hash_algo->rawsz;
374 +
375 + return store_get_one(s, &key, out);
376 +}
377 +
378 +/*
379 + * A recorded hunk sequence must satisfy the provider interface's
380 + * shared check (diff_provider_check_hunk()) before it may be replayed:
381 + * coordinates decode from be32 into long, which is 32-bit on some
382 + * platforms, so a crafted value can decode negative or out of order.
383 + * An entry that fails reads as a miss, so the caller recomputes.
384 + */
385 +static int replayable_hunks(const struct precomputed_entry *e)
386 +{
387 + struct diff_provider_hunks_check c = { 0 };
388 + uint32_t i;
389 +
390 + /*
391 + * Replaying a record with no hunks would assert the blob pair
392 + * equivalent, a claim the store must never make (the writer
393 + * refuses to record one), so such a record is invalid.
394 + */
395 + if (!e->num_hunks)
396 + return 0;
397 + for (i = 0; i < e->num_hunks; i++) {
398 + struct precomputed_hunk h;
399 + nth_precomputed_hunk(e, i, &h);
400 + if (diff_provider_check_hunk(&c, h.old_start, h.old_count,
401 + h.new_start, h.new_count))
402 + return 0;
403 + }
404 + return 1;
405 +}
406 +
407 +int diff_hunks_replay(struct diff_hunks_store *s,
408 + const struct object_id *old_oid,
409 + const struct object_id *new_oid,
410 + int xdl_opts,
411 + xdl_emit_hunk_consume_func_t hunk_func, void *cb_data)
412 +{
413 + struct precomputed_entry e;
414 + uint32_t i;
415 +
416 + if (!diff_hunks_store_get(s, old_oid, new_oid, xdl_opts, &e) ||
417 + !replayable_hunks(&e))
418 + return 0;
419 + for (i = 0; i < e.num_hunks; i++) {
420 + struct precomputed_hunk h;
421 + nth_precomputed_hunk(&e, i, &h);
422 + hunk_func(h.old_start, h.old_count,
423 + h.new_start, h.new_count, cb_data);
424 + }
425 + return 1;
426 +}
427 +
428 +/* Validate one store file. Returns 0 if valid or absent, -1 on any error. */
429 +static int verify_store_at(struct repository *r, const char *fname)
430 +{
431 + struct diff_hunks_store *s;
432 + size_t entry_size;
433 + uint32_t i;
434 + int fd;
435 + int ret = 0;
436 +
437 + /*
438 + * A file that cannot be opened is not evidence of corruption:
439 + * report the open error, and reserve the corruption diagnostics
440 + * below for a file that was read and failed to parse.
441 + */
442 + fd = git_open(fname);
443 + if (fd < 0) {
444 + if (errno == ENOENT)
445 + return 0; /* absent is valid */
446 + return error_errno(_("unable to open diff-hunks store %s"),
447 + fname);
448 + }
449 + close(fd);
450 + s = load_store_at(r->hash_algo, fname);
451 + if (!s)
452 + return error(_("diff-hunks store failed to load (corrupt "
453 + "header or hash mismatch): %s"), fname);
454 + if (!hashfile_checksum_valid(r->hash_algo, s->data, s->data_len)) {
455 + error(_("diff-hunks store has incorrect checksum and is "
456 + "likely corrupt: %s"), fname);
457 + free_store(s);
458 + return -1;
459 + }
460 +
461 + entry_size = store_index_entry_size(s->hash_algo);
462 + for (i = 0; i < s->num_entries; i++) {
463 + const unsigned char *ep = s->index + st_mult(entry_size, i);
464 + size_t keysz = store_index_key_size(s->hash_algo);
465 + uint32_t offset = index_entry_hdat_offset(ep, keysz);
466 + struct precomputed_entry pe;
467 +
468 + /*
469 + * Keyed by (old_oid, new_oid, xdl_opts), increasing. memcmp
470 + * matches cmp_store_index_key's integer comparison of
471 + * xdl_opts because it is non-negative, so its big-endian
472 + * bytes order the same as its value.
473 + */
474 + if (i > 0 && memcmp(ep - entry_size, ep, keysz) >= 0) {
475 + error(_("diff-hunks entry %u not in sorted order"), i);
476 + ret = -1;
477 + }
478 + if (!precomputed_entry_at(s, offset, &pe)) {
479 + error(_("diff-hunks entry %u has out-of-bounds hunk "
480 + "data"), i);
481 + ret = -1;
482 + } else if (!replayable_hunks(&pe)) {
483 + error(_("diff-hunks entry %u holds an invalid hunk "
484 + "sequence"), i);
485 + ret = -1;
486 + }
487 + }
488 +
489 + free_store(s);
490 + return ret;
491 +}
492 +
493 +int diff_hunks_verify(struct repository *r)
494 +{
495 + char *fname = diff_hunks_store_path(r);
496 + int ret = 0;
497 +
498 + if (verify_store_at(r, fname))
499 + ret = -1;
500 + free(fname);
501 + return ret;
502 +}
503 +
504 +int diff_hunks_clear(struct repository *r)
505 +{
506 + char *fname = diff_hunks_store_path(r);
507 + int ret = 0;
508 +
509 + if (unlink(fname) && errno != ENOENT)
510 + ret = error_errno(_("unable to remove %s"), fname);
511 + free(fname);
512 + return ret;
513 +}
514 +
515 +struct writer_entry {
516 + struct object_id old_oid;
517 + struct object_id new_oid;
518 + int xdl_opts;
519 + uint32_t hdat_offset;
520 +};
521 +
522 +struct diff_hunks_writer {
523 + struct repository *r;
524 + struct writer_entry *entries;
525 + size_t nr, alloc;
526 + size_t seed_nr; /* nr after seeding; finish skips a no-op flush */
527 + unsigned force_flush : 1; /* seed pruned: rewrite even a no-op warm */
528 + struct strbuf hdat;
529 + struct hashmap dedup; /* hunk block content -> offset in hdat */
530 +};
531 +
532 +/* A record of one distinct hunk block already present in hdat. */
533 +struct dedup_entry {
534 + struct hashmap_entry ent;
535 + uint32_t offset;
536 + uint32_t len;
537 +};
538 +
539 +static int dedup_cmp(const void *cmp_data,
540 + const struct hashmap_entry *a,
541 + const struct hashmap_entry *b,
542 + const void *keydata UNUSED)
543 +{
544 + const struct diff_hunks_writer *writer = cmp_data;
545 + const struct dedup_entry *ea = container_of(a, const struct dedup_entry, ent);
546 + const struct dedup_entry *eb = container_of(b, const struct dedup_entry, ent);
547 +
548 + if (ea->len != eb->len)
549 + return 1;
550 + return memcmp(writer->hdat.buf + ea->offset,
551 + writer->hdat.buf + eb->offset, ea->len);
552 +}
553 +
554 +static struct diff_hunks_writer *diff_hunks_writer_new(struct repository *r)
555 +{
556 + struct diff_hunks_writer *w;
557 +
558 + CALLOC_ARRAY(w, 1);
559 + w->r = r;
560 + strbuf_init(&w->hdat, 0);
561 + hashmap_init(&w->dedup, dedup_cmp, w, 0);
562 + return w;
563 +}
564 +
565 +static void strbuf_put_be32(struct strbuf *sb, uint32_t val)
566 +{
567 + unsigned char buf[4];
568 + put_be32(buf, val);
569 + strbuf_add(sb, buf, 4);
570 +}
571 +
572 +/*
573 + * The hunk block just appended at `start` is deduplicated: if an
574 + * identical block is already in hdat, this copy is dropped and the
575 + * earlier offset returned; otherwise it is kept and remembered.
576 + * Distinct keys that diff to the same hunks then share one block.
577 + */
578 +static uint32_t intern_block(struct diff_hunks_writer *w, size_t start)
579 +{
580 + size_t len = w->hdat.len - start;
581 + struct dedup_entry key, *found, *added;
582 +
583 + hashmap_entry_init(&key.ent, memhash(w->hdat.buf + start, len));
584 + key.offset = (uint32_t)start;
585 + key.len = (uint32_t)len;
586 +
587 + found = hashmap_get_entry(&w->dedup, &key, ent, NULL);
588 + if (found) {
589 + strbuf_setlen(&w->hdat, start);
590 + return found->offset;
591 + }
592 +
593 + added = xmalloc(sizeof(*added));
594 + hashmap_entry_init(&added->ent, key.ent.hash);
595 + added->offset = key.offset;
596 + added->len = key.len;
597 + hashmap_add(&w->dedup, &added->ent);
598 + return key.offset;
599 +}
600 +
601 +int diff_hunks_writer_add(struct diff_hunks_writer *w,
602 + const struct object_id *old_oid,
603 + const struct object_id *new_oid,
604 + int xdl_opts,
605 + const struct precomputed_hunk *hunks,
606 + size_t nr_hunks)
607 +{
608 + struct writer_entry *e;
609 + size_t i, block_start;
610 +
611 + if (!w)
612 + return 0;
613 + /*
614 + * The block appended for this entry is sizeof(uint32_t) +
615 + * nr_hunks * DIFF_HUNKS_HUNK_SIZE bytes. Bound nr_hunks so that
616 + * length fits the uint32_t the dedup index records (and so the
617 + * count itself fits the uint32_t written to the store).
618 + */
619 + if (!nr_hunks ||
620 + nr_hunks > (UINT32_MAX - sizeof(uint32_t)) / DIFF_HUNKS_HUNK_SIZE ||
621 + is_null_oid(old_oid) || is_null_oid(new_oid))
622 + return 0;
623 + if (w->hdat.len > UINT32_MAX)
624 + return 0;
625 + /*
626 + * Coordinates are stored as 32-bit values; a result that cannot
627 + * round-trip is dropped rather than silently truncated.
628 + */
629 + for (i = 0; i < nr_hunks; i++)
630 + if ((uintmax_t)hunks[i].old_start > (uintmax_t)INT32_MAX ||
631 + (uintmax_t)hunks[i].old_count > (uintmax_t)INT32_MAX ||
632 + (uintmax_t)hunks[i].new_start > (uintmax_t)INT32_MAX ||
633 + (uintmax_t)hunks[i].new_count > (uintmax_t)INT32_MAX)
634 + return 0;
635 +
636 + ALLOC_GROW(w->entries, w->nr + 1, w->alloc);
637 + e = &w->entries[w->nr++];
638 + oidcpy(&e->old_oid, old_oid);
639 + oidcpy(&e->new_oid, new_oid);
640 + e->xdl_opts = xdl_opts;
641 +
642 + block_start = w->hdat.len;
643 + strbuf_put_be32(&w->hdat, (uint32_t)nr_hunks);
644 + for (i = 0; i < nr_hunks; i++) {
645 + strbuf_put_be32(&w->hdat, hunks[i].old_start);
646 + strbuf_put_be32(&w->hdat, hunks[i].old_count);
647 + strbuf_put_be32(&w->hdat, hunks[i].new_start);
648 + strbuf_put_be32(&w->hdat, hunks[i].new_count);
649 + }
650 + e->hdat_offset = intern_block(w, block_start);
651 + return 1;
652 +}
653 +
654 +/*
655 + * Seed the writer with fname's entries so a rewrite preserves them,
656 + * setting *pruned when the rewrite will not carry the whole file
657 + * forward: the file failed its checksum and was discarded outright, or
658 + * individual entries were dropped because they failed the replayable
659 + * check or the writer refused them (a key naming no blob). A
660 + * rewrite re-checksums, so corruption must not be carried forward:
661 + * that would launder it into a checksum-valid file that verify can no
662 + * longer catch. This path already reads the whole file, so verify the
663 + * checksum here (the reader keeps trusting committed files, without
664 + * re-checksumming); an invalid
665 + * entry reads as a miss anyway, so dropping it heals the store rather
666 + * than losing anything a reader could use.
667 + */
668 +static void diff_hunks_writer_seed(struct diff_hunks_writer *w,
669 + const char *fname, int *pruned)
670 +{
671 + struct diff_hunks_store *s = load_store_at(w->r->hash_algo, fname);
672 + unsigned int rawsz;
673 + size_t entry_size, keysz;
674 + struct precomputed_hunk *hunks = NULL;
675 + size_t hunks_alloc = 0;
676 + uint32_t i, dropped = 0;
677 +
678 + if (!s)
679 + return;
680 + if (!hashfile_checksum_valid(w->r->hash_algo, s->data, s->data_len)) {
681 + warning(_("diff-hunks store %s failed its checksum; "
682 + "discarding it"), fname);
683 + free_store(s);
684 + *pruned = 1;
685 + return;
686 + }
687 + rawsz = s->hash_algo->rawsz;
688 + entry_size = store_index_entry_size(s->hash_algo);
689 + keysz = store_index_key_size(s->hash_algo);
690 +
691 + for (i = 0; i < s->num_entries; i++) {
692 + const unsigned char *ep = s->index + st_mult(entry_size, i);
693 + const unsigned char *old_hash, *new_hash;
694 + struct object_id old_oid, new_oid;
695 + uint32_t xdl_opts, j;
696 + struct precomputed_entry pe;
697 +
698 + decode_store_index_key(ep, rawsz, &old_hash, &new_hash,
699 + &xdl_opts);
700 + oidread(&old_oid, old_hash, s->hash_algo);
701 + oidread(&new_oid, new_hash, s->hash_algo);
702 + if (!precomputed_entry_at(s, index_entry_hdat_offset(ep, keysz), &pe) ||
703 + !replayable_hunks(&pe)) {
704 + dropped++;
705 + continue;
706 + }
707 + ALLOC_GROW(hunks, pe.num_hunks, hunks_alloc);
708 + for (j = 0; j < pe.num_hunks; j++)
709 + nth_precomputed_hunk(&pe, j, &hunks[j]);
710 + if (!diff_hunks_writer_add(w, &old_oid, &new_oid,
711 + (int)xdl_opts, hunks, pe.num_hunks))
712 + dropped++;
713 + }
714 + if (dropped) {
715 + warning(Q_("diff-hunks store %s: dropping %u invalid entry",
716 + "diff-hunks store %s: dropping %u invalid entries",
717 + dropped), fname, dropped);
718 + *pruned = 1;
719 + }
720 + free(hunks);
721 + free_store(s);
722 +}
723 +
724 +/*
725 + * Writing is off by default. It is enabled per invocation by the
726 + * GIT_DIFF_HUNKS_WRITE environment variable, or persistently by the
727 + * diffHunks.write config, with the environment variable winning when
728 + * set. Only a warming run (a diff or log the repository owner chooses
729 + * to run with writing on) enables it, so ordinary reads never mutate
730 + * the store.
731 + */
732 +static int diff_hunks_write_enabled(struct repository *r)
733 +{
734 + const char *env = getenv("GIT_DIFF_HUNKS_WRITE");
735 + int val;
736 +
737 + if (env) {
738 + /*
739 + * This is a warming opt-in, so an unparseable value must not
740 + * abort an ordinary read command: treat it as disabled.
741 + */
742 + val = git_parse_maybe_bool(env);
743 + return val < 0 ? 0 : val;
744 + }
745 + if (!repo_config_get_bool(r, "diffhunks.write", &val))
746 + return val;
747 + return 0;
748 +}
749 +
750 +struct diff_hunks_writer *diff_hunks_writer_maybe_new(struct repository *r)
751 +{
752 + struct diff_hunks_writer *w;
753 + char *fname;
754 + int pruned;
755 +
756 + if (!diff_hunks_write_enabled(r))
757 + return NULL;
758 + /*
759 + * Seed from the existing store so a flush merges with it rather
760 + * than replacing it: a later warm adds newly computed pairs
761 + * without discarding what earlier warms recorded.
762 + */
763 + w = diff_hunks_writer_new(r);
764 + fname = diff_hunks_store_path(r);
765 + pruned = 0;
766 + diff_hunks_writer_seed(w, fname, &pruned);
767 + free(fname);
768 + w->seed_nr = w->nr;
769 + /*
770 + * A pruning seed means the file on disk holds material the
771 + * rewrite must not preserve; flush even if this warm computes
772 + * nothing new, so the store on disk is repaired rather than
773 + * left serving what the seed refused.
774 + */
775 + w->force_flush = !!pruned;
776 + return w;
777 +}
778 +
779 +static int writer_entry_cmp(const void *va, const void *vb, void *ctx)
780 +{
781 + const struct writer_entry *a = va, *b = vb;
782 + unsigned int rawsz = *(const unsigned int *)ctx;
783 + return cmp_store_index_key(a->old_oid.hash, a->new_oid.hash,
784 + (uint32_t)a->xdl_opts,
785 + b->old_oid.hash, b->new_oid.hash,
786 + (uint32_t)b->xdl_opts,
787 + rawsz);
788 +}
789 +
790 +struct write_ctx {
791 + struct diff_hunks_writer *w;
792 + unsigned int rawsz;
793 +};
794 +
795 +static int write_index_chunk(struct hashfile *f, void *data)
796 +{
797 + struct write_ctx *ctx = data;
798 + size_t i;
799 +
800 + for (i = 0; i < ctx->w->nr; i++) {
801 + hashwrite(f, ctx->w->entries[i].old_oid.hash, ctx->rawsz);
802 + hashwrite(f, ctx->w->entries[i].new_oid.hash, ctx->rawsz);
803 + hashwrite_be32(f, ctx->w->entries[i].xdl_opts);
804 + hashwrite_be32(f, ctx->w->entries[i].hdat_offset);
805 + }
806 + return 0;
807 +}
808 +
809 +static int write_data_chunk(struct hashfile *f, void *data)
810 +{
811 + struct write_ctx *ctx = data;
812 + hashwrite(f, ctx->w->hdat.buf, ctx->w->hdat.len);
813 + return 0;
814 +}
815 +
816 +/* Sort, dedup, and write the accumulated entries to the file at fname. */
817 +static int diff_hunks_writer_flush(struct diff_hunks_writer *w, char *fname)
818 +{
819 + struct lock_file lk = LOCK_INIT;
820 + struct hashfile *f;
821 + struct chunkfile *cf;
822 + unsigned int rawsz = w->r->hash_algo->rawsz;
823 + struct write_ctx ctx = { w, rawsz };
824 + size_t entry_size;
825 +
826 + QSORT_S(w->entries, w->nr, writer_entry_cmp, &rawsz);
827 +
828 + /*
829 + * The same blob pair recurs across history (reverts, cherry-
830 + * picks); identical keys carry identical hunks, so keep one of
831 + * each. The index must stay duplicate-free for binary search.
832 + */
833 + if (w->nr > 1) {
834 + size_t kept = 1, i;
835 + for (i = 1; i < w->nr; i++)
836 + if (writer_entry_cmp(&w->entries[kept - 1],
837 + &w->entries[i], &rawsz))
838 + w->entries[kept++] = w->entries[i];
839 + w->nr = kept;
840 + }
841 +
842 + if (safe_create_leading_directories(w->r, fname)) {
843 + error(_("unable to create directory for %s"), fname);
844 + return -1;
845 + }
846 + if (hold_lock_file_for_update(&lk, fname, 0) < 0) {
847 + error_errno(_("unable to lock %s"), fname);
848 + return -1;
849 + }
850 + adjust_shared_perm(w->r, get_lock_file_path(&lk));
851 + f = hashfd(w->r->hash_algo, get_lock_file_fd(&lk),
852 + get_lock_file_path(&lk));
853 +
854 + entry_size = store_index_entry_size(w->r->hash_algo);
855 + cf = init_chunkfile(f);
856 + add_chunk(cf, DIFF_HUNKS_CHUNKID_INDEX, w->nr * entry_size,
857 + write_index_chunk);
858 + add_chunk(cf, DIFF_HUNKS_CHUNKID_DATA, w->hdat.len, write_data_chunk);
859 +
860 + hashwrite_be32(f, DIFF_HUNKS_SIGNATURE);
861 + hashwrite_u8(f, DIFF_HUNKS_VERSION);
862 + hashwrite_u8(f, oid_version(w->r->hash_algo));
863 + hashwrite_u8(f, get_num_chunks(cf));
864 + hashwrite_u8(f, 0); /* reserved */
865 +
866 + write_chunkfile(cf, &ctx);
867 + free_chunkfile(cf);
868 +
869 + /*
870 + * fsync per the user's configuration (like commit-graph and the
871 + * multi-pack-index), then commit atomically. Readers trust the
872 + * committed file rather than re-checksumming it; diff_hunks_verify()
873 + * checks the checksum separately.
874 + */
875 + finalize_hashfile(f, NULL, FSYNC_COMPONENT_DIFF_HUNKS,
876 + CSUM_HASH_IN_STREAM | CSUM_FSYNC);
877 + /*
878 + * This same process may hold the current store mmapped (a warm
879 + * that also reads); the commit below renames over it, which must
880 + * never land on a live mapping (Windows refuses it). Close the
881 + * store and clear the load-attempted flag first, so the next
882 + * read loads the committed file.
883 + */
884 + if (w->r->objects) {
885 + close_diff_hunks_store(w->r->objects);
886 + w->r->objects->diff_hunks_store_attempted = 0;
887 + }
888 + if (commit_lock_file(&lk)) {
889 + error_errno(_("unable to write %s"), fname);
890 + return -1;
891 + }
892 + return 0;
893 +}
894 +
895 +static void diff_hunks_writer_free(struct diff_hunks_writer *w)
896 +{
897 + if (!w)
898 + return;
899 + hashmap_clear_and_free(&w->dedup, struct dedup_entry, ent);
900 + free(w->entries);
901 + strbuf_release(&w->hdat);
902 + free(w);
903 +}
904 +
905 +void diff_hunks_writer_finish(struct diff_hunks_writer *w)
906 +{
907 + if (!w)
908 + return;
909 + /* Skip the flush when the warm recorded nothing beyond its seed. */
910 + if (w->nr != w->seed_nr || w->force_flush) {
911 + char *fname = diff_hunks_store_path(w->r);
912 + diff_hunks_writer_flush(w, fname);
913 + free(fname);
914 + }
915 + diff_hunks_writer_free(w);
916 +}
diff-hunks.h new
+117
@@ -0,0 +1,117 @@
1 +#ifndef DIFF_HUNKS_H
2 +#define DIFF_HUNKS_H
3 +
4 +#include "hash.h"
5 +#include "xdiff-interface.h" /* xdl_emit_hunk_consume_func_t */
6 +
7 +struct object_id;
8 +struct repository;
9 +struct object_database;
10 +
11 +/*
12 + * A persistent store of precomputed diff hunk coordinates, at
13 + * .git/objects/info/diff-hunks. Entries are keyed by the two blobs diffed
14 + * and the xdl_opts they were diffed under, so a cached result is valid
15 + * in any context that key recurs in, independent of path. The xdl_opts
16 + * key component mirrors the (always non-negative) diff_options field it
17 + * projects from, and is serialized and compared as a 4-byte big-endian
18 + * integer.
19 + *
20 + * The hunks a pair produces are not unique. They vary with the xdiff
21 + * algorithm and ignore flags (xdl_opts, part of the key), and with
22 + * whether the diff was trimmed: a zero-context diff runs
23 + * trim_common_tail, which can pick a different but equally valid set of
24 + * hunks than an untrimmed diff. The store holds one entry per key, so a
25 + * pair is recorded only when its trimmed and untrimmed diffs are
26 + * identical (the recording caller checks); such an entry serves a
27 + * consumer at any context. The rare pair where the two diffs differ is
28 + * never recorded and is always computed.
29 + *
30 + * The store is a cache: ordinary commands read it and fall back to
31 + * computing the diff when it is absent, stale, or corrupt. It is filled
32 + * as a side effect of diff and log runs, but only when writing is
33 + * enabled (such a write-enabled run is a warming run); writing is off
34 + * by default, so an ordinary command reads the store without recording
35 + * into it.
36 + */
37 +
38 +/*
39 + * A hunk's coordinates. The type is long to match the xdiff emit
40 + * callback; the values are a diff's line numbers and counts, always
41 + * within the int32 range the on-disk format stores (see
42 + * diff_hunks_writer_add()).
43 + */
44 +struct precomputed_hunk {
45 + long old_start;
46 + long old_count;
47 + long new_start;
48 + long new_count;
49 +};
50 +
51 +/*
52 + * The repository's store, loaded once on first use and cached on the
53 + * object database. Returns NULL when reading is disabled
54 + * (core.diffHunks=false), the store is absent, or it fails to parse
55 + * (wrong signature, version, or object hash, or a corrupt structure).
56 + * The lookup functions below accept a NULL store and treat it as
57 + * empty (every lookup misses), so callers need not check for NULL.
58 + * The object database owns the store; callers must not free it.
59 + */
60 +struct diff_hunks_store *repo_diff_hunks_store(struct repository *r);
61 +
62 +/* Free the repository's cached store, at object-database teardown. */
63 +void close_diff_hunks_store(struct object_database *o);
64 +
65 +/*
66 + * Replay the recorded hunks of an (old blob, new blob) pair diffed
67 + * under xdl_opts through hunk_func. The sequence is validated before
68 + * any callback runs: on a hit (return 1) every hunk is emitted, on a
69 + * miss (return 0: absent pair, xdl_opts mismatch, or an entry that
70 + * fails validation) nothing is emitted, so a caller may accumulate
71 + * directly into its result.
72 + */
73 +int diff_hunks_replay(struct diff_hunks_store *s,
74 + const struct object_id *old_oid,
75 + const struct object_id *new_oid,
76 + int xdl_opts,
77 + xdl_emit_hunk_consume_func_t hunk_func, void *cb_data);
78 +
79 +/*
80 + * A warming run's writer: it accumulates the hunks it computes in memory
81 + * and flushes them to the store in one pass at finish.
82 + */
83 +struct diff_hunks_writer;
84 +
85 +/*
86 + * Return a writer for a warming run, or NULL when writing is disabled
87 + * (the default). diff_hunks_writer_add() tolerates a NULL writer, so a
88 + * caller may attach the result unconditionally. Pair with
89 + * diff_hunks_writer_finish().
90 + */
91 +struct diff_hunks_writer *diff_hunks_writer_maybe_new(struct repository *r);
92 +
93 +/*
94 + * Record a blob pair's hunks as computed under xdl_opts; a later lookup
95 + * with a matching key is served these hunks. The caller must have
96 + * checked that the pair's trimmed and untrimmed diffs are identical
97 + * (see the top of this file), so the entry answers at any context.
98 + * NULL-safe. Returns 1 when the entry was recorded, 0 when the writer
99 + * refused it (no hunks, a null object id, or values the on-disk
100 + * 32-bit fields cannot hold).
101 + */
102 +int diff_hunks_writer_add(struct diff_hunks_writer *w,
103 + const struct object_id *old_oid,
104 + const struct object_id *new_oid,
105 + int xdl_opts,
106 + const struct precomputed_hunk *hunks,
107 + size_t nr_hunks);
108 +
109 +/* Flush the accumulated entries to the store and free the writer. NULL-safe. */
110 +void diff_hunks_writer_finish(struct diff_hunks_writer *w);
111 +
112 +/* Remove the store file. Returns 0 (incl. absent) or -1. */
113 +int diff_hunks_clear(struct repository *r);
114 +/* Validate the store. Returns 0 if valid/absent, -1 if corrupt. */
115 +int diff_hunks_verify(struct repository *r);
116 +
117 +#endif /* DIFF_HUNKS_H */
environment.c
+1
@@ -239,6 +239,7 @@ static const struct fsync_component_name {
239 { "pack", FSYNC_COMPONENT_PACK },
240 { "pack-metadata", FSYNC_COMPONENT_PACK_METADATA },
241 { "commit-graph", FSYNC_COMPONENT_COMMIT_GRAPH },
242 + { "diff-hunks", FSYNC_COMPONENT_DIFF_HUNKS },
243 { "index", FSYNC_COMPONENT_INDEX },
244 { "objects", FSYNC_COMPONENTS_OBJECTS },
245 { "reference", FSYNC_COMPONENT_REFERENCE },
git.c
+1
@@ -566,6 +566,7 @@ static struct cmd_struct commands[] = {
566 { "diagnose", cmd_diagnose, RUN_SETUP_GENTLY },
567 { "diff", cmd_diff, NO_PARSEOPT },
568 { "diff-files", cmd_diff_files, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
569 + { "diff-hunks", cmd_diff_hunks, RUN_SETUP },
570 { "diff-index", cmd_diff_index, RUN_SETUP | NO_PARSEOPT },
571 { "diff-pairs", cmd_diff_pairs, RUN_SETUP | NO_PARSEOPT },
572 { "diff-tree", cmd_diff_tree, RUN_SETUP | NO_PARSEOPT },
meson.build
+2
@@ -364,6 +364,7 @@ libgit_sources = [
364 'diffcore-pickaxe.c',
365 'diffcore-rename.c',
366 'diffcore-rotate.c',
367 + 'diff-hunks.c',
368 'dir-iterator.c',
369 'dir.c',
370 'editor.c',
@@ -633,6 +634,7 @@ builtin_sources = [
634 'builtin/describe.c',
635 'builtin/diagnose.c',
636 'builtin/diff-files.c',
637 + 'builtin/diff-hunks.c',
638 'builtin/diff-index.c',
639 'builtin/diff-pairs.c',
640 'builtin/diff-tree.c',
odb.c
+2
@@ -2,6 +2,7 @@
2 #include "abspath.h"
3 #include "commit-graph.h"
4 #include "config.h"
5 +#include "diff-hunks.h"
6 #include "dir.h"
7 #include "environment.h"
8 #include "gettext.h"
@@ -1033,6 +1034,7 @@ void odb_close(struct object_database *o)
1034 for (source = o->sources; source; source = source->next)
1035 odb_source_close(source);
1036 close_commit_graph(o);
1037 + close_diff_hunks_store(o);
1038 }
1039
1040 static void odb_free_sources(struct object_database *o)
odb.h
+4
@@ -8,6 +8,7 @@
8 #include "thread-utils.h"
9
10 struct cached_object_entry;
11 +struct diff_hunks_store;
12 struct list_objects_filter_options;
13 struct odb_source_inmemory;
14 struct packed_git;
@@ -76,6 +77,9 @@ struct object_database {
77 struct commit_graph *commit_graph;
78 unsigned commit_graph_attempted : 1; /* if loading has been attempted */
79
80 + struct diff_hunks_store *diff_hunks_store;
81 + unsigned diff_hunks_store_attempted : 1; /* if loading has been attempted */
82 +
83 /*
84 * This is meant to hold a *small* number of objects that you would
85 * want odb_read_object() to be able to return, but yet you do not want
repo-settings.c
+1
@@ -77,6 +77,7 @@ void prepare_repo_settings(struct repository *r)
77 repo_cfg_bool(r, "pack.usesparse", &r->settings.pack_use_sparse, 1);
78 repo_cfg_bool(r, "pack.usepathwalk", &r->settings.pack_use_path_walk, 0);
79 repo_cfg_bool(r, "core.multipackindex", &r->settings.core_multi_pack_index, 1);
80 + repo_cfg_bool(r, "core.diffhunks", &r->settings.core_diff_hunks, 1);
81 repo_cfg_bool(r, "index.sparse", &r->settings.sparse_index, 0);
82 repo_cfg_bool(r, "index.skiphash", &r->settings.index_skip_hash, r->settings.index_skip_hash);
83 repo_cfg_bool(r, "pack.readreverseindex", &r->settings.pack_read_reverse_index, 1);
repo-settings.h
+1
@@ -22,6 +22,7 @@ struct repo_settings {
22 int core_commit_graph;
23 int commit_graph_generation_version;
24 int commit_graph_changed_paths_version;
25 + int core_diff_hunks;
26 int gc_write_commit_graph;
27 int fetch_write_commit_graph;
28 int command_requires_full_index;
write-or-die.h
+5 -2
@@ -22,13 +22,15 @@ enum fsync_component {
22 FSYNC_COMPONENT_INDEX = 1 << 4,
23 FSYNC_COMPONENT_REFERENCE = 1 << 5,
24 FSYNC_COMPONENT_OBJECT_MAP = 1 << 6,
25 + FSYNC_COMPONENT_DIFF_HUNKS = 1 << 7,
26 };
27
28 #define FSYNC_COMPONENTS_OBJECTS (FSYNC_COMPONENT_LOOSE_OBJECT | \
29 FSYNC_COMPONENT_PACK)
30
31 #define FSYNC_COMPONENTS_DERIVED_METADATA (FSYNC_COMPONENT_PACK_METADATA | \
31 - FSYNC_COMPONENT_COMMIT_GRAPH)
32 + FSYNC_COMPONENT_COMMIT_GRAPH | \
33 + FSYNC_COMPONENT_DIFF_HUNKS)
34
35 #define FSYNC_COMPONENTS_DEFAULT ((FSYNC_COMPONENTS_OBJECTS | \
36 FSYNC_COMPONENTS_DERIVED_METADATA) & \
@@ -46,7 +48,8 @@ enum fsync_component {
48 FSYNC_COMPONENT_COMMIT_GRAPH | \
49 FSYNC_COMPONENT_INDEX | \
50 FSYNC_COMPONENT_REFERENCE | \
49 - FSYNC_COMPONENT_OBJECT_MAP)
51 + FSYNC_COMPONENT_OBJECT_MAP | \
52 + FSYNC_COMPONENT_DIFF_HUNKS)
53
54 #ifndef FSYNC_COMPONENTS_PLATFORM_DEFAULT
55 #define FSYNC_COMPONENTS_PLATFORM_DEFAULT FSYNC_COMPONENTS_DEFAULT