sparse-checkout: add 'cone' mode
The sparse-checkout feature can have quadratic performance as the number of patterns and number of entries in the index grow. If there are 1,000 patterns and 1,000,000 entries, this time can be very significant. Create a new Boolean config option, core.sparseCheckoutCone, to indicate that we expect the sparse-checkout file to contain a more limited set of patterns. This is a separate config setting from core.sparseCheckout to avoid breaking older clients by introducing a tri-state option. The config option does nothing right now, but will be expanded upon in a later commit. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Derrick Stolee committed
Nov 21, 2019 at 22:04 UTC
879321eb0bec25779386445d65242452825155be
6 files changed
+85
-4
Documentation/config/core.txt
+8
-2
@@ -593,8 +593,14 @@ core.multiPackIndex::
593
multi-pack-index design document].
594
595
core.sparseCheckout::
596
- Enable "sparse checkout" feature. See section "Sparse checkout" in
597
- linkgit:git-read-tree[1] for more information.
596
+ Enable "sparse checkout" feature. See linkgit:git-sparse-checkout[1]
597
+ for more information.
598
+
599
+core.sparseCheckoutCone::
600
+ Enables the "cone mode" of the sparse checkout feature. When the
601
+ sparse-checkout file contains a limited set of patterns, then this
602
+ mode provides significant performance advantages. See
603
+ linkgit:git-sparse-checkout[1] for more information.
604
605
core.abbrev::
606
Set the length object names are abbreviated to. If
Documentation/git-sparse-checkout.txt
+54
-1
@@ -80,7 +80,9 @@ the sparse-checkout file.
80
To repopulate the working directory with all files, use the
81
`git sparse-checkout disable` command.
82
83
-## FULL PATTERN SET
83
+
84
+FULL PATTERN SET
85
+----------------
86
87
By default, the sparse-checkout file uses the same syntax as `.gitignore`
88
files.
@@ -95,6 +97,57 @@ using negative patterns. For example, to remove the file `unwanted`:
97
----------------
98
99
100
+CONE PATTERN SET
101
+----------------
102
+
103
+The full pattern set allows for arbitrary pattern matches and complicated
104
+inclusion/exclusion rules. These can result in O(N*M) pattern matches when
105
+updating the index, where N is the number of patterns and M is the number
106
+of paths in the index. To combat this performance issue, a more restricted
107
+pattern set is allowed when `core.spareCheckoutCone` is enabled.
108
+
109
+The accepted patterns in the cone pattern set are:
110
+
111
+1. *Recursive:* All paths inside a directory are included.
112
+
113
+2. *Parent:* All files immediately inside a directory are included.
114
+
115
+In addition to the above two patterns, we also expect that all files in the
116
+root directory are included. If a recursive pattern is added, then all
117
+leading directories are added as parent patterns.
118
+
119
+By default, when running `git sparse-checkout init`, the root directory is
120
+added as a parent pattern. At this point, the sparse-checkout file contains
121
+the following patterns:
122
+
123
+----------------
124
+/*
125
+!/*/
126
+----------------
127
+
128
+This says "include everything in root, but nothing two levels below root."
129
+If we then add the folder `A/B/C` as a recursive pattern, the folders `A` and
130
+`A/B` are added as parent patterns. The resulting sparse-checkout file is
131
+now
132
+
133
+----------------
134
+/*
135
+!/*/
136
+/A/
137
+!/A/*/
138
+/A/B/
139
+!/A/B/*/
140
+/A/B/C/
141
+----------------
142
+
143
+Here, order matters, so the negative patterns are overridden by the positive
144
+patterns that appear lower in the file.
145
+
146
+If `core.sparseCheckoutCone=true`, then Git will parse the sparse-checkout file
147
+expecting patterns of these types. Git will warn if the patterns do not match.
148
+If the patterns do match the expected format, then Git will use faster hash-
149
+based algorithms to compute inclusion in the sparse-checkout.
150
+
151
SEE ALSO
152
--------
153
cache.h
+3
-1
@@ -918,12 +918,14 @@ extern char *git_replace_ref_base;
918
919
extern int fsync_object_files;
920
extern int core_preload_index;
921
-extern int core_apply_sparse_checkout;
921
extern int precomposed_unicode;
922
extern int protect_hfs;
923
extern int protect_ntfs;
924
extern const char *core_fsmonitor;
925
926
+int core_apply_sparse_checkout;
927
+int core_sparse_checkout_cone;
928
+
929
/*
930
* Include broken refs in all ref iterations, which will
931
* generally choke dangerous operations rather than letting
config.c
+5
@@ -1364,6 +1364,11 @@ static int git_default_core_config(const char *var, const char *value, void *cb)
1364
return 0;
1365
}
1366
1367
+ if (!strcmp(var, "core.sparsecheckoutcone")) {
1368
+ core_sparse_checkout_cone = git_config_bool(var, value);
1369
+ return 0;
1370
+ }
1371
+
1372
if (!strcmp(var, "core.precomposeunicode")) {
1373
precomposed_unicode = git_config_bool(var, value);
1374
return 0;
environment.c
+1
@@ -67,6 +67,7 @@ enum object_creation_mode object_creation_mode = OBJECT_CREATION_MODE;
67
char *notes_ref_name;
68
int grafts_replace_parents = 1;
69
int core_apply_sparse_checkout;
70
+int core_sparse_checkout_cone;
71
int merge_log_config = -1;
72
int precomposed_unicode = -1; /* see probe_utf8_pathname_composition() */
73
unsigned long pack_size_limit_cfg;
t/t1091-sparse-checkout-builtin.sh
+14
@@ -148,6 +148,20 @@ test_expect_success 'set sparse-checkout using --stdin' '
148
test_cmp expect dir
149
'
150
151
+test_expect_success 'cone mode: match patterns' '
152
+ git -C repo config --worktree core.sparseCheckoutCone true &&
153
+ rm -rf repo/a repo/folder1 repo/folder2 &&
154
+ git -C repo read-tree -mu HEAD &&
155
+ git -C repo reset --hard &&
156
+ ls repo >dir &&
157
+ cat >expect <<-EOF &&
158
+ a
159
+ folder1
160
+ folder2
161
+ EOF
162
+ test_cmp expect dir
163
+'
164
+
165
test_expect_success 'sparse-checkout disable' '
166
git -C repo sparse-checkout disable &&
167
test_path_is_missing repo/.git/info/sparse-checkout &&