checkout: stop expanding sparse indexes

Previous changes did the necessary improvements to unpack-trees.c and diff-lib.c in order to modify a sparse index based on its comparision with a tree. The only remaining work is to remove some ensure_full_index() calls and add tests that verify that the index is not expanded in our interesting cases. Include 'switch' and 'restore' in these tests, as they share a base implementation with 'checkout'. Here are the relevant performance results from p2000-sparse-operations.sh: Test HEAD~1 HEAD -------------------------------------------------------------------------------- 2000.18: git checkout -f - (full-v3) 0.49(0.43+0.03) 0.47(0.39+0.05) -4.1% 2000.19: git checkout -f - (full-v4) 0.45(0.37+0.06) 0.42(0.37+0.05) -6.7% 2000.20: git checkout -f - (sparse-v3) 0.76(0.71+0.07) 0.04(0.03+0.04) -94.7% 2000.21: git checkout -f - (sparse-v4) 0.75(0.72+0.04) 0.05(0.06+0.04) -93.3% It is important to compare the full index case to the sparse index case, as the previous results for the sparse index were inflated by the index expansion. For index v4, this is an 88% improvement. On an internal repository with over two million paths at HEAD and a sparse-checkout definition containing ~60,000 of those paths, 'git checkout' went from 3.5s to 297ms with this change. The theoretical optimum where only those ~60,000 paths exist was 275ms, so the extra sparse directory entries contribute a 22ms overhead. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Derrick Stolee committed Jun 29, 2021 at 02:13 UTC 1ba5f45132f987e630497c09b74af687bf4f863b
2 files changed +12 -6
builtin/checkout.c
+3 -5
@@ -378,9 +378,6 @@ static int checkout_worktree(const struct checkout_opts *opts,
378 if (pc_workers > 1)
379 init_parallel_checkout();
380
381 - /* TODO: audit for interaction with sparse-index. */
382 - ensure_full_index(&the_index);
383 -
381 for (pos = 0; pos < active_nr; pos++) {
382 struct cache_entry *ce = active_cache[pos];
383 if (ce->ce_flags & CE_MATCHED) {
@@ -530,8 +527,6 @@ static int checkout_paths(const struct checkout_opts *opts,
527 * Make sure all pathspecs participated in locating the paths
528 * to be checked out.
529 */
533 - /* TODO: audit for interaction with sparse-index. */
534 - ensure_full_index(&the_index);
530 for (pos = 0; pos < active_nr; pos++)
531 if (opts->overlay_mode)
532 mark_ce_for_checkout_overlay(active_cache[pos],
@@ -1593,6 +1588,9 @@ static int checkout_main(int argc, const char **argv, const char *prefix,
1588
1589 git_config(git_checkout_config, opts);
1590
1591 + prepare_repo_settings(the_repository);
1592 + the_repository->settings.command_requires_full_index = 0;
1593 +
1594 opts->track = BRANCH_TRACK_UNSPECIFIED;
1595
1596 if (!opts->accept_pathspec && !opts->accept_ref)
t/t1092-sparse-checkout-compatibility.sh
+9 -1
@@ -560,7 +560,15 @@ test_expect_success 'sparse-index is not expanded' '
560 echo >>sparse-index/a &&
561 ensure_not_expanded commit --include a -m a &&
562 echo >>sparse-index/deep/deeper1/a &&
563 - ensure_not_expanded commit --include deep/deeper1/a -m deeper
563 + ensure_not_expanded commit --include deep/deeper1/a -m deeper &&
564 + ensure_not_expanded checkout rename-out-to-out &&
565 + ensure_not_expanded checkout - &&
566 + ensure_not_expanded switch rename-out-to-out &&
567 + ensure_not_expanded switch - &&
568 + git -C sparse-index reset --hard &&
569 + ensure_not_expanded checkout rename-out-to-out -- deep/deeper1 &&
570 + git -C sparse-index reset --hard &&
571 + ensure_not_expanded restore -s rename-out-to-out -- deep/deeper1
572 '
573
574 # NEEDSWORK: a sparse-checkout behaves differently from a full checkout