status: use sparse-index throughout

By testing 'git -c core.fsmonitor= status -uno', we can check for the simplest index operations that can be made sparse-aware. The necessary implementation details are already integrated with sparse-checkout, so modify command_requires_full_index to be zero for cmd_status(). In refresh_index(), we loop through the index entries to refresh their stat() information. However, sparse directories have no stat() information to populate. Ignore these entries. This allows 'git status' to no longer expand a sparse index to a full one. This is further tested by dropping the "-uno" option and adding an untracked file into the worktree. The performance test p2000-sparse-checkout-operations.sh demonstrates these improvements: Test HEAD~1 HEAD ----------------------------------------------------------------------------- 2000.2: git status (full-index-v3) 0.31(0.30+0.05) 0.31(0.29+0.06) +0.0% 2000.3: git status (full-index-v4) 0.31(0.29+0.07) 0.34(0.30+0.08) +9.7% 2000.4: git status (sparse-index-v3) 2.35(2.28+0.10) 0.04(0.04+0.05) -98.3% 2000.5: git status (sparse-index-v4) 2.35(2.24+0.15) 0.05(0.04+0.06) -97.9% Note that since HEAD~1 was expanding the sparse index by parsing trees, it was artificially slower than the full index case. Thus, the 98% improvement is misleading, and instead we should celebrate the 0.34s to 0.05s improvement of 85%. This is more indicative of the peformance gains we are expecting by using a sparse index. Note: we are dropping the assignment of core.fsmonitor here. This is not necessary for the test script as we are not altering the config any other way. Correct integration with FS Monitor will be validated in later changes. Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Derrick Stolee committed Jul 14, 2021 at 13:12 UTC d76723ee5313d0176e8777af4aa104d7562543c1
3 files changed +20 -6
builtin/commit.c
+3
@@ -1510,6 +1510,9 @@ int cmd_status(int argc, const char **argv, const char *prefix)
1510 if (argc == 2 && !strcmp(argv[1], "-h"))
1511 usage_with_options(builtin_status_usage, builtin_status_options);
1512
1513 + prepare_repo_settings(the_repository);
1514 + the_repository->settings.command_requires_full_index = 0;
1515 +
1516 status_init_config(&s, git_status_config);
1517 argc = parse_options(argc, argv, prefix,
1518 builtin_status_options,
read-cache.c
+8 -2
@@ -1585,8 +1585,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,
1585 */
1586 preload_index(istate, pathspec, 0);
1587 trace2_region_enter("index", "refresh", NULL);
1588 - /* TODO: audit for interaction with sparse-index. */
1589 - ensure_full_index(istate);
1588 +
1589 for (i = 0; i < istate->cache_nr; i++) {
1590 struct cache_entry *ce, *new_entry;
1591 int cache_errno = 0;
@@ -1601,6 +1600,13 @@ int refresh_index(struct index_state *istate, unsigned int flags,
1600 if (ignore_skip_worktree && ce_skip_worktree(ce))
1601 continue;
1602
1603 + /*
1604 + * If this entry is a sparse directory, then there isn't
1605 + * any stat() information to update. Ignore the entry.
1606 + */
1607 + if (S_ISSPARSEDIR(ce->ce_mode))
1608 + continue;
1609 +
1610 if (pathspec && !ce_path_match(istate, ce, pathspec, seen))
1611 filtered = 1;
1612
t/t1092-sparse-checkout-compatibility.sh
+9 -4
@@ -511,12 +511,17 @@ test_expect_success 'sparse-index is expanded and converted back' '
511 GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \
512 git -C sparse-index -c core.fsmonitor="" reset --hard &&
513 test_region index convert_to_sparse trace2.txt &&
514 - test_region index ensure_full_index trace2.txt &&
514 + test_region index ensure_full_index trace2.txt
515 +'
516
516 - rm trace2.txt &&
517 +test_expect_success 'sparse-index is not expanded' '
518 + init_repos &&
519 +
520 + rm -f trace2.txt &&
521 + echo >>sparse-index/untracked.txt &&
522 GIT_TRACE2_EVENT="$(pwd)/trace2.txt" GIT_TRACE2_EVENT_NESTING=10 \
518 - git -C sparse-index -c core.fsmonitor="" status -uno &&
519 - test_region index ensure_full_index trace2.txt
523 + git -C sparse-index status &&
524 + test_region ! index ensure_full_index trace2.txt
525 '
526
527 test_done