prune: save reachable-from-recent objects with bitmaps

We pass our prune expiration to mark_reachable_objects(), which will traverse not only the reachable objects, but consider any recent ones as tips for reachability; see d3038d22f9 (prune: keep objects reachable from recent objects, 2014-10-15) for details. However, this interacts badly with the bitmap code path added in fde67d6896 (prune: use bitmaps for reachability traversal, 2019-02-13). If we hit the bitmap-optimized path, we return immediately to avoid the regular traversal, accidentally skipping the "also traverse recent" code. Instead, we should do an if-else for the bitmap versus regular traversal, and then follow up with the "recent" traversal in either case. This reuses the "rev_info" for a bitmap and then a regular traversal, but that should work OK (the bitmap code clears the pending array in the usual way, just like a regular traversal would). Note that I dropped the comment above the regular traversal here. It has little explanatory value, and makes the if-else logic much harder to read. Here are a few variants that I rejected: - it seems like both the reachability and recent traversals could be done in a single traversal. This was rejected by d3038d22f9 (prune: keep objects reachable from recent objects, 2014-10-15), though the balance may be different when using bitmaps. However, there's a subtle correctness issue, too: we use revs->ignore_missing_links for the recent traversal, but not the reachability one. - we could try using bitmaps for the recent traversal, too, which could possibly improve performance. But it would require some fixes in the bitmap code, which uses ignore_missing_links for its own purposes. Plus it would probably not help all that much in practice. We use the reachable tips to generate bitmaps, so those objects are likely not covered by bitmaps (unless they just became unreachable). And in general, we expect the set of unreachable objects to be much smaller anyway, so there's less to gain. The test in t5304 detects the bug and confirms the fix. I also beefed up the tests in t6501, which covers the mtime-checking code more thoroughly, to handle the bitmap case (in addition to just "loose" and "packed" cases). Interestingly, this test doesn't actually detect the bug, because it is running "git gc", and not "prune" directly. And "gc" will call "repack" first, which does not suffer the same bug. So the old-but-reachable-from-recent objects get scooped up into the new pack along with the actually-recent objects, which gives both a recent mtime. But it seemed prudent to get more coverage of the bitmap case for related code. Reported-by: David Emett <dave@sp4m.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Jeff King committed Apr 28, 2021 at 11:42 UTC 2ba582ba4c62fae506174ce326deab082d962e63
3 files changed +36 -15
reachable.c
+4 -9
@@ -227,17 +227,12 @@ void mark_reachable_objects(struct rev_info *revs, int mark_reflog,
227 if (bitmap_git) {
228 traverse_bitmap_commit_list(bitmap_git, revs, mark_object_seen);
229 free_bitmap_index(bitmap_git);
230 - return;
230 + } else {
231 + if (prepare_revision_walk(revs))
232 + die("revision walk setup failed");
233 + traverse_commit_list(revs, mark_commit, mark_object, &cp);
234 }
235
233 - /*
234 - * Set up the revision walk - this will move all commits
235 - * from the pending list to the commit walking list.
236 - */
237 - if (prepare_revision_walk(revs))
238 - die("revision walk setup failed");
239 - traverse_commit_list(revs, mark_commit, mark_object, &cp);
240 -
236 if (mark_recent) {
237 revs->ignore_missing_links = 1;
238 if (add_unseen_recent_objects_to_traversal(revs, mark_recent))
t/t5304-prune.sh
+16
@@ -352,4 +352,20 @@ test_expect_success 'trivial prune with bitmaps enabled' '
352 test_must_fail git cat-file -e $blob
353 '
354
355 +test_expect_success 'old reachable-from-recent retained with bitmaps' '
356 + git repack -adb &&
357 + to_drop=$(echo bitmap-from-recent-1 | git hash-object -w --stdin) &&
358 + test-tool chmtime -86400 .git/objects/$(test_oid_to_path $to_drop) &&
359 + to_save=$(echo bitmap-from-recent-2 | git hash-object -w --stdin) &&
360 + test-tool chmtime -86400 .git/objects/$(test_oid_to_path $to_save) &&
361 + tree=$(printf "100644 blob $to_save\tfile\n" | git mktree) &&
362 + test-tool chmtime -86400 .git/objects/$(test_oid_to_path $tree) &&
363 + commit=$(echo foo | git commit-tree $tree) &&
364 + git prune --expire=12.hours.ago &&
365 + git cat-file -e $commit &&
366 + git cat-file -e $tree &&
367 + git cat-file -e $to_save &&
368 + test_must_fail git cat-file -e $to_drop
369 +'
370 +
371 test_done
t/t6501-freshen-objects.sh
+16 -6
@@ -43,15 +43,25 @@ commit () {
43 }
44
45 maybe_repack () {
46 - if test -n "$repack"; then
46 + case "$title" in
47 + loose)
48 + : skip repack
49 + ;;
50 + repack)
51 git repack -ad
48 - fi
52 + ;;
53 + bitmap)
54 + git repack -adb
55 + ;;
56 + *)
57 + echo >&2 "unknown test type in maybe_repack"
58 + return 1
59 + ;;
60 + esac
61 }
62
51 -for repack in '' true; do
52 - title=${repack:+repack}
53 - title=${title:-loose}
54 -
63 +for title in loose repack bitmap
64 +do
65 test_expect_success "make repo completely empty ($title)" '
66 rm -rf .git &&
67 git init