@matheusbsbs / matheus-bagliano / commits / 9639474b6d
Fork 0

pack-bitmap: pass object filter to fill-in traversal

Sometimes a bitmap traversal still has to walk some commits manually, because those commits aren't included in the bitmap packfile (e.g., due to a push or commit since the last full repack). If we're given an object filter, we don't pass it down to this traversal. It's not necessary for correctness because the bitmap code has its own filters to post-process the bitmap result (which it must, to filter out the objects that _are_ mentioned in the bitmapped packfile). And with blob filters, there was no performance reason to pass along those filters, either. The fill-in traversal could omit them from the result, but it wouldn't save us any time to do so, since we'd still have to walk each tree entry to see if it's a blob or not. But now that we support tree filters, there's opportunity for savings. A tree:depth=0 filter means we can avoid accessing trees entirely, since we know we won't them (or any of the subtrees or blobs they point to). The new test in p5310 shows this off (the "partial bitmap" state is one where HEAD~100 and its ancestors are all in a bitmapped pack, but HEAD~100..HEAD are not). Here are the results (run against linux.git): Test HEAD^ HEAD ------------------------------------------------------------------------------------------------- [...] 5310.16: rev-list with tree filter (partial bitmap) 0.19(0.17+0.02) 0.03(0.02+0.01) -84.2% The absolute number of savings isn't _huge_, but keep in mind that we only omitted 100 first-parent links (in the version of linux.git here, that's 894 actual commits). In a more pathological case, we might have a much larger proportion of non-bitmapped commits. I didn't bother creating such a case in the perf script because the setup is expensive, and this is plenty to show the savings as a percentage. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Jeff King committed May 4, 2020 at 17:12 UTC 9639474b6dd126bae29dba70e2fbe11169d60e20
2 files changed +14 -5
pack-bitmap.c
+9 -5
@@ -506,7 +506,8 @@ static int should_include(struct commit *commit, void *_data)
506 static struct bitmap *find_objects(struct bitmap_index *bitmap_git,
507 struct rev_info *revs,
508 struct object_list *roots,
509 - struct bitmap *seen)
509 + struct bitmap *seen,
510 + struct list_objects_filter_options *filter)
511 {
512 struct bitmap *base = NULL;
513 int needs_walk = 0;
@@ -599,8 +600,9 @@ static struct bitmap *find_objects(struct bitmap_index *bitmap_git,
600 show_data.bitmap_git = bitmap_git;
601 show_data.base = base;
602
602 - traverse_commit_list(revs, show_commit, show_object,
603 - &show_data);
603 + traverse_commit_list_filtered(filter, revs,
604 + show_commit, show_object,
605 + &show_data, NULL);
606 }
607
608 return base;
@@ -999,7 +1001,8 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs,
1001
1002 if (haves) {
1003 revs->ignore_missing_links = 1;
1002 - haves_bitmap = find_objects(bitmap_git, revs, haves, NULL);
1004 + haves_bitmap = find_objects(bitmap_git, revs, haves, NULL,
1005 + filter);
1006 reset_revision_walk();
1007 revs->ignore_missing_links = 0;
1008
@@ -1007,7 +1010,8 @@ struct bitmap_index *prepare_bitmap_walk(struct rev_info *revs,
1010 BUG("failed to perform bitmap walk");
1011 }
1012
1010 - wants_bitmap = find_objects(bitmap_git, revs, wants, haves_bitmap);
1013 + wants_bitmap = find_objects(bitmap_git, revs, wants, haves_bitmap,
1014 + filter);
1015
1016 if (!wants_bitmap)
1017 BUG("failed to perform bitmap walk");
t/perf/p5310-pack-bitmaps.sh
+5
@@ -91,4 +91,9 @@ test_perf 'pack to file (partial bitmap)' '
91 git pack-objects --use-bitmap-index --all pack2b </dev/null >/dev/null
92 '
93
94 +test_perf 'rev-list with tree filter (partial bitmap)' '
95 + git rev-list --use-bitmap-index --count --objects --all \
96 + --filter=tree:0 >/dev/null
97 +'
98 +
99 test_done