midx.c: protect against disappearing packs

When a packed object is stored in a multi-pack index, but that pack has racily gone away, the MIDX code simply calls die(), when it could be returning an error to the caller, which would in turn lead to re-scanning the pack directory. A pack can racily disappear, for example, due to a simultaneous 'git repack -ad', You can also reproduce this with two terminals, where one is running: git init while true; do git commit -q --allow-empty -m foo git repack -ad git multi-pack-index write done (in effect, constantly writing new MIDXs), and the other is running: obj=$(git rev-parse HEAD) while true; do echo $obj | git cat-file --batch-check='%(objectsize:disk)' || break done That will sometimes hit the error preparing packfile from multi-pack-index message, which this patch fixes. Right now, that path to discovering a missing pack looks something like 'find_pack_entry()' calling 'fill_midx_entry()' and eventually making its way to call 'nth_midxed_pack_entry()'. 'nth_midxed_pack_entry()' already checks 'is_pack_valid()' and propagates an error if the pack is invalid. So, this works if the pack has gone away between calling 'prepare_midx_pack()' and before calling 'is_pack_valid()', but not if it disappears before then. Catch the case where the pack has already disappeared before 'prepare_midx_pack()' by returning an error in that case, too. Co-authored-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Taylor Blau committed Nov 25, 2020 at 12:17 UTC 506ec2fbda5334c4fc60fbd9f425fff3916a2066
2 files changed +8 -2
midx.c
+1 -1
@@ -285,7 +285,7 @@ static int nth_midxed_pack_entry(struct repository *r,
285 pack_int_id = nth_midxed_pack_int_id(m, pos);
286
287 if (prepare_midx_pack(r, m, pack_int_id))
288 - die(_("error preparing packfile from multi-pack-index"));
288 + return 0;
289 p = m->packs[pack_int_id];
290
291 /*
t/t5319-multi-pack-index.sh
+7 -1
@@ -547,7 +547,7 @@ test_expect_success 'repack --batch-size=0 repacks everything' '
547 )
548 '
549
550 -test_expect_success 'load reverse index when missing .idx' '
550 +test_expect_success 'load reverse index when missing .idx, .pack' '
551 git init repo &&
552 test_when_finished "rm -fr repo" &&
553 (
@@ -560,9 +560,15 @@ test_expect_success 'load reverse index when missing .idx' '
560 git multi-pack-index write &&
561
562 git rev-parse HEAD >tip &&
563 + pack=$(ls .git/objects/pack/pack-*.pack) &&
564 idx=$(ls .git/objects/pack/pack-*.idx) &&
565
566 mv $idx $idx.bak &&
567 + git cat-file --batch-check="%(objectsize:disk)" <tip &&
568 +
569 + mv $idx.bak $idx &&
570 +
571 + mv $pack $pack.bak &&
572 git cat-file --batch-check="%(objectsize:disk)" <tip
573 )
574 '