midx-write.c: pass `start_pack` to `compute_sorted_entries()`

The function `compute_sorted_entries()` is broadly responsible for building an array of the objects to be written into a MIDX based on the provided list of packs. If we have loaded an existing MIDX, however, we may not use all of its packs, despite loading them into the ctx->info array. The existing implementation simply skips past the first ctx->m->num_packs (if ctx->m is non-NULL, indicating that we loaded an existing MIDX). This is because we read objects in packs from an existing MIDX via the MIDX itself, rather than from the pack-level fanout to guarantee a de-duplicated result (see: a40498a1265 (midx: use existing midx when writing new one, 2018-07-12)). Future changes (outside the scope of this patch series) to the MIDX code will require us to skip *at most* that number[^1]. We could tag each pack with a bit that indicates the pack's contents should be included in the MIDX. But we can just as easily determine the number of packs to skip by passing in the number of packs we learned about after processing an existing MIDX. [^1]: Kind of. The real number will be bounded by the number of packs in a MIDX layer, and the number of packs in its base layer(s), but that concept hasn't been fully defined yet. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Taylor Blau committed May 29, 2024 at 18:55 UTC 33e9218ffbc98618896587d72eab18178f483a17
1 file changed +6 -4
midx-write.c
+6 -4
@@ -299,12 +299,12 @@ static void midx_fanout_add_pack_fanout(struct midx_fanout *fanout,
299 * Copy only the de-duplicated entries (selected by most-recent modified time
300 * of a packfile containing the object).
301 */
302 -static void compute_sorted_entries(struct write_midx_context *ctx)
302 +static void compute_sorted_entries(struct write_midx_context *ctx,
303 + uint32_t start_pack)
304 {
305 uint32_t cur_fanout, cur_pack, cur_object;
306 size_t alloc_objects, total_objects = 0;
307 struct midx_fanout fanout = { 0 };
307 - uint32_t start_pack = ctx->m ? ctx->m->num_packs : 0;
308
309 for (cur_pack = start_pack; cur_pack < ctx->nr; cur_pack++)
310 total_objects = st_add(total_objects,
@@ -883,7 +883,7 @@ static int write_midx_internal(const char *object_dir,
883 {
884 struct strbuf midx_name = STRBUF_INIT;
885 unsigned char midx_hash[GIT_MAX_RAWSZ];
886 - uint32_t i;
886 + uint32_t i, start_pack;
887 struct hashfile *f = NULL;
888 struct lock_file lk;
889 struct write_midx_context ctx = { 0 };
@@ -951,6 +951,8 @@ static int write_midx_internal(const char *object_dir,
951 }
952 }
953
954 + start_pack = ctx.nr;
955 +
956 ctx.pack_paths_checked = 0;
957 if (flags & MIDX_PROGRESS)
958 ctx.progress = start_delayed_progress(_("Adding packfiles to multi-pack-index"), 0);
@@ -1048,7 +1050,7 @@ static int write_midx_internal(const char *object_dir,
1050 }
1051 }
1052
1051 - compute_sorted_entries(&ctx);
1053 + compute_sorted_entries(&ctx, start_pack);
1054
1055 ctx.large_offsets_needed = 0;
1056 for (i = 0; i < ctx.entries_nr; i++) {