@matheusbsbs / matheus-bagliano / commits / cff9711616

multi-pack-index: prepare for 'expire' subcommand

The multi-pack-index tracks objects in a collection of pack-files. Only one copy of each object is indexed, using the modified time of the pack-files to determine tie-breakers. It is possible to have a pack-file with no referenced objects because all objects have a duplicate in a newer pack-file. Introduce a new 'expire' subcommand to the multi-pack-index builtin. This subcommand will delete these unused pack-files and rewrite the multi-pack-index to no longer refer to those files. More details about the specifics will follow as the method is implemented. Add a test that verifies the 'expire' subcommand is correctly wired, but will still be valid when the verb is implemented. Specifically, create a set of packs that should all have referenced objects and should not be removed during an 'expire' operation. The packs are created carefully to ensure they have a specific order when sorted by size. This will be important in a later test. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Derrick Stolee committed Jun 10, 2019 at 16:35 UTC cff97116160e57741c9a954a51bfcb0057b3e89d
5 files changed +63 -1
Documentation/git-multi-pack-index.txt
+5
@@ -31,6 +31,11 @@ write::
31 verify::
32 Verify the contents of the MIDX file.
33
34 +expire::
35 + Delete the pack-files that are tracked by the MIDX file, but
36 + have no objects referenced by the MIDX. Rewrite the MIDX file
37 + afterward to remove all references to these pack-files.
38 +
39
40 EXAMPLES
41 --------
builtin/multi-pack-index.c
+3 -1
@@ -6,7 +6,7 @@
6 #include "trace2.h"
7
8 static char const * const builtin_multi_pack_index_usage[] = {
9 - N_("git multi-pack-index [--object-dir=<dir>] (write|verify)"),
9 + N_("git multi-pack-index [--object-dir=<dir>] (write|verify|expire)"),
10 NULL
11 };
12
@@ -47,6 +47,8 @@ int cmd_multi_pack_index(int argc, const char **argv,
47 return write_midx_file(opts.object_dir);
48 if (!strcmp(argv[0], "verify"))
49 return verify_midx_file(the_repository, opts.object_dir);
50 + if (!strcmp(argv[0], "expire"))
51 + return expire_midx_packs(the_repository, opts.object_dir);
52
53 die(_("unrecognized verb: %s"), argv[0]);
54 }
midx.c
+5
@@ -1140,3 +1140,8 @@ int verify_midx_file(struct repository *r, const char *object_dir)
1140
1141 return verify_midx_error;
1142 }
1143 +
1144 +int expire_midx_packs(struct repository *r, const char *object_dir)
1145 +{
1146 + return 0;
1147 +}
midx.h
+1
@@ -50,6 +50,7 @@ int prepare_multi_pack_index_one(struct repository *r, const char *object_dir, i
50 int write_midx_file(const char *object_dir);
51 void clear_midx_file(struct repository *r);
52 int verify_midx_file(struct repository *r, const char *object_dir);
53 +int expire_midx_packs(struct repository *r, const char *object_dir);
54
55 void close_midx(struct multi_pack_index *m);
56
t/t5319-multi-pack-index.sh
+49
@@ -363,4 +363,53 @@ test_expect_success 'verify incorrect 64-bit offset' '
363 "incorrect object offset"
364 '
365
366 +test_expect_success 'setup expire tests' '
367 + mkdir dup &&
368 + (
369 + cd dup &&
370 + git init &&
371 + test-tool genrandom "data" 4096 >large_file.txt &&
372 + git update-index --add large_file.txt &&
373 + for i in $(test_seq 1 20)
374 + do
375 + test_commit $i
376 + done &&
377 + git branch A HEAD &&
378 + git branch B HEAD~8 &&
379 + git branch C HEAD~13 &&
380 + git branch D HEAD~16 &&
381 + git branch E HEAD~18 &&
382 + git pack-objects --revs .git/objects/pack/pack-A <<-EOF &&
383 + refs/heads/A
384 + ^refs/heads/B
385 + EOF
386 + git pack-objects --revs .git/objects/pack/pack-B <<-EOF &&
387 + refs/heads/B
388 + ^refs/heads/C
389 + EOF
390 + git pack-objects --revs .git/objects/pack/pack-C <<-EOF &&
391 + refs/heads/C
392 + ^refs/heads/D
393 + EOF
394 + git pack-objects --revs .git/objects/pack/pack-D <<-EOF &&
395 + refs/heads/D
396 + ^refs/heads/E
397 + EOF
398 + git pack-objects --revs .git/objects/pack/pack-E <<-EOF &&
399 + refs/heads/E
400 + EOF
401 + git multi-pack-index write
402 + )
403 +'
404 +
405 +test_expect_success 'expire does not remove any packs' '
406 + (
407 + cd dup &&
408 + ls .git/objects/pack >expect &&
409 + git multi-pack-index expire &&
410 + ls .git/objects/pack >actual &&
411 + test_cmp expect actual
412 + )
413 +'
414 +
415 test_done