@matheusbsbs / matheus-bagliano / commits / ee34a2bead

repack: add `repack.packKeptObjects` config var

The git-repack command always passes `--honor-pack-keep` to pack-objects. This has traditionally been a good thing, as we do not want to duplicate those objects in a new pack, and we are not going to delete the old pack. However, when bitmaps are in use, it is important for a full repack to include all reachable objects, even if they may be duplicated in a .keep pack. Otherwise, we cannot generate the bitmaps, as the on-disk format requires the set of objects in the pack to be fully closed. Even if the repository does not generally have .keep files, a simultaneous push could cause a race condition in which a .keep file exists at the moment of a repack. The repack may try to include those objects in one of two situations: 1. The pushed .keep pack contains objects that were already in the repository (e.g., blobs due to a revert of an old commit). 2. Receive-pack updates the refs, making the objects reachable, but before it removes the .keep file, the repack runs. In either case, we may prefer to duplicate some objects in the new, full pack, and let the next repack (after the .keep file is cleaned up) take care of removing them. This patch introduces both a command-line and config option to disable the `--honor-pack-keep` option. By default, it is triggered when pack.writeBitmaps (or `--write-bitmap-index` is turned on), but specifying it explicitly can override the behavior (e.g., in cases where you prefer .keep files to bitmaps, but only when they are present). Note that this option just disables the pack-objects behavior. We still leave packs with a .keep in place, as we do not necessarily know that we have duplicated all of their objects. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Jeff King committed Mar 3, 2014 at 15:04 UTC ee34a2beadb94a9595f09af719e3c09b485ca797

4 files changed +44 -2

Documentation/config.txt

@@ -2128,6 +2128,13 @@ repack.usedeltabaseoffset::

2128 "false" and repack. Access from old Git versions over the

2129 native protocol are unaffected by this option.

2130

2131 +repack.packKeptObjects::

2132 + If set to true, makes `git repack` act as if

2133 + `--pack-kept-objects` was passed. See linkgit:git-repack[1] for

2134 + details. Defaults to `false` normally, but `true` if a bitmap

2135 + index is being written (either via `--write-bitmap-index` or

2136 + `pack.writeBitmaps`).

2137 +

2138 rerere.autoupdate::

2139 When set to true, `git-rerere` updates the index with the

2140 resulting contents after it cleanly resolves conflicts using

Documentation/git-repack.txt

@@ -117,6 +117,14 @@ other objects in that pack they already have locally.

117 must be able to refer to all reachable objects. This option

118 overrides the setting of `pack.writebitmaps`.

119

120 +--pack-kept-objects::

121 + Include objects in `.keep` files when repacking. Note that we

122 + still do not delete `.keep` packs after `pack-objects` finishes.

123 + This means that we may duplicate objects, but this makes the

124 + option safe to use when there are concurrent pushes or fetches.

125 + This option is generally only useful if you are writing bitmaps

126 + with `-b` or `pack.writebitmaps`, as it ensures that the

127 + bitmapped packfile has the necessary objects.

128

129 Configuration

130 -------------

builtin/repack.c

+12 -1

@@ -9,6 +9,7 @@

9 #include "argv-array.h"

11 static int delta_base_offset = 1;

12 +static int pack_kept_objects = -1;

13 static char *packdir, *packtmp;

15 static const char *const git_repack_usage[] = {

@@ -22,6 +23,10 @@ static int repack_config(const char *var, const char *value, void *cb)

23 delta_base_offset = git_config_bool(var, value);

24 return 0;

25 }

26 + if (!strcmp(var, "repack.packkeptobjects")) {

27 + pack_kept_objects = git_config_bool(var, value);

28 + return 0;

29 + }

30 return git_default_config(var, value, cb);

31 }

@@ -175,6 +180,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix)

180 N_("limits the maximum delta depth")),

181 OPT_INTEGER(0, "max-pack-size", &max_pack_size,

182 N_("maximum size of each packfile")),

183 + OPT_BOOL(0, "pack-kept-objects", &pack_kept_objects,

184 + N_("repack objects in packs marked with .keep")),

185 OPT_END()

186 };

187

@@ -183,6 +190,9 @@ int cmd_repack(int argc, const char **argv, const char *prefix)

190 argc = parse_options(argc, argv, prefix, builtin_repack_options,

191 git_repack_usage, 0);

192

193 + if (pack_kept_objects < 0)

194 + pack_kept_objects = write_bitmap;

195 +

196 packdir = mkpathdup("%s/pack", get_object_directory());

197 packtmp = mkpathdup("%s/.tmp-%d-pack", packdir, (int)getpid());

198

@@ -190,7 +200,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix)

200

201 argv_array_push(&cmd_args, "pack-objects");

202 argv_array_push(&cmd_args, "--keep-true-parents");

193 - argv_array_push(&cmd_args, "--honor-pack-keep");

203 + if (!pack_kept_objects)

204 + argv_array_push(&cmd_args, "--honor-pack-keep");

205 argv_array_push(&cmd_args, "--non-empty");

206 argv_array_push(&cmd_args, "--all");

207 argv_array_push(&cmd_args, "--reflog");

t/t7700-repack.sh

+17 -1

@@ -21,7 +21,7 @@ test_expect_success 'objects in packs marked .keep are not repacked' '

21 objsha1=$(git verify-pack -v pack-$packsha1.idx | head -n 1 |

22 sed -e "s/^$[0-9a-f]\{40\}$.*/\1/") &&

23 mv pack-* .git/objects/pack/ &&

24 - git repack -A -d -l &&

24 + git repack --no-pack-kept-objects -A -d -l &&

25 git prune-packed &&

26 for p in .git/objects/pack/*.idx; do

27 idx=$(basename $p)

@@ -35,6 +35,22 @@ test_expect_success 'objects in packs marked .keep are not repacked' '

35 test -z "$found_duplicate_object"

36 '

38 +test_expect_success 'writing bitmaps can duplicate .keep objects' '

39 + # build on $objsha1, $packsha1, and .keep state from previous

40 + git repack -Adl &&

41 + test_when_finished "found_duplicate_object=" &&

42 + for p in .git/objects/pack/*.idx; do

43 + idx=$(basename $p)

44 + test "pack-$packsha1.idx" = "$idx" && continue

45 + if git verify-pack -v $p | egrep "^$objsha1"; then

46 + found_duplicate_object=1

47 + echo "DUPLICATE OBJECT FOUND"

48 + break

49 + fi

50 + done &&

51 + test "$found_duplicate_object" = 1

52 +'

53 +

54 test_expect_success 'loose objects in alternate ODB are not repacked' '

55 mkdir alt_objects &&

56 echo `pwd`/alt_objects > .git/objects/info/alternates &&