@matheusbsbs / matheus-bagliano / commits / 1385bb7ba3

reachable: only mark local objects as recent

When pruning and repacking a repository that has an alternate object store configured, we may traverse a large number of objects in the alternate. This serves no purpose, and may be expensive to do. A longer explanation is below. Commits d3038d2 and abcb865 taught prune and pack-objects (respectively) to treat "recent" objects as tips for reachability, so that we keep whole chunks of history. They built on the object traversal in 660c889 (sha1_file: add for_each iterators for loose and packed objects, 2014-10-15), which covers both local and alternate objects. In both cases, covering alternate objects is unnecessary, as both commands can only drop objects from the local repository. In the case of prune, we traverse only the local object directory. And in the case of repacking, while we may or may not include local objects in our pack, we will never reach into the alternate with "repack -d". The "-l" option is only a question of whether we are migrating objects from the alternate into our repository, or leaving them untouched. It is possible that we may drop an object that is depended upon by another object in the alternate. For example, imagine two repositories, A and B, with A pointing to B as an alternate. Now imagine a commit that is in B which references a tree that is only in A. Traversing from recent objects in B might prevent A from dropping that tree. But this case isn't worth covering. Repo B should take responsibility for its own objects. It would never have had the commit in the first place if it did not also have the tree, and assuming it is using the same "keep recent chunks of history" scheme, then it would itself keep the tree, as well. So checking the alternate objects is not worth doing, and come with a significant performance impact. In both cases, we skip any recent objects that have already been marked SEEN (i.e., that we know are already reachable for prune, or included in the pack for a repack). So there is a slight waste of time in opening the alternate packs at all, only to notice that we have already considered each object. But much worse, the alternate repository may have a large number of objects that are not reachable from the local repository at all, and we end up adding them to the traversal. We can fix this by considering only local unseen objects. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Jeff King committed Mar 27, 2015 at 07:32 UTC 1385bb7ba39128de0a5bc4ff6e8a5ad03fc49205

3 files changed +16 -7

cache.h

+5 -3

@@ -1283,14 +1283,16 @@ int for_each_loose_file_in_objdir_buf(struct strbuf *path,

1283

1284 /*

1285 * Iterate over loose and packed objects in both the local

1286 - * repository and any alternates repositories.

1286 + * repository and any alternates repositories (unless the

1287 + * LOCAL_ONLY flag is set).

1288 */

1289 +#define FOR_EACH_OBJECT_LOCAL_ONLY 0x1

1290 typedef int each_packed_object_fn(const unsigned char *sha1,

1291 struct packed_git *pack,

1292 uint32_t pos,

1293 void *data);

1292 -extern int for_each_loose_object(each_loose_object_fn, void *);

1293 -extern int for_each_packed_object(each_packed_object_fn, void *);

1294 +extern int for_each_loose_object(each_loose_object_fn, void *, unsigned flags);

1295 +extern int for_each_packed_object(each_packed_object_fn, void *, unsigned flags);

1296

1297 struct object_info {

1298 /* Request */

reachable.c

+4 -2

@@ -142,10 +142,12 @@ int add_unseen_recent_objects_to_traversal(struct rev_info *revs,

142 data.revs = revs;

143 data.timestamp = timestamp;

144

145 - r = for_each_loose_object(add_recent_loose, &data);

145 + r = for_each_loose_object(add_recent_loose, &data,

146 + FOR_EACH_OBJECT_LOCAL_ONLY);

147 if (r)

148 return r;

148 - return for_each_packed_object(add_recent_packed, &data);

149 + return for_each_packed_object(add_recent_packed, &data,

150 + FOR_EACH_OBJECT_LOCAL_ONLY);

151 }

152

153 void mark_reachable_objects(struct rev_info *revs, int mark_reflog,

sha1_file.c

+7 -2

@@ -3418,7 +3418,7 @@ static int loose_from_alt_odb(struct alternate_object_database *alt,

3418 return r;

3419 }

3420

3421 -int for_each_loose_object(each_loose_object_fn cb, void *data)

3421 +int for_each_loose_object(each_loose_object_fn cb, void *data, unsigned flags)

3422 {

3423 struct loose_alt_odb_data alt;

3424 int r;

@@ -3428,6 +3428,9 @@ int for_each_loose_object(each_loose_object_fn cb, void *data)

3428 if (r)

3429 return r;

3430

3431 + if (flags & FOR_EACH_OBJECT_LOCAL_ONLY)

3432 + return 0;

3433 +

3434 alt.cb = cb;

3435 alt.data = data;

3436 return foreach_alt_odb(loose_from_alt_odb, &alt);

@@ -3452,13 +3455,15 @@ static int for_each_object_in_pack(struct packed_git *p, each_packed_object_fn c

3455 return r;

3456 }

3457

3455 -int for_each_packed_object(each_packed_object_fn cb, void *data)

3458 +int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)

3459 {

3460 struct packed_git *p;

3461 int r = 0;

3462

3463 prepare_packed_git();

3464 for (p = packed_git; p; p = p->next) {

3465 + if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)

3466 + continue;

3467 r = for_each_object_in_pack(p, cb, data);

3468 if (r)

3469 break;