@matheusbsbs / matheus-bagliano / commits / a937b37e76

revision: quit pruning diff more quickly when possible

When the revision traversal machinery is given a pathspec, we must compute the parent-diff for each commit to determine which ones are TREESAME. We set the QUICK diff flag to avoid looking at more entries than we need; we really just care whether there are any changes at all. But there is one case where we want to know a bit more: if --remove-empty is set, we care about finding cases where the change consists only of added entries (in which case we may prune the parent in try_to_simplify_commit()). To cover that case, our file_add_remove() callback does not quit the diff upon seeing an added entry; it keeps looking for other types of entries. But this means when --remove-empty is not set (and it is not by default), we compute more of the diff than is necessary. You can see this in a pathological case where a commit adds a very large number of entries, and we limit based on a broad pathspec. E.g.: perl -e ' chomp(my $blob = `git hash-object -w --stdin </dev/null`); for my $a (1..1000) { for my $b (1..1000) { print "100644 $blob\t$a/$b\n"; } } ' | git update-index --index-info git commit -qm add git rev-list HEAD -- . This case takes about 100ms now, but after this patch only needs 6ms. That's not a huge improvement, but it's easy to get and it protects us against even more pathological cases (e.g., going from 1 million to 10 million files would take ten times as long with the current code, but not increase at all after this patch). This is reported to minorly speed-up pathspec limiting in real world repositories (like the 100-million-file Windows repository), but probably won't make a noticeable difference outside of pathological setups. This patch actually covers the case without --remove-empty, and the case where we see only deletions. See the in-code comment for details. Note that we have to add a new member to the diff_options struct so that our callback can see the value of revs->remove_empty_trees. This callback parameter could be passed to the "add_remove" and "change" callbacks, but there's not much point. They already receive the diff_options struct, and doing it this way avoids having to update the function signature of the other callbacks (arguably the format_callback and output_prefix functions could benefit from the same simplification). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Jeff King committed Oct 13, 2017 at 11:27 UTC a937b37e766479c8e780b17cce9c4b252fd97e40

2 files changed +14 -3

diff.h

@@ -180,6 +180,7 @@ struct diff_options {

180 pathchange_fn_t pathchange;

181 change_fn_t change;

182 add_remove_fn_t add_remove;

183 + void *change_fn_data;

184 diff_format_fn_t format_callback;

185 void *format_callback_data;

186 diff_prefix_fn_t output_prefix;

revision.c

+13 -3

@@ -394,8 +394,16 @@ static struct commit *one_relevant_parent(const struct rev_info *revs,

394 * if the whole diff is removal of old data, and otherwise

395 * REV_TREE_DIFFERENT (of course if the trees are the same we

396 * want REV_TREE_SAME).

397 - * That means that once we get to REV_TREE_DIFFERENT, we do not

398 - * have to look any further.

397 + *

398 + * The only time we care about the distinction is when

399 + * remove_empty_trees is in effect, in which case we care only about

400 + * whether the whole change is REV_TREE_NEW, or if there's another type

401 + * of change. Which means we can stop the diff early in either of these

402 + * cases:

403 + *

404 + * 1. We're not using remove_empty_trees at all.

405 + *

406 + * 2. We saw anything except REV_TREE_NEW.

407 */

408 static int tree_difference = REV_TREE_SAME;

409

@@ -406,9 +414,10 @@ static void file_add_remove(struct diff_options *options,

414 const char *fullpath, unsigned dirty_submodule)

415 {

416 int diff = addremove == '+' ? REV_TREE_NEW : REV_TREE_OLD;

417 + struct rev_info *revs = options->change_fn_data;

418

419 tree_difference |= diff;

411 - if (tree_difference == REV_TREE_DIFFERENT)

420 + if (!revs->remove_empty_trees || tree_difference != REV_TREE_NEW)

421 DIFF_OPT_SET(options, HAS_CHANGES);

422 }

423

@@ -1346,6 +1355,7 @@ void init_revisions(struct rev_info *revs, const char *prefix)

1355 DIFF_OPT_SET(&revs->pruning, QUICK);

1356 revs->pruning.add_remove = file_add_remove;

1357 revs->pruning.change = file_change;

1358 + revs->pruning.change_fn_data = revs;

1359 revs->sort_order = REV_SORT_IN_GRAPH_ORDER;

1360 revs->dense = 1;

1361 revs->prefix = prefix;