@matheusbsbs / matheus-bagliano / commits / 890226ccb5

commit-graph write: add itermediate progress

Add progress output to sections of code between "Annotating[...]" and "Computing[...]generation numbers". This can collectively take 5-10 seconds on a large enough repository. On a test repository with I have with ~7 million commits and ~50 million objects we'll now emit: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (124763727/124763727), done. Loading known commits in commit graph: 100% (18989461/18989461), done. Expanding reachable commits in commit graph: 100% (18989507/18989461), done. Clearing commit marks in commit graph: 100% (18989507/18989507), done. Counting distinct commits in commit graph: 100% (18989507/18989507), done. Finding extra edges in commit graph: 100% (18989507/18989507), done. Computing commit graph generation numbers: 100% (7250302/7250302), done. Writing out commit graph in 4 passes: 100% (29001208/29001208), done. Whereas on a medium-sized repository such as linux.git these new progress bars won't have time to kick in and as before and we'll still emit output like: $ ~/g/git/git --exec-path=$HOME/g/git commit-graph write Finding commits for commit graph among packed objects: 100% (6529159/6529159), done. Expanding reachable commits in commit graph: 815990, done. Computing commit graph generation numbers: 100% (815983/815983), done. Writing out commit graph in 4 passes: 100% (3263932/3263932), done. The "Counting distinct commits in commit graph" phase will spend most of its time paused at "0/*" as we QSORT(...) the list. That's not optimal, but at least we don't seem to be stalling anymore most of the time. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Ævar Arnfjörð Bjarmason committed Jan 19, 2019 at 21:21 UTC 890226ccb57d6f9657c29aadfe1c106b939afbc8

1 file changed +13

commit-graph.c

+13

@@ -895,12 +895,19 @@ void write_commit_graph(const char *obj_dir,

895

896 close_reachable(&oids, report_progress);

897

898 + if (report_progress)

899 + progress = start_delayed_progress(

900 + _("Counting distinct commits in commit graph"),

901 + oids.nr);

902 + display_progress(progress, 0); /* TODO: Measure QSORT() progress */

903 QSORT(oids.list, oids.nr, commit_compare);

904 count_distinct = 1;

905 for (i = 1; i < oids.nr; i++) {

906 + display_progress(progress, i + 1);

907 if (!oideq(&oids.list[i - 1], &oids.list[i]))

908 count_distinct++;

909 }

910 + stop_progress(&progress);

911

912 if (count_distinct >= GRAPH_PARENT_MISSING)

913 die(_("the commit graph format cannot write %d commits"), count_distinct);

@@ -910,8 +917,13 @@ void write_commit_graph(const char *obj_dir,

917 ALLOC_ARRAY(commits.list, commits.alloc);

918

919 num_extra_edges = 0;

920 + if (report_progress)

921 + progress = start_delayed_progress(

922 + _("Finding extra edges in commit graph"),

923 + oids.nr);

924 for (i = 0; i < oids.nr; i++) {

925 int num_parents = 0;

926 + display_progress(progress, i + 1);

927 if (i > 0 && oideq(&oids.list[i - 1], &oids.list[i]))

928 continue;

929

@@ -928,6 +940,7 @@ void write_commit_graph(const char *obj_dir,

940 commits.nr++;

941 }

942 num_chunks = num_extra_edges ? 4 : 3;

943 + stop_progress(&progress);

944

945 if (commits.nr >= GRAPH_PARENT_MISSING)

946 die(_("too many commits to write graph"));