@matheusbsbs / matheus-bagliano / commits / 3ce9e5f293

p5313: add performance tests for --path-walk

The previous change added a --path-walk option to 'git pack-objects'. Create a performance test that demonstrates the time and space benefits of the feature. In order to get an appropriate comparison, we need to avoid reusing deltas and recompute them from scratch. Compare the creation of a thin pack representing a small push and the creation of a relatively large non-thin pack. Running on my copy of the Git repository results in this data (removing the repack tests for --name-hash-version): Test this tree ------------------------------------------------------------------------ 5313.2: thin pack with --name-hash-version=1 0.02(0.01+0.01) 5313.3: thin pack size with --name-hash-version=1 1.6K 5313.4: big pack with --name-hash-version=1 2.55(4.20+0.26) 5313.5: big pack size with --name-hash-version=1 16.4M 5313.6: shallow fetch pack with --name-hash-version=1 1.24(2.03+0.08) 5313.7: shallow pack size with --name-hash-version=1 12.2M 5313.10: thin pack with --name-hash-version=2 0.03(0.01+0.01) 5313.11: thin pack size with --name-hash-version=2 1.6K 5313.12: big pack with --name-hash-version=2 1.91(3.23+0.20) 5313.13: big pack size with --name-hash-version=2 16.4M 5313.14: shallow fetch pack with --name-hash-version=2 1.06(1.57+0.10) 5313.15: shallow pack size with --name-hash-version=2 12.5M 5313.18: thin pack with --path-walk 0.03(0.01+0.01) 5313.19: thin pack size with --path-walk 1.6K 5313.20: big pack with --path-walk 2.05(3.24+0.27) 5313.21: big pack size with --path-walk 16.3M 5313.22: shallow fetch pack with --path-walk 1.08(1.66+0.07) 5313.23: shallow pack size with --path-walk 12.4M This can be reformatted as follows: Pack Type Hash v1 Hash v2 Path Walk --------------------------------------------------- thin pack (time) 0.02s 0.03s 0.03s (size) 1.6K 1.6K 1.6K big pack (time) 2.55s 1.91s 2.05s (size) 16.4M 16.4M 16.3M shallow pack (time) 1.24s 1.06s 1.08s (size) 12.2M 12.5M 12.4M Note that the timing is slower because there is no threading in the --path-walk case (yet). Also, the shallow pack cases are really not using the --path-walk logic right now because it is disabled until some additions are made to the path walk API. The cases where the --path-walk option really shines is when the default name-hash is overwhelmed with unhelpful collisions. An open source example can be found in the microsoft/fluentui repo [1] at a certain commit [2]. [1] https://github.com/microsoft/fluentui [2] e70848ebac1cd720875bccaa3026f4a9ed700e08 Running the tests on this repo results in the following comparison table: Pack Type Hash v1 Hash v2 Path Walk --------------------------------------------------- thin pack (time) 0.36s 0.12s 0.08s (size) 1.2M 22.0K 18.4K big pack (time) 2.00s 2.90s 2.21s (size) 20.4M 25.9M 19.5M shallow pack (time) 1.41s 1.80s 1.65s (size) 34.4M 33.7M 33.6M Notice in particular that in the small thin pack, the time performance has improved from 0.36s for --name-hash-version=1 to 0.08s and this is likely due to the improved size of the resulting pack: 18.4K instead of 1.2M. The relatively new --name-hash-version=2 is competitive with --path-walk (0.12s and 22.0K) but not quite as successful. Finally, running this on a copy of the Linux kernel repository results in these data points: Pack Type Hash v1 Hash v2 Path Walk --------------------------------------------------- thin pack (time) 0.03s 0.13s 0.03s (size) 4.6K 4.6K 4.6K big pack (time) 15.29s 12.32s 13.92s (size) 201.1M 159.1M 158.5M shallow pack (time) 10.88s 22.93s 22.74s (size) 269.2M 273.8M 267.7M Signed-off-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Derrick Stolee committed May 16, 2025 at 18:11 UTC 3ce9e5f2934f88921bac00548b3c422365764da2
1 file changed +23 -14
t/perf/p5313-pack-objects.sh
+23 -14
@@ -25,46 +25,55 @@ test_expect_success 'create rev input' '
25 EOF
26 '
27
28 -for version in 1 2
29 -do
30 - export version
28 +test_all_with_args () {
29 + parameter=$1
30 + export parameter
31
32 - test_perf "thin pack with version $version" '
32 + test_perf "thin pack with $parameter" '
33 git pack-objects --thin --stdout --revs --sparse \
34 - --name-hash-version=$version <in-thin >out
34 + $parameter <in-thin >out
35 '
36
37 - test_size "thin pack size with version $version" '
37 + test_size "thin pack size with $parameter" '
38 test_file_size out
39 '
40
41 - test_perf "big pack with version $version" '
41 + test_perf "big pack with $parameter" '
42 git pack-objects --stdout --revs --sparse \
43 - --name-hash-version=$version <in-big >out
43 + $parameter <in-big >out
44 '
45
46 - test_size "big pack size with version $version" '
46 + test_size "big pack size with $parameter" '
47 test_file_size out
48 '
49
50 - test_perf "shallow fetch pack with version $version" '
50 + test_perf "shallow fetch pack with $parameter" '
51 git pack-objects --stdout --revs --sparse --shallow \
52 - --name-hash-version=$version <in-shallow >out
52 + $parameter <in-shallow >out
53 '
54
55 - test_size "shallow pack size with version $version" '
55 + test_size "shallow pack size with $parameter" '
56 test_file_size out
57 '
58 +}
59
59 - test_perf "repack with version $version" '
60 +for version in 1 2
61 +do
62 + export version
63 +
64 + test_all_with_args --name-hash-version=$version
65 +
66 + test_perf "repack with --name-hash-version=$version" '
67 git repack -adf --name-hash-version=$version
68 '
69
63 - test_size "repack size with version $version" '
70 + test_size "repack size with --name-hash-version=$version" '
71 gitdir=$(git rev-parse --git-dir) &&
72 pack=$(ls $gitdir/objects/pack/pack-*.pack) &&
73 test_file_size "$pack"
74 '
75 done
76
77 +test_all_with_args --path-walk
78 +
79 test_done