@matheusbsbs / matheus-bagliano / commits / 89afd5f5ad

t/perf: add fsmonitor perf test for git diff

Results for the git-diff fsmonitor optimization in patch in the parent-rev (using a 400k file repo to test) As you can see here - git diff with fsmonitor running is significantly better with this patch series (80% faster on my workload)! GIT_PERF_LARGE_REPO=~/src/server ./run v2.29.0-rc1 . -- p7519-fsmonitor.sh Test v2.29.0-rc1 this tree ----------------------------------------------------------------------------------------------------------------- 7519.2: status (fsmonitor=.git/hooks/fsmonitor-watchman) 1.46(0.82+0.64) 1.47(0.83+0.62) +0.7% 7519.3: status -uno (fsmonitor=.git/hooks/fsmonitor-watchman) 0.16(0.12+0.04) 0.17(0.12+0.05) +6.3% 7519.4: status -uall (fsmonitor=.git/hooks/fsmonitor-watchman) 1.36(0.73+0.62) 1.37(0.76+0.60) +0.7% 7519.5: diff (fsmonitor=.git/hooks/fsmonitor-watchman) 0.85(0.22+0.63) 0.14(0.10+0.05) -83.5% 7519.6: diff -- 0_files (fsmonitor=.git/hooks/fsmonitor-watchman) 0.12(0.08+0.05) 0.13(0.11+0.02) +8.3% 7519.7: diff -- 10_files (fsmonitor=.git/hooks/fsmonitor-watchman) 0.12(0.08+0.04) 0.13(0.09+0.04) +8.3% 7519.8: diff -- 100_files (fsmonitor=.git/hooks/fsmonitor-watchman) 0.12(0.07+0.05) 0.13(0.07+0.06) +8.3% 7519.9: diff -- 1000_files (fsmonitor=.git/hooks/fsmonitor-watchman) 0.12(0.09+0.04) 0.13(0.08+0.05) +8.3% 7519.10: diff -- 10000_files (fsmonitor=.git/hooks/fsmonitor-watchman) 0.14(0.09+0.05) 0.13(0.10+0.03) -7.1% 7519.12: status (fsmonitor=) 1.67(0.93+1.49) 1.67(0.99+1.42) +0.0% 7519.13: status -uno (fsmonitor=) 0.37(0.30+0.82) 0.37(0.33+0.79) +0.0% 7519.14: status -uall (fsmonitor=) 1.58(0.97+1.35) 1.57(0.86+1.45) -0.6% 7519.15: diff (fsmonitor=) 0.34(0.28+0.83) 0.34(0.27+0.83) +0.0% 7519.16: diff -- 0_files (fsmonitor=) 0.09(0.06+0.04) 0.09(0.08+0.02) +0.0% 7519.17: diff -- 10_files (fsmonitor=) 0.09(0.07+0.03) 0.09(0.06+0.05) +0.0% 7519.18: diff -- 100_files (fsmonitor=) 0.09(0.06+0.04) 0.09(0.06+0.04) +0.0% 7519.19: diff -- 1000_files (fsmonitor=) 0.09(0.06+0.04) 0.09(0.05+0.05) +0.0% 7519.20: diff -- 10000_files (fsmonitor=) 0.10(0.08+0.04) 0.10(0.06+0.05) +0.0% I also added a benchmark for a tiny git diff workload w/ a pathspec. I see an approximately .02 second overhead added w/ and w/o fsmonitor From looking at these results, I suspected that refresh_fsmonitor is already happening during git diff - independent of this patch series' optimization. Confirmed that suspicion by breaking on refresh_fsmonitor. (gdb) bt [simplified] 0 refresh_fsmonitor at fsmonitor.c:176 1 ie_match_stat at read-cache.c:375 2 match_stat_with_submodule at diff-lib.c:237 4 builtin_diff_files at builtin/diff.c:260 5 cmd_diff at builtin/diff.c:541 6 run_builtin at git.c:450 7 handle_builtin at git.c:700 8 run_argv at git.c:767 9 cmd_main at git.c:898 10 main at common-main.c:52 Signed-off-by: Nipunn Koorapati <nipunn@dropbox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Nipunn Koorapati committed Oct 20, 2020 at 13:41 UTC 89afd5f5ad920d1b5f01548bf5c8aeca8e56e899
1 file changed +71
t/perf/p7519-fsmonitor.sh
+71
@@ -115,6 +115,13 @@ test_expect_success "setup for fsmonitor" '
115
116 git config core.fsmonitor "$INTEGRATION_SCRIPT" &&
117 git update-index --fsmonitor &&
118 + mkdir 1_file 10_files 100_files 1000_files 10000_files &&
119 + for i in $(test_seq 1 10); do touch 10_files/$i; done &&
120 + for i in $(test_seq 1 100); do touch 100_files/$i; done &&
121 + for i in $(test_seq 1 1000); do touch 1000_files/$i; done &&
122 + for i in $(test_seq 1 10000); do touch 10000_files/$i; done &&
123 + git add 1_file 10_files 100_files 1000_files 10000_files &&
124 + git commit -m "Add files" &&
125 git status # Warm caches
126 '
127
@@ -142,6 +149,38 @@ test_perf "status -uall (fsmonitor=$INTEGRATION_SCRIPT)" '
149 git status -uall
150 '
151
152 +if test -n "$GIT_PERF_7519_DROP_CACHE"; then
153 + test-tool drop-caches
154 +fi
155 +
156 +test_perf "diff (fsmonitor=$INTEGRATION_SCRIPT)" '
157 + git diff
158 +'
159 +
160 +if test -n "$GIT_PERF_7519_DROP_CACHE"; then
161 + test-tool drop-caches
162 +fi
163 +
164 +test_perf "diff -- 0_files (fsmonitor=$INTEGRATION_SCRIPT)" '
165 + git diff -- 1_file
166 +'
167 +
168 +test_perf "diff -- 10_files (fsmonitor=$INTEGRATION_SCRIPT)" '
169 + git diff -- 10_files
170 +'
171 +
172 +test_perf "diff -- 100_files (fsmonitor=$INTEGRATION_SCRIPT)" '
173 + git diff -- 100_files
174 +'
175 +
176 +test_perf "diff -- 1000_files (fsmonitor=$INTEGRATION_SCRIPT)" '
177 + git diff -- 1000_files
178 +'
179 +
180 +test_perf "diff -- 10000_files (fsmonitor=$INTEGRATION_SCRIPT)" '
181 + git diff -- 10000_files
182 +'
183 +
184 test_expect_success "setup without fsmonitor" '
185 unset INTEGRATION_SCRIPT &&
186 git config --unset core.fsmonitor &&
@@ -172,6 +211,38 @@ test_perf "status -uall (fsmonitor=$INTEGRATION_SCRIPT)" '
211 git status -uall
212 '
213
214 +if test -n "$GIT_PERF_7519_DROP_CACHE"; then
215 + test-tool drop-caches
216 +fi
217 +
218 +test_perf "diff (fsmonitor=$INTEGRATION_SCRIPT)" '
219 + git diff
220 +'
221 +
222 +if test -n "$GIT_PERF_7519_DROP_CACHE"; then
223 + test-tool drop-caches
224 +fi
225 +
226 +test_perf "diff -- 0_files (fsmonitor=$INTEGRATION_SCRIPT)" '
227 + git diff -- 1_file
228 +'
229 +
230 +test_perf "diff -- 10_files (fsmonitor=$INTEGRATION_SCRIPT)" '
231 + git diff -- 10_files
232 +'
233 +
234 +test_perf "diff -- 100_files (fsmonitor=$INTEGRATION_SCRIPT)" '
235 + git diff -- 100_files
236 +'
237 +
238 +test_perf "diff -- 1000_files (fsmonitor=$INTEGRATION_SCRIPT)" '
239 + git diff -- 1000_files
240 +'
241 +
242 +test_perf "diff -- 10000_files (fsmonitor=$INTEGRATION_SCRIPT)" '
243 + git diff -- 10000_files
244 +'
245 +
246 if test_have_prereq WATCHMAN
247 then
248 watchman watch-del "$GIT_WORK_TREE" >/dev/null 2>&1 &&