t/test-lib: allow skipping leak checks for passing tests

With `GIT_TEST_PASSING_SANITIZE_LEAK=check`, one can double check whether a memory leak fix caused some test suites to become leak free. This is done by running all tests with the leak checker enabled. If a test suite does not declare `TEST_PASSES_SANITIZE_LEAK=true` but still finishes successfully with the leak checker enabled, then this indicates that the test is leak free and thus missing the annotation. It is somewhat slow to execute though because it runs all of our test suites with the leak sanitizer enabled. It is also pointless in most cases, because the only test suites that need to be checked are those which _aren't_ yet marked with `TEST_PASSES_SANITIZE_LEAK=true`. Introduce a new value "check-failing". When set, we behave the same as if "check" was passed, except that we only check those tests which do not have `TEST_PASSES_SANITIZE_LEAK=true` set. This is significantly faster than running all test suites but still fulfills the usecase of finding newly-leak-free test suites. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Patrick Steinhardt committed Sep 5, 2024 at 12:08 UTC a9539a993a2b4dbfb5540aebb02bfcfd5be4c24f
2 files changed +13 -1
t/README
+3
@@ -386,6 +386,9 @@ GIT_TEST_PASSING_SANITIZE_LEAK=check when combined with "--immediate"
386 will run to completion faster, and result in the same failing
387 tests.
388
389 +GIT_TEST_PASSING_SANITIZE_LEAK=check-failing behaves the same as "check",
390 +but skips all tests which are already marked as leak-free.
391 +
392 GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version'
393 default to n.
394
t/test-lib.sh
+10 -1
@@ -1558,8 +1558,16 @@ then
1558 passes_sanitize_leak=t
1559 fi
1560
1561 - if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check"
1561 + if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" ||
1562 + test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check-failing"
1563 then
1564 + if test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check-failing" &&
1565 + test -n "$passes_sanitize_leak"
1566 + then
1567 + skip_all="skipping leak-free $this_test under GIT_TEST_PASSING_SANITIZE_LEAK=check-failing"
1568 + test_done
1569 + fi
1570 +
1571 sanitize_leak_check=t
1572 if test -n "$invert_exit_code"
1573 then
@@ -1597,6 +1605,7 @@ then
1605 export LSAN_OPTIONS
1606
1607 elif test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check" ||
1608 + test "$GIT_TEST_PASSING_SANITIZE_LEAK" = "check-failing" ||
1609 test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false
1610 then
1611 BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_PASSING_SANITIZE_LEAK=true"