repack: use tempfiles for signal cleanup

When git-repack exits due to a signal, it tries to clean up by calling its remove_temporary_files() function, which walks through the packs dir looking for ".tmp-$$-pack-*" files to delete (where "$$" is the pid of the current process). The biggest problem here is that remove_temporary_files() is not safe to call in a signal handler. It uses opendir(), which isn't on the POSIX async-signal-safe list. The details will be platform-specific, but a likely issue is that it needs to allocate memory; if we receive a signal while inside malloc(), etc, we'll conflict on the allocator lock and deadlock with ourselves. We can fix this by just cleaning up the files directly, without walking the directory. We already know the complete list of .tmp-* files that were generated, because we recorded them via populate_pack_exts(). When we find files there, we can use register_tempfile() to record the filenames. If we receive a signal, then the tempfile API will clean them up for us, and it's async-safe and pretty battle-tested. Note that this is slightly racier than the existing scheme. We don't record the filenames until pack-objects tells us the hash over stdout. So during the period between it generating the file and reporting the hash, we'd fail to clean up. However, that period is very small. During most of the pack generation process pack-objects is using its own internal tempfiles. It's only at the very end that it moves them into the names git-repack expects, and then it immediately reports the name to us. Given that cleanup like this is best effort (after all, we may get SIGKILL), this level of race is acceptable. When we register the tempfiles, we'll record them locally and use the result to call rename_tempfile(), rather than renaming by hand. This isn't strictly necessary, as once we've renamed the files they're gone, and the tempfile API's cleanup unlink() would simply become a pointless noop. But managing the lifetimes of the tempfile objects is the cleanest thing to do, and the tempfile pointers naturally fill the same role as the old booleans. This patch also fixes another small problem. We only hook signals, and don't set up an atexit handler. So if we see an error that causes us to die(), we'll leave the .tmp-* files in place. But since the tempfile API handles this for us, this is now fixed for free. The new test covers this by stimulating a failure of pack-objects when generating a cruft pack. Before this patch, the .tmp-* file for the main pack would have been left, but now we correctly clean it up. Two small subtleties on the implementation: - in the renaming loop, we can stop re-constructing fname_old; we only use it when we have a tempfile to rename, so we can just ask the tempfile for its path (which, barring bugs, should be identical) - when renaming fails, our error message mentions fname_old. But since a failed rename_tempfile() invalidates the tempfile struct, we'll lose access to that string. Instead, let's mention the destination filename, which is what most other callers do. Reported-by: Jan Pokorný <poki@fnusa.cz> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Jeff King committed Oct 21, 2022 at 20:21 UTC 9cf10d8786c47c18ced6cb00e140cc18db8a7509
2 files changed +16 -18
builtin/repack.c
+8 -18
@@ -122,13 +122,6 @@ static void remove_temporary_files(void)
122 strbuf_release(&buf);
123 }
124
125 -static void remove_pack_on_signal(int signo)
126 -{
127 - remove_temporary_files();
128 - sigchain_pop(signo);
129 - raise(signo);
130 -}
131 -
125 /*
126 * Adds all packs hex strings to either fname_nonkept_list or
127 * fname_kept_list based on whether each pack has a corresponding
@@ -248,7 +241,7 @@ static struct {
241 };
242
243 struct generated_pack_data {
251 - char exts[ARRAY_SIZE(exts)];
244 + struct tempfile *tempfiles[ARRAY_SIZE(exts)];
245 };
246
247 static struct generated_pack_data *populate_pack_exts(const char *name)
@@ -265,7 +258,7 @@ static struct generated_pack_data *populate_pack_exts(const char *name)
258 if (stat(path.buf, &statbuf))
259 continue;
260
268 - data->exts[i] = 1;
261 + data->tempfiles[i] = register_tempfile(path.buf);
262 }
263
264 strbuf_release(&path);
@@ -867,8 +860,6 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
860 split_pack_geometry(geometry, geometric_factor);
861 }
862
870 - sigchain_push_common(remove_pack_on_signal);
871 -
863 prepare_pack_objects(&cmd, &po_args);
864
865 show_progress = !po_args.quiet && isatty(2);
@@ -1013,22 +1004,22 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
1004 struct generated_pack_data *data = item->util;
1005
1006 for (ext = 0; ext < ARRAY_SIZE(exts); ext++) {
1016 - char *fname, *fname_old;
1007 + char *fname;
1008
1009 fname = mkpathdup("%s/pack-%s%s",
1010 packdir, item->string, exts[ext].name);
1020 - fname_old = mkpathdup("%s-%s%s",
1021 - packtmp, item->string, exts[ext].name);
1011
1023 - if (data->exts[ext]) {
1012 + if (data->tempfiles[ext]) {
1013 + const char *fname_old = get_tempfile_path(data->tempfiles[ext]);
1014 struct stat statbuffer;
1015 +
1016 if (!stat(fname_old, &statbuffer)) {
1017 statbuffer.st_mode &= ~(S_IWUSR | S_IWGRP | S_IWOTH);
1018 chmod(fname_old, statbuffer.st_mode);
1019 }
1020
1030 - if (rename(fname_old, fname))
1031 - die_errno(_("renaming '%s' failed"), fname_old);
1021 + if (rename_tempfile(&data->tempfiles[ext], fname))
1022 + die_errno(_("renaming pack to '%s' failed"), fname);
1023 } else if (!exts[ext].optional)
1024 die(_("pack-objects did not write a '%s' file for pack %s-%s"),
1025 exts[ext].name, packtmp, item->string);
@@ -1036,7 +1027,6 @@ int cmd_repack(int argc, const char **argv, const char *prefix)
1027 die_errno(_("could not unlink: %s"), fname);
1028
1029 free(fname);
1039 - free(fname_old);
1030 }
1031 }
1032 /* End of pack replacement. */
t/t7700-repack.sh
+8
@@ -432,6 +432,14 @@ test_expect_success TTY '--quiet disables progress' '
432 test_must_be_empty stderr
433 '
434
435 +test_expect_success 'clean up .tmp-* packs on error' '
436 + test_must_fail git \
437 + -c repack.cruftwindow=bogus \
438 + repack -ad --cruft &&
439 + find $objdir/pack -name '.tmp-*' >tmpfiles &&
440 + test_must_be_empty tmpfiles
441 +'
442 +
443 test_expect_success 'setup for update-server-info' '
444 git init update-server-info &&
445 test_commit -C update-server-info message