maintenance: add support for systemd timers on Linux
The existing mechanism for scheduling background maintenance is done through cron. On Linux systems managed by systemd, systemd provides an alternative to schedule recurring tasks: systemd timers. The main motivations to implement systemd timers in addition to cron are: * cron is optional and Linux systems running systemd might not have it installed. * The execution of `crontab -l` can tell us if cron is installed but not if the daemon is actually running. * With systemd, each service is run in its own cgroup and its logs are tagged by the service inside journald. With cron, all scheduled tasks are running in the cron daemon cgroup and all the logs of the user-scheduled tasks are pretended to belong to the system cron service. Concretely, a user that doesn’t have access to the system logs won’t have access to the log of their own tasks scheduled by cron whereas they will have access to the log of their own tasks scheduled by systemd timer. Although `cron` attempts to send email, that email may go unseen by the user because these days, local mailboxes are not heavily used anymore. In order to schedule git maintenance, we need two unit template files: * ~/.config/systemd/user/git-maintenance@.service to define the command to be started by systemd and * ~/.config/systemd/user/git-maintenance@.timer to define the schedule at which the command should be run. Those units are templates that are parameterized by the frequency. Based on those templates, 3 timers are started: * git-maintenance@hourly.timer * git-maintenance@daily.timer * git-maintenance@weekly.timer The command launched by those three timers are the same as with the other scheduling methods: /path/to/git for-each-repo --exec-path=/path/to --config=maintenance.repo maintenance run --schedule=%i with the full path for git to ensure that the version of git launched for the scheduled maintenance is the same as the one used to run `maintenance start`. The timer unit contains `Persistent=true` so that, if the computer is powered down when a maintenance task should run, the task will be run when the computer is back powered on. Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr> Acked-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Lénaïc Huard committed
Sep 4, 2021 at 22:55 UTC
b681b191f923267aab80ae7f7ab2f85a692e8833
3 files changed
+330
-11
Documentation/git-maintenance.txt
+53
-5
@@ -179,14 +179,16 @@ OPTIONS
179
`maintenance.<task>.enabled` configured as `true` are considered.
180
See the 'TASKS' section for the list of accepted `<task>` values.
181
182
---scheduler=auto|crontab|launchctl|schtasks::
182
+--scheduler=auto|crontab|systemd-timer|launchctl|schtasks::
183
When combined with the `start` subcommand, specify the scheduler
184
for running the hourly, daily and weekly executions of
185
`git maintenance run`.
186
- Possible values for `<scheduler>` are `auto`, `crontab` (POSIX),
187
- `launchctl` (macOS), and `schtasks` (Windows).
188
- When `auto` is specified, the appropriate platform-specific
189
- scheduler is used. Default is `auto`.
186
+ Possible values for `<scheduler>` are `auto`, `crontab`
187
+ (POSIX), `systemd-timer` (Linux), `launchctl` (macOS), and
188
+ `schtasks` (Windows). When `auto` is specified, the
189
+ appropriate platform-specific scheduler is used; on Linux,
190
+ `systemd-timer` is used if available, otherwise
191
+ `crontab`. Default is `auto`.
192
193
194
TROUBLESHOOTING
@@ -286,6 +288,52 @@ schedule to ensure you are executing the correct binaries in your
288
schedule.
289
290
291
+BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS
292
+-----------------------------------------------
293
+
294
+While Linux supports `cron`, depending on the distribution, `cron` may
295
+be an optional package not necessarily installed. On modern Linux
296
+distributions, systemd timers are superseding it.
297
+
298
+If user systemd timers are available, they will be used as a replacement
299
+of `cron`.
300
+
301
+In this case, `git maintenance start` will create user systemd timer units
302
+and start the timers. The current list of user-scheduled tasks can be found
303
+by running `systemctl --user list-timers`. The timers written by `git
304
+maintenance start` are similar to this:
305
+
306
+-----------------------------------------------------------------------
307
+$ systemctl --user list-timers
308
+NEXT LEFT LAST PASSED UNIT ACTIVATES
309
+Thu 2021-04-29 19:00:00 CEST 42min left Thu 2021-04-29 18:00:11 CEST 17min ago git-maintenance@hourly.timer git-maintenance@hourly.service
310
+Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago git-maintenance@daily.timer git-maintenance@daily.service
311
+Mon 2021-05-03 00:00:00 CEST 3 days left Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service
312
+-----------------------------------------------------------------------
313
+
314
+One timer is registered for each `--schedule=<frequency>` option.
315
+
316
+The definition of the systemd units can be inspected in the following files:
317
+
318
+-----------------------------------------------------------------------
319
+~/.config/systemd/user/git-maintenance@.timer
320
+~/.config/systemd/user/git-maintenance@.service
321
+~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer
322
+~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer
323
+~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer
324
+-----------------------------------------------------------------------
325
+
326
+`git maintenance start` will overwrite these files and start the timer
327
+again with `systemctl --user`, so any customization should be done by
328
+creating a drop-in file, i.e. a `.conf` suffixed file in the
329
+`~/.config/systemd/user/git-maintenance@.service.d` directory.
330
+
331
+`git maintenance stop` will stop the user systemd timers and delete
332
+the above mentioned files.
333
+
334
+For more details, see `systemd.timer(5)`.
335
+
336
+
337
BACKGROUND MAINTENANCE ON MACOS SYSTEMS
338
---------------------------------------
339
builtin/gc.c
+216
@@ -2074,10 +2074,210 @@ done_editing:
2074
return result;
2075
}
2076
2077
+static int real_is_systemd_timer_available(void)
2078
+{
2079
+ struct child_process child = CHILD_PROCESS_INIT;
2080
+
2081
+ strvec_pushl(&child.args, "systemctl", "--user", "list-timers", NULL);
2082
+ child.no_stdin = 1;
2083
+ child.no_stdout = 1;
2084
+ child.no_stderr = 1;
2085
+ child.silent_exec_failure = 1;
2086
+
2087
+ if (start_command(&child))
2088
+ return 0;
2089
+ if (finish_command(&child))
2090
+ return 0;
2091
+ return 1;
2092
+}
2093
+
2094
+static int is_systemd_timer_available(void)
2095
+{
2096
+ const char *cmd = "systemctl";
2097
+ int is_available;
2098
+
2099
+ if (get_schedule_cmd(&cmd, &is_available))
2100
+ return is_available;
2101
+
2102
+ return real_is_systemd_timer_available();
2103
+}
2104
+
2105
+static char *xdg_config_home_systemd(const char *filename)
2106
+{
2107
+ return xdg_config_home_for("systemd/user", filename);
2108
+}
2109
+
2110
+static int systemd_timer_enable_unit(int enable,
2111
+ enum schedule_priority schedule)
2112
+{
2113
+ const char *cmd = "systemctl";
2114
+ struct child_process child = CHILD_PROCESS_INIT;
2115
+ const char *frequency = get_frequency(schedule);
2116
+
2117
+ /*
2118
+ * Disabling the systemd unit while it is already disabled makes
2119
+ * systemctl print an error.
2120
+ * Let's ignore it since it means we already are in the expected state:
2121
+ * the unit is disabled.
2122
+ *
2123
+ * On the other hand, enabling a systemd unit which is already enabled
2124
+ * produces no error.
2125
+ */
2126
+ if (!enable)
2127
+ child.no_stderr = 1;
2128
+
2129
+ get_schedule_cmd(&cmd, NULL);
2130
+ strvec_split(&child.args, cmd);
2131
+ strvec_pushl(&child.args, "--user", enable ? "enable" : "disable",
2132
+ "--now", NULL);
2133
+ strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency);
2134
+
2135
+ if (start_command(&child))
2136
+ return error(_("failed to start systemctl"));
2137
+ if (finish_command(&child))
2138
+ /*
2139
+ * Disabling an already disabled systemd unit makes
2140
+ * systemctl fail.
2141
+ * Let's ignore this failure.
2142
+ *
2143
+ * Enabling an enabled systemd unit doesn't fail.
2144
+ */
2145
+ if (enable)
2146
+ return error(_("failed to run systemctl"));
2147
+ return 0;
2148
+}
2149
+
2150
+static int systemd_timer_delete_unit_templates(void)
2151
+{
2152
+ int ret = 0;
2153
+ char *filename = xdg_config_home_systemd("git-maintenance@.timer");
2154
+ if (unlink(filename) && !is_missing_file_error(errno))
2155
+ ret = error_errno(_("failed to delete '%s'"), filename);
2156
+ FREE_AND_NULL(filename);
2157
+
2158
+ filename = xdg_config_home_systemd("git-maintenance@.service");
2159
+ if (unlink(filename) && !is_missing_file_error(errno))
2160
+ ret = error_errno(_("failed to delete '%s'"), filename);
2161
+
2162
+ free(filename);
2163
+ return ret;
2164
+}
2165
+
2166
+static int systemd_timer_delete_units(void)
2167
+{
2168
+ return systemd_timer_enable_unit(0, SCHEDULE_HOURLY) ||
2169
+ systemd_timer_enable_unit(0, SCHEDULE_DAILY) ||
2170
+ systemd_timer_enable_unit(0, SCHEDULE_WEEKLY) ||
2171
+ systemd_timer_delete_unit_templates();
2172
+}
2173
+
2174
+static int systemd_timer_write_unit_templates(const char *exec_path)
2175
+{
2176
+ char *filename;
2177
+ FILE *file;
2178
+ const char *unit;
2179
+
2180
+ filename = xdg_config_home_systemd("git-maintenance@.timer");
2181
+ if (safe_create_leading_directories(filename)) {
2182
+ error(_("failed to create directories for '%s'"), filename);
2183
+ goto error;
2184
+ }
2185
+ file = fopen_or_warn(filename, "w");
2186
+ if (file == NULL)
2187
+ goto error;
2188
+
2189
+ unit = "# This file was created and is maintained by Git.\n"
2190
+ "# Any edits made in this file might be replaced in the future\n"
2191
+ "# by a Git command.\n"
2192
+ "\n"
2193
+ "[Unit]\n"
2194
+ "Description=Optimize Git repositories data\n"
2195
+ "\n"
2196
+ "[Timer]\n"
2197
+ "OnCalendar=%i\n"
2198
+ "Persistent=true\n"
2199
+ "\n"
2200
+ "[Install]\n"
2201
+ "WantedBy=timers.target\n";
2202
+ if (fputs(unit, file) == EOF) {
2203
+ error(_("failed to write to '%s'"), filename);
2204
+ fclose(file);
2205
+ goto error;
2206
+ }
2207
+ if (fclose(file) == EOF) {
2208
+ error_errno(_("failed to flush '%s'"), filename);
2209
+ goto error;
2210
+ }
2211
+ free(filename);
2212
+
2213
+ filename = xdg_config_home_systemd("git-maintenance@.service");
2214
+ file = fopen_or_warn(filename, "w");
2215
+ if (file == NULL)
2216
+ goto error;
2217
+
2218
+ unit = "# This file was created and is maintained by Git.\n"
2219
+ "# Any edits made in this file might be replaced in the future\n"
2220
+ "# by a Git command.\n"
2221
+ "\n"
2222
+ "[Unit]\n"
2223
+ "Description=Optimize Git repositories data\n"
2224
+ "\n"
2225
+ "[Service]\n"
2226
+ "Type=oneshot\n"
2227
+ "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n"
2228
+ "LockPersonality=yes\n"
2229
+ "MemoryDenyWriteExecute=yes\n"
2230
+ "NoNewPrivileges=yes\n"
2231
+ "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n"
2232
+ "RestrictNamespaces=yes\n"
2233
+ "RestrictRealtime=yes\n"
2234
+ "RestrictSUIDSGID=yes\n"
2235
+ "SystemCallArchitectures=native\n"
2236
+ "SystemCallFilter=@system-service\n";
2237
+ if (fprintf(file, unit, exec_path, exec_path) < 0) {
2238
+ error(_("failed to write to '%s'"), filename);
2239
+ fclose(file);
2240
+ goto error;
2241
+ }
2242
+ if (fclose(file) == EOF) {
2243
+ error_errno(_("failed to flush '%s'"), filename);
2244
+ goto error;
2245
+ }
2246
+ free(filename);
2247
+ return 0;
2248
+
2249
+error:
2250
+ free(filename);
2251
+ systemd_timer_delete_unit_templates();
2252
+ return -1;
2253
+}
2254
+
2255
+static int systemd_timer_setup_units(void)
2256
+{
2257
+ const char *exec_path = git_exec_path();
2258
+
2259
+ int ret = systemd_timer_write_unit_templates(exec_path) ||
2260
+ systemd_timer_enable_unit(1, SCHEDULE_HOURLY) ||
2261
+ systemd_timer_enable_unit(1, SCHEDULE_DAILY) ||
2262
+ systemd_timer_enable_unit(1, SCHEDULE_WEEKLY);
2263
+ if (ret)
2264
+ systemd_timer_delete_units();
2265
+ return ret;
2266
+}
2267
+
2268
+static int systemd_timer_update_schedule(int run_maintenance, int fd)
2269
+{
2270
+ if (run_maintenance)
2271
+ return systemd_timer_setup_units();
2272
+ else
2273
+ return systemd_timer_delete_units();
2274
+}
2275
+
2276
enum scheduler {
2277
SCHEDULER_INVALID = -1,
2278
SCHEDULER_AUTO,
2279
SCHEDULER_CRON,
2280
+ SCHEDULER_SYSTEMD,
2281
SCHEDULER_LAUNCHCTL,
2282
SCHEDULER_SCHTASKS,
2283
};
@@ -2092,6 +2292,11 @@ static const struct {
2292
.is_available = is_crontab_available,
2293
.update_schedule = crontab_update_schedule,
2294
},
2295
+ [SCHEDULER_SYSTEMD] = {
2296
+ .name = "systemctl",
2297
+ .is_available = is_systemd_timer_available,
2298
+ .update_schedule = systemd_timer_update_schedule,
2299
+ },
2300
[SCHEDULER_LAUNCHCTL] = {
2301
.name = "launchctl",
2302
.is_available = is_launchctl_available,
@@ -2112,6 +2317,9 @@ static enum scheduler parse_scheduler(const char *value)
2317
return SCHEDULER_AUTO;
2318
else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab"))
2319
return SCHEDULER_CRON;
2320
+ else if (!strcasecmp(value, "systemd") ||
2321
+ !strcasecmp(value, "systemd-timer"))
2322
+ return SCHEDULER_SYSTEMD;
2323
else if (!strcasecmp(value, "launchctl"))
2324
return SCHEDULER_LAUNCHCTL;
2325
else if (!strcasecmp(value, "schtasks"))
@@ -2148,6 +2356,14 @@ static enum scheduler resolve_scheduler(enum scheduler scheduler)
2356
#elif defined(GIT_WINDOWS_NATIVE)
2357
return SCHEDULER_SCHTASKS;
2358
2359
+#elif defined(__linux__)
2360
+ if (is_systemd_timer_available())
2361
+ return SCHEDULER_SYSTEMD;
2362
+ else if (is_crontab_available())
2363
+ return SCHEDULER_CRON;
2364
+ else
2365
+ die(_("neither systemd timers nor crontab are available"));
2366
+
2367
#else
2368
return SCHEDULER_CRON;
2369
#endif
t/t7900-maintenance.sh
+61
-6
@@ -20,6 +20,18 @@ test_xmllint () {
20
fi
21
}
22
23
+test_lazy_prereq SYSTEMD_ANALYZE '
24
+ systemd-analyze --help >out &&
25
+ grep verify out
26
+'
27
+
28
+test_systemd_analyze_verify () {
29
+ if test_have_prereq SYSTEMD_ANALYZE
30
+ then
31
+ systemd-analyze verify "$@"
32
+ fi
33
+}
34
+
35
test_expect_success 'help text' '
36
test_expect_code 129 git maintenance -h 2>err &&
37
test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -634,15 +646,56 @@ test_expect_success 'start and stop Windows maintenance' '
646
test_cmp expect args
647
'
648
649
+test_expect_success 'start and stop Linux/systemd maintenance' '
650
+ write_script print-args <<-\EOF &&
651
+ printf "%s\n" "$*" >>args
652
+ EOF
653
+
654
+ XDG_CONFIG_HOME="$PWD" &&
655
+ export XDG_CONFIG_HOME &&
656
+ rm -f args &&
657
+ GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start --scheduler=systemd-timer &&
658
+
659
+ # start registers the repo
660
+ git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
661
+
662
+ test_systemd_analyze_verify "systemd/user/git-maintenance@.service" &&
663
+
664
+ printf -- "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
665
+ test_cmp expect args &&
666
+
667
+ rm -f args &&
668
+ GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop &&
669
+
670
+ # stop does not unregister the repo
671
+ git config --get --global --fixed-value maintenance.repo "$(pwd)" &&
672
+
673
+ test_path_is_missing "systemd/user/git-maintenance@.timer" &&
674
+ test_path_is_missing "systemd/user/git-maintenance@.service" &&
675
+
676
+ printf -- "--user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
677
+ test_cmp expect args
678
+'
679
+
680
test_expect_success 'start and stop when several schedulers are available' '
681
write_script print-args <<-\EOF &&
682
printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:\(schtasks /create .* /xml\).*:\1:;" >>args
683
EOF
684
685
rm -f args &&
643
- GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
644
- printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
686
+ GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=systemd-timer &&
687
+ printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
688
hourly daily weekly >expect &&
689
+ printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
690
+ hourly daily weekly >>expect &&
691
+ printf -- "systemctl --user enable --now git-maintenance@%s.timer\n" hourly daily weekly >>expect &&
692
+ test_cmp expect args &&
693
+
694
+ rm -f args &&
695
+ GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl &&
696
+ printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
697
+ printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
698
+ hourly daily weekly >>expect &&
699
for frequency in hourly daily weekly
700
do
701
PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
@@ -652,17 +705,19 @@ test_expect_success 'start and stop when several schedulers are available' '
705
test_cmp expect args &&
706
707
rm -f args &&
655
- GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
708
+ GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks &&
709
+ printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
710
printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
657
- hourly daily weekly >expect &&
711
+ hourly daily weekly >>expect &&
712
printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \
713
hourly daily weekly >>expect &&
714
test_cmp expect args &&
715
716
rm -f args &&
663
- GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
717
+ GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop &&
718
+ printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect &&
719
printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
665
- hourly daily weekly >expect &&
720
+ hourly daily weekly >>expect &&
721
printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \
722
hourly daily weekly >>expect &&
723
test_cmp expect args