| 1 | git-maintenance(1) |
| 2 | ================== |
| 3 | |
| 4 | NAME |
| 5 | ---- |
| 6 | git-maintenance - Run tasks to optimize Git repository data |
| 7 | |
| 8 | |
| 9 | SYNOPSIS |
| 10 | -------- |
| 11 | [verse] |
| 12 | 'git maintenance' run [<options>] |
| 13 | 'git maintenance' start [--scheduler=<scheduler>] |
| 14 | 'git maintenance' (stop|register|unregister) [<options>] |
| 15 | 'git maintenance' is-needed [<options>] |
| 16 | |
| 17 | |
| 18 | DESCRIPTION |
| 19 | ----------- |
| 20 | Run tasks to optimize Git repository data, speeding up other Git commands |
| 21 | and reducing storage requirements for the repository. |
| 22 | |
| 23 | Git commands that add repository data, such as `git add` or `git fetch`, |
| 24 | are optimized for a responsive user experience. These commands do not take |
| 25 | time to optimize the Git data, since such optimizations scale with the full |
| 26 | size of the repository while these user commands each perform a relatively |
| 27 | small action. |
| 28 | |
| 29 | The `git maintenance` command provides flexibility for how to optimize the |
| 30 | Git repository. |
| 31 | |
| 32 | SUBCOMMANDS |
| 33 | ----------- |
| 34 | |
| 35 | run:: |
| 36 | Run one or more maintenance tasks. If one or more `--task` options |
| 37 | are specified, then those tasks are run in that order. Otherwise, |
| 38 | the tasks are determined by which `maintenance.<task>.enabled` |
| 39 | config options are true. By default, only `maintenance.gc.enabled` |
| 40 | is true. |
| 41 | |
| 42 | start:: |
| 43 | Start running maintenance on the current repository. This performs |
| 44 | the same config updates as the `register` subcommand, then updates |
| 45 | the background scheduler to run `git maintenance run --scheduled` |
| 46 | on an hourly basis. |
| 47 | |
| 48 | stop:: |
| 49 | Halt the background maintenance schedule. The current repository |
| 50 | is not removed from the list of maintained repositories, in case |
| 51 | the background maintenance is restarted later. |
| 52 | |
| 53 | register:: |
| 54 | Initialize Git config values so any scheduled maintenance will start |
| 55 | running on this repository. This adds the repository to the |
| 56 | `maintenance.repo` config variable in the current user's global config, |
| 57 | or the config specified by --config-file option, and enables some |
| 58 | recommended configuration values for `maintenance.<task>.schedule`. The |
| 59 | tasks that are enabled are safe for running in the background without |
| 60 | disrupting foreground processes. |
| 61 | + |
| 62 | The `register` subcommand will also set the `maintenance.strategy` config |
| 63 | value to `incremental`, if this value is not previously set. The |
| 64 | `incremental` strategy uses the following schedule for each maintenance |
| 65 | task: |
| 66 | + |
| 67 | -- |
| 68 | * `gc`: disabled. |
| 69 | * `commit-graph`: hourly. |
| 70 | * `prefetch`: hourly. |
| 71 | * `loose-objects`: daily. |
| 72 | * `incremental-repack`: daily. |
| 73 | -- |
| 74 | + |
| 75 | `git maintenance register` will also disable foreground maintenance by |
| 76 | setting `maintenance.auto = false` in the current repository. This config |
| 77 | setting will remain after a `git maintenance unregister` command. |
| 78 | |
| 79 | unregister:: |
| 80 | Remove the current repository from background maintenance. This |
| 81 | only removes the repository from the configured list. It does not |
| 82 | stop the background maintenance processes from running. |
| 83 | + |
| 84 | The `unregister` subcommand will report an error if the current repository |
| 85 | is not already registered. Use the `--force` option to return success even |
| 86 | when the current repository is not registered. |
| 87 | |
| 88 | is-needed:: |
| 89 | Check whether maintenance needs to be run without actually running it. |
| 90 | Exits with a 0 status code if maintenance needs to be run, 1 otherwise. |
| 91 | Ideally used with the '--auto' flag. |
| 92 | + |
| 93 | If one or more `--task` options are specified, then those tasks are checked |
| 94 | in that order. Otherwise, the tasks are determined by which |
| 95 | `maintenance.<task>.enabled` config options are true. By default, only |
| 96 | `maintenance.gc.enabled` is true. |
| 97 | |
| 98 | TASKS |
| 99 | ----- |
| 100 | |
| 101 | commit-graph:: |
| 102 | The `commit-graph` job updates the `commit-graph` files incrementally, |
| 103 | then verifies that the written data is correct. The incremental |
| 104 | write is safe to run alongside concurrent Git processes since it |
| 105 | will not expire `.graph` files that were in the previous |
| 106 | `commit-graph-chain` file. They will be deleted by a later run based |
| 107 | on the expiration delay. |
| 108 | |
| 109 | prefetch:: |
| 110 | The `prefetch` task updates the object directory with the latest |
| 111 | objects from all registered remotes. For each remote, a `git fetch` |
| 112 | command is run. The configured refspec is modified to place all |
| 113 | requested refs within `refs/prefetch/`. Also, tags are not updated. |
| 114 | + |
| 115 | This is done to avoid disrupting the remote-tracking branches. The end users |
| 116 | expect these refs to stay unmoved unless they initiate a fetch. However, |
| 117 | with the prefetch task, the objects necessary to complete a later real fetch |
| 118 | would already be obtained, making the real fetch faster. In the ideal case, |
| 119 | it will just become an update to a bunch of remote-tracking branches without |
| 120 | any object transfer. |
| 121 | + |
| 122 | The `remote.<name>.skipFetchAll` configuration can be used to |
| 123 | exclude a particular remote from getting prefetched. |
| 124 | |
| 125 | gc:: |
| 126 | Clean up unnecessary files and optimize the local repository. "GC" |
| 127 | stands for "garbage collection," but this task performs many |
| 128 | smaller tasks. This task can be expensive for large repositories, |
| 129 | as it repacks all Git objects into a single pack-file. It can also |
| 130 | be disruptive in some situations, as it deletes stale data. See |
| 131 | linkgit:git-gc[1] for more details on garbage collection in Git. |
| 132 | |
| 133 | loose-objects:: |
| 134 | The `loose-objects` job cleans up loose objects and places them into |
| 135 | pack-files. In order to prevent race conditions with concurrent Git |
| 136 | commands, it follows a two-step process. First, it deletes any loose |
| 137 | objects that already exist in a pack-file; concurrent Git processes |
| 138 | will examine the pack-file for the object data instead of the loose |
| 139 | object. Second, it creates a new pack-file (starting with "loose-") |
| 140 | containing a batch of loose objects. |
| 141 | + |
| 142 | The batch size defaults to fifty thousand objects to prevent the job from |
| 143 | taking too long on a repository with many loose objects. Use the |
| 144 | `maintenance.loose-objects.batchSize` config option to adjust this size, |
| 145 | including a value of `0` to remove the limit. |
| 146 | + |
| 147 | The `gc` task writes unreachable objects as loose objects to be cleaned up |
| 148 | by a later step only if they are not re-added to a pack-file; for this |
| 149 | reason it is not advisable to enable both the `loose-objects` and `gc` |
| 150 | tasks at the same time. |
| 151 | |
| 152 | incremental-repack:: |
| 153 | The `incremental-repack` job repacks the object directory |
| 154 | using the `multi-pack-index` feature. In order to prevent race |
| 155 | conditions with concurrent Git commands, it follows a two-step |
| 156 | process. First, it calls `git multi-pack-index expire` to delete |
| 157 | pack-files unreferenced by the `multi-pack-index` file. Second, it |
| 158 | calls `git multi-pack-index repack` to select several small |
| 159 | pack-files and repack them into a bigger one, and then update the |
| 160 | `multi-pack-index` entries that refer to the small pack-files to |
| 161 | refer to the new pack-file. This prepares those small pack-files |
| 162 | for deletion upon the next run of `git multi-pack-index expire`. |
| 163 | The selection of the small pack-files is such that the expected |
| 164 | size of the big pack-file is at least the batch size; see the |
| 165 | `--batch-size` option for the `repack` subcommand in |
| 166 | linkgit:git-multi-pack-index[1]. The default batch-size is zero, |
| 167 | which is a special case that attempts to repack all pack-files |
| 168 | into a single pack-file. |
| 169 | |
| 170 | pack-refs:: |
| 171 | The `pack-refs` task collects the loose reference files and |
| 172 | collects them into a single file. This speeds up operations that |
| 173 | need to iterate across many references. See linkgit:git-pack-refs[1] |
| 174 | for more information. |
| 175 | |
| 176 | reflog-expire:: |
| 177 | The `reflog-expire` task deletes any entries in the reflog older than the |
| 178 | expiry threshold. See linkgit:git-reflog[1] for more information. |
| 179 | |
| 180 | rerere-gc:: |
| 181 | The `rerere-gc` task invokes garbage collection for stale entries in |
| 182 | the rerere cache. See linkgit:git-rerere[1] for more information. |
| 183 | |
| 184 | worktree-prune:: |
| 185 | The `worktree-prune` task deletes stale or broken worktrees. See |
| 186 | linkgit:git-worktree[1] for more information. |
| 187 | |
| 188 | OPTIONS |
| 189 | ------- |
| 190 | --auto:: |
| 191 | When combined with the `run` subcommand, run maintenance tasks |
| 192 | only if certain thresholds are met. For example, the `gc` task |
| 193 | runs when the number of loose objects exceeds the number stored |
| 194 | in the `gc.auto` config setting, or when the number of pack-files |
| 195 | exceeds the `gc.autoPackLimit` config setting. Not compatible with |
| 196 | the `--schedule` option. |
| 197 | When combined with the `is-needed` subcommand, check if the required |
| 198 | thresholds are met without actually running maintenance. |
| 199 | |
| 200 | --schedule:: |
| 201 | When combined with the `run` subcommand, run maintenance tasks |
| 202 | only if certain time conditions are met, as specified by the |
| 203 | `maintenance.<task>.schedule` config value for each `<task>`. |
| 204 | This config value specifies a number of seconds since the last |
| 205 | time that task ran, according to the `maintenance.<task>.lastRun` |
| 206 | config value. The tasks that are tested are those provided by |
| 207 | the `--task=<task>` option(s) or those with |
| 208 | `maintenance.<task>.enabled` set to true. |
| 209 | |
| 210 | --quiet:: |
| 211 | Do not report progress or other information over `stderr`. |
| 212 | |
| 213 | --task=<task>:: |
| 214 | If this option is specified one or more times, then only run the |
| 215 | specified tasks in the specified order. If no `--task=<task>` |
| 216 | arguments are specified, then only the tasks with |
| 217 | `maintenance.<task>.enabled` configured as `true` are considered. |
| 218 | See the 'TASKS' section for the list of accepted `<task>` values. |
| 219 | |
| 220 | --scheduler=auto|crontab|systemd-timer|launchctl|schtasks:: |
| 221 | When combined with the `start` subcommand, specify the scheduler |
| 222 | for running the hourly, daily and weekly executions of |
| 223 | `git maintenance run`. |
| 224 | Possible values for `<scheduler>` are `auto`, `crontab` |
| 225 | (POSIX), `systemd-timer` (Linux), `launchctl` (macOS), and |
| 226 | `schtasks` (Windows). When `auto` is specified, the |
| 227 | appropriate platform-specific scheduler is used; on Linux, |
| 228 | `systemd-timer` is used if available, otherwise |
| 229 | `crontab`. Default is `auto`. |
| 230 | |
| 231 | |
| 232 | TROUBLESHOOTING |
| 233 | --------------- |
| 234 | The `git maintenance` command is designed to simplify the repository |
| 235 | maintenance patterns while minimizing user wait time during Git commands. |
| 236 | A variety of configuration options are available to allow customizing this |
| 237 | process. The default maintenance options focus on operations that complete |
| 238 | quickly, even on large repositories. |
| 239 | |
| 240 | Users may find some cases where scheduled maintenance tasks do not run as |
| 241 | frequently as intended. Each `git maintenance run` command takes a lock on |
| 242 | the repository's object database, and this prevents other concurrent |
| 243 | `git maintenance run` commands from running on the same repository. Without |
| 244 | this safeguard, competing processes could leave the repository in an |
| 245 | unpredictable state. |
| 246 | |
| 247 | The background maintenance schedule runs `git maintenance run` processes |
| 248 | on an hourly basis. Each run executes the "hourly" tasks. At midnight, |
| 249 | that process also executes the "daily" tasks. At midnight on the first day |
| 250 | of the week, that process also executes the "weekly" tasks. A single |
| 251 | process iterates over each registered repository, performing the scheduled |
| 252 | tasks for that frequency. The processes are scheduled to a random minute of |
| 253 | the hour per client to spread out the load that multiple clients might |
| 254 | generate (e.g. from prefetching). Depending on the number of registered |
| 255 | repositories and their sizes, this process may take longer than an hour. |
| 256 | In this case, multiple `git maintenance run` commands may run on the same |
| 257 | repository at the same time, colliding on the object database lock. This |
| 258 | results in one of the two tasks not running. |
| 259 | |
| 260 | If you find that some maintenance windows are taking longer than one hour |
| 261 | to complete, then consider reducing the complexity of your maintenance |
| 262 | tasks. For example, the `gc` task is much slower than the |
| 263 | `incremental-repack` task. However, this comes at a cost of a slightly |
| 264 | larger object database. Consider moving more expensive tasks to be run |
| 265 | less frequently. |
| 266 | |
| 267 | Expert users may consider scheduling their own maintenance tasks using a |
| 268 | different schedule than is available through `git maintenance start` and |
| 269 | Git configuration options. These users should be aware of the object |
| 270 | database lock and how concurrent `git maintenance run` commands behave. |
| 271 | Further, the `git gc` command should not be combined with |
| 272 | `git maintenance run` commands. `git gc` modifies the object database |
| 273 | but does not take the lock in the same way as `git maintenance run`. If |
| 274 | possible, use `git maintenance run --task=gc` instead of `git gc`. |
| 275 | |
| 276 | The following sections describe the mechanisms put in place to run |
| 277 | background maintenance by `git maintenance start` and how to customize |
| 278 | them. |
| 279 | |
| 280 | BACKGROUND MAINTENANCE ON POSIX SYSTEMS |
| 281 | --------------------------------------- |
| 282 | |
| 283 | The standard mechanism for scheduling background tasks on POSIX systems |
| 284 | is cron(8). This tool executes commands based on a given schedule. The |
| 285 | current list of user-scheduled tasks can be found by running `crontab -l`. |
| 286 | The schedule written by `git maintenance start` is similar to this: |
| 287 | |
| 288 | ----------------------------------------------------------------------- |
| 289 | # BEGIN GIT MAINTENANCE SCHEDULE |
| 290 | # The following schedule was created by Git |
| 291 | # Any edits made in this region might be |
| 292 | # replaced in the future by a Git command. |
| 293 | |
| 294 | 0 1-23 * * * "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly |
| 295 | 0 0 * * 1-6 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=daily |
| 296 | 0 0 * * 0 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly |
| 297 | |
| 298 | # END GIT MAINTENANCE SCHEDULE |
| 299 | ----------------------------------------------------------------------- |
| 300 | |
| 301 | The comments are used as a region to mark the schedule as written by Git. |
| 302 | Any modifications within this region will be completely deleted by |
| 303 | `git maintenance stop` or overwritten by `git maintenance start`. |
| 304 | |
| 305 | The `crontab` entry specifies the full path of the `git` executable to |
| 306 | ensure that the executed `git` command is the same one with which |
| 307 | `git maintenance start` was issued independent of `PATH`. If the same user |
| 308 | runs `git maintenance start` with multiple Git executables, then only the |
| 309 | latest executable is used. |
| 310 | |
| 311 | These commands use `git for-each-repo --config=maintenance.repo` to run |
| 312 | `git maintenance run --schedule=<frequency>` on each repository listed in |
| 313 | the multi-valued `maintenance.repo` config option. These are typically |
| 314 | loaded from the user-specific global config. The `git maintenance` process |
| 315 | then determines which maintenance tasks are configured to run on each |
| 316 | repository with each `<frequency>` using the `maintenance.<task>.schedule` |
| 317 | config options. These values are loaded from the global or repository |
| 318 | config values. |
| 319 | |
| 320 | If the config values are insufficient to achieve your desired background |
| 321 | maintenance schedule, then you can create your own schedule. If you run |
| 322 | `crontab -e`, then an editor will load with your user-specific `cron` |
| 323 | schedule. In that editor, you can add your own schedule lines. You could |
| 324 | start by adapting the default schedule listed earlier, or you could read |
| 325 | the crontab(5) documentation for advanced scheduling techniques. Please |
| 326 | do use the full path and `--exec-path` techniques from the default |
| 327 | schedule to ensure you are executing the correct binaries in your |
| 328 | schedule. |
| 329 | |
| 330 | |
| 331 | BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS |
| 332 | ----------------------------------------------- |
| 333 | |
| 334 | While Linux supports `cron`, depending on the distribution, `cron` may |
| 335 | be an optional package not necessarily installed. On modern Linux |
| 336 | distributions, systemd timers are superseding it. |
| 337 | |
| 338 | If user systemd timers are available, they will be used as a replacement |
| 339 | of `cron`. |
| 340 | |
| 341 | In this case, `git maintenance start` will create user systemd timer units |
| 342 | and start the timers. The current list of user-scheduled tasks can be found |
| 343 | by running `systemctl --user list-timers`. The timers written by `git |
| 344 | maintenance start` are similar to this: |
| 345 | |
| 346 | ----------------------------------------------------------------------- |
| 347 | $ systemctl --user list-timers |
| 348 | NEXT LEFT LAST PASSED UNIT ACTIVATES |
| 349 | Thu 2021-04-29 19:00:00 CEST 42min left Thu 2021-04-29 18:00:11 CEST 17min ago git-maintenance@hourly.timer git-maintenance@hourly.service |
| 350 | Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago git-maintenance@daily.timer git-maintenance@daily.service |
| 351 | Mon 2021-05-03 00:00:00 CEST 3 days left Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service |
| 352 | ----------------------------------------------------------------------- |
| 353 | |
| 354 | One timer is registered for each `--schedule=<frequency>` option. |
| 355 | |
| 356 | The definition of the systemd units can be inspected in the following files: |
| 357 | |
| 358 | ----------------------------------------------------------------------- |
| 359 | ~/.config/systemd/user/git-maintenance@.timer |
| 360 | ~/.config/systemd/user/git-maintenance@.service |
| 361 | ~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer |
| 362 | ~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer |
| 363 | ~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer |
| 364 | ----------------------------------------------------------------------- |
| 365 | |
| 366 | `git maintenance start` will overwrite these files and start the timer |
| 367 | again with `systemctl --user`, so any customization should be done by |
| 368 | creating a drop-in file, i.e. a `.conf` suffixed file in the |
| 369 | `~/.config/systemd/user/git-maintenance@.service.d` directory. |
| 370 | |
| 371 | `git maintenance stop` will stop the user systemd timers and delete |
| 372 | the above mentioned files. |
| 373 | |
| 374 | For more details, see `systemd.timer(5)`. |
| 375 | |
| 376 | |
| 377 | BACKGROUND MAINTENANCE ON MACOS SYSTEMS |
| 378 | --------------------------------------- |
| 379 | |
| 380 | While macOS technically supports `cron`, using `crontab -e` requires |
| 381 | elevated privileges and the executed process does not have a full user |
| 382 | context. Without a full user context, Git and its credential helpers |
| 383 | cannot access stored credentials, so some maintenance tasks are not |
| 384 | functional. |
| 385 | |
| 386 | Instead, `git maintenance start` interacts with the `launchctl` tool, |
| 387 | which is the recommended way to schedule timed jobs in macOS. Scheduling |
| 388 | maintenance through `git maintenance (start|stop)` requires some |
| 389 | `launchctl` features available only in macOS 10.11 or later. |
| 390 | |
| 391 | Your user-specific scheduled tasks are stored as XML-formatted `.plist` |
| 392 | files in `~/Library/LaunchAgents/`. You can see the currently-registered |
| 393 | tasks using the following command: |
| 394 | |
| 395 | ----------------------------------------------------------------------- |
| 396 | $ ls ~/Library/LaunchAgents/org.git-scm.git* |
| 397 | org.git-scm.git.daily.plist |
| 398 | org.git-scm.git.hourly.plist |
| 399 | org.git-scm.git.weekly.plist |
| 400 | ----------------------------------------------------------------------- |
| 401 | |
| 402 | One task is registered for each `--schedule=<frequency>` option. To |
| 403 | inspect how the XML format describes each schedule, open one of these |
| 404 | `.plist` files in an editor and inspect the `<array>` element following |
| 405 | the `<key>StartCalendarInterval</key>` element. |
| 406 | |
| 407 | `git maintenance start` will overwrite these files and register the |
| 408 | tasks again with `launchctl`, so any customizations should be done by |
| 409 | creating your own `.plist` files with distinct names. Similarly, the |
| 410 | `git maintenance stop` command will unregister the tasks with `launchctl` |
| 411 | and delete the `.plist` files. |
| 412 | |
| 413 | To create more advanced customizations to your background tasks, see |
| 414 | launchctl.plist(5) for more information. |
| 415 | |
| 416 | |
| 417 | BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS |
| 418 | ----------------------------------------- |
| 419 | |
| 420 | Windows does not support `cron` and instead has its own system for |
| 421 | scheduling background tasks. The `git maintenance start` command uses |
| 422 | the `schtasks` command to submit tasks to this system. You can inspect |
| 423 | all background tasks using the Task Scheduler application. The tasks |
| 424 | added by Git have names of the form `Git Maintenance (<frequency>)`. |
| 425 | The Task Scheduler GUI has ways to inspect these tasks, but you can also |
| 426 | export the tasks to XML files and view the details there. |
| 427 | |
| 428 | Note that since Git is a console application, these background tasks |
| 429 | create a console window visible to the current user. This can be changed |
| 430 | manually by selecting the "Run whether user is logged in or not" option |
| 431 | in Task Scheduler. This change requires a password input, which is why |
| 432 | `git maintenance start` does not select it by default. |
| 433 | |
| 434 | If you want to customize the background tasks, please rename the tasks |
| 435 | so future calls to `git maintenance (start|stop)` do not overwrite your |
| 436 | custom tasks. |
| 437 | |
| 438 | CONFIGURATION |
| 439 | ------------- |
| 440 | |
| 441 | include::includes/cmd-config-section-all.adoc[] |
| 442 | |
| 443 | include::config/maintenance.adoc[] |
| 444 | |
| 445 | |
| 446 | GIT |
| 447 | --- |
| 448 | Part of the linkgit:git[1] suite |