maintenance: add loose-objects.batchSize config
The 'loose-objects' task of 'git maintenance run' first deletes loose objects that exit within packfiles and then collects loose objects into a packfile. This second step uses an implicit limit of fifty thousand that cannot be modified by users. Add a new config option that allows this limit to be adjusted or ignored entirely. While creating tests for this option, I noticed that actually there was an off-by-one error due to the strict comparison in the limit check. I considered making the limit check turn true on equality, but instead I thought to use INT_MAX as a "no limit" barrier which should mean it's never possible to hit the limit. Thus, a new decrement to the limit is provided if the value is positive. (The restriction to positive values is to avoid underflow if INT_MIN is configured.) Signed-off-by: Derrick Stolee <stolee@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Derrick Stolee committed
Mar 24, 2025 at 00:51 UTC
6540560fd6c91091f6cf1eaedd034bc1827e1506
4 files changed
+54
-7
Documentation/config/maintenance.adoc
+5
@@ -61,6 +61,11 @@ maintenance.loose-objects.auto::
61
loose objects is at least the value of `maintenance.loose-objects.auto`.
62
The default value is 100.
63
64
+maintenance.loose-objects.batchSize::
65
+ This integer config option controls the maximum number of loose objects
66
+ written into a packfile during the `loose-objects` task. The default is
67
+ fifty thousand. Use value `0` to indicate no limit.
68
+
69
maintenance.incremental-repack.auto::
70
This integer config option controls how often the `incremental-repack`
71
task should be run as part of `git maintenance run --auto`. If zero,
Documentation/git-maintenance.adoc
+11
-7
@@ -126,13 +126,17 @@ loose-objects::
126
objects that already exist in a pack-file; concurrent Git processes
127
will examine the pack-file for the object data instead of the loose
128
object. Second, it creates a new pack-file (starting with "loose-")
129
- containing a batch of loose objects. The batch size is limited to 50
130
- thousand objects to prevent the job from taking too long on a
131
- repository with many loose objects. The `gc` task writes unreachable
132
- objects as loose objects to be cleaned up by a later step only if
133
- they are not re-added to a pack-file; for this reason it is not
134
- advisable to enable both the `loose-objects` and `gc` tasks at the
135
- same time.
129
+ containing a batch of loose objects.
130
++
131
+The batch size defaults to fifty thousand objects to prevent the job from
132
+taking too long on a repository with many loose objects. Use the
133
+`maintenance.loose-objects.batchSize` config option to adjust this size,
134
+including a value of `0` to remove the limit.
135
++
136
+The `gc` task writes unreachable objects as loose objects to be cleaned up
137
+by a later step only if they are not re-added to a pack-file; for this
138
+reason it is not advisable to enable both the `loose-objects` and `gc`
139
+tasks at the same time.
140
141
incremental-repack::
142
The `incremental-repack` job repacks the object directory
builtin/gc.c
+10
@@ -1163,6 +1163,7 @@ static int write_loose_object_to_stdin(const struct object_id *oid,
1163
1164
fprintf(d->in, "%s\n", oid_to_hex(oid));
1165
1166
+ /* If batch_size is INT_MAX, then this will return 0 always. */
1167
return ++(d->count) > d->batch_size;
1168
}
1169
@@ -1208,6 +1209,15 @@ static int pack_loose(struct maintenance_run_opts *opts)
1209
data.count = 0;
1210
data.batch_size = 50000;
1211
1212
+ repo_config_get_int(r, "maintenance.loose-objects.batchSize",
1213
+ &data.batch_size);
1214
+
1215
+ /* If configured as 0, then remove limit. */
1216
+ if (!data.batch_size)
1217
+ data.batch_size = INT_MAX;
1218
+ else if (data.batch_size > 0)
1219
+ data.batch_size--; /* Decrease for equality on limit. */
1220
+
1221
for_each_loose_file_in_objdir(r->objects->odb->path,
1222
write_loose_object_to_stdin,
1223
NULL,
t/t7900-maintenance.sh
+28
@@ -306,6 +306,34 @@ test_expect_success 'maintenance.loose-objects.auto' '
306
test_subcommand git prune-packed --quiet <trace-loC
307
'
308
309
+test_expect_success 'maintenance.loose-objects.batchSize' '
310
+ git init loose-batch &&
311
+
312
+ # This creates three objects per commit.
313
+ test_commit_bulk -C loose-batch 34 &&
314
+ pack=$(ls loose-batch/.git/objects/pack/pack-*.pack) &&
315
+ index="${pack%pack}idx" &&
316
+ rm "$index" &&
317
+ git -C loose-batch unpack-objects <"$pack" &&
318
+ git -C loose-batch config maintenance.loose-objects.batchSize 50 &&
319
+
320
+ GIT_PROGRESS_DELAY=0 \
321
+ git -C loose-batch maintenance run --no-quiet --task=loose-objects 2>err &&
322
+ grep "Enumerating objects: 50, done." err &&
323
+
324
+ GIT_PROGRESS_DELAY=0 \
325
+ git -C loose-batch maintenance run --no-quiet --task=loose-objects 2>err &&
326
+ grep "Enumerating objects: 50, done." err &&
327
+
328
+ GIT_PROGRESS_DELAY=0 \
329
+ git -C loose-batch maintenance run --no-quiet --task=loose-objects 2>err &&
330
+ grep "Enumerating objects: 2, done." err &&
331
+
332
+ GIT_PROGRESS_DELAY=0 \
333
+ git -C loose-batch maintenance run --no-quiet --task=loose-objects 2>err &&
334
+ test_must_be_empty err
335
+'
336
+
337
test_expect_success 'incremental-repack task' '
338
packDir=.git/objects/pack &&
339
for i in $(test_seq 1 5)