@matheusbsbs / matheus-bagliano / commits / 425d290ce5

unpack-objects: use the bulk-checkin infrastructure

The unpack-objects functionality is used by fetch, push, and fast-import to turn the transfered data into object database entries when there are fewer objects than the 'unpacklimit' setting. By enabling an odb-transaction when unpacking objects, we can take advantage of batched fsyncs. Here are some performance numbers to justify batch mode for unpack-objects, collected on a WSL2 Ubuntu VM. Fsync Mode | Time for 90 objects (ms) ------------------------------------- Off | 170 On,fsync | 760 On,batch | 230 Note that the default unpackLimit is 100 objects, so there's a 3x benefit in the worst case. The non-batch mode fsync scales linearly with the number of objects, so there are significant benefits even with smaller numbers of objects. Signed-off-by: Neeraj Singh <neerajsi@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Neeraj Singh committed Apr 4, 2022 at 22:20 UTC 425d290ce564ada84a5322545dbbd1866f2a29c4

1 file changed +3

builtin/unpack-objects.c

@@ -1,5 +1,6 @@

1 #include "builtin.h"

2 #include "cache.h"

3 +#include "bulk-checkin.h"

4 #include "config.h"

5 #include "object-store.h"

6 #include "object.h"

@@ -503,10 +504,12 @@ static void unpack_all(void)

504 if (!quiet)

505 progress = start_progress(_("Unpacking objects"), nr_objects);

506 CALLOC_ARRAY(obj_list, nr_objects);

507 + begin_odb_transaction();

508 for (i = 0; i < nr_objects; i++) {

509 unpack_one(i);

510 display_progress(progress, i + 1);

511 }

512 + end_odb_transaction();

513 stop_progress(&progress);

514

515 if (delta_list)