@matheusbsbs / matheus-bagliano / commits / c0f4752ed2

core.fsyncmethod: batched disk flushes for loose-objects

When adding many objects to a repo with `core.fsync=loose-object`, the cost of fsync'ing each object file can become prohibitive. One major source of the cost of fsync is the implied flush of the hardware writeback cache within the disk drive. This commit introduces a new `core.fsyncMethod=batch` option that batches up hardware flushes. It hooks into the bulk-checkin odb-transaction functionality, takes advantage of tmp-objdir, and uses the writeout-only support code. When the new mode is enabled, we do the following for each new object: 1a. Create the object in a tmp-objdir. 2a. Issue a pagecache writeback request and wait for it to complete. At the end of the entire transaction when unplugging bulk checkin: 1b. Issue an fsync against a dummy file to flush the log and hardware writeback cache, which should by now have seen the tmp-objdir writes. 2b. Rename all of the tmp-objdir files to their final names. 3b. When updating the index and/or refs, we assume that Git will issue another fsync internal to that operation. This is not the default today, but the user now has the option of syncing the index and there is a separate patch series to implement syncing of refs. On a filesystem with a singular journal that is updated during name operations (e.g. create, link, rename, etc), such as NTFS, HFS+, or XFS we would expect the fsync to trigger a journal writeout so that this sequence is enough to ensure that the user's data is durable by the time the git command returns. This sequence also ensures that no object files appear in the main object store unless they are fsync-durable. Batch mode is only enabled if core.fsync includes loose-objects. If the legacy core.fsyncObjectFiles setting is enabled, but core.fsync does not include loose-objects, we will use file-by-file fsyncing. In step (1a) of the sequence, the tmp-objdir is created lazily to avoid work if no loose objects are ever added to the ODB. We use a tmp-objdir to maintain the invariant that no loose-objects are visible in the main ODB unless they are properly fsync-durable. This is important since future ODB operations that try to create an object with specific contents will silently drop the new data if an object with the target hash exists without checking that the loose-object contents match the hash. Only a full git-fsck would restore the ODB to a functional state where dataloss doesn't occur. In step (1b) of the sequence, we issue a fsync against a dummy file created specifically for the purpose. This method has a little higher cost than using one of the input object files, but makes adding new callers of this mechanism easier, since we don't need to figure out which object file is "last" or risk sharing violations by caching the fd of the last object file. _Performance numbers_: Linux - Hyper-V VM running Kernel 5.11 (Ubuntu 20.04) on a fast SSD. Mac - macOS 11.5.1 running on a Mac mini on a 1TB Apple SSD. Windows - Same host as Linux, a preview version of Windows 11. Adding 500 files to the repo with 'git add' Times reported in seconds. object file syncing | Linux | Mac | Windows --------------------|-------|-------|-------- disabled | 0.06 | 0.35 | 0.61 fsync | 1.88 | 11.18 | 2.47 batch | 0.15 | 0.41 | 1.53 Signed-off-by: Neeraj Singh <neerajsi@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Neeraj Singh committed Apr 4, 2022 at 22:20 UTC c0f4752ed2ffb89d10a9e469a83f089cc7e9df9d

6 files changed +97 -2

Documentation/config/core.txt

@@ -595,6 +595,14 @@ core.fsyncMethod::

595 * `writeout-only` issues pagecache writeback requests, but depending on the

596 filesystem and storage hardware, data added to the repository may not be

597 durable in the event of a system crash. This is the default mode on macOS.

598 +* `batch` enables a mode that uses writeout-only flushes to stage multiple

599 + updates in the disk writeback cache and then does a single full fsync of

600 + a dummy file to trigger the disk cache flush at the end of the operation.

601 ++

602 +Currently `batch` mode only applies to loose-object files. Other repository

603 +data is made durable as if `fsync` was specified. This mode is expected to

604 +be as safe as `fsync` on macOS for repos stored on HFS+ or APFS filesystems

605 +and on Windows for repos stored on NTFS or ReFS filesystems.

606

607 core.fsyncObjectFiles::

608 This boolean will enable 'fsync()' when writing object files.

bulk-checkin.c

+71

@@ -3,15 +3,20 @@

3 */

4 #include "cache.h"

5 #include "bulk-checkin.h"

6 +#include "lockfile.h"

7 #include "repository.h"

8 #include "csum-file.h"

9 #include "pack.h"

10 #include "strbuf.h"

11 +#include "string-list.h"

12 +#include "tmp-objdir.h"

13 #include "packfile.h"

14 #include "object-store.h"

16 static int odb_transaction_nesting;

18 +static struct tmp_objdir *bulk_fsync_objdir;

19 +

20 static struct bulk_checkin_packfile {

21 char *pack_tmp_name;

22 struct hashfile *f;

@@ -80,6 +85,40 @@ clear_exit:

85 reprepare_packed_git(the_repository);

86 }

88 +/*

89 + * Cleanup after batch-mode fsync_object_files.

90 + */

91 +static void flush_batch_fsync(void)

92 +{

93 + struct strbuf temp_path = STRBUF_INIT;

94 + struct tempfile *temp;

95 +

96 + if (!bulk_fsync_objdir)

97 + return;

98 +

99 + /*

100 + * Issue a full hardware flush against a temporary file to ensure

101 + * that all objects are durable before any renames occur. The code in

102 + * fsync_loose_object_bulk_checkin has already issued a writeout

103 + * request, but it has not flushed any writeback cache in the storage

104 + * hardware or any filesystem logs. This fsync call acts as a barrier

105 + * to ensure that the data in each new object file is durable before

106 + * the final name is visible.

107 + */

108 + strbuf_addf(&temp_path, "%s/bulk_fsync_XXXXXX", get_object_directory());

109 + temp = xmks_tempfile(temp_path.buf);

110 + fsync_or_die(get_tempfile_fd(temp), get_tempfile_path(temp));

111 + delete_tempfile(&temp);

112 + strbuf_release(&temp_path);

113 +

114 + /*

115 + * Make the object files visible in the primary ODB after their data is

116 + * fully durable.

117 + */

118 + tmp_objdir_migrate(bulk_fsync_objdir);

119 + bulk_fsync_objdir = NULL;

120 +}

121 +

122 static int already_written(struct bulk_checkin_packfile *state, struct object_id *oid)

123 {

124 int i;

@@ -274,6 +313,37 @@ static int deflate_to_pack(struct bulk_checkin_packfile *state,

313 return 0;

314 }

315

316 +void prepare_loose_object_bulk_checkin(void)

317 +{

318 + /*

319 + * We lazily create the temporary object directory

320 + * the first time an object might be added, since

321 + * callers may not know whether any objects will be

322 + * added at the time they call begin_odb_transaction.

323 + */

324 + if (!odb_transaction_nesting || bulk_fsync_objdir)

325 + return;

326 +

327 + bulk_fsync_objdir = tmp_objdir_create("bulk-fsync");

328 + if (bulk_fsync_objdir)

329 + tmp_objdir_replace_primary_odb(bulk_fsync_objdir, 0);

330 +}

331 +

332 +void fsync_loose_object_bulk_checkin(int fd, const char *filename)

333 +{

334 + /*

335 + * If we have an active ODB transaction, we issue a call that

336 + * cleans the filesystem page cache but avoids a hardware flush

337 + * command. Later on we will issue a single hardware flush

338 + * before renaming the objects to their final names as part of

339 + * flush_batch_fsync.

340 + */

341 + if (!bulk_fsync_objdir ||

342 + git_fsync(fd, FSYNC_WRITEOUT_ONLY) < 0) {

343 + fsync_or_die(fd, filename);

344 + }

345 +}

346 +

347 int index_bulk_checkin(struct object_id *oid,

348 int fd, size_t size, enum object_type type,

349 const char *path, unsigned flags)

@@ -292,6 +362,7 @@ void begin_odb_transaction(void)

362

363 void flush_odb_transaction(void)

364 {

365 + flush_batch_fsync();

366 flush_bulk_checkin_packfile(&bulk_checkin_packfile);

367 }

368

bulk-checkin.h

@@ -6,6 +6,9 @@

7 #include "cache.h"

9 +void prepare_loose_object_bulk_checkin(void);

10 +void fsync_loose_object_bulk_checkin(int fd, const char *filename);

11 +

12 int index_bulk_checkin(struct object_id *oid,

13 int fd, size_t size, enum object_type type,

14 const char *path, unsigned flags);

cache.h

+7 -1

@@ -1037,7 +1037,8 @@ extern int use_fsync;

1037

1038 enum fsync_method {

1039 FSYNC_METHOD_FSYNC,

1040 - FSYNC_METHOD_WRITEOUT_ONLY

1040 + FSYNC_METHOD_WRITEOUT_ONLY,

1041 + FSYNC_METHOD_BATCH,

1042 };

1043

1044 extern enum fsync_method fsync_method;

@@ -1750,6 +1751,11 @@ void fsync_or_die(int fd, const char *);

1751 int fsync_component(enum fsync_component component, int fd);

1752 void fsync_component_or_die(enum fsync_component component, int fd, const char *msg);

1753

1754 +static inline int batch_fsync_enabled(enum fsync_component component)

1755 +{

1756 + return (fsync_components & component) && (fsync_method == FSYNC_METHOD_BATCH);

1757 +}

1758 +

1759 ssize_t read_in_full(int fd, void *buf, size_t count);

1760 ssize_t write_in_full(int fd, const void *buf, size_t count);

1761 ssize_t pread_in_full(int fd, void *buf, size_t count, off_t offset);

config.c

@@ -1687,6 +1687,8 @@ static int git_default_core_config(const char *var, const char *value, void *cb)

1687 fsync_method = FSYNC_METHOD_FSYNC;

1688 else if (!strcmp(value, "writeout-only"))

1689 fsync_method = FSYNC_METHOD_WRITEOUT_ONLY;

1690 + else if (!strcmp(value, "batch"))

1691 + fsync_method = FSYNC_METHOD_BATCH;

1692 else

1693 warning(_("ignoring unknown core.fsyncMethod value '%s'"), value);

1694

object-file.c

+6 -1

@@ -1852,7 +1852,9 @@ static void close_loose_object(int fd)

1852 if (the_repository->objects->odb->will_destroy)

1853 goto out;

1854

1855 - if (fsync_object_files > 0)

1855 + if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT))

1856 + fsync_loose_object_bulk_checkin(fd, "loose object file");

1857 + else if (fsync_object_files > 0)

1858 fsync_or_die(fd, "loose object file");

1859 else

1860 fsync_component_or_die(FSYNC_COMPONENT_LOOSE_OBJECT, fd,

@@ -1920,6 +1922,9 @@ static int write_loose_object(const struct object_id *oid, char *hdr,

1922 static struct strbuf tmp_file = STRBUF_INIT;

1923 static struct strbuf filename = STRBUF_INIT;

1924

1925 + if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT))

1926 + prepare_loose_object_bulk_checkin();

1927 +

1928 loose_object_path(the_repository, &filename, oid);

1929

1930 fd = create_tmpfile(&tmp_file, filename.buf);