@matheusbsbs / matheus-bagliano / commits / 740d24e541

fast-import: use writev(3p) to send cat-blob responses

When answering a `cat-blob` command, `cat_blob()` issues three separate calls to write(3p) on the cat-blob fd: one for the header line, one for the full blob payload, and one for the trailing newline. Frontends like git-filter-repo issue these commands in bulk, once per rewritten blob, so the syscall overhead adds up. Use `writev_in_full()` to send all three parts with a single syscall. This can be benchmarked with the following setup: $ git cat-file --unordered --filter=object:type=blob --batch-check='cat-blob %(objectname)' --batch-all-objects >request $ git fast-import --cat-blob-fd=3 <request Executing this with 100,000 objects in linux.git: Benchmark 1: HEAD~ Time (mean ± σ): 1.320 s ± 0.003 s [User: 1.154 s, System: 0.161 s] Range (min … max): 1.314 s … 1.324 s 10 runs Benchmark 2: HEAD Time (mean ± σ): 1.270 s ± 0.022 s [User: 1.133 s, System: 0.132 s] Range (min … max): 1.209 s … 1.282 s 10 runs Summary HEAD ran 1.04 ± 0.02 times faster than HEAD~ Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Patrick Steinhardt committed Jul 16, 2026 at 09:52 UTC 740d24e5416182d45cecfdf5eaa1074ee6005ec1

1 file changed +15 -3

builtin/fast-import.c

+15 -3

@@ -3332,6 +3332,7 @@ static void cat_blob_write(const char *buf, unsigned long size)

3332 static void cat_blob(struct object_entry *oe, struct object_id *oid)

3333 {

3334 struct strbuf line = STRBUF_INIT;

3335 + struct iovec iov[3];

3336 unsigned long size;

3337 enum object_type type = 0;

3338 char *buf;

@@ -3365,10 +3366,21 @@ static void cat_blob(struct object_entry *oe, struct object_id *oid)

3366 strbuf_reset(&line);

3367 strbuf_addf(&line, "%s %s %"PRIuMAX"\n", oid_to_hex(oid),

3368 type_name(type), (uintmax_t)size);

3368 - cat_blob_write(line.buf, line.len);

3369 +

3370 + /*

3371 + * Write the header, the payload and the trailing newline with a

3372 + * single writev(3p) call instead of three separate write(3p) calls.

3373 + */

3374 + iov[0].iov_base = line.buf;

3375 + iov[0].iov_len = line.len;

3376 + iov[1].iov_base = buf;

3377 + iov[1].iov_len = size;

3378 + iov[2].iov_base = (void *) "\n";

3379 + iov[2].iov_len = 1;

3380 +

3381 + if (writev_in_full(cat_blob_fd, iov, ARRAY_SIZE(iov)) < 0)

3382 + die_errno(_("write to frontend failed"));

3383 strbuf_release(&line);

3370 - cat_blob_write(buf, size);

3371 - cat_blob_write("\n", 1);

3384 if (oe && oe->pack_id == pack_id) {

3385 last_blob.offset = oe->idx.offset;

3386 strbuf_attach(&last_blob.data, buf, size, size + 1);