refs: reuse iterators when determining refname availability

When verifying whether refnames are available we have to verify whether any reference exists that is nested under the current reference. E.g. given a reference "refs/heads/foo", we must make sure that there is no other reference "refs/heads/foo/*". This check is performed using a ref iterator with the prefix set to the nested reference namespace. Until now it used to not be possible to reseek iterators, so we always had to reallocate the iterator for every single reference we're about to check. This keeps us from reusing state that the iterator may have and that may make it work more efficiently. Refactor the logic to reseek iterators. This leads to a sizeable speedup with the "reftable" backend: Benchmark 1: update-ref: create many refs (refformat = reftable, preexisting = 100000, new = 10000, revision = HEAD~) Time (mean ± σ): 39.8 ms ± 0.9 ms [User: 29.7 ms, System: 9.8 ms] Range (min … max): 38.4 ms … 42.0 ms 62 runs Benchmark 2: update-ref: create many refs (refformat = reftable, preexisting = 100000, new = 10000, revision = HEAD) Time (mean ± σ): 31.9 ms ± 1.1 ms [User: 27.0 ms, System: 4.5 ms] Range (min … max): 29.8 ms … 34.3 ms 74 runs Summary update-ref: create many refs (refformat = reftable, preexisting = 100000, new = 10000, revision = HEAD) ran 1.25 ± 0.05 times faster than update-ref: create many refs (refformat = reftable, preexisting = 100000, new = 10000, revision = HEAD~) The "files" backend doesn't really show a huge impact: Benchmark 1: update-ref: create many refs (refformat = files, preexisting = 100000, new = 10000, revision = HEAD~) Time (mean ± σ): 392.3 ms ± 7.1 ms [User: 59.7 ms, System: 328.8 ms] Range (min … max): 384.6 ms … 404.5 ms 10 runs Benchmark 2: update-ref: create many refs (refformat = files, preexisting = 100000, new = 10000, revision = HEAD) Time (mean ± σ): 387.7 ms ± 7.4 ms [User: 54.6 ms, System: 329.6 ms] Range (min … max): 377.0 ms … 397.7 ms 10 runs Summary update-ref: create many refs (refformat = files, preexisting = 100000, new = 10000, revision = HEAD) ran 1.01 ± 0.03 times faster than update-ref: create many refs (refformat = files, preexisting = 100000, new = 10000, revision = HEAD~) This is mostly because it is way slower to begin with because it has to create a separate file for each new reference, so the milliseconds we shave off by reseeking the iterator doesn't really translate into a significant relative improvement. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Patrick Steinhardt committed Mar 12, 2025 at 16:56 UTC 87d297f48367737444810f8c3e76ef88cb6aa4e3
1 file changed +7 -5
refs.c
+7 -5
@@ -2564,8 +2564,13 @@ int refs_verify_refnames_available(struct ref_store *refs,
2564 if (!initial_transaction) {
2565 int ok;
2566
2567 - iter = refs_ref_iterator_begin(refs, dirname.buf, NULL, 0,
2568 - DO_FOR_EACH_INCLUDE_BROKEN);
2567 + if (!iter) {
2568 + iter = refs_ref_iterator_begin(refs, dirname.buf, NULL, 0,
2569 + DO_FOR_EACH_INCLUDE_BROKEN);
2570 + } else if (ref_iterator_seek(iter, dirname.buf) < 0) {
2571 + goto cleanup;
2572 + }
2573 +
2574 while ((ok = ref_iterator_advance(iter)) == ITER_OK) {
2575 if (skip &&
2576 string_list_has_string(skip, iter->refname))
@@ -2578,9 +2583,6 @@ int refs_verify_refnames_available(struct ref_store *refs,
2583
2584 if (ok != ITER_DONE)
2585 BUG("error while iterating over references");
2581 -
2582 - ref_iterator_free(iter);
2583 - iter = NULL;
2586 }
2587
2588 extra_refname = find_descendant_ref(dirname.buf, extras, skip);