@matheusbsbs / matheus-bagliano / commits / 2b98478c6f
Fork 0

connected: always use partial clone optimization

With 50033772d5 ("connected: verify promisor-ness of partial clone", 2020-01-30), the fast path (checking promisor packs) in check_connected() now passes a subset of the slow path (rev-list) - if all objects to be checked are found in promisor packs, both the fast path and the slow path will pass; otherwise, the fast path will definitely not pass. This means that we can always attempt the fast path whenever we need to do the slow path. The fast path is currently guarded by a flag; therefore, remove that flag. Also, make the fast path fallback to the slow path - if the fast path fails, the failing OID and all remaining OIDs will be passed to rev-list. The main user-visible benefit is the performance of fetch from a partial clone - specifically, the speedup of the connectivity check done before the fetch. In particular, a no-op fetch into a partial clone on my computer was sped up from 7 seconds to 0.01 seconds. This is a complement to the work in 2df1aa239c ("fetch: forgo full connectivity check if --filter", 2020-01-30), which is the child of the aforementioned 50033772d5. In that commit, the connectivity check *after* the fetch was sped up. The addition of the fast path might cause performance reductions in these cases: - If a partial clone or a fetch into a partial clone fails, Git will fruitlessly run rev-list (it is expected that everything fetched would go into promisor packs, so if that didn't happen, it is most likely that rev-list will fail too). - Any connectivity checks done by receive-pack, in the (in my opinion, unlikely) event that a partial clone serves receive-pack. I think that these cases are rare enough, and the performance reduction in this case minor enough (additional object DB access), that the benefit of avoiding a flag outweighs these. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Jonathan Tan committed Mar 20, 2020 at 15:00 UTC 2b98478c6fc32b04434981c783b0472e0137cfad
4 files changed +9 -23
builtin/clone.c
+2 -5
@@ -672,8 +672,7 @@ static void update_remote_refs(const struct ref *refs,
672 const char *branch_top,
673 const char *msg,
674 struct transport *transport,
675 - int check_connectivity,
676 - int check_refs_are_promisor_objects_only)
675 + int check_connectivity)
676 {
677 const struct ref *rm = mapped_refs;
678
@@ -682,8 +681,6 @@ static void update_remote_refs(const struct ref *refs,
681
682 opt.transport = transport;
683 opt.progress = transport->progress;
685 - opt.check_refs_are_promisor_objects_only =
686 - !!check_refs_are_promisor_objects_only;
684
685 if (check_connected(iterate_ref_map, &rm, &opt))
686 die(_("remote did not send all necessary objects"));
@@ -1270,7 +1267,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
1267
1268 update_remote_refs(refs, mapped_refs, remote_head_points_at,
1269 branch_top.buf, reflog_msg.buf, transport,
1273 - !is_local, filter_options.choice);
1270 + !is_local);
1271
1272 update_head(our_head_points_at, remote_head, reflog_msg.buf);
1273
builtin/fetch.c
-7
@@ -908,13 +908,6 @@ static int store_updated_refs(const char *raw_url, const char *remote_name,
908 if (!connectivity_checked) {
909 struct check_connected_options opt = CHECK_CONNECTED_INIT;
910
911 - if (filter_options.choice)
912 - /*
913 - * Since a filter is specified, objects indirectly
914 - * referenced by refs are allowed to be absent.
915 - */
916 - opt.check_refs_are_promisor_objects_only = 1;
917 -
911 rm = ref_map;
912 if (check_connected(iterate_ref_map, &rm, &opt)) {
913 rc = error(_("%s did not send all necessary objects\n"), url);
connected.c
+7 -2
@@ -52,7 +52,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
52 strbuf_release(&idx_file);
53 }
54
55 - if (opt->check_refs_are_promisor_objects_only) {
55 + if (has_promisor_remote()) {
56 /*
57 * For partial clones, we don't want to have to do a regular
58 * connectivity check because we have to enumerate and exclude
@@ -71,13 +71,18 @@ int check_connected(oid_iterate_fn fn, void *cb_data,
71 if (find_pack_entry_one(oid.hash, p))
72 goto promisor_pack_found;
73 }
74 - return 1;
74 + /*
75 + * Fallback to rev-list with oid and the rest of the
76 + * object IDs provided by fn.
77 + */
78 + goto no_promisor_pack_found;
79 promisor_pack_found:
80 ;
81 } while (!fn(cb_data, &oid));
82 return 0;
83 }
84
85 +no_promisor_pack_found:
86 if (opt->shallow_file) {
87 argv_array_push(&rev_list.args, "--shallow-file");
88 argv_array_push(&rev_list.args, opt->shallow_file);
connected.h
-9
@@ -46,15 +46,6 @@ struct check_connected_options {
46 * during a fetch.
47 */
48 unsigned is_deepening_fetch : 1;
49 -
50 - /*
51 - * If non-zero, only check that the top-level objects referenced by the
52 - * wanted refs (passed in as cb_data) are promisor objects. This is
53 - * useful for partial clones, where enumerating and excluding all
54 - * promisor objects is very slow and the commit-walk itself becomes a
55 - * no-op.
56 - */
57 - unsigned check_refs_are_promisor_objects_only : 1;
49 };
50
51 #define CHECK_CONNECTED_INIT { 0 }