upload-pack: look up "want" lines via commit-graph

During packfile negotiation the client will send "want" and "want-ref" lines to the server to tell it which objects it is interested in. The server-side parses each of those and looks them up to see whether it actually has requested objects. This lookup is performed by calling `parse_object()` directly, which thus hits the object database. In the general case though most of the objects the client requests will be commits. We can thus try to look up the object via the commit-graph opportunistically, which is much faster than doing the same via the object database. Refactor parsing of both "want" and "want-ref" lines to do so. The following benchmark is executed in a repository with a huge number of references. It uses cached request from git-fetch(1) as input to git-upload-pack(1) that contains about 876,000 "want" lines: Benchmark 1: HEAD~ Time (mean ± σ): 7.113 s ± 0.028 s [User: 6.900 s, System: 0.662 s] Range (min … max): 7.072 s … 7.168 s 10 runs Benchmark 2: HEAD Time (mean ± σ): 6.622 s ± 0.061 s [User: 6.452 s, System: 0.650 s] Range (min … max): 6.535 s … 6.727 s 10 runs Summary 'HEAD' ran 1.07 ± 0.01 times faster than 'HEAD~' Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Patrick Steinhardt committed Mar 1, 2022 at 10:33 UTC 4de656263aa195080495fc0a103351b9eaac8160
1 file changed +17 -3
upload-pack.c
+17 -3
@@ -1400,13 +1400,19 @@ static int parse_want(struct packet_writer *writer, const char *line,
1400 const char *arg;
1401 if (skip_prefix(line, "want ", &arg)) {
1402 struct object_id oid;
1403 + struct commit *commit;
1404 struct object *o;
1405
1406 if (get_oid_hex(arg, &oid))
1407 die("git upload-pack: protocol error, "
1408 "expected to get oid, not '%s'", line);
1409
1409 - o = parse_object(the_repository, &oid);
1410 + commit = lookup_commit_in_graph(the_repository, &oid);
1411 + if (commit)
1412 + o = &commit->object;
1413 + else
1414 + o = parse_object(the_repository, &oid);
1415 +
1416 if (!o) {
1417 packet_writer_error(writer,
1418 "upload-pack: not our ref %s",
@@ -1434,7 +1440,7 @@ static int parse_want_ref(struct packet_writer *writer, const char *line,
1440 if (skip_prefix(line, "want-ref ", &refname_nons)) {
1441 struct object_id oid;
1442 struct string_list_item *item;
1437 - struct object *o;
1443 + struct object *o = NULL;
1444 struct strbuf refname = STRBUF_INIT;
1445
1446 strbuf_addf(&refname, "%s%s", get_git_namespace(), refname_nons);
@@ -1448,7 +1454,15 @@ static int parse_want_ref(struct packet_writer *writer, const char *line,
1454 item = string_list_append(wanted_refs, refname_nons);
1455 item->util = oiddup(&oid);
1456
1451 - o = parse_object_or_die(&oid, refname_nons);
1457 + if (!starts_with(refname_nons, "refs/tags/")) {
1458 + struct commit *commit = lookup_commit_in_graph(the_repository, &oid);
1459 + if (commit)
1460 + o = &commit->object;
1461 + }
1462 +
1463 + if (!o)
1464 + o = parse_object_or_die(&oid, refname_nons);
1465 +
1466 if (!(o->flags & WANTED)) {
1467 o->flags |= WANTED;
1468 add_object_array(o, NULL, want_obj);