@matheusbsbs / matheus-bagliano / commits / b90d9f7632

fetch: fetch unpopulated, changed submodules

"git fetch --recurse-submodules" only considers populated submodules (i.e. submodules that can be found by iterating the index), which makes "git fetch" behave differently based on which commit is checked out. As a result, even if the user has initialized all submodules correctly, they may not fetch the necessary submodule commits, and commands like "git checkout --recurse-submodules" might fail. Teach "git fetch" to fetch cloned, changed submodules regardless of whether they are populated. This is in addition to the current behavior of fetching populated submodules (which is always attempted regardless of what was fetched in the superproject, or even if nothing was fetched in the superproject). A submodule may be encountered multiple times (via the list of populated submodules or via the list of changed submodules). When this happens, "git fetch" only reads the 'populated copy' and ignores the 'changed copy'. Amend the verify_fetch_result() test helper so that we can assert on which 'copy' is being read. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Glen Choo committed Mar 7, 2022 at 16:14 UTC b90d9f7632d380d3f16197ae657ab57075acd1eb
6 files changed +477 -46
Documentation/fetch-options.txt
+17 -9
@@ -186,15 +186,23 @@ endif::git-pull[]
186 ifndef::git-pull[]
187 --recurse-submodules[=yes|on-demand|no]::
188 This option controls if and under what conditions new commits of
189 - populated submodules should be fetched too. It can be used as a
190 - boolean option to completely disable recursion when set to 'no' or to
191 - unconditionally recurse into all populated submodules when set to
192 - 'yes', which is the default when this option is used without any
193 - value. Use 'on-demand' to only recurse into a populated submodule
194 - when the superproject retrieves a commit that updates the submodule's
195 - reference to a commit that isn't already in the local submodule
196 - clone. By default, 'on-demand' is used, unless
197 - `fetch.recurseSubmodules` is set (see linkgit:git-config[1]).
189 + submodules should be fetched too. When recursing through submodules,
190 + `git fetch` always attempts to fetch "changed" submodules, that is, a
191 + submodule that has commits that are referenced by a newly fetched
192 + superproject commit but are missing in the local submodule clone. A
193 + changed submodule can be fetched as long as it is present locally e.g.
194 + in `$GIT_DIR/modules/` (see linkgit:gitsubmodules[7]); if the upstream
195 + adds a new submodule, that submodule cannot be fetched until it is
196 + cloned e.g. by `git submodule update`.
197 ++
198 +When set to 'on-demand', only changed submodules are fetched. When set
199 +to 'yes', all populated submodules are fetched and submodules that are
200 +both unpopulated and changed are fetched. When set to 'no', submodules
201 +are never fetched.
202 ++
203 +When unspecified, this uses the value of `fetch.recurseSubmodules` if it
204 +is set (see linkgit:git-config[1]), defaulting to 'on-demand' if unset.
205 +When this option is used without any value, it defaults to 'yes'.
206 endif::git-pull[]
207
208 -j::
Documentation/git-fetch.txt
+4 -6
@@ -287,12 +287,10 @@ include::transfer-data-leaks.txt[]
287
288 BUGS
289 ----
290 -Using --recurse-submodules can only fetch new commits in already checked
291 -out submodules right now. When e.g. upstream added a new submodule in the
292 -just fetched commits of the superproject the submodule itself cannot be
293 -fetched, making it impossible to check out that submodule later without
294 -having to do a fetch again. This is expected to be fixed in a future Git
295 -version.
290 +Using --recurse-submodules can only fetch new commits in submodules that are
291 +present locally e.g. in `$GIT_DIR/modules/`. If the upstream adds a new
292 +submodule, that submodule cannot be fetched until it is cloned e.g. by `git
293 +submodule update`. This is expected to be fixed in a future Git version.
294
295 SEE ALSO
296 --------
builtin/fetch.c
+7 -7
@@ -2178,13 +2178,13 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
2178 max_children = fetch_parallel_config;
2179
2180 add_options_to_argv(&options);
2181 - result = fetch_populated_submodules(the_repository,
2182 - &options,
2183 - submodule_prefix,
2184 - recurse_submodules,
2185 - recurse_submodules_default,
2186 - verbosity < 0,
2187 - max_children);
2181 + result = fetch_submodules(the_repository,
2182 + &options,
2183 + submodule_prefix,
2184 + recurse_submodules,
2185 + recurse_submodules_default,
2186 + verbosity < 0,
2187 + max_children);
2188 strvec_clear(&options);
2189 }
2190
submodule.c
+177 -16
@@ -808,9 +808,40 @@ static const char *default_name_or_path(const char *path_or_name)
808
809 /*
810 * Holds relevant information for a changed submodule. Used as the .util
811 - * member of the changed submodule string_list_item.
811 + * member of the changed submodule name string_list_item.
812 + *
813 + * (super_oid, path) allows the submodule config to be read from _some_
814 + * .gitmodules file. We store this information the first time we find a
815 + * superproject commit that points to the submodule, but this is
816 + * arbitrary - we can choose any (super_oid, path) that matches the
817 + * submodule's name.
818 + *
819 + * NEEDSWORK: Storing an arbitrary commit is undesirable because we can't
820 + * guarantee that we're reading the commit that the user would expect. A better
821 + * scheme would be to just fetch a submodule by its name. This requires two
822 + * steps:
823 + * - Create a function that behaves like repo_submodule_init(), but accepts a
824 + * submodule name instead of treeish_name and path. This should be easy
825 + * because repo_submodule_init() internally uses the submodule's name.
826 + *
827 + * - Replace most instances of 'struct submodule' (which is the .gitmodules
828 + * config) with just the submodule name. This is OK because we expect
829 + * submodule settings to be stored in .git/config (via "git submodule init"),
830 + * not .gitmodules. This also lets us delete get_non_gitmodules_submodule(),
831 + * which constructs a bogus 'struct submodule' for the sake of giving a
832 + * placeholder name to a gitlink.
833 */
834 struct changed_submodule_data {
835 + /*
836 + * The first superproject commit in the rev walk that points to
837 + * the submodule.
838 + */
839 + const struct object_id *super_oid;
840 + /*
841 + * Path to the submodule in the superproject commit referenced
842 + * by 'super_oid'.
843 + */
844 + char *path;
845 /* The submodule commits that have changed in the rev walk. */
846 struct oid_array new_commits;
847 };
@@ -818,6 +849,7 @@ struct changed_submodule_data {
849 static void changed_submodule_data_clear(struct changed_submodule_data *cs_data)
850 {
851 oid_array_clear(&cs_data->new_commits);
852 + free(cs_data->path);
853 }
854
855 static void collect_changed_submodules_cb(struct diff_queue_struct *q,
@@ -862,9 +894,14 @@ static void collect_changed_submodules_cb(struct diff_queue_struct *q,
894 continue;
895
896 item = string_list_insert(changed, name);
865 - if (!item->util)
897 + if (item->util)
898 + cs_data = item->util;
899 + else {
900 item->util = xcalloc(1, sizeof(struct changed_submodule_data));
867 - cs_data = item->util;
901 + cs_data = item->util;
902 + cs_data->super_oid = commit_oid;
903 + cs_data->path = xstrdup(p->two->path);
904 + }
905 oid_array_append(&cs_data->new_commits, &p->two->oid);
906 }
907 }
@@ -1253,14 +1290,36 @@ void check_for_new_submodule_commits(struct object_id *oid)
1290 oid_array_append(&ref_tips_after_fetch, oid);
1291 }
1292
1293 +/*
1294 + * Returns 1 if there is at least one submodule gitdir in
1295 + * $GIT_DIR/modules and 0 otherwise. This follows
1296 + * submodule_name_to_gitdir(), which looks for submodules in
1297 + * $GIT_DIR/modules, not $GIT_COMMON_DIR.
1298 + *
1299 + * A submodule can be moved to $GIT_DIR/modules manually by running "git
1300 + * submodule absorbgitdirs", or it may be initialized there by "git
1301 + * submodule update".
1302 + */
1303 +static int repo_has_absorbed_submodules(struct repository *r)
1304 +{
1305 + int ret;
1306 + struct strbuf buf = STRBUF_INIT;
1307 +
1308 + strbuf_repo_git_path(&buf, r, "modules/");
1309 + ret = file_exists(buf.buf) && !is_empty_dir(buf.buf);
1310 + strbuf_release(&buf);
1311 + return ret;
1312 +}
1313 +
1314 static void calculate_changed_submodule_paths(struct repository *r,
1315 struct string_list *changed_submodule_names)
1316 {
1317 struct strvec argv = STRVEC_INIT;
1318 struct string_list_item *name;
1319
1262 - /* No need to check if there are no submodules configured */
1263 - if (!submodule_from_path(r, NULL, NULL))
1320 + /* No need to check if no submodules would be fetched */
1321 + if (!submodule_from_path(r, NULL, NULL) &&
1322 + !repo_has_absorbed_submodules(r))
1323 return;
1324
1325 strvec_push(&argv, "--"); /* argv[0] program name */
@@ -1333,7 +1392,16 @@ int submodule_touches_in_range(struct repository *r,
1392 }
1393
1394 struct submodule_parallel_fetch {
1336 - int count;
1395 + /*
1396 + * The index of the last index entry processed by
1397 + * get_fetch_task_from_index().
1398 + */
1399 + int index_count;
1400 + /*
1401 + * The index of the last string_list entry processed by
1402 + * get_fetch_task_from_changed().
1403 + */
1404 + int changed_count;
1405 struct strvec args;
1406 struct repository *r;
1407 const char *prefix;
@@ -1342,7 +1410,16 @@ struct submodule_parallel_fetch {
1410 int quiet;
1411 int result;
1412
1413 + /*
1414 + * Names of submodules that have new commits. Generated by
1415 + * walking the newly fetched superproject commits.
1416 + */
1417 struct string_list changed_submodule_names;
1418 + /*
1419 + * Names of submodules that have already been processed. Lets us
1420 + * avoid fetching the same submodule more than once.
1421 + */
1422 + struct string_list seen_submodule_names;
1423
1424 /* Pending fetches by OIDs */
1425 struct fetch_task **oid_fetch_tasks;
@@ -1353,6 +1430,7 @@ struct submodule_parallel_fetch {
1430 #define SPF_INIT { \
1431 .args = STRVEC_INIT, \
1432 .changed_submodule_names = STRING_LIST_INIT_DUP, \
1433 + .seen_submodule_names = STRING_LIST_INIT_DUP, \
1434 .submodules_with_errors = STRBUF_INIT, \
1435 }
1436
@@ -1390,6 +1468,7 @@ struct fetch_task {
1468 const struct submodule *sub;
1469 unsigned free_sub : 1; /* Do we need to free the submodule? */
1470 const char *default_argv; /* The default fetch mode. */
1471 + struct strvec git_args; /* Args for the child git process. */
1472
1473 struct oid_array *commits; /* Ensure these commits are fetched */
1474 };
@@ -1425,6 +1504,8 @@ static void fetch_task_release(struct fetch_task *p)
1504 if (p->repo)
1505 repo_clear(p->repo);
1506 FREE_AND_NULL(p->repo);
1507 +
1508 + strvec_clear(&p->git_args);
1509 }
1510
1511 static struct repository *get_submodule_repo_for(struct repository *r,
@@ -1463,6 +1544,9 @@ static struct fetch_task *fetch_task_create(struct submodule_parallel_fetch *spf
1544 task->free_sub = 1;
1545 }
1546
1547 + if (string_list_lookup(&spf->seen_submodule_names, task->sub->name))
1548 + goto cleanup;
1549 +
1550 switch (get_fetch_recurse_config(task->sub, spf))
1551 {
1552 default:
@@ -1493,10 +1577,12 @@ static struct fetch_task *fetch_task_create(struct submodule_parallel_fetch *spf
1577 }
1578
1579 static struct fetch_task *
1496 -get_fetch_task(struct submodule_parallel_fetch *spf, struct strbuf *err)
1580 +get_fetch_task_from_index(struct submodule_parallel_fetch *spf,
1581 + struct strbuf *err)
1582 {
1498 - for (; spf->count < spf->r->index->cache_nr; spf->count++) {
1499 - const struct cache_entry *ce = spf->r->index->cache[spf->count];
1583 + for (; spf->index_count < spf->r->index->cache_nr; spf->index_count++) {
1584 + const struct cache_entry *ce =
1585 + spf->r->index->cache[spf->index_count];
1586 struct fetch_task *task;
1587
1588 if (!S_ISGITLINK(ce->ce_mode))
@@ -1511,7 +1597,7 @@ get_fetch_task(struct submodule_parallel_fetch *spf, struct strbuf *err)
1597 strbuf_addf(err, _("Fetching submodule %s%s\n"),
1598 spf->prefix, ce->name);
1599
1514 - spf->count++;
1600 + spf->index_count++;
1601 return task;
1602 } else {
1603 struct strbuf empty_submodule_path = STRBUF_INIT;
@@ -1539,11 +1625,83 @@ get_fetch_task(struct submodule_parallel_fetch *spf, struct strbuf *err)
1625 return NULL;
1626 }
1627
1628 +static struct fetch_task *
1629 +get_fetch_task_from_changed(struct submodule_parallel_fetch *spf,
1630 + struct strbuf *err)
1631 +{
1632 + for (; spf->changed_count < spf->changed_submodule_names.nr;
1633 + spf->changed_count++) {
1634 + struct string_list_item item =
1635 + spf->changed_submodule_names.items[spf->changed_count];
1636 + struct changed_submodule_data *cs_data = item.util;
1637 + struct fetch_task *task;
1638 +
1639 + if (!is_tree_submodule_active(spf->r, cs_data->super_oid,cs_data->path))
1640 + continue;
1641 +
1642 + task = fetch_task_create(spf, cs_data->path,
1643 + cs_data->super_oid);
1644 + if (!task)
1645 + continue;
1646 +
1647 + if (!task->repo) {
1648 + strbuf_addf(err, _("Could not access submodule '%s' at commit %s\n"),
1649 + cs_data->path,
1650 + find_unique_abbrev(cs_data->super_oid, DEFAULT_ABBREV));
1651 +
1652 + fetch_task_release(task);
1653 + free(task);
1654 + continue;
1655 + }
1656 +
1657 + if (!spf->quiet)
1658 + strbuf_addf(err,
1659 + _("Fetching submodule %s%s at commit %s\n"),
1660 + spf->prefix, task->sub->path,
1661 + find_unique_abbrev(cs_data->super_oid,
1662 + DEFAULT_ABBREV));
1663 +
1664 + spf->changed_count++;
1665 + /*
1666 + * NEEDSWORK: Submodules set/unset a value for
1667 + * core.worktree when they are populated/unpopulated by
1668 + * "git checkout" (and similar commands, see
1669 + * submodule_move_head() and
1670 + * connect_work_tree_and_git_dir()), but if the
1671 + * submodule is unpopulated in another way (e.g. "git
1672 + * rm", "rm -r"), core.worktree will still be set even
1673 + * though the directory doesn't exist, and the child
1674 + * process will crash while trying to chdir into the
1675 + * nonexistent directory.
1676 + *
1677 + * In this case, we know that the submodule has no
1678 + * working tree, so we can work around this by
1679 + * setting "--work-tree=." (--bare does not work because
1680 + * worktree settings take precedence over bare-ness).
1681 + * However, this is not necessarily true in other cases,
1682 + * so a generalized solution is still necessary.
1683 + *
1684 + * Possible solutions:
1685 + * - teach "git [add|rm]" to unset core.worktree and
1686 + * discourage users from removing submodules without
1687 + * using a Git command.
1688 + * - teach submodule child processes to ignore stale
1689 + * core.worktree values.
1690 + */
1691 + strvec_push(&task->git_args, "--work-tree=.");
1692 + return task;
1693 + }
1694 + return NULL;
1695 +}
1696 +
1697 static int get_next_submodule(struct child_process *cp, struct strbuf *err,
1698 void *data, void **task_cb)
1699 {
1700 struct submodule_parallel_fetch *spf = data;
1546 - struct fetch_task *task = get_fetch_task(spf, err);
1701 + struct fetch_task *task =
1702 + get_fetch_task_from_index(spf, err);
1703 + if (!task)
1704 + task = get_fetch_task_from_changed(spf, err);
1705
1706 if (task) {
1707 struct strbuf submodule_prefix = STRBUF_INIT;
@@ -1553,6 +1711,8 @@ static int get_next_submodule(struct child_process *cp, struct strbuf *err,
1711 prepare_submodule_repo_env_in_gitdir(&cp->env_array);
1712 cp->git_cmd = 1;
1713 strvec_init(&cp->args);
1714 + if (task->git_args.nr)
1715 + strvec_pushv(&cp->args, task->git_args.v);
1716 strvec_pushv(&cp->args, spf->args.v);
1717 strvec_push(&cp->args, task->default_argv);
1718 strvec_push(&cp->args, "--submodule-prefix");
@@ -1564,6 +1724,7 @@ static int get_next_submodule(struct child_process *cp, struct strbuf *err,
1724 *task_cb = task;
1725
1726 strbuf_release(&submodule_prefix);
1727 + string_list_insert(&spf->seen_submodule_names, task->sub->name);
1728 return 1;
1729 }
1730
@@ -1678,11 +1839,11 @@ out:
1839 return 0;
1840 }
1841
1681 -int fetch_populated_submodules(struct repository *r,
1682 - const struct strvec *options,
1683 - const char *prefix, int command_line_option,
1684 - int default_option,
1685 - int quiet, int max_parallel_jobs)
1842 +int fetch_submodules(struct repository *r,
1843 + const struct strvec *options,
1844 + const char *prefix, int command_line_option,
1845 + int default_option,
1846 + int quiet, int max_parallel_jobs)
1847 {
1848 int i;
1849 struct submodule_parallel_fetch spf = SPF_INIT;
submodule.h
+6 -6
@@ -88,12 +88,12 @@ int should_update_submodules(void);
88 */
89 const struct submodule *submodule_from_ce(const struct cache_entry *ce);
90 void check_for_new_submodule_commits(struct object_id *oid);
91 -int fetch_populated_submodules(struct repository *r,
92 - const struct strvec *options,
93 - const char *prefix,
94 - int command_line_option,
95 - int default_option,
96 - int quiet, int max_parallel_jobs);
91 +int fetch_submodules(struct repository *r,
92 + const struct strvec *options,
93 + const char *prefix,
94 + int command_line_option,
95 + int default_option,
96 + int quiet, int max_parallel_jobs);
97 unsigned is_submodule_modified(const char *path, int ignore_untracked);
98 int submodule_uses_gitfile(const char *path);
99
t/t5526-fetch-submodules.sh
+266 -2
@@ -12,17 +12,29 @@ pwd=$(pwd)
12
13 write_expected_sub () {
14 NEW_HEAD=$1 &&
15 + SUPER_HEAD=$2 &&
16 cat >"$pwd/expect.err.sub" <<-EOF
16 - Fetching submodule submodule
17 + Fetching submodule submodule${SUPER_HEAD:+ at commit $SUPER_HEAD}
18 From $pwd/submodule
19 OLD_HEAD..$NEW_HEAD sub -> origin/sub
20 EOF
21 }
22
23 +write_expected_sub2 () {
24 + NEW_HEAD=$1 &&
25 + SUPER_HEAD=$2 &&
26 + cat >"$pwd/expect.err.sub2" <<-EOF
27 + Fetching submodule submodule2${SUPER_HEAD:+ at commit $SUPER_HEAD}
28 + From $pwd/submodule2
29 + OLD_HEAD..$NEW_HEAD sub2 -> origin/sub2
30 + EOF
31 +}
32 +
33 write_expected_deep () {
34 NEW_HEAD=$1 &&
35 + SUB_HEAD=$2 &&
36 cat >"$pwd/expect.err.deep" <<-EOF
25 - Fetching submodule submodule/subdir/deepsubmodule
37 + Fetching submodule submodule/subdir/deepsubmodule${SUB_HEAD:+ at commit $SUB_HEAD}
38 From $pwd/deepsubmodule
39 OLD_HEAD..$NEW_HEAD deep -> origin/deep
40 EOF
@@ -106,6 +118,10 @@ verify_fetch_result () {
118 then
119 cat expect.err.deep >>expect.err.combined
120 fi &&
121 + if test -f expect.err.sub2
122 + then
123 + cat expect.err.sub2 >>expect.err.combined
124 + fi &&
125 sed -e 's/[0-9a-f][0-9a-f]*\.\./OLD_HEAD\.\./' "$ACTUAL_ERR" >actual.err.cmp &&
126 test_cmp expect.err.combined actual.err.cmp
127 }
@@ -419,6 +435,147 @@ test_expect_success "'--recurse-submodules=on-demand' recurses as deep as necess
435 verify_fetch_result actual.err
436 '
437
438 +# These tests verify that we can fetch submodules that aren't in the
439 +# index.
440 +#
441 +# First, test the simple case where the index is empty and we only fetch
442 +# submodules that are not in the index.
443 +test_expect_success 'setup downstream branch without submodules' '
444 + (
445 + cd downstream &&
446 + git checkout --recurse-submodules -b no-submodules &&
447 + git rm .gitmodules &&
448 + git rm submodule &&
449 + git commit -m "no submodules" &&
450 + git checkout --recurse-submodules super
451 + )
452 +'
453 +
454 +test_expect_success "'--recurse-submodules=on-demand' should fetch submodule commits if the submodule is changed but the index has no submodules" '
455 + add_submodule_commits &&
456 + add_superproject_commits &&
457 + # Fetch the new superproject commit
458 + (
459 + cd downstream &&
460 + git switch --recurse-submodules no-submodules &&
461 + git fetch --recurse-submodules=on-demand >../actual.out 2>../actual.err
462 + ) &&
463 + super_head=$(git rev-parse --short HEAD) &&
464 + sub_head=$(git -C submodule rev-parse --short HEAD) &&
465 + deep_head=$(git -C submodule/subdir/deepsubmodule rev-parse --short HEAD) &&
466 +
467 + # assert that these are fetched from commits, not the index
468 + write_expected_sub $sub_head $super_head &&
469 + write_expected_deep $deep_head $sub_head &&
470 +
471 + test_must_be_empty actual.out &&
472 + verify_fetch_result actual.err
473 +'
474 +
475 +test_expect_success "'--recurse-submodules' should fetch submodule commits if the submodule is changed but the index has no submodules" '
476 + add_submodule_commits &&
477 + add_superproject_commits &&
478 + # Fetch the new superproject commit
479 + (
480 + cd downstream &&
481 + git switch --recurse-submodules no-submodules &&
482 + git fetch --recurse-submodules >../actual.out 2>../actual.err
483 + ) &&
484 + super_head=$(git rev-parse --short HEAD) &&
485 + sub_head=$(git -C submodule rev-parse --short HEAD) &&
486 + deep_head=$(git -C submodule/subdir/deepsubmodule rev-parse --short HEAD) &&
487 +
488 + # assert that these are fetched from commits, not the index
489 + write_expected_sub $sub_head $super_head &&
490 + write_expected_deep $deep_head $sub_head &&
491 +
492 + test_must_be_empty actual.out &&
493 + verify_fetch_result actual.err
494 +'
495 +
496 +test_expect_success "'--recurse-submodules' should ignore changed, inactive submodules" '
497 + add_submodule_commits &&
498 + add_superproject_commits &&
499 +
500 + # Fetch the new superproject commit
501 + (
502 + cd downstream &&
503 + git switch --recurse-submodules no-submodules &&
504 + git -c submodule.submodule.active=false fetch --recurse-submodules >../actual.out 2>../actual.err
505 + ) &&
506 + test_must_be_empty actual.out &&
507 + super_head=$(git rev-parse --short HEAD) &&
508 + write_expected_super $super_head &&
509 + # Neither should be fetched because the submodule is inactive
510 + rm expect.err.sub &&
511 + rm expect.err.deep &&
512 + verify_fetch_result actual.err
513 +'
514 +
515 +# Now that we know we can fetch submodules that are not in the index,
516 +# test that we can fetch index and non-index submodules in the same
517 +# operation.
518 +test_expect_success 'setup downstream branch with other submodule' '
519 + mkdir submodule2 &&
520 + (
521 + cd submodule2 &&
522 + git init &&
523 + echo sub2content >sub2file &&
524 + git add sub2file &&
525 + git commit -a -m new &&
526 + git branch -M sub2
527 + ) &&
528 + git checkout -b super-sub2-only &&
529 + git submodule add "$pwd/submodule2" submodule2 &&
530 + git commit -m "add sub2" &&
531 + git checkout super &&
532 + (
533 + cd downstream &&
534 + git fetch --recurse-submodules origin &&
535 + git checkout super-sub2-only &&
536 + # Explicitly run "git submodule update" because sub2 is new
537 + # and has not been cloned.
538 + git submodule update --init &&
539 + git checkout --recurse-submodules super
540 + )
541 +'
542 +
543 +test_expect_success "'--recurse-submodules' should fetch submodule commits in changed submodules and the index" '
544 + test_when_finished "rm expect.err.sub2" &&
545 + # Create new commit in origin/super
546 + add_submodule_commits &&
547 + add_superproject_commits &&
548 +
549 + # Create new commit in origin/super-sub2-only
550 + git checkout super-sub2-only &&
551 + (
552 + cd submodule2 &&
553 + test_commit --no-tag foo
554 + ) &&
555 + git add submodule2 &&
556 + git commit -m "new submodule2" &&
557 +
558 + git checkout super &&
559 + (
560 + cd downstream &&
561 + git fetch --recurse-submodules >../actual.out 2>../actual.err
562 + ) &&
563 + test_must_be_empty actual.out &&
564 + sub2_head=$(git -C submodule2 rev-parse --short HEAD) &&
565 + super_head=$(git rev-parse --short super) &&
566 + super_sub2_only_head=$(git rev-parse --short super-sub2-only) &&
567 + write_expected_sub2 $sub2_head $super_sub2_only_head &&
568 +
569 + # write_expected_super cannot handle >1 branch. Since this is a
570 + # one-off, construct expect.err.super manually.
571 + cat >"$pwd/expect.err.super" <<-EOF &&
572 + From $pwd/.
573 + OLD_HEAD..$super_head super -> origin/super
574 + OLD_HEAD..$super_sub2_only_head super-sub2-only -> origin/super-sub2-only
575 + EOF
576 + verify_fetch_result actual.err
577 +'
578 +
579 test_expect_success "'--recurse-submodules=on-demand' stops when no new submodule commits are found in the superproject (and ignores config)" '
580 add_submodule_commits &&
581 echo a >> file &&
@@ -861,4 +1018,111 @@ test_expect_success 'recursive fetch after deinit a submodule' '
1018 test_cmp expect actual
1019 '
1020
1021 +test_expect_success 'setup repo with upstreams that share a submodule name' '
1022 + mkdir same-name-1 &&
1023 + (
1024 + cd same-name-1 &&
1025 + git init -b main &&
1026 + test_commit --no-tag a
1027 + ) &&
1028 + git clone same-name-1 same-name-2 &&
1029 + # same-name-1 and same-name-2 both add a submodule with the
1030 + # name "submodule"
1031 + (
1032 + cd same-name-1 &&
1033 + mkdir submodule &&
1034 + git -C submodule init -b main &&
1035 + test_commit -C submodule --no-tag a1 &&
1036 + git submodule add "$pwd/same-name-1/submodule" &&
1037 + git add submodule &&
1038 + git commit -m "super-a1"
1039 + ) &&
1040 + (
1041 + cd same-name-2 &&
1042 + mkdir submodule &&
1043 + git -C submodule init -b main &&
1044 + test_commit -C submodule --no-tag a2 &&
1045 + git submodule add "$pwd/same-name-2/submodule" &&
1046 + git add submodule &&
1047 + git commit -m "super-a2"
1048 + ) &&
1049 + git clone same-name-1 -o same-name-1 same-name-downstream &&
1050 + (
1051 + cd same-name-downstream &&
1052 + git remote add same-name-2 ../same-name-2 &&
1053 + git fetch --all &&
1054 + # init downstream with same-name-1
1055 + git submodule update --init
1056 + )
1057 +'
1058 +
1059 +test_expect_success 'fetch --recurse-submodules updates name-conflicted, populated submodule' '
1060 + test_when_finished "git -C same-name-downstream checkout main" &&
1061 + (
1062 + cd same-name-1 &&
1063 + test_commit -C submodule --no-tag b1 &&
1064 + git add submodule &&
1065 + git commit -m "super-b1"
1066 + ) &&
1067 + (
1068 + cd same-name-2 &&
1069 + test_commit -C submodule --no-tag b2 &&
1070 + git add submodule &&
1071 + git commit -m "super-b2"
1072 + ) &&
1073 + (
1074 + cd same-name-downstream &&
1075 + # even though the .gitmodules is correct, we cannot
1076 + # fetch from same-name-2
1077 + git checkout same-name-2/main &&
1078 + git fetch --recurse-submodules same-name-1 &&
1079 + test_must_fail git fetch --recurse-submodules same-name-2
1080 + ) &&
1081 + super_head1=$(git -C same-name-1 rev-parse HEAD) &&
1082 + git -C same-name-downstream cat-file -e $super_head1 &&
1083 +
1084 + super_head2=$(git -C same-name-2 rev-parse HEAD) &&
1085 + git -C same-name-downstream cat-file -e $super_head2 &&
1086 +
1087 + sub_head1=$(git -C same-name-1/submodule rev-parse HEAD) &&
1088 + git -C same-name-downstream/submodule cat-file -e $sub_head1 &&
1089 +
1090 + sub_head2=$(git -C same-name-2/submodule rev-parse HEAD) &&
1091 + test_must_fail git -C same-name-downstream/submodule cat-file -e $sub_head2
1092 +'
1093 +
1094 +test_expect_success 'fetch --recurse-submodules updates name-conflicted, unpopulated submodule' '
1095 + (
1096 + cd same-name-1 &&
1097 + test_commit -C submodule --no-tag c1 &&
1098 + git add submodule &&
1099 + git commit -m "super-c1"
1100 + ) &&
1101 + (
1102 + cd same-name-2 &&
1103 + test_commit -C submodule --no-tag c2 &&
1104 + git add submodule &&
1105 + git commit -m "super-c2"
1106 + ) &&
1107 + (
1108 + cd same-name-downstream &&
1109 + git checkout main &&
1110 + git rm .gitmodules &&
1111 + git rm submodule &&
1112 + git commit -m "no submodules" &&
1113 + git fetch --recurse-submodules same-name-1
1114 + ) &&
1115 + head1=$(git -C same-name-1/submodule rev-parse HEAD) &&
1116 + head2=$(git -C same-name-2/submodule rev-parse HEAD) &&
1117 + (
1118 + cd same-name-downstream/.git/modules/submodule &&
1119 + # The submodule has core.worktree pointing to the "git
1120 + # rm"-ed directory, overwrite the invalid value. See
1121 + # comment in get_fetch_task_from_changed() for more
1122 + # information.
1123 + git --work-tree=. cat-file -e $head1 &&
1124 + test_must_fail git --work-tree=. cat-file -e $head2
1125 + )
1126 +'
1127 +
1128 test_done