@matheusbsbs / matheus-bagliano / commits / f954c7b8ff

fsmonitor: never set CE_FSMONITOR_VALID on submodules

Never set CE_FSMONITOR_VALID on the cache-entry of submodule directories. During a client command like 'git status', we may need to recurse into each submodule to compute a status summary for the submodule. Since the purpose of the ce_flag is to let Git avoid scanning a cache-entry, setting the flag causes the recursive call to be avoided and we report incorrect (no status) for the submodule. We created an OS watch on the root directory of our working directory and we receive events for everything in the cone under it. When submodules are present inside our working directory, we receive events for both our repo (the super) and any subs within it. Since our index doesn't have any information for items within the submodules, we can't use those events. We could try to truncate the paths of those events back to the submodule boundary and mark the GITLINK as dirty, but that feels expensive since we would have to prefix compare every FS event that we receive against a list of submodule roots. And it still wouldn't be sufficient to correctly report status on the submodule, since we don't have any space in the cache-entry to cache the submodule's status (the 'SCMU' bits in porcelain V2 speak). That is, the CE_FSMONITOR_VALID bit just says that we don't need to scan/inspect it because we already know the answer -- it doesn't say that the item is clean -- and we don't have space in the cache-entry to store those answers. So we should always do the recursive scan. Therefore, we should never set the flag on GITLINK cache-entries. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Jeff Hostetler committed May 26, 2022 at 21:47 UTC f954c7b8ff3f1443dfd52b5d0c59208a8d1d87ec
3 files changed +124
fsmonitor.c
+2
@@ -580,6 +580,8 @@ void tweak_fsmonitor(struct index_state *istate)
580 if (fsmonitor_enabled) {
581 /* Mark all entries valid */
582 for (i = 0; i < istate->cache_nr; i++) {
583 + if (S_ISGITLINK(istate->cache[i]->ce_mode))
584 + continue;
585 istate->cache[i]->ce_flags |= CE_FSMONITOR_VALID;
586 }
587
fsmonitor.h
+11
@@ -68,6 +68,15 @@ static inline int is_fsmonitor_refreshed(const struct index_state *istate)
68 * Set the given cache entries CE_FSMONITOR_VALID bit. This should be
69 * called any time the cache entry has been updated to reflect the
70 * current state of the file on disk.
71 + *
72 + * However, never mark submodules as valid. When commands like "git
73 + * status" run they might need to recurse into the submodule (using a
74 + * child process) to get a summary of the submodule state. We don't
75 + * have (and don't want to create) the facility to translate every
76 + * FS event that we receive and that happens to be deep inside of a
77 + * submodule back to the submodule root, so we cannot correctly keep
78 + * track of this bit on the gitlink directory. Therefore, we never
79 + * set it on submodules.
80 */
81 static inline void mark_fsmonitor_valid(struct index_state *istate, struct cache_entry *ce)
82 {
@@ -75,6 +84,8 @@ static inline void mark_fsmonitor_valid(struct index_state *istate, struct cache
84
85 if (fsm_mode > FSMONITOR_MODE_DISABLED &&
86 !(ce->ce_flags & CE_FSMONITOR_VALID)) {
87 + if (S_ISGITLINK(ce->ce_mode))
88 + return;
89 istate->cache_changed = 1;
90 ce->ce_flags |= CE_FSMONITOR_VALID;
91 trace_printf_key(&trace_fsmonitor, "mark_fsmonitor_clean '%s'", ce->name);
t/t7527-builtin-fsmonitor.sh
+111
@@ -721,4 +721,115 @@ do
721 '
722 done
723
724 +# Test fsmonitor interaction with submodules.
725 +#
726 +# If we start the daemon in the super, it will see FS events for
727 +# everything in the working directory cone and this includes any
728 +# files/directories contained *within* the submodules.
729 +#
730 +# A `git status` at top level will get events for items within the
731 +# submodule and ignore them, since they aren't named in the index
732 +# of the super repo. This makes the fsmonitor response a little
733 +# noisy, but it doesn't alter the correctness of the state of the
734 +# super-proper.
735 +#
736 +# When we have submodules, `git status` normally does a recursive
737 +# status on each of the submodules and adds a summary row for any
738 +# dirty submodules. (See the "S..." bits in porcelain V2 output.)
739 +#
740 +# It is therefore important that the top level status not be tricked
741 +# by the FSMonitor response to skip those recursive calls. That is,
742 +# even if FSMonitor says that the mtime of the submodule directory
743 +# hasn't changed and it could be implicitly marked valid, we must
744 +# not take that shortcut. We need to force the recusion into the
745 +# submodule so that we get a summary of the status *within* the
746 +# submodule.
747 +
748 +create_super () {
749 + super="$1" &&
750 +
751 + git init "$super" &&
752 + echo x >"$super/file_1" &&
753 + echo y >"$super/file_2" &&
754 + echo z >"$super/file_3" &&
755 + mkdir "$super/dir_1" &&
756 + echo a >"$super/dir_1/file_11" &&
757 + echo b >"$super/dir_1/file_12" &&
758 + mkdir "$super/dir_1/dir_2" &&
759 + echo a >"$super/dir_1/dir_2/file_21" &&
760 + echo b >"$super/dir_1/dir_2/file_22" &&
761 + git -C "$super" add . &&
762 + git -C "$super" commit -m "initial $super commit"
763 +}
764 +
765 +create_sub () {
766 + sub="$1" &&
767 +
768 + git init "$sub" &&
769 + echo x >"$sub/file_x" &&
770 + echo y >"$sub/file_y" &&
771 + echo z >"$sub/file_z" &&
772 + mkdir "$sub/dir_x" &&
773 + echo a >"$sub/dir_x/file_a" &&
774 + echo b >"$sub/dir_x/file_b" &&
775 + mkdir "$sub/dir_x/dir_y" &&
776 + echo a >"$sub/dir_x/dir_y/file_a" &&
777 + echo b >"$sub/dir_x/dir_y/file_b" &&
778 + git -C "$sub" add . &&
779 + git -C "$sub" commit -m "initial $sub commit"
780 +}
781 +
782 +my_match_and_clean () {
783 + git -C super --no-optional-locks status --porcelain=v2 >actual.with &&
784 + git -C super --no-optional-locks -c core.fsmonitor=false \
785 + status --porcelain=v2 >actual.without &&
786 + test_cmp actual.with actual.without &&
787 +
788 + git -C super/dir_1/dir_2/sub reset --hard &&
789 + git -C super/dir_1/dir_2/sub clean -d -f
790 +}
791 +
792 +test_expect_success 'submodule always visited' '
793 + test_when_finished "git -C super fsmonitor--daemon stop; \
794 + rm -rf super; \
795 + rm -rf sub" &&
796 +
797 + create_super super &&
798 + create_sub sub &&
799 +
800 + git -C super submodule add ../sub ./dir_1/dir_2/sub &&
801 + git -C super commit -m "add sub" &&
802 +
803 + start_daemon -C super &&
804 + git -C super config core.fsmonitor true &&
805 + git -C super update-index --fsmonitor &&
806 + git -C super status &&
807 +
808 + # Now run pairs of commands w/ and w/o FSMonitor while we make
809 + # some dirt in the submodule and confirm matching output.
810 +
811 + # Completely clean status.
812 + my_match_and_clean &&
813 +
814 + # .M S..U
815 + echo z >super/dir_1/dir_2/sub/dir_x/dir_y/foobar_u &&
816 + my_match_and_clean &&
817 +
818 + # .M S.M.
819 + echo z >super/dir_1/dir_2/sub/dir_x/dir_y/foobar_m &&
820 + git -C super/dir_1/dir_2/sub add . &&
821 + my_match_and_clean &&
822 +
823 + # .M S.M.
824 + echo z >>super/dir_1/dir_2/sub/dir_x/dir_y/file_a &&
825 + git -C super/dir_1/dir_2/sub add . &&
826 + my_match_and_clean &&
827 +
828 + # .M SC..
829 + echo z >>super/dir_1/dir_2/sub/dir_x/dir_y/file_a &&
830 + git -C super/dir_1/dir_2/sub add . &&
831 + git -C super/dir_1/dir_2/sub commit -m "SC.." &&
832 + my_match_and_clean
833 +'
834 +
835 test_done