Avoid conflicts when merging branches with mixed normalization
Currently, merging across changes in line ending normalization is painful since files containing CRLF will conflict with normalized files, even if the only difference between the two versions is the line endings. Additionally, any "real" merge conflicts that exist are obscured because every line in the file has a conflict. Assume you start out with a repo that has a lot of text files with CRLF checked in (A): o---C / \ A---B---D B: Add "* text=auto" to .gitattributes and normalize all files to LF-only C: Modify some of the text files D: Try to merge C You will get a ridiculous number of LF/CRLF conflicts when trying to merge C into D, since the repository contents for C are "wrong" wrt the new .gitattributes file. Fix ll-merge so that the "base", "theirs" and "ours" stages are passed through convert_to_worktree() and convert_to_git() before a three-way merge. This ensures that all three stages are normalized in the same way, removing from consideration differences that are only due to normalization. This feature is optional for now since it changes a low-level mechanism and is not necessary for the majority of users. The "merge.renormalize" config variable enables it. Signed-off-by: Eyvind Bernhardsen <eyvind.bernhardsen@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Eyvind Bernhardsen committed
Jul 2, 2010 at 21:20 UTC
f217f0e86dc7bacc5dc127982eaadca758b558ce
8 files changed
+143
-2
Documentation/gitattributes.txt
+34
@@ -317,6 +317,17 @@ command is "cat").
317
smudge = cat
318
------------------------
319
320
+For best results, `clean` should not alter its output further if it is
321
+run twice ("clean->clean" should be equivalent to "clean"), and
322
+multiple `smudge` commands should not alter `clean`'s output
323
+("smudge->smudge->clean" should be equivalent to "clean"). See the
324
+section on merging below.
325
+
326
+The "indent" filter is well-behaved in this regard: it will not modify
327
+input that is already correctly indented. In this case, the lack of a
328
+smudge filter means that the clean filter _must_ accept its own output
329
+without modifying it.
330
+
331
332
Interaction between checkin/checkout attributes
333
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -331,6 +342,29 @@ In the check-out codepath, the blob content is first converted
342
with `text`, and then `ident` and fed to `filter`.
343
344
345
+Merging branches with differing checkin/checkout attributes
346
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
347
+
348
+If you have added attributes to a file that cause the canonical
349
+repository format for that file to change, such as adding a
350
+clean/smudge filter or text/eol/ident attributes, merging anything
351
+where the attribute is not in place would normally cause merge
352
+conflicts.
353
+
354
+To prevent these unnecessary merge conflicts, git can be told to run a
355
+virtual check-out and check-in of all three stages of a file when
356
+resolving a three-way merge by setting the `merge.renormalize`
357
+configuration variable. This prevents changes caused by check-in
358
+conversion from causing spurious merge conflicts when a converted file
359
+is merged with an unconverted file.
360
+
361
+As long as a "smudge->clean" results in the same output as a "clean"
362
+even on files that are already smudged, this strategy will
363
+automatically resolve all filter-related conflicts. Filters that do
364
+not act in this way may cause additional merge conflicts that must be
365
+resolved manually.
366
+
367
+
368
Generating diff text
369
~~~~~~~~~~~~~~~~~~~~
370
Documentation/merge-config.txt
+10
@@ -15,6 +15,16 @@ merge.renameLimit::
15
during a merge; if not specified, defaults to the value of
16
diff.renameLimit.
17
18
+merge.renormalize::
19
+ Tell git that canonical representation of files in the
20
+ repository has changed over time (e.g. earlier commits record
21
+ text files with CRLF line endings, but recent ones use LF line
22
+ endings). In such a repository, git can convert the data
23
+ recorded in commits to a canonical form before performing a
24
+ merge to reduce unnecessary conflicts. For more information,
25
+ see section "Merging branches with differing checkin/checkout
26
+ attributes" in linkgit:gitattributes[5].
27
+
28
merge.stat::
29
Whether to print the diffstat between ORIG_HEAD and the merge result
30
at the end of the merge. True by default.
builtin/merge.c
+3
@@ -503,6 +503,9 @@ static int git_merge_config(const char *k, const char *v, void *cb)
503
return git_config_string(&pull_octopus, k, v);
504
else if (!strcmp(k, "merge.log") || !strcmp(k, "merge.summary"))
505
option_log = git_config_bool(k, v);
506
+ else if (!strcmp(k, "merge.renormalize")) {
507
+ merge_renormalize = git_config_bool(k, v);
508
+ }
509
return git_diff_ui_config(k, v, cb);
510
}
511
cache.h
+2
@@ -551,6 +551,7 @@ extern int read_replace_refs;
551
extern int fsync_object_files;
552
extern int core_preload_index;
553
extern int core_apply_sparse_checkout;
554
+extern int merge_renormalize;
555
556
enum safe_crlf {
557
SAFE_CRLF_FALSE = 0,
@@ -1054,6 +1055,7 @@ extern void trace_argv_printf(const char **argv, const char *format, ...);
1055
extern int convert_to_git(const char *path, const char *src, size_t len,
1056
struct strbuf *dst, enum safe_crlf checksafe);
1057
extern int convert_to_working_tree(const char *path, const char *src, size_t len, struct strbuf *dst);
1058
+extern int renormalize_buffer(const char *path, const char *src, size_t len, struct strbuf *dst);
1059
1060
/* add */
1061
/*
convert.c
+14
-2
@@ -93,7 +93,8 @@ static int is_binary(unsigned long size, struct text_stat *stats)
93
return 0;
94
}
95
96
-static enum eol determine_output_conversion(enum action action) {
96
+static enum eol determine_output_conversion(enum action action)
97
+{
98
switch (action) {
99
case CRLF_BINARY:
100
return EOL_UNSET;
@@ -693,7 +694,8 @@ static int git_path_check_ident(const char *path, struct git_attr_check *check)
694
return !!ATTR_TRUE(value);
695
}
696
696
-enum action determine_action(enum action text_attr, enum eol eol_attr) {
697
+static enum action determine_action(enum action text_attr, enum eol eol_attr)
698
+{
699
if (text_attr == CRLF_BINARY)
700
return CRLF_BINARY;
701
if (eol_attr == EOL_LF)
@@ -773,3 +775,13 @@ int convert_to_working_tree(const char *path, const char *src, size_t len, struc
775
}
776
return ret | apply_filter(path, src, len, dst, filter);
777
}
778
+
779
+int renormalize_buffer(const char *path, const char *src, size_t len, struct strbuf *dst)
780
+{
781
+ int ret = convert_to_working_tree(path, src, len, dst);
782
+ if (ret) {
783
+ src = dst->buf;
784
+ len = dst->len;
785
+ }
786
+ return ret | convert_to_git(path, src, len, dst, 0);
787
+}
environment.c
+1
@@ -53,6 +53,7 @@ enum object_creation_mode object_creation_mode = OBJECT_CREATION_MODE;
53
char *notes_ref_name;
54
int grafts_replace_parents = 1;
55
int core_apply_sparse_checkout;
56
+int merge_renormalize;
57
58
/* Parallel index stat data preload? */
59
int core_preload_index = 0;
ll-merge.c
+15
@@ -321,6 +321,16 @@ static int git_path_check_merge(const char *path, struct git_attr_check check[2]
321
return git_checkattr(path, 2, check);
322
}
323
324
+static void normalize_file(mmfile_t *mm, const char *path)
325
+{
326
+ struct strbuf strbuf = STRBUF_INIT;
327
+ if (renormalize_buffer(path, mm->ptr, mm->size, &strbuf)) {
328
+ free(mm->ptr);
329
+ mm->size = strbuf.len;
330
+ mm->ptr = strbuf_detach(&strbuf, NULL);
331
+ }
332
+}
333
+
334
int ll_merge(mmbuffer_t *result_buf,
335
const char *path,
336
mmfile_t *ancestor, const char *ancestor_label,
@@ -334,6 +344,11 @@ int ll_merge(mmbuffer_t *result_buf,
344
const struct ll_merge_driver *driver;
345
int virtual_ancestor = flag & 01;
346
347
+ if (merge_renormalize) {
348
+ normalize_file(ancestor, path);
349
+ normalize_file(ours, path);
350
+ normalize_file(theirs, path);
351
+ }
352
if (!git_path_check_merge(path, check)) {
353
ll_driver_name = check[0].value;
354
if (check[1].value) {
t/t6038-merge-text-auto.sh
new
+64
@@ -0,0 +1,64 @@
1
+#!/bin/sh
2
+
3
+test_description='CRLF merge conflict across text=auto change'
4
+
5
+. ./test-lib.sh
6
+
7
+test_expect_success setup '
8
+ git config merge.renormalize true &&
9
+ git config core.autocrlf false &&
10
+ echo first line | append_cr >file &&
11
+ echo first line >control_file &&
12
+ echo only line >inert_file &&
13
+ git add file control_file inert_file &&
14
+ git commit -m "Initial" &&
15
+ git tag initial &&
16
+ git branch side &&
17
+ echo "* text=auto" >.gitattributes &&
18
+ touch file &&
19
+ git add .gitattributes file &&
20
+ git commit -m "normalize file" &&
21
+ echo same line | append_cr >>file &&
22
+ echo same line >>control_file &&
23
+ git add file control_file &&
24
+ git commit -m "add line from a" &&
25
+ git tag a &&
26
+ git rm .gitattributes &&
27
+ rm file &&
28
+ git checkout file &&
29
+ git commit -m "remove .gitattributes" &&
30
+ git tag c &&
31
+ git checkout side &&
32
+ echo same line | append_cr >>file &&
33
+ echo same line >>control_file &&
34
+ git add file control_file &&
35
+ git commit -m "add line from b" &&
36
+ git tag b &&
37
+ git checkout master
38
+'
39
+
40
+test_expect_success 'Check merging after setting text=auto' '
41
+ git reset --hard a &&
42
+ git merge b &&
43
+ cat file | remove_cr >file.temp &&
44
+ test_cmp file file.temp
45
+'
46
+
47
+test_expect_success 'Check merging addition of text=auto' '
48
+ git reset --hard b &&
49
+ git merge a &&
50
+ cat file | remove_cr >file.temp &&
51
+ test_cmp file file.temp
52
+'
53
+
54
+test_expect_failure 'Test delete/normalize conflict' '
55
+ git checkout side &&
56
+ git reset --hard initial &&
57
+ git rm file &&
58
+ git commit -m "remove file" &&
59
+ git checkout master &&
60
+ git reset --hard a^ &&
61
+ git merge side
62
+'
63
+
64
+test_done