Add per-repository eol normalization
Change the semantics of the "crlf" attribute so that it enables end-of-line normalization when it is set, regardless of "core.autocrlf". Add a new setting for "crlf": "auto", which enables end-of-line conversion but does not override the automatic text file detection. Add a new attribute "eol" with possible values "crlf" and "lf". When set, this attribute enables normalization and forces git to use CRLF or LF line endings in the working directory, respectively. The line ending style to be used for normalized text files in the working directory is set using "core.autocrlf". When it is set to "true", CRLFs are used in the working directory; when set to "input" or "false", LFs are used. Signed-off-by: Eyvind Bernhardsen <eyvind.bernhardsen@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Eyvind Bernhardsen committed
May 19, 2010 at 22:43 UTC
fd6cce9e89ab5ac1125a3b5f5611048ad22379e7
7 files changed
+215
-69
Documentation/config.txt
+2
-2
@@ -203,8 +203,8 @@ core.autocrlf::
203
based on the file's contents. See linkgit:gitattributes[5].
204
205
core.safecrlf::
206
- If true, makes git check if converting `CRLF` as controlled by
207
- `core.autocrlf` is reversible. Git will verify if a command
206
+ If true, makes git check if converting `CRLF` is reversible when
207
+ end-of-line conversion is active. Git will verify if a command
208
modifies a file in the work tree either directly or indirectly.
209
For example, committing a file followed by checking out the
210
same file should yield the original file in the work tree. If
Documentation/gitattributes.txt
+113
-25
@@ -95,50 +95,138 @@ repository upon 'git add' and 'git commit'.
95
`crlf`
96
^^^^^^
97
98
-This attribute controls the line-ending convention.
98
+This attribute enables and controls end-of-line normalization. When a
99
+text file is normalized, its line endings are converted to LF in the
100
+repository. To control what line ending style is used in the working
101
+directory, use the `eol` attribute for a single file and the
102
+`core.autocrlf` configuration variable for all text files.
103
104
Set::
105
102
- Setting the `crlf` attribute on a path is meant to mark
103
- the path as a "text" file. 'core.autocrlf' conversion
104
- takes place without guessing the content type by
105
- inspection.
106
+ Setting the `crlf` attribute on a path enables end-of-line
107
+ normalization and marks the path as a text file. End-of-line
108
+ conversion takes place without guessing the content type.
109
110
Unset::
111
112
Unsetting the `crlf` attribute on a path tells git not to
113
attempt any end-of-line conversion upon checkin or checkout.
114
115
+Set to string value "auto"::
116
+
117
+ When `crlf` is set to "auto", the path is marked for automatic
118
+ end-of-line normalization. If git decides that the content is
119
+ text, its line endings are normalized to LF on checkin.
120
+
121
Unspecified::
122
114
- Unspecified `crlf` attribute tells git to apply the
115
- `core.autocrlf` conversion when the file content looks
116
- like text.
123
+ If the `crlf` attribute is unspecified, git uses the `eol`
124
+ attribute and the `core.autocrlf` configuration variable to
125
+ determine if the file should be converted.
126
118
-Set to string value "input"::
127
+Any other value causes git to act as if `crlf` has been left
128
+unspecified.
129
120
- This is similar to setting the attribute to `true`, but
121
- also forces git to act as if `core.autocrlf` is set to
122
- `input` for the path.
130
+`eol`
131
+^^^^^
132
124
-Any other value set to `crlf` attribute is ignored and git acts
125
-as if the attribute is left unspecified.
133
+This attribute sets a specific line-ending style to be used in the
134
+working directory. It enables end-of-line normalization without any
135
+content checks, similar to setting the `crlf` attribute.
136
137
+Set to string value "crlf"::
138
128
-The `core.autocrlf` conversion
129
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
139
+ This setting forces git to normalize line endings on checkin
140
+ and convert them to CRLF when the file is checked out,
141
+ regardless of `crlf` and `core.autocrlf`.
142
+
143
+Set to string value "lf"::
144
+
145
+ This setting forces git to normalize line endings to LF on
146
+ checkin and prevents conversion to CRLF when the file is
147
+ checked out, regardless of `crlf` and `core.autocrlf`.
148
+ `crlf=input` is a backwards compatibility alias for `eol=lf`.
149
+
150
+End-of-line conversion
151
+^^^^^^^^^^^^^^^^^^^^^^
152
+
153
+While git normally leaves file contents alone, it can be configured to
154
+normalize line endings to LF in the repository and, optionally, to
155
+convert them to CRLF when files are checked out.
156
+
157
+Here is an example that will make git normalize .txt, .vcproj and .sh
158
+files, ensure that .vcproj files have CRLF and .sh files have LF in
159
+the working directory, and prevent .jpg files from being normalized
160
+regardless of their content.
161
+
162
+------------------------
163
+*.txt crlf
164
+*.vcproj eol=crlf
165
+*.sh eol=lf
166
+*.jpg -crlf
167
+------------------------
168
+
169
+Other source code management systems normalize all text files in their
170
+repositories, and there are two ways to enable similar automatic
171
+normalization in git.
172
+
173
+If you simply want to have CRLF line endings in your working directory
174
+regardless of the repository you are working with, you can set the
175
+config variable "core.autocrlf" without changing any attributes.
176
+
177
+------------------------
178
+[core]
179
+ autocrlf = true
180
+------------------------
181
+
182
+This does not force normalization of all text files, but does ensure
183
+that text files that you introduce to the repository have their line
184
+endings normalized to LF when they are added, and that files that are
185
+already normalized in the repository stay normalized. You can also
186
+set `autocrlf` to "input" to have automatic normalization of new text
187
+files without conversion to CRLF in the working directory.
188
+
189
+If you want to interoperate with a source code management system that
190
+enforces end-of-line normalization, or you simply want all text files
191
+in your repository to be normalized, you should instead set the `crlf`
192
+attribute to "auto" for _all_ files.
193
+
194
+------------------------
195
+* crlf=auto
196
+------------------------
197
+
198
+This ensures that all files that git considers to be text will have
199
+normalized (LF) line endings in the repository.
200
131
-If the configuration variable `core.autocrlf` is false, no
132
-conversion is done.
201
+NOTE: When `crlf=auto` normalization is enabled in an existing
202
+repository, any text files containing CRLFs should be normalized. If
203
+they are not they will be normalized the next time someone tries to
204
+change them, causing unfortunate misattribution. From a clean working
205
+directory:
206
134
-When `core.autocrlf` is true, it means that the platform wants
135
-CRLF line endings for files in the working tree, and you want to
136
-convert them back to the normal LF line endings when checking
137
-in to the repository.
207
+-------------------------------------------------
208
+$ echo "* crlf=auto" >>.gitattributes
209
+$ rm .git/index # Remove the index to force git to
210
+$ git reset # re-scan the working directory
211
+$ git status # Show files that will be normalized
212
+$ git add -u
213
+$ git add .gitattributes
214
+$ git commit -m "Introduce end-of-line normalization"
215
+-------------------------------------------------
216
139
-When `core.autocrlf` is set to "input", line endings are
140
-converted to LF upon checkin, but there is no conversion done
141
-upon checkout.
217
+If any files that should not be normalized show up in 'git status',
218
+unset their `crlf` attribute before running 'git add -u'.
219
+
220
+------------------------
221
+manual.pdf -crlf
222
+------------------------
223
+
224
+Conversely, text files that git does not detect can have normalization
225
+enabled manually.
226
+
227
+------------------------
228
+weirdchars.txt crlf
229
+------------------------
230
231
If `core.safecrlf` is set to "true" or "warn", git verifies if
232
the conversion is reversible for the current setting of
cache.h
+8
-1
@@ -535,7 +535,6 @@ extern int core_compression_seen;
535
extern size_t packed_git_window_size;
536
extern size_t packed_git_limit;
537
extern size_t delta_base_cache_limit;
538
-extern int auto_crlf;
538
extern int read_replace_refs;
539
extern int fsync_object_files;
540
extern int core_preload_index;
@@ -549,6 +548,14 @@ enum safe_crlf {
548
549
extern enum safe_crlf safe_crlf;
550
551
+enum auto_crlf {
552
+ AUTO_CRLF_FALSE = 0,
553
+ AUTO_CRLF_TRUE = 1,
554
+ AUTO_CRLF_INPUT = -1,
555
+};
556
+
557
+extern enum auto_crlf auto_crlf;
558
+
559
enum branch_track {
560
BRANCH_TRACK_UNSPECIFIED = -1,
561
BRANCH_TRACK_NEVER = 0,
config.c
+1
-1
@@ -461,7 +461,7 @@ static int git_default_core_config(const char *var, const char *value)
461
462
if (!strcmp(var, "core.autocrlf")) {
463
if (value && !strcasecmp(value, "input")) {
464
- auto_crlf = -1;
464
+ auto_crlf = AUTO_CRLF_INPUT;
465
return 0;
466
}
467
auto_crlf = git_config_bool(var, value);
convert.c
+84
-33
@@ -8,13 +8,23 @@
8
* This should use the pathname to decide on whether it wants to do some
9
* more interesting conversions (automatic gzip/unzip, general format
10
* conversions etc etc), but by default it just does automatic CRLF<->LF
11
- * translation when the "auto_crlf" option is set.
11
+ * translation when the "crlf" attribute or "auto_crlf" option is set.
12
*/
13
14
-#define CRLF_GUESS (-1)
15
-#define CRLF_BINARY 0
16
-#define CRLF_TEXT 1
17
-#define CRLF_INPUT 2
14
+enum action {
15
+ CRLF_GUESS = -1,
16
+ CRLF_BINARY = 0,
17
+ CRLF_TEXT,
18
+ CRLF_INPUT,
19
+ CRLF_CRLF,
20
+ CRLF_AUTO,
21
+};
22
+
23
+enum eol {
24
+ EOL_UNSET,
25
+ EOL_LF,
26
+ EOL_CRLF,
27
+};
28
29
struct text_stat {
30
/* NUL, CR, LF and CRLF counts */
@@ -89,13 +99,14 @@ static int is_binary(unsigned long size, struct text_stat *stats)
99
return 0;
100
}
101
92
-static void check_safe_crlf(const char *path, int action,
102
+static void check_safe_crlf(const char *path, enum action action,
103
struct text_stat *stats, enum safe_crlf checksafe)
104
{
105
if (!checksafe)
106
return;
107
98
- if (action == CRLF_INPUT || auto_crlf <= 0) {
108
+ if (action == CRLF_INPUT ||
109
+ (action == CRLF_GUESS && auto_crlf == AUTO_CRLF_INPUT)) {
110
/*
111
* CRLFs would not be restored by checkout:
112
* check if we'd remove CRLFs
@@ -106,7 +117,8 @@ static void check_safe_crlf(const char *path, int action,
117
else /* i.e. SAFE_CRLF_FAIL */
118
die("CRLF would be replaced by LF in %s.", path);
119
}
109
- } else if (auto_crlf > 0) {
120
+ } else if (action == CRLF_CRLF ||
121
+ (action == CRLF_GUESS && auto_crlf == AUTO_CRLF_TRUE)) {
122
/*
123
* CRLFs would be added by checkout:
124
* check if we have "naked" LFs
@@ -158,17 +170,18 @@ static int has_cr_in_index(const char *path)
170
}
171
172
static int crlf_to_git(const char *path, const char *src, size_t len,
161
- struct strbuf *buf, int action, enum safe_crlf checksafe)
173
+ struct strbuf *buf, enum action action, enum safe_crlf checksafe)
174
{
175
struct text_stat stats;
176
char *dst;
177
166
- if ((action == CRLF_BINARY) || !auto_crlf || !len)
178
+ if (action == CRLF_BINARY ||
179
+ (action == CRLF_GUESS && auto_crlf == AUTO_CRLF_FALSE) || !len)
180
return 0;
181
182
gather_stats(src, len, &stats);
183
171
- if (action == CRLF_GUESS) {
184
+ if (action == CRLF_AUTO || action == CRLF_GUESS) {
185
/*
186
* We're currently not going to even try to convert stuff
187
* that has bare CR characters. Does anybody do that crazy
@@ -183,12 +196,14 @@ static int crlf_to_git(const char *path, const char *src, size_t len,
196
if (is_binary(len, &stats))
197
return 0;
198
186
- /*
187
- * If the file in the index has any CR in it, do not convert.
188
- * This is the new safer autocrlf handling.
189
- */
190
- if (has_cr_in_index(path))
191
- return 0;
199
+ if (action == CRLF_GUESS) {
200
+ /*
201
+ * If the file in the index has any CR in it, do not convert.
202
+ * This is the new safer autocrlf handling.
203
+ */
204
+ if (has_cr_in_index(path))
205
+ return 0;
206
+ }
207
}
208
209
check_safe_crlf(path, action, &stats, checksafe);
@@ -201,7 +216,7 @@ static int crlf_to_git(const char *path, const char *src, size_t len,
216
if (strbuf_avail(buf) + buf->len < len)
217
strbuf_grow(buf, len - buf->len);
218
dst = buf->buf;
204
- if (action == CRLF_GUESS) {
219
+ if (action == CRLF_AUTO || action == CRLF_GUESS) {
220
/*
221
* If we guessed, we already know we rejected a file with
222
* lone CR, and we can strip a CR without looking at what
@@ -224,13 +239,13 @@ static int crlf_to_git(const char *path, const char *src, size_t len,
239
}
240
241
static int crlf_to_worktree(const char *path, const char *src, size_t len,
227
- struct strbuf *buf, int action)
242
+ struct strbuf *buf, enum action action)
243
{
244
char *to_free = NULL;
245
struct text_stat stats;
246
247
if ((action == CRLF_BINARY) || (action == CRLF_INPUT) ||
233
- auto_crlf <= 0)
248
+ (action != CRLF_CRLF && auto_crlf != AUTO_CRLF_TRUE))
249
return 0;
250
251
if (!len)
@@ -246,11 +261,13 @@ static int crlf_to_worktree(const char *path, const char *src, size_t len,
261
if (stats.lf == stats.crlf)
262
return 0;
263
249
- if (action == CRLF_GUESS) {
250
- /* If we have any CR or CRLF line endings, we do not touch it */
251
- /* This is the new safer autocrlf-handling */
252
- if (stats.cr > 0 || stats.crlf > 0)
253
- return 0;
264
+ if (action == CRLF_AUTO || action == CRLF_GUESS) {
265
+ if (action == CRLF_GUESS) {
266
+ /* If we have any CR or CRLF line endings, we do not touch it */
267
+ /* This is the new safer autocrlf-handling */
268
+ if (stats.cr > 0 || stats.crlf > 0)
269
+ return 0;
270
+ }
271
272
/* If we have any bare CR characters, we're not going to touch it */
273
if (stats.cr != stats.crlf)
@@ -423,11 +440,13 @@ static int read_convert_config(const char *var, const char *value, void *cb)
440
static void setup_convert_check(struct git_attr_check *check)
441
{
442
static struct git_attr *attr_crlf;
443
+ static struct git_attr *attr_eol;
444
static struct git_attr *attr_ident;
445
static struct git_attr *attr_filter;
446
447
if (!attr_crlf) {
448
attr_crlf = git_attr("crlf");
449
+ attr_eol = git_attr("eol");
450
attr_ident = git_attr("ident");
451
attr_filter = git_attr("filter");
452
user_convert_tail = &user_convert;
@@ -436,6 +455,7 @@ static void setup_convert_check(struct git_attr_check *check)
455
check[0].attr = attr_crlf;
456
check[1].attr = attr_ident;
457
check[2].attr = attr_filter;
458
+ check[3].attr = attr_eol;
459
}
460
461
static int count_ident(const char *cp, unsigned long size)
@@ -592,9 +612,24 @@ static int git_path_check_crlf(const char *path, struct git_attr_check *check)
612
;
613
else if (!strcmp(value, "input"))
614
return CRLF_INPUT;
615
+ else if (!strcmp(value, "auto"))
616
+ return CRLF_AUTO;
617
return CRLF_GUESS;
618
}
619
620
+static int git_path_check_eol(const char *path, struct git_attr_check *check)
621
+{
622
+ const char *value = check->value;
623
+
624
+ if (ATTR_UNSET(value))
625
+ ;
626
+ else if (!strcmp(value, "lf"))
627
+ return EOL_LF;
628
+ else if (!strcmp(value, "crlf"))
629
+ return EOL_CRLF;
630
+ return EOL_UNSET;
631
+}
632
+
633
static struct convert_driver *git_path_check_convert(const char *path,
634
struct git_attr_check *check)
635
{
@@ -616,20 +651,32 @@ static int git_path_check_ident(const char *path, struct git_attr_check *check)
651
return !!ATTR_TRUE(value);
652
}
653
654
+enum action determine_action(enum action crlf_attr, enum eol eol_attr) {
655
+ if (crlf_attr == CRLF_BINARY)
656
+ return CRLF_BINARY;
657
+ if (eol_attr == EOL_LF)
658
+ return CRLF_INPUT;
659
+ if (eol_attr == EOL_CRLF)
660
+ return CRLF_CRLF;
661
+ return crlf_attr;
662
+}
663
+
664
int convert_to_git(const char *path, const char *src, size_t len,
665
struct strbuf *dst, enum safe_crlf checksafe)
666
{
622
- struct git_attr_check check[3];
623
- int crlf = CRLF_GUESS;
667
+ struct git_attr_check check[4];
668
+ enum action action = CRLF_GUESS;
669
+ enum eol eol = EOL_UNSET;
670
int ident = 0, ret = 0;
671
const char *filter = NULL;
672
673
setup_convert_check(check);
674
if (!git_checkattr(path, ARRAY_SIZE(check), check)) {
675
struct convert_driver *drv;
630
- crlf = git_path_check_crlf(path, check + 0);
676
+ action = git_path_check_crlf(path, check + 0);
677
ident = git_path_check_ident(path, check + 1);
678
drv = git_path_check_convert(path, check + 2);
679
+ eol = git_path_check_eol(path, check + 3);
680
if (drv && drv->clean)
681
filter = drv->clean;
682
}
@@ -639,7 +686,8 @@ int convert_to_git(const char *path, const char *src, size_t len,
686
src = dst->buf;
687
len = dst->len;
688
}
642
- ret |= crlf_to_git(path, src, len, dst, crlf, checksafe);
689
+ action = determine_action(action, eol);
690
+ ret |= crlf_to_git(path, src, len, dst, action, checksafe);
691
if (ret) {
692
src = dst->buf;
693
len = dst->len;
@@ -649,17 +697,19 @@ int convert_to_git(const char *path, const char *src, size_t len,
697
698
int convert_to_working_tree(const char *path, const char *src, size_t len, struct strbuf *dst)
699
{
652
- struct git_attr_check check[3];
653
- int crlf = CRLF_GUESS;
700
+ struct git_attr_check check[4];
701
+ enum action action = CRLF_GUESS;
702
+ enum eol eol = EOL_UNSET;
703
int ident = 0, ret = 0;
704
const char *filter = NULL;
705
706
setup_convert_check(check);
707
if (!git_checkattr(path, ARRAY_SIZE(check), check)) {
708
struct convert_driver *drv;
660
- crlf = git_path_check_crlf(path, check + 0);
709
+ action = git_path_check_crlf(path, check + 0);
710
ident = git_path_check_ident(path, check + 1);
711
drv = git_path_check_convert(path, check + 2);
712
+ eol = git_path_check_eol(path, check + 3);
713
if (drv && drv->smudge)
714
filter = drv->smudge;
715
}
@@ -669,7 +719,8 @@ int convert_to_working_tree(const char *path, const char *src, size_t len, struc
719
src = dst->buf;
720
len = dst->len;
721
}
672
- ret |= crlf_to_worktree(path, src, len, dst, crlf);
722
+ action = determine_action(action, eol);
723
+ ret |= crlf_to_worktree(path, src, len, dst, action);
724
if (ret) {
725
src = dst->buf;
726
len = dst->len;
environment.c
+1
-1
@@ -38,7 +38,7 @@ const char *pager_program;
38
int pager_use_color = 1;
39
const char *editor_program;
40
const char *excludes_file;
41
-int auto_crlf = 0; /* 1: both ways, -1: only when adding git objects */
41
+enum auto_crlf auto_crlf = AUTO_CRLF_FALSE;
42
int read_replace_refs = 1;
43
enum safe_crlf safe_crlf = SAFE_CRLF_WARN;
44
unsigned whitespace_rule_cfg = WS_DEFAULT_RULE;
t/t0025-crlf-auto.sh
+6
-6
@@ -39,7 +39,7 @@ test_expect_success 'default settings cause no changes' '
39
test -z "$onediff" -a -z "$twodiff" -a -z "$threediff"
40
'
41
42
-test_expect_failure 'crlf=true causes a CRLF file to be normalized' '
42
+test_expect_success 'crlf=true causes a CRLF file to be normalized' '
43
44
rm -f .gitattributes tmp one two three &&
45
echo "two crlf" > .gitattributes &&
@@ -51,7 +51,7 @@ test_expect_failure 'crlf=true causes a CRLF file to be normalized' '
51
test -n "$twodiff"
52
'
53
54
-test_expect_failure 'eol=crlf gives a normalized file CRLFs with autocrlf=false' '
54
+test_expect_success 'eol=crlf gives a normalized file CRLFs with autocrlf=false' '
55
56
rm -f .gitattributes tmp one two three &&
57
git config core.autocrlf false &&
@@ -63,7 +63,7 @@ test_expect_failure 'eol=crlf gives a normalized file CRLFs with autocrlf=false'
63
test -z "$onediff"
64
'
65
66
-test_expect_failure 'eol=crlf gives a normalized file CRLFs with autocrlf=input' '
66
+test_expect_success 'eol=crlf gives a normalized file CRLFs with autocrlf=input' '
67
68
rm -f .gitattributes tmp one two three &&
69
git config core.autocrlf input &&
@@ -75,7 +75,7 @@ test_expect_failure 'eol=crlf gives a normalized file CRLFs with autocrlf=input'
75
test -z "$onediff"
76
'
77
78
-test_expect_failure 'eol=lf gives a normalized file LFs with autocrlf=true' '
78
+test_expect_success 'eol=lf gives a normalized file LFs with autocrlf=true' '
79
80
rm -f .gitattributes tmp one two three &&
81
git config core.autocrlf true &&
@@ -101,7 +101,7 @@ test_expect_success 'autocrlf=true does not normalize CRLF files' '
101
test -z "$onediff" -a -z "$twodiff" -a -z "$threediff"
102
'
103
104
-test_expect_failure 'crlf=auto, autocrlf=true _does_ normalize CRLF files' '
104
+test_expect_success 'crlf=auto, autocrlf=true _does_ normalize CRLF files' '
105
106
rm -f .gitattributes tmp one two three &&
107
git config core.autocrlf true &&
@@ -128,7 +128,7 @@ test_expect_success 'crlf=auto, autocrlf=true does not normalize binary files' '
128
test -z "$threediff"
129
'
130
131
-test_expect_failure 'eol=crlf _does_ normalize binary files' '
131
+test_expect_success 'eol=crlf _does_ normalize binary files' '
132
133
rm -f .gitattributes tmp one two three &&
134
echo "three eol=crlf" > .gitattributes &&