grep/pcre2: use PCRE2_UTF even with ASCII patterns
compile_pcre2_pattern() currently uses the option PCRE2_UTF only for patterns with non-ASCII characters. Patterns with ASCII wildcards can match non-ASCII strings, though. Without that option PCRE2 mishandles UTF-8 input, though -- it matches parts of multi-byte characters. Fix that by using PCRE2_UTF even for ASCII-only patterns. This is a remake of the reverted ae39ba431a (grep/pcre2: fix an edge case concerning ascii patterns and UTF-8 data, 2021-10-15). The change to the condition and the test are simplified and more targeted. Original-patch-by: Hamza Mahfooz <someguy@effective-light.com> Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
René Scharfe committed
Dec 18, 2021 at 20:50 UTC
dc2c44fbb100fa609174d9069a70e2b54b0591ca
2 files changed
+7
-1
grep.c
+1
-1
@@ -382,7 +382,7 @@ static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt
382
}
383
options |= PCRE2_CASELESS;
384
}
385
- if (!opt->ignore_locale && is_utf8_locale() && has_non_ascii(p->pattern) &&
385
+ if (!opt->ignore_locale && is_utf8_locale() &&
386
!(!opt->ignore_case && (p->fixed || p->is_fixed)))
387
options |= (PCRE2_UTF | PCRE2_MATCH_INVALID_UTF);
388
t/t7812-grep-icase-non-ascii.sh
+6
@@ -123,4 +123,10 @@ test_expect_success GETTEXT_LOCALE,LIBPCRE2,PCRE2_MATCH_INVALID_UTF 'PCRE v2: gr
123
test_cmp invalid-0xe5 actual
124
'
125
126
+test_expect_success GETTEXT_LOCALE,LIBPCRE2 'PCRE v2: grep non-literal ASCII from UTF-8' '
127
+ git grep --perl-regexp -h -o -e ll. file >actual &&
128
+ echo "lló" >expected &&
129
+ test_cmp expected actual
130
+'
131
+
132
test_done