@matheusbsbs / matheus-bagliano / commits / 0c8424c259

t: work around multibyte bug in quoted heredocs with Dash v0.5.13

When executing our test suite with Dash v0.5.13.2 one can observe several test failures that all have the same symptoms: we have a quoted heredoc that contains multibyte characters, but the final data does not match what we actually wanted to write. One such example is in t0300, where we see the diffs like the following: --- expect-stdout 2026-04-01 07:25:45.249919440 +0000 +++ stdout 2026-04-01 07:25:45.254919509 +0000 @@ -1,5 +1,5 @@ protocol=https host=example.com -path=perú.git +path=perú.git username=foo password=bar While seemingly the same, the data that we've written via the heredoc contains some invisible bytes. The expected hex representation of the string is: 7065 72c3 ba2e 6769 74 per...git But what we actually get instead is this string: 7065 7285 02c3 ba02 852e 6769 74 per.......git What's important to note here is that the multibyte character exists in both versions. But in the broken version we see that the bytes are wrapped in a sequence of "85 02" and "02 85". This is the CTLMBCHAR byte sequence of Dash, which it uses internally to quote multibyte sequences. As it turns out, this bug was introduced in c5bf970 (expand: Add multi-byte support to pmatch, 2024-06-02), which adds multibyte support to more contexts of Dash. One of these contexts seems to be in heredocs, and Dash _does_ correctly unquote these multibyte sequences when using an unquoted heredoc. But the bug seems to be that this unquoting does not happen in quoted heredocs, and the bug still exists on the latest "master" branch. For now, work around the bug by using unquoted heredocs instead. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Patrick Steinhardt committed Apr 2, 2026 at 08:51 UTC 0c8424c259b417c6aadc23f5398e55edd7b047a2
6 files changed +40 -18
t/t0300-credentials.sh
+3 -1
@@ -675,7 +675,9 @@ test_expect_success 'match percent-encoded values' '
675 test_expect_success 'match percent-encoded UTF-8 values in path' '
676 test_config credential.https://example.com.useHttpPath true &&
677 test_config credential.https://example.com/perú.git.helper "$HELPER" &&
678 - check fill <<-\EOF
678 + # NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
679 + # that contain multibyte chars.
680 + check fill <<-EOF
681 url=https://example.com/per%C3%BA.git
682 --
683 protocol=https
t/t3430-rebase-merges.sh
+4 -2
@@ -507,9 +507,11 @@ test_expect_success 'octopus merges' '
507 git rebase -i --force-rebase -r HEAD^^ &&
508 test "Hank" = "$(git show -s --format=%an HEAD)" &&
509 test "$before" != $(git rev-parse HEAD) &&
510 - test_cmp_graph HEAD^^.. <<-\EOF
510 + # NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
511 + # that contain multibyte chars.
512 + test_cmp_graph HEAD^^.. <<-EOF
513 *-. Tüntenfüsch
512 - |\ \
514 + |\\ \\
515 | | * three
516 | * | two
517 | |/
t/t3902-quoted.sh
+9 -7
@@ -60,16 +60,18 @@ With SP in it
60 "\346\277\261\351\207\216\347\264\224"
61 EOF
62
63 -cat >expect.raw <<\EOF
63 +# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
64 +# that contain multibyte chars.
65 +cat >expect.raw <<EOF
66 Name
65 -"Name and a\nLF"
66 -"Name and an\tHT"
67 -"Name\""
67 +"Name and a\\nLF"
68 +"Name and an\\tHT"
69 +"Name\\""
70 With SP in it
69 -"濱野\t純"
70 -"濱野\n純"
71 +"濱野\\t純"
72 +"濱野\\n純"
73 濱野 純
72 -"濱野\"純"
74 +"濱野\\"純"
75 濱野/file
76 濱野純
77 EOF
t/t4014-format-patch.sh
+12 -4
@@ -1285,7 +1285,9 @@ test_expect_success 'format-patch wraps extremely long from-header (rfc2047)' '
1285 check_author "Foö Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar"
1286 '
1287
1288 -cat >expect <<'EOF'
1288 +# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1289 +# that contain multibyte chars.
1290 +cat >expect <<EOF
1291 From: Foö Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar
1292 Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo
1293 Bar Foo Bar Foo Bar Foo Bar <author@example.com>
@@ -1300,7 +1302,9 @@ test_expect_success 'format-patch wraps extremely long from-header (non-ASCII wi
1302 test_cmp expect actual
1303 '
1304
1303 -cat >expect <<'EOF'
1305 +# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1306 +# that contain multibyte chars.
1307 +cat >expect <<EOF
1308 Subject: [PATCH] Foö
1309 EOF
1310 test_expect_success 'subject lines are unencoded with --no-encode-email-headers' '
@@ -1312,7 +1316,9 @@ test_expect_success 'subject lines are unencoded with --no-encode-email-headers'
1316 test_cmp expect actual
1317 '
1318
1315 -cat >expect <<'EOF'
1319 +# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1320 +# that contain multibyte chars.
1321 +cat >expect <<EOF
1322 Subject: [PATCH] Foö
1323 EOF
1324 test_expect_success 'subject lines are unencoded with format.encodeEmailHeaders=false' '
@@ -1523,7 +1529,9 @@ test_expect_success 'in-body headers trigger content encoding' '
1529 test_env GIT_AUTHOR_NAME="éxötìc" test_commit exotic &&
1530 test_when_finished "git reset --hard HEAD^" &&
1531 git format-patch -1 --stdout --from >patch &&
1526 - cat >expect <<-\EOF &&
1532 + # NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1533 + # that contain multibyte chars.
1534 + cat >expect <<-EOF &&
1535 From: C O Mitter <committer@example.com>
1536 Content-Type: text/plain; charset=UTF-8
1537
t/t4201-shortlog.sh
+3 -1
@@ -105,7 +105,9 @@ test_expect_success 'output from user-defined format is re-wrapped' '
105 '
106
107 test_expect_success !MINGW,ICONV 'shortlog wrapping' '
108 - cat >expect <<\EOF &&
108 + # NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
109 + # that contain multibyte chars.
110 + cat >expect <<EOF &&
111 A U Thor (5):
112 Test
113 This is a very, very long first line for the commit message to see if
t/t9001-send-email.sh
+9 -3
@@ -1649,7 +1649,9 @@ test_expect_success $PREREQ 'To headers from files reset each patch' '
1649 '
1650
1651 test_expect_success $PREREQ 'setup expect' '
1652 -cat >email-using-8bit <<\EOF
1652 +# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1653 +# that contain multibyte chars.
1654 +cat >email-using-8bit <<EOF
1655 From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
1656 Message-ID: <bogus-message-id@example.com>
1657 From: author@example.com
@@ -1735,7 +1737,9 @@ test_expect_success $PREREQ '--8bit-encoding overrides sendemail.8bitEncoding' '
1737 '
1738
1739 test_expect_success $PREREQ 'setup expect' '
1738 - cat >email-using-8bit <<-\EOF
1740 + # NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1741 + # that contain multibyte chars.
1742 + cat >email-using-8bit <<-EOF
1743 From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
1744 Message-ID: <bogus-message-id@example.com>
1745 From: author@example.com
@@ -1764,7 +1768,9 @@ test_expect_success $PREREQ '--8bit-encoding also treats subject' '
1768 '
1769
1770 test_expect_success $PREREQ 'setup expect' '
1767 - cat >email-using-8bit <<-\EOF
1771 + # NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
1772 + # that contain multibyte chars.
1773 + cat >email-using-8bit <<-EOF
1774 From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
1775 Message-ID: <bogus-message-id@example.com>
1776 From: A U Thor <author@example.com>