@matheusbsbs / matheus-bagliano / commits / 596b5e77c9

clean/smudge: allow clean filters to process extremely large files

The filter system allows for alterations to file contents when they're moved between the database and the worktree. We already made sure that it is possible for smudge filters to produce contents that are larger than `unsigned long` can represent (which matters on systems where `unsigned long` is narrower than `size_t`, most notably 64-bit Windows). Now we make sure that clean filters can _consume_ contents that are larger than that. Note that this commit only allows clean filters' _input_ to be larger than can be represented by `unsigned long`. This change makes only a very minute dent into the much larger project to teach Git to use `size_t` instead of `unsigned long` wherever appropriate. Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Matt Cooper <vtbassmatt@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Matt Cooper committed Nov 2, 2021 at 15:46 UTC 596b5e77c960cc57ad2e68407b298411ec5e8cb8

2 files changed +12 -1

convert.c

+1 -1

@@ -613,7 +613,7 @@ static int crlf_to_worktree(const char *src, size_t len, struct strbuf *buf,

613

614 struct filter_params {

615 const char *src;

616 - unsigned long size;

616 + size_t size;

617 int fd;

618 const char *cmd;

619 const char *path;

t/t1051-large-conversion.sh

+11

@@ -98,4 +98,15 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT \

98 test "$size" -eq $((5 * 1024 * 1024 * 1024 + $small_size))

99 '

100

101 +# This clean filter writes down the size of input it receives. By checking against

102 +# the actual size, we ensure that cleaning doesn't mangle large files on 64-bit Windows.

103 +test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT \

104 + 'files over 4GB convert on input' '

105 + test-tool genzeros $((5*1024*1024*1024)) >big &&

106 + test_config filter.checklarge.clean "wc -c >big.size" &&

107 + echo "big filter=checklarge" >.gitattributes &&

108 + git add big &&

109 + test $(test_file_size big) -eq $(cat big.size)

110 +'

111 +

112 test_done