@matheusbsbs / matheus-bagliano / commits / e9aa762cc7

odb: teach read_blob_entry to use size_t

There is mixed use of size_t and unsigned long to deal with sizes in the codebase. Recall that Windows defines unsigned long as 32 bits even on 64-bit platforms, meaning that converting size_t to unsigned long narrows the range. This mostly doesn't cause a problem since Git rarely deals with files larger than 2^32 bytes. But adjunct systems such as Git LFS, which use smudge/clean filters to keep huge files out of the repository, may have huge file contents passed through some of the functions in entry.c and convert.c. On Windows, this results in a truncated file being written to the workdir. I traced this to one specific use of unsigned long in write_entry (and a similar instance in write_pc_item_to_fd for parallel checkout). That appeared to be for the call to read_blob_entry, which expects a pointer to unsigned long. By altering the signature of read_blob_entry to expect a size_t, write_entry can be switched to use size_t internally (which all of its callers and most of its callees already used). To avoid touching dozens of additional files, read_blob_entry uses a local unsigned long to call a chain of functions which aren't prepared to accept size_t. Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Matt Cooper <vtbassmatt@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Matt Cooper committed Nov 2, 2021 at 15:46 UTC e9aa762cc72e6cf8fd76fefe5ca2b5064be1a821

4 files changed +8 -6

entry.c

+5 -3

@@ -82,11 +82,13 @@ static int create_file(const char *path, unsigned int mode)

82 return open(path, O_WRONLY | O_CREAT | O_EXCL, mode);

83 }

85 -void *read_blob_entry(const struct cache_entry *ce, unsigned long *size)

85 +void *read_blob_entry(const struct cache_entry *ce, size_t *size)

86 {

87 enum object_type type;

88 - void *blob_data = read_object_file(&ce->oid, &type, size);

88 + unsigned long ul;

89 + void *blob_data = read_object_file(&ce->oid, &type, &ul);

91 + *size = ul;

92 if (blob_data) {

93 if (type == OBJ_BLOB)

94 return blob_data;

@@ -270,7 +272,7 @@ static int write_entry(struct cache_entry *ce, char *path, struct conv_attrs *ca

272 int fd, ret, fstat_done = 0;

273 char *new_blob;

274 struct strbuf buf = STRBUF_INIT;

273 - unsigned long size;

275 + size_t size;

276 ssize_t wrote;

277 size_t newsize = 0;

278 struct stat st;

entry.h

+1 -1

@@ -51,7 +51,7 @@ int finish_delayed_checkout(struct checkout *state, int *nr_checkouts);

51 */

52 void unlink_entry(const struct cache_entry *ce);

54 -void *read_blob_entry(const struct cache_entry *ce, unsigned long *size);

54 +void *read_blob_entry(const struct cache_entry *ce, size_t *size);

55 int fstat_checkout_output(int fd, const struct checkout *state, struct stat *st);

56 void update_ce_after_write(const struct checkout *state, struct cache_entry *ce,

57 struct stat *st);

parallel-checkout.c

+1 -1

@@ -261,7 +261,7 @@ static int write_pc_item_to_fd(struct parallel_checkout_item *pc_item, int fd,

261 struct stream_filter *filter;

262 struct strbuf buf = STRBUF_INIT;

263 char *blob;

264 - unsigned long size;

264 + size_t size;

265 ssize_t wrote;

266

267 /* Sanity check */

t/t1051-large-conversion.sh

+1 -1

@@ -85,7 +85,7 @@ test_expect_success 'ident converts on output' '

86 # This smudge filter prepends 5GB of zeros to the file it checks out. This

87 # ensures that smudging doesn't mangle large files on 64-bit Windows.

88 -test_expect_failure EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT \

88 +test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT \

89 'files over 4GB convert on output' '

90 test_commit test small "a small file" &&

91 small_size=$(test_file_size small) &&