@matheusbsbs / matheus-bagliano / commits / 96e41f58fe

fsck: report invalid object type-path combinations

Improve the error that's emitted in cases where we find a loose object we parse, but which isn't at the location we expect it to be. Before this change we'd prefix the error with a not-a-OID derived from the path at which the object was found, due to an emergent behavior in how we'd end up with an "OID" in these codepaths. Now we'll instead say what object we hashed, and what path it was found at. Before this patch series e.g.: $ git hash-object --stdin -w -t blob </dev/null e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 $ mv objects/e6/ objects/e7 Would emit ("[...]" used to abbreviate the OIDs): git fsck error: hash mismatch for ./objects/e7/9d[...] (expected e79d[...]) error: e79d[...]: object corrupt or missing: ./objects/e7/9d[...] Now we'll instead emit: error: e69d[...]: hash-path mismatch, found at: ./objects/e7/9d[...] Furthermore, we'll do the right thing when the object type and its location are bad. I.e. this case: $ git hash-object --stdin -w -t garbage --literally </dev/null 8315a83d2acc4c174aed59430f9a9c4ed926440f $ mv objects/83 objects/84 As noted in an earlier commits we'd simply die early in those cases, until preceding commits fixed the hard die on invalid object type: $ git fsck fatal: invalid object type Now we'll instead emit sensible error messages: $ git fsck error: 8315[...]: hash-path mismatch, found at: ./objects/84/15[...] error: 8315[...]: object is of unknown type 'garbage': ./objects/84/15[...] In both fsck.c and object-file.c we're using null_oid as a sentinel value for checking whether we got far enough to be certain that the issue was indeed this OID mismatch. We need to add the "object corrupt or missing" special-case to deal with cases where read_loose_object() will return an error before completing check_object_signature(), e.g. if we have an error in unpack_loose_rest() because we find garbage after the valid gzip content: $ git hash-object --stdin -w -t blob </dev/null e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 $ chmod 755 objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391 $ echo garbage >>objects/e6/9de29bb2d1d6434b8b29ae775ad8c2e48c5391 $ git fsck error: garbage at end of loose object 'e69d[...]' error: unable to unpack contents of ./objects/e6/9d[...] error: e69d[...]: object corrupt or missing: ./objects/e6/9d[...] There is currently some weird messaging in the edge case when the two are combined, i.e. because we're not explicitly passing along an error state about this specific scenario from check_stream_oid() via read_loose_object() we'll end up printing the null OID if an object is of an unknown type *and* it can't be unpacked by zlib, e.g.: $ git hash-object --stdin -w -t garbage --literally </dev/null 8315a83d2acc4c174aed59430f9a9c4ed926440f $ chmod 755 objects/83/15a83d2acc4c174aed59430f9a9c4ed926440f $ echo garbage >>objects/83/15a83d2acc4c174aed59430f9a9c4ed926440f $ /usr/bin/git fsck fatal: invalid object type $ ~/g/git/git fsck error: garbage at end of loose object '8315a83d2acc4c174aed59430f9a9c4ed926440f' error: unable to unpack contents of ./objects/83/15a83d2acc4c174aed59430f9a9c4ed926440f error: 8315a83d2acc4c174aed59430f9a9c4ed926440f: object corrupt or missing: ./objects/83/15a83d2acc4c174aed59430f9a9c4ed926440f error: 0000000000000000000000000000000000000000: object is of unknown type 'garbage': ./objects/83/15a83d2acc4c174aed59430f9a9c4ed926440f [...] I think it's OK to leave that for future improvements, which would involve enum-ifying more error state as we've done with "enum unpack_loose_header_result" in preceding commits. In these increasingly more obscure cases the worst that can happen is that we'll get slightly nonsensical or inapplicable error messages. There's other such potential edge cases, all of which might produce some confusing messaging, but still be handled correctly as far as passing along errors goes. E.g. if check_object_signature() returns and oideq(real_oid, null_oid()) is true, which could happen if it returns -1 due to the read_istream() call having failed. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Ævar Arnfjörð Bjarmason committed Oct 1, 2021 at 11:16 UTC 96e41f58fe1a5aeadf2bf1c1850c53a1c1144bbc

11 files changed +38 -26

builtin/fast-export.c

+1 -1

@@ -312,7 +312,7 @@ static void export_blob(const struct object_id *oid)

312 if (!buf)

313 die("could not read blob %s", oid_to_hex(oid));

314 if (check_object_signature(the_repository, oid, buf, size,

315 - type_name(type)) < 0)

315 + type_name(type), NULL) < 0)

316 die("oid mismatch in blob %s", oid_to_hex(oid));

317 object = parse_object_buffer(the_repository, oid, type,

318 size, buf, &eaten);

builtin/fsck.c

+11 -4

@@ -607,6 +607,7 @@ static int fsck_loose(const struct object_id *oid, const char *path, void *data)

607 void *contents;

608 int eaten;

609 struct object_info oi = OBJECT_INFO_INIT;

610 + struct object_id real_oid = *null_oid();

611 int err = 0;

612

613 strbuf_reset(&cb_data->obj_type);

@@ -614,12 +615,18 @@ static int fsck_loose(const struct object_id *oid, const char *path, void *data)

615 oi.sizep = &size;

616 oi.typep = &type;

617

617 - if (read_loose_object(path, oid, &contents, &oi) < 0)

618 - err = error(_("%s: object corrupt or missing: %s"),

619 - oid_to_hex(oid), path);

618 + if (read_loose_object(path, oid, &real_oid, &contents, &oi) < 0) {

619 + if (contents && !oideq(&real_oid, oid))

620 + err = error(_("%s: hash-path mismatch, found at: %s"),

621 + oid_to_hex(&real_oid), path);

622 + else

623 + err = error(_("%s: object corrupt or missing: %s"),

624 + oid_to_hex(oid), path);

625 + }

626 if (type != OBJ_NONE && type < 0)

627 err = error(_("%s: object is of unknown type '%s': %s"),

622 - oid_to_hex(oid), cb_data->obj_type.buf, path);

628 + oid_to_hex(&real_oid), cb_data->obj_type.buf,

629 + path);

630 if (err < 0) {

631 errors_found |= ERROR_OBJECT;

632 return 0; /* keep checking other objects */

builtin/index-pack.c

+1 -1

@@ -1421,7 +1421,7 @@ static void fix_unresolved_deltas(struct hashfile *f)

1421

1422 if (check_object_signature(the_repository, &d->oid,

1423 data, size,

1424 - type_name(type)))

1424 + type_name(type), NULL))

1425 die(_("local object %s is corrupt"), oid_to_hex(&d->oid));

1426

1427 /*

builtin/mktag.c

+2 -1

@@ -62,7 +62,8 @@ static int verify_object_in_tag(struct object_id *tagged_oid, int *tagged_type)

63 repl = lookup_replace_object(the_repository, tagged_oid);

64 ret = check_object_signature(the_repository, repl,

65 - buffer, size, type_name(*tagged_type));

65 + buffer, size, type_name(*tagged_type),

66 + NULL);

67 free(buffer);

69 return ret;

cache.h

+2 -1

@@ -1344,7 +1344,8 @@ struct object_info;

1344 int parse_loose_header(const char *hdr, struct object_info *oi);

1345

1346 int check_object_signature(struct repository *r, const struct object_id *oid,

1347 - void *buf, unsigned long size, const char *type);

1347 + void *buf, unsigned long size, const char *type,

1348 + struct object_id *real_oidp);

1349

1350 int finalize_object_file(const char *tmpfile, const char *filename);

1351

object-file.c

+10 -11

@@ -1039,9 +1039,11 @@ void *xmmap(void *start, size_t length,

1039 * the streaming interface and rehash it to do the same.

1040 */

1041 int check_object_signature(struct repository *r, const struct object_id *oid,

1042 - void *map, unsigned long size, const char *type)

1042 + void *map, unsigned long size, const char *type,

1043 + struct object_id *real_oidp)

1044 {

1044 - struct object_id real_oid;

1045 + struct object_id tmp;

1046 + struct object_id *real_oid = real_oidp ? real_oidp : &tmp;

1047 enum object_type obj_type;

1048 struct git_istream *st;

1049 git_hash_ctx c;

@@ -1049,8 +1051,8 @@ int check_object_signature(struct repository *r, const struct object_id *oid,

1051 int hdrlen;

1052

1053 if (map) {

1052 - hash_object_file(r->hash_algo, map, size, type, &real_oid);

1053 - return !oideq(oid, &real_oid) ? -1 : 0;

1054 + hash_object_file(r->hash_algo, map, size, type, real_oid);

1055 + return !oideq(oid, real_oid) ? -1 : 0;

1056 }

1057

1058 st = open_istream(r, oid, &obj_type, &size, NULL);

@@ -1075,9 +1077,9 @@ int check_object_signature(struct repository *r, const struct object_id *oid,

1077 break;

1078 r->hash_algo->update_fn(&c, buf, readlen);

1079 }

1078 - r->hash_algo->final_oid_fn(&real_oid, &c);

1080 + r->hash_algo->final_oid_fn(real_oid, &c);

1081 close_istream(st);

1080 - return !oideq(oid, &real_oid) ? -1 : 0;

1082 + return !oideq(oid, real_oid) ? -1 : 0;

1083 }

1084

1085 int git_open_cloexec(const char *name, int flags)

@@ -2520,6 +2522,7 @@ static int check_stream_oid(git_zstream *stream,

2522

2523 int read_loose_object(const char *path,

2524 const struct object_id *expected_oid,

2525 + struct object_id *real_oid,

2526 void **contents,

2527 struct object_info *oi)

2528 {

@@ -2530,8 +2533,6 @@ int read_loose_object(const char *path,

2533 char hdr[MAX_HEADER_LEN];

2534 unsigned long *size = oi->sizep;

2535

2533 - *contents = NULL;

2534 -

2536 map = map_loose_object_1(the_repository, path, NULL, &mapsize);

2537 if (!map) {

2538 error_errno(_("unable to mmap %s"), path);

@@ -2561,9 +2562,7 @@ int read_loose_object(const char *path,

2562 goto out;

2563 }

2564 if (check_object_signature(the_repository, expected_oid,

2564 - *contents, *size, oi->type_name->buf)) {

2565 - error(_("hash mismatch for %s (expected %s)"), path,

2566 - oid_to_hex(expected_oid));

2565 + *contents, *size, oi->type_name->buf, real_oid)) {

2566 free(*contents);

2567 goto out;

2568 }

object-store.h

@@ -244,6 +244,7 @@ int force_object_loose(const struct object_id *oid, time_t mtime);

244 */

245 int read_loose_object(const char *path,

246 const struct object_id *expected_oid,

247 + struct object_id *real_oid,

248 void **contents,

249 struct object_info *oi);

250

object.c

+2 -2

@@ -261,7 +261,7 @@ struct object *parse_object(struct repository *r, const struct object_id *oid)

261 if ((obj && obj->type == OBJ_BLOB && repo_has_object_file(r, oid)) ||

262 (!obj && repo_has_object_file(r, oid) &&

263 oid_object_info(r, oid, NULL) == OBJ_BLOB)) {

264 - if (check_object_signature(r, repl, NULL, 0, NULL) < 0) {

264 + if (check_object_signature(r, repl, NULL, 0, NULL, NULL) < 0) {

265 error(_("hash mismatch %s"), oid_to_hex(oid));

266 return NULL;

267 }

@@ -272,7 +272,7 @@ struct object *parse_object(struct repository *r, const struct object_id *oid)

272 buffer = repo_read_object_file(r, oid, &type, &size);

273 if (buffer) {

274 if (check_object_signature(r, repl, buffer, size,

275 - type_name(type)) < 0) {

275 + type_name(type), NULL) < 0) {

276 free(buffer);

277 error(_("hash mismatch %s"), oid_to_hex(repl));

278 return NULL;

pack-check.c

+2 -1

@@ -142,7 +142,8 @@ static int verify_packfile(struct repository *r,

142 err = error("cannot unpack %s from %s at offset %"PRIuMAX"",

143 oid_to_hex(&oid), p->pack_name,

144 (uintmax_t)entries[i].offset);

145 - else if (check_object_signature(r, &oid, data, size, type_name(type)))

145 + else if (check_object_signature(r, &oid, data, size,

146 + type_name(type), NULL))

147 err = error("packed %s from %s is corrupt",

148 oid_to_hex(&oid), p->pack_name);

149 else if (fn) {

t/t1006-cat-file.sh

+1 -1

@@ -512,7 +512,7 @@ test_expect_success 'cat-file -t and -s on corrupt loose object' '

512 # Swap the two to corrupt the repository

513 mv -f "$other_path" "$empty_path" &&

514 test_must_fail git fsck 2>err.fsck &&

515 - grep "hash mismatch" err.fsck &&

515 + grep "hash-path mismatch" err.fsck &&

516

517 # confirm that cat-file is reading the new swapped-in

518 # blob...

t/t1450-fsck.sh

+5 -3

@@ -54,6 +54,7 @@ test_expect_success 'object with hash mismatch' '

54 cd hash-mismatch &&

56 oid=$(echo blob | git hash-object -w --stdin) &&

57 + oldoid=$oid &&

58 old=$(test_oid_to_path "$oid") &&

59 new=$(dirname $old)/$(test_oid ff_2) &&

60 oid="$(dirname $new)$(basename $new)" &&

@@ -65,7 +66,7 @@ test_expect_success 'object with hash mismatch' '

66 git update-ref refs/heads/bogus $cmt &&

68 test_must_fail git fsck 2>out &&

68 - grep "$oid.*corrupt" out

69 + grep "$oldoid: hash-path mismatch, found at: .*$new" out

70 )

71 '

@@ -75,6 +76,7 @@ test_expect_success 'object with hash and type mismatch' '

76 cd hash-type-mismatch &&

78 oid=$(echo blob | git hash-object -w --stdin -t garbage --literally) &&

79 + oldoid=$oid &&

80 old=$(test_oid_to_path "$oid") &&

81 new=$(dirname $old)/$(test_oid ff_2) &&

82 oid="$(dirname $new)$(basename $new)" &&

@@ -87,8 +89,8 @@ test_expect_success 'object with hash and type mismatch' '

91 test_must_fail git fsck 2>out &&

90 - grep "^error: hash mismatch for " out &&

91 - grep "^error: $oid: object is of unknown type '"'"'garbage'"'"'" out

92 + grep "^error: $oldoid: hash-path mismatch, found at: .*$new" out &&

93 + grep "^error: $oldoid: object is of unknown type '"'"'garbage'"'"'" out

94 )

95 '