@matheusbsbs / matheus-bagliano / commits / df1b059d8d
Fork 0

Document pack .idx file format upgrade strategy.

Way back when Junio developed the 64 bit index topic he came up with a means of changing the .idx file format so that older Git clients would recognize that they don't understand the file and refuse to read it, while newer clients could tell the difference between the old-style and new-style .idx files. Unfortunately this wasn't recorded anywhere. This change documents how we might go about changing the .idx file format by using a special signature in the first four bytes. Credit (and possible blame) goes completely to Junio for thinking up this technique. The change also modifies the error message of the current Git code so that users get a recommendation to upgrade their Git software should this version or later encounter a new-style .idx which it cannot process. We already do this for the .pack files, but since we usually process the .idx files first its important that these files are recognized and encourage an upgrade. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <junkio@cox.net>

Shawn O. Pearce committed Jan 17, 2007 at 20:43 UTC df1b059d8dd5c16edaa98d7da842b2a32c01e526
2 files changed +42 -3
pack.h
+28
@@ -15,5 +15,33 @@ struct pack_header {
15 unsigned int hdr_entries;
16 };
17
18 +/*
19 + * Packed object index header
20 + *
21 + * struct pack_idx_header {
22 + * uint32_t idx_signature;
23 + * uint32_t idx_version;
24 + * };
25 + *
26 + * Note: this header isn't active yet. In future versions of git
27 + * we may change the index file format. At that time we would start
28 + * the first four bytes of the new index format with this signature,
29 + * as all older git binaries would find this value illegal and abort
30 + * reading the file.
31 + *
32 + * This is the case because the number of objects in a packfile
33 + * cannot exceed 1,431,660,000 as every object would need at least
34 + * 3 bytes of data and the overall packfile cannot exceed 4 GiB due
35 + * to the 32 bit offsets used by the index. Clearly the signature
36 + * exceeds this maximum.
37 + *
38 + * Very old git binaries will also compare the first 4 bytes to the
39 + * next 4 bytes in the index and abort with a "non-monotonic index"
40 + * error if the second 4 byte word is smaller than the first 4
41 + * byte word. This would be true in the proposed future index
42 + * format as idx_signature would be greater than idx_version.
43 + */
44 +#define PACK_IDX_SIGNATURE 0xff744f63 /* "\377tOc" */
45 +
46 extern int verify_pack(struct packed_git *, int);
47 #endif
sha1_file.c
+14 -3
@@ -456,12 +456,23 @@ static int check_packed_git_idx(const char *path, unsigned long *idx_size_,
456
457 /* check index map */
458 if (idx_size < 4*256 + 20 + 20)
459 - return error("index file too small");
459 + return error("index file %s is too small", path);
460 +
461 + /* a future index format would start with this, as older git
462 + * binaries would fail the non-monotonic index check below.
463 + * give a nicer warning to the user if we can.
464 + */
465 + if (index[0] == htonl(PACK_IDX_SIGNATURE))
466 + return error("index file %s is a newer version"
467 + " and is not supported by this binary"
468 + " (try upgrading GIT to a newer version)",
469 + path);
470 +
471 nr = 0;
472 for (i = 0; i < 256; i++) {
473 unsigned int n = ntohl(index[i]);
474 if (n < nr)
464 - return error("non-monotonic index");
475 + return error("non-monotonic index %s", path);
476 nr = n;
477 }
478
@@ -473,7 +484,7 @@ static int check_packed_git_idx(const char *path, unsigned long *idx_size_,
484 * - 20-byte SHA1 file checksum
485 */
486 if (idx_size != 4*256 + nr * 24 + 20 + 20)
476 - return error("wrong index file size");
487 + return error("wrong index file size in %s", path);
488
489 return 0;
490 }