@matheusbsbs / matheus-bagliano / commits / 71362bd552
Fork 0

Documentation: describe pack idx v2

Lifted from the log message of c553ca25bd60dc9fd50b8bc7bd329601b81cee66 (pack-objects: learn about pack index version 2). Acked-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>

linux@horizon.com committed Dec 14, 2007 at 06:28 UTC 71362bd55280bb0f4aa67f89435308a4b26213b5
1 file changed +61 -33
Documentation/technical/pack-format.txt
+61 -33
@@ -1,9 +1,9 @@
1 GIT pack format
2 ===============
3
4 -= pack-*.pack file has the following format:
4 += pack-*.pack files have the following format:
5
6 - - The header appears at the beginning and consists of the following:
6 + - A header appears at the beginning and consists of the following:
7
8 4-byte signature:
9 The signature is: {'P', 'A', 'C', 'K'}
@@ -34,18 +34,14 @@ GIT pack format
34
35 - The trailer records 20-byte SHA1 checksum of all of the above.
36
37 -= pack-*.idx file has the following format:
37 += Original (version 1) pack-*.idx files have the following format:
38
39 - The header consists of 256 4-byte network byte order
40 integers. N-th entry of this table records the number of
41 objects in the corresponding pack, the first byte of whose
42 - object name are smaller than N. This is called the
42 + object name is less than or equal to N. This is called the
43 'first-level fan-out' table.
44
45 - Observation: we would need to extend this to an array of
46 - 8-byte integers to go beyond 4G objects per pack, but it is
47 - not strictly necessary.
48 -
45 - The header is followed by sorted 24-byte entries, one entry
46 per object in the pack. Each entry is:
47
@@ -55,10 +51,6 @@ GIT pack format
51
52 20-byte object name.
53
58 - Observation: we would definitely need to extend this to
59 - 8-byte integer plus 20-byte object name to handle a packfile
60 - that is larger than 4GB.
61 -
54 - The file is concluded with a trailer:
55
56 A copy of the 20-byte SHA1 checksum at the end of
@@ -68,31 +60,30 @@ GIT pack format
60
61 Pack Idx file:
62
71 - idx
72 - +--------------------------------+
73 - | fanout[0] = 2 |-.
74 - +--------------------------------+ |
63 + -- +--------------------------------+
64 +fanout | fanout[0] = 2 (for example) |-.
65 +table +--------------------------------+ |
66 | fanout[1] | |
67 +--------------------------------+ |
68 | fanout[2] | |
69 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
79 - | fanout[255] | |
80 - +--------------------------------+ |
81 -main | offset | |
82 -index | object name 00XXXXXXXXXXXXXXXX | |
83 -table +--------------------------------+ |
84 - | offset | |
85 - | object name 00XXXXXXXXXXXXXXXX | |
86 - +--------------------------------+ |
87 - .-| offset |<+
88 - | | object name 01XXXXXXXXXXXXXXXX |
89 - | +--------------------------------+
90 - | | offset |
91 - | | object name 01XXXXXXXXXXXXXXXX |
92 - | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
93 - | | offset |
94 - | | object name FFXXXXXXXXXXXXXXXX |
95 - | +--------------------------------+
70 + | fanout[255] = total objects |---.
71 + -- +--------------------------------+ | |
72 +main | offset | | |
73 +index | object name 00XXXXXXXXXXXXXXXX | | |
74 +table +--------------------------------+ | |
75 + | offset | | |
76 + | object name 00XXXXXXXXXXXXXXXX | | |
77 + +--------------------------------+<+ |
78 + .-| offset | |
79 + | | object name 01XXXXXXXXXXXXXXXX | |
80 + | +--------------------------------+ |
81 + | | offset | |
82 + | | object name 01XXXXXXXXXXXXXXXX | |
83 + | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
84 + | | offset | |
85 + | | object name FFXXXXXXXXXXXXXXXX | |
86 + --| +--------------------------------+<--+
87 trailer | | packfile checksum |
88 | +--------------------------------+
89 | | idxfile checksum |
@@ -116,3 +107,40 @@ Pack file entry: <+
107 20-byte base object name SHA1 (the size above is the
108 size of the delta data that follows).
109 delta data, deflated.
110 +
111 +
112 += Version 2 pack-*.idx files support packs larger than 4 GiB, and
113 + have some other reorganizations. They have the format:
114 +
115 + - A 4-byte magic number '\377tOc' which is an unreasonable
116 + fanout[0] value.
117 +
118 + - A 4-byte version number (= 2)
119 +
120 + - A 256-entry fan-out table just like v1.
121 +
122 + - A table of sorted 20-byte SHA1 object names. These are
123 + packed together without offset values to reduce the cache
124 + footprint of the binary search for a specific object name.
125 +
126 + - A table of 4-byte CRC32 values of the packed object data.
127 + This is new in v2 so compressed data can be copied directly
128 + from pack to pack during repacking withough undetected
129 + data corruption.
130 +
131 + - A table of 4-byte offset values (in network byte order).
132 + These are usually 31-bit pack file offsets, but large
133 + offsets are encoded as an index into the next table with
134 + the msbit set.
135 +
136 + - A table of 8-byte offset entries (empty for pack files less
137 + than 2 GiB). Pack files are organized with heavily used
138 + objects toward the front, so most object references should
139 + not need to refer to this table.
140 +
141 + - The same trailer as a v1 pack file:
142 +
143 + A copy of the 20-byte SHA1 checksum at the end of
144 + corresponding packfile.
145 +
146 + 20-byte SHA1-checksum of all of the above.