multi-pack-index: add format details
The multi-pack-index feature generalizes the existing pack-index feature by indexing objects across multiple pack-files. Describe the basic file format, using a 12-byte header followed by a lookup table for a list of "chunks" which will be described later. The file ends with a footer containing a checksum using the hash algorithm. The header allows later versions to create breaking changes by advancing the version number. We can also change the hash algorithm using a different version value. We will add the individual chunk format information as we introduce the code that writes that information. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Derrick Stolee committed
Jul 12, 2018 at 15:39 UTC
e0d1bcf82590aea8ee007ab087772f1c82f6890b
1 file changed
+49
Documentation/technical/pack-format.txt
+49
@@ -252,3 +252,52 @@ Pack file entry: <+
252
corresponding packfile.
253
254
20-byte SHA-1-checksum of all of the above.
255
+
256
+== multi-pack-index (MIDX) files have the following format:
257
+
258
+The multi-pack-index files refer to multiple pack-files and loose objects.
259
+
260
+In order to allow extensions that add extra data to the MIDX, we organize
261
+the body into "chunks" and provide a lookup table at the beginning of the
262
+body. The header includes certain length values, such as the number of packs,
263
+the number of base MIDX files, hash lengths and types.
264
+
265
+All 4-byte numbers are in network order.
266
+
267
+HEADER:
268
+
269
+ 4-byte signature:
270
+ The signature is: {'M', 'I', 'D', 'X'}
271
+
272
+ 1-byte version number:
273
+ Git only writes or recognizes version 1.
274
+
275
+ 1-byte Object Id Version
276
+ Git only writes or recognizes version 1 (SHA1).
277
+
278
+ 1-byte number of "chunks"
279
+
280
+ 1-byte number of base multi-pack-index files:
281
+ This value is currently always zero.
282
+
283
+ 4-byte number of pack files
284
+
285
+CHUNK LOOKUP:
286
+
287
+ (C + 1) * 12 bytes providing the chunk offsets:
288
+ First 4 bytes describe chunk id. Value 0 is a terminating label.
289
+ Other 8 bytes provide offset in current file for chunk to start.
290
+ (Chunks are provided in file-order, so you can infer the length
291
+ using the next chunk position if necessary.)
292
+
293
+ The remaining data in the body is described one chunk at a time, and
294
+ these chunks may be given in any order. Chunks are required unless
295
+ otherwise specified.
296
+
297
+CHUNK DATA:
298
+
299
+ (This section intentionally left incomplete.)
300
+
301
+TRAILER:
302
+
303
+ 20-byte SHA1-checksum of the above contents.