@matheusbsbs / matheus-bagliano / commits / 5a3b130cad

doc: add corrected commit date info

With generation data chunk and corrected commit dates implemented, let's update the technical documentation for commit-graph. Signed-off-by: Abhishek Kumar <abhishekkumar8222@gmail.com> Reviewed-by: Taylor Blau <me@ttaylorr.com> Reviewed-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Abhishek Kumar committed Jan 16, 2021 at 18:11 UTC 5a3b130cad0d5c770f766e3af6d32b41766374c0

2 files changed +86 -19

Documentation/technical/commit-graph-format.txt

+22 -6

@@ -4,11 +4,7 @@ Git commit graph format

4 The Git commit graph stores a list of commit OIDs and some associated

5 metadata, including:

7 -- The generation number of the commit. Commits with no parents have

8 - generation number 1; commits with parents have generation number

9 - one more than the maximum generation number of its parents. We

10 - reserve zero as special, and can be used to mark a generation

11 - number invalid or as "not computed".

7 +- The generation number of the commit.

9 - The root tree OID.

@@ -86,13 +82,33 @@ CHUNK DATA:

82 position. If there are more than two parents, the second value

83 has its most-significant bit on and the other bits store an array

84 position into the Extra Edge List chunk.

89 - * The next 8 bytes store the generation number of the commit and

85 + * The next 8 bytes store the topological level (generation number v1)

86 + of the commit and

87 the commit time in seconds since EPOCH. The generation number

88 uses the higher 30 bits of the first 4 bytes, while the commit

89 time uses the 32 bits of the second 4 bytes, along with the lowest

90 2 bits of the lowest byte, storing the 33rd and 34th bit of the

91 commit time.

93 + Generation Data (ID: {'G', 'D', 'A', 'T' }) (N * 4 bytes) [Optional]

94 + * This list of 4-byte values store corrected commit date offsets for the

95 + commits, arranged in the same order as commit data chunk.

96 + * If the corrected commit date offset cannot be stored within 31 bits,

97 + the value has its most-significant bit on and the other bits store

98 + the position of corrected commit date into the Generation Data Overflow

99 + chunk.

100 + * Generation Data chunk is present only when commit-graph file is written

101 + by compatible versions of Git and in case of split commit-graph chains,

102 + the topmost layer also has Generation Data chunk.

103 +

104 + Generation Data Overflow (ID: {'G', 'D', 'O', 'V' }) [Optional]

105 + * This list of 8-byte values stores the corrected commit date offsets

106 + for commits with corrected commit date offsets that cannot be

107 + stored within 31 bits.

108 + * Generation Data Overflow chunk is present only when Generation Data

109 + chunk is present and atleast one corrected commit date offset cannot

110 + be stored within 31 bits.

111 +

112 Extra Edge List (ID: {'E', 'D', 'G', 'E'}) [Optional]

113 This list of 4-byte values store the second through nth parents for

114 all octopus merges. The second parent value in the commit data stores

Documentation/technical/commit-graph.txt

+64 -13

@@ -38,14 +38,31 @@ A consumer may load the following info for a commit from the graph:

39 Values 1-4 satisfy the requirements of parse_commit_gently().

41 -Define the "generation number" of a commit recursively as follows:

41 +There are two definitions of generation number:

42 +1. Corrected committer dates (generation number v2)

43 +2. Topological levels (generation nummber v1)

43 - * A commit with no parents (a root commit) has generation number one.

45 +Define "corrected committer date" of a commit recursively as follows:

45 - * A commit with at least one parent has generation number one more than

46 - the largest generation number among its parents.

47 + * A commit with no parents (a root commit) has corrected committer date

48 + equal to its committer date.

48 -Equivalently, the generation number of a commit A is one more than the

50 + * A commit with at least one parent has corrected committer date equal to

51 + the maximum of its commiter date and one more than the largest corrected

52 + committer date among its parents.

53 +

54 + * As a special case, a root commit with timestamp zero has corrected commit

55 + date of 1, to be able to distinguish it from GENERATION_NUMBER_ZERO

56 + (that is, an uncomputed corrected commit date).

57 +

58 +Define the "topological level" of a commit recursively as follows:

59 +

60 + * A commit with no parents (a root commit) has topological level of one.

61 +

62 + * A commit with at least one parent has topological level one more than

63 + the largest topological level among its parents.

64 +

65 +Equivalently, the topological level of a commit A is one more than the

66 length of a longest path from A to a root commit. The recursive definition

67 is easier to use for computation and observing the following property:

@@ -60,6 +77,9 @@ is easier to use for computation and observing the following property:

77 generation numbers, then we always expand the boundary commit with highest

78 generation number and can easily detect the stopping condition.

80 +The property applies to both versions of generation number, that is both

81 +corrected committer dates and topological levels.

82 +

83 This property can be used to significantly reduce the time it takes to

84 walk commits and determine topological relationships. Without generation

85 numbers, the general heuristic is the following:

@@ -67,7 +87,9 @@ numbers, the general heuristic is the following:

87 If A and B are commits with commit time X and Y, respectively, and

88 X < Y, then A _probably_ cannot reach B.

70 -This heuristic is currently used whenever the computation is allowed to

90 +In absence of corrected commit dates (for example, old versions of Git or

91 +mixed generation graph chains),

92 +this heuristic is currently used whenever the computation is allowed to

93 violate topological relationships due to clock skew (such as "git log"

94 with default order), but is not used when the topological order is

95 required (such as merge base calculations, "git log --graph").

@@ -77,7 +99,7 @@ in the commit graph. We can treat these commits as having "infinite"

99 generation number and walk until reaching commits with known generation

100 number.

101

80 -We use the macro GENERATION_NUMBER_INFINITY = 0xFFFFFFFF to mark commits not

102 +We use the macro GENERATION_NUMBER_INFINITY to mark commits not

103 in the commit-graph file. If a commit-graph file was written by a version

104 of Git that did not compute generation numbers, then those commits will

105 have generation number represented by the macro GENERATION_NUMBER_ZERO = 0.

@@ -93,12 +115,12 @@ fully-computed generation numbers. Using strict inequality may result in

115 walking a few extra commits, but the simplicity in dealing with commits

116 with generation number *_INFINITY or *_ZERO is valuable.

117

96 -We use the macro GENERATION_NUMBER_MAX = 0x3FFFFFFF to for commits whose

97 -generation numbers are computed to be at least this value. We limit at

98 -this value since it is the largest value that can be stored in the

99 -commit-graph file using the 30 bits available to generation numbers. This

100 -presents another case where a commit can have generation number equal to

101 -that of a parent.

118 +We use the macro GENERATION_NUMBER_V1_MAX = 0x3FFFFFFF for commits whose

119 +topological levels (generation number v1) are computed to be at least

120 +this value. We limit at this value since it is the largest value that

121 +can be stored in the commit-graph file using the 30 bits available

122 +to topological levels. This presents another case where a commit can

123 +have generation number equal to that of a parent.

124

125 Design Details

126 --------------

@@ -267,6 +289,35 @@ The merge strategy values (2 for the size multiple, 64,000 for the maximum

289 number of commits) could be extracted into config settings for full

290 flexibility.

291

292 +## Handling Mixed Generation Number Chains

293 +

294 +With the introduction of generation number v2 and generation data chunk, the

295 +following scenario is possible:

296 +

297 +1. "New" Git writes a commit-graph with the corrected commit dates.

298 +2. "Old" Git writes a split commit-graph on top without corrected commit dates.

299 +

300 +A naive approach of using the newest available generation number from

301 +each layer would lead to violated expectations: the lower layer would

302 +use corrected commit dates which are much larger than the topological

303 +levels of the higher layer. For this reason, Git inspects the topmost

304 +layer to see if the layer is missing corrected commit dates. In such a case

305 +Git only uses topological level for generation numbers.

306 +

307 +When writing a new layer in split commit-graph, we write corrected commit

308 +dates if the topmost layer has corrected commit dates written. This

309 +guarantees that if a layer has corrected commit dates, all lower layers

310 +must have corrected commit dates as well.

311 +

312 +When merging layers, we do not consider whether the merged layers had corrected

313 +commit dates. Instead, the new layer will have corrected commit dates if the

314 +layer below the new layer has corrected commit dates.

315 +

316 +While writing or merging layers, if the new layer is the only layer, it will

317 +have corrected commit dates when written by compatible versions of Git. Thus,

318 +rewriting split commit-graph as a single file (`--split=replace`) creates a

319 +single layer with corrected commit dates.

320 +

321 ## Deleting graph-{hash} files

322

323 After a new tip file is written, some `graph-{hash}` files may no longer