1
-////////////////////////////////////////////////////////////////
2
-
3
- GIT - the stupid content tracker
4
-
5
-////////////////////////////////////////////////////////////////
6
-
7
-"git" can mean anything, depending on your mood.
8
-
9
- - random three-letter combination that is pronounceable, and not
10
- actually used by any common UNIX command. The fact that it is a
11
- mispronunciation of "get" may or may not be relevant.
12
- - stupid. contemptible and despicable. simple. Take your pick from the
13
- dictionary of slang.
14
- - "global information tracker": you're in a good mood, and it actually
15
- works for you. Angels sing, and a light suddenly fills the room.
16
- - "goddamn idiotic truckload of sh*t": when it breaks
17
-
18
-This is a (not so) stupid but extremely fast directory content manager.
19
-It doesn't do a whole lot at its core, but what it 'does' do is track
20
-directory contents efficiently.
21
-
22
-There are two object abstractions: the "object database", and the
23
-"current directory cache" aka "index".
24
-
25
-The Object Database
26
-~~~~~~~~~~~~~~~~~~~
27
-The object database is literally just a content-addressable collection
28
-of objects. All objects are named by their content, which is
29
-approximated by the SHA1 hash of the object itself. Objects may refer
30
-to other objects (by referencing their SHA1 hash), and so you can
31
-build up a hierarchy of objects.
32
-
33
-All objects have a statically determined "type" aka "tag", which is
34
-determined at object creation time, and which identifies the format of
35
-the object (i.e. how it is used, and how it can refer to other
36
-objects). There are currently four different object types: "blob",
37
-"tree", "commit" and "tag".
38
-
39
-A "blob" object cannot refer to any other object, and is, like the type
40
-implies, a pure storage object containing some user data. It is used to
41
-actually store the file data, i.e. a blob object is associated with some
42
-particular version of some file.
43
-
44
-A "tree" object is an object that ties one or more "blob" objects into a
45
-directory structure. In addition, a tree object can refer to other tree
46
-objects, thus creating a directory hierarchy.
47
-
48
-A "commit" object ties such directory hierarchies together into
49
-a DAG of revisions - each "commit" is associated with exactly one tree
50
-(the directory hierarchy at the time of the commit). In addition, a
51
-"commit" refers to one or more "parent" commit objects that describe the
52
-history of how we arrived at that directory hierarchy.
53
-
54
-As a special case, a commit object with no parents is called the "root"
55
-object, and is the point of an initial project commit. Each project
56
-must have at least one root, and while you can tie several different
57
-root objects together into one project by creating a commit object which
58
-has two or more separate roots as its ultimate parents, that's probably
59
-just going to confuse people. So aim for the notion of "one root object
60
-per project", even if git itself does not enforce that.
61
-
62
-A "tag" object symbolically identifies and can be used to sign other
63
-objects. It contains the identifier and type of another object, a
64
-symbolic name (of course!) and, optionally, a signature.
65
-
66
-Regardless of object type, all objects share the following
67
-characteristics: they are all deflated with zlib, and have a header
68
-that not only specifies their type, but also provides size information
69
-about the data in the object. It's worth noting that the SHA1 hash
70
-that is used to name the object is the hash of the original data
71
-plus this header, so `sha1sum` 'file' does not match the object name
72
-for 'file'.
73
-(Historical note: in the dawn of the age of git the hash
74
-was the sha1 of the 'compressed' object.)
75
-
76
-As a result, the general consistency of an object can always be tested
77
-independently of the contents or the type of the object: all objects can
78
-be validated by verifying that (a) their hashes match the content of the
79
-file and (b) the object successfully inflates to a stream of bytes that
80
-forms a sequence of <ascii type without space> + <space> + <ascii decimal
81
-size> + <byte\0> + <binary object data>.
82
-
83
-The structured objects can further have their structure and
84
-connectivity to other objects verified. This is generally done with
85
-the `git-fsck` program, which generates a full dependency graph
86
-of all objects, and verifies their internal consistency (in addition
87
-to just verifying their superficial consistency through the hash).
88
-
89
-The object types in some more detail:
90
-
91
-Blob Object
92
-~~~~~~~~~~~
93
-A "blob" object is nothing but a binary blob of data, and doesn't
94
-refer to anything else. There is no signature or any other
95
-verification of the data, so while the object is consistent (it 'is'
96
-indexed by its sha1 hash, so the data itself is certainly correct), it
97
-has absolutely no other attributes. No name associations, no
98
-permissions. It is purely a blob of data (i.e. normally "file
99
-contents").
100
-
101
-In particular, since the blob is entirely defined by its data, if two
102
-files in a directory tree (or in multiple different versions of the
103
-repository) have the same contents, they will share the same blob
104
-object. The object is totally independent of its location in the
105
-directory tree, and renaming a file does not change the object that
106
-file is associated with in any way.
107
-
108
-A blob is typically created when gitlink:git-update-index[1]
109
-(or gitlink:git-add[1]) is run, and its data can be accessed by
110
-gitlink:git-cat-file[1].
111
-
112
-Tree Object
113
-~~~~~~~~~~~
114
-The next hierarchical object type is the "tree" object. A tree object
115
-is a list of mode/name/blob data, sorted by name. Alternatively, the
116
-mode data may specify a directory mode, in which case instead of
117
-naming a blob, that name is associated with another TREE object.
118
-
119
-Like the "blob" object, a tree object is uniquely determined by the
120
-set contents, and so two separate but identical trees will always
121
-share the exact same object. This is true at all levels, i.e. it's
122
-true for a "leaf" tree (which does not refer to any other trees, only
123
-blobs) as well as for a whole subdirectory.
124
-
125
-For that reason a "tree" object is just a pure data abstraction: it
126
-has no history, no signatures, no verification of validity, except
127
-that since the contents are again protected by the hash itself, we can
128
-trust that the tree is immutable and its contents never change.
129
-
130
-So you can trust the contents of a tree to be valid, the same way you
131
-can trust the contents of a blob, but you don't know where those
132
-contents 'came' from.
133
-
134
-Side note on trees: since a "tree" object is a sorted list of
135
-"filename+content", you can create a diff between two trees without
136
-actually having to unpack two trees. Just ignore all common parts,
137
-and your diff will look right. In other words, you can effectively
138
-(and efficiently) tell the difference between any two random trees by
139
-O(n) where "n" is the size of the difference, rather than the size of
140
-the tree.
141
-
142
-Side note 2 on trees: since the name of a "blob" depends entirely and
143
-exclusively on its contents (i.e. there are no names or permissions
144
-involved), you can see trivial renames or permission changes by
145
-noticing that the blob stayed the same. However, renames with data
146
-changes need a smarter "diff" implementation.
147
-
148
-A tree is created with gitlink:git-write-tree[1] and
149
-its data can be accessed by gitlink:git-ls-tree[1].
150
-Two trees can be compared with gitlink:git-diff-tree[1].
151
-
152
-Commit Object
153
-~~~~~~~~~~~~~
154
-The "commit" object is an object that introduces the notion of
155
-history into the picture. In contrast to the other objects, it
156
-doesn't just describe the physical state of a tree, it describes how
157
-we got there, and why.
158
-
159
-A "commit" is defined by the tree-object that it results in, the
160
-parent commits (zero, one or more) that led up to that point, and a
161
-comment on what happened. Again, a commit is not trusted per se:
162
-the contents are well-defined and "safe" due to the cryptographically
163
-strong signatures at all levels, but there is no reason to believe
164
-that the tree is "good" or that the merge information makes sense.
165
-The parents do not have to actually have any relationship with the
166
-result, for example.
167
-
168
-Note on commits: unlike real SCM's, commits do not contain
169
-rename information or file mode change information. All of that is
170
-implicit in the trees involved (the result tree, and the result trees
171
-of the parents), and describing that makes no sense in this idiotic
172
-file manager.
173
-
174
-A commit is created with gitlink:git-commit-tree[1] and
175
-its data can be accessed by gitlink:git-cat-file[1].
176
-
177
-Trust
178
-~~~~~
179
-An aside on the notion of "trust". Trust is really outside the scope
180
-of "git", but it's worth noting a few things. First off, since
181
-everything is hashed with SHA1, you 'can' trust that an object is
182
-intact and has not been messed with by external sources. So the name
183
-of an object uniquely identifies a known state - just not a state that
184
-you may want to trust.
185
-
186
-Furthermore, since the SHA1 signature of a commit refers to the
187
-SHA1 signatures of the tree it is associated with and the signatures
188
-of the parent, a single named commit specifies uniquely a whole set
189
-of history, with full contents. You can't later fake any step of the
190
-way once you have the name of a commit.
191
-
192
-So to introduce some real trust in the system, the only thing you need
193
-to do is to digitally sign just 'one' special note, which includes the
194
-name of a top-level commit. Your digital signature shows others
195
-that you trust that commit, and the immutability of the history of
196
-commits tells others that they can trust the whole history.
197
-
198
-In other words, you can easily validate a whole archive by just
199
-sending out a single email that tells the people the name (SHA1 hash)
200
-of the top commit, and digitally sign that email using something
201
-like GPG/PGP.
202
-
203
-To assist in this, git also provides the tag object...
204
-
205
-Tag Object
206
-~~~~~~~~~~
207
-Git provides the "tag" object to simplify creating, managing and
208
-exchanging symbolic and signed tokens. The "tag" object at its
209
-simplest simply symbolically identifies another object by containing
210
-the sha1, type and symbolic name.
211
-
212
-However it can optionally contain additional signature information
213
-(which git doesn't care about as long as there's less than 8k of
214
-it). This can then be verified externally to git.
215
-
216
-Note that despite the tag features, "git" itself only handles content
217
-integrity; the trust framework (and signature provision and
218
-verification) has to come from outside.
219
-
220
-A tag is created with gitlink:git-mktag[1],
221
-its data can be accessed by gitlink:git-cat-file[1],
222
-and the signature can be verified by
223
-gitlink:git-verify-tag[1].
224
-
225
-
226
-The "index" aka "Current Directory Cache"
227
------------------------------------------
228
-The index is a simple binary file, which contains an efficient
229
-representation of a virtual directory content at some random time. It
230
-does so by a simple array that associates a set of names, dates,
231
-permissions and content (aka "blob") objects together. The cache is
232
-always kept ordered by name, and names are unique (with a few very
233
-specific rules) at any point in time, but the cache has no long-term
234
-meaning, and can be partially updated at any time.
235
-
236
-In particular, the index certainly does not need to be consistent with
237
-the current directory contents (in fact, most operations will depend on
238
-different ways to make the index 'not' be consistent with the directory
239
-hierarchy), but it has three very important attributes:
240
-
241
-'(a) it can re-generate the full state it caches (not just the
242
-directory structure: it contains pointers to the "blob" objects so
243
-that it can regenerate the data too)'
244
-
245
-As a special case, there is a clear and unambiguous one-way mapping
246
-from a current directory cache to a "tree object", which can be
247
-efficiently created from just the current directory cache without
248
-actually looking at any other data. So a directory cache at any one
249
-time uniquely specifies one and only one "tree" object (but has
250
-additional data to make it easy to match up that tree object with what
251
-has happened in the directory)
252
-
253
-'(b) it has efficient methods for finding inconsistencies between that
254
-cached state ("tree object waiting to be instantiated") and the
255
-current state.'
256
-
257
-'(c) it can additionally efficiently represent information about merge
258
-conflicts between different tree objects, allowing each pathname to be
259
-associated with sufficient information about the trees involved that
260
-you can create a three-way merge between them.'
261
-
262
-Those are the three ONLY things that the directory cache does. It's a
263
-cache, and the normal operation is to re-generate it completely from a
264
-known tree object, or update/compare it with a live tree that is being
265
-developed. If you blow the directory cache away entirely, you generally
266
-haven't lost any information as long as you have the name of the tree
267
-that it described.
268
-
269
-At the same time, the index is at the same time also the
270
-staging area for creating new trees, and creating a new tree always
271
-involves a controlled modification of the index file. In particular,
272
-the index file can have the representation of an intermediate tree that
273
-has not yet been instantiated. So the index can be thought of as a
274
-write-back cache, which can contain dirty information that has not yet
275
-been written back to the backing store.
276
-
277
-
278
-
279
-The Workflow
280
-------------
281
-Generally, all "git" operations work on the index file. Some operations
282
-work *purely* on the index file (showing the current state of the
283
-index), but most operations move data to and from the index file. Either
284
-from the database or from the working directory. Thus there are four
285
-main combinations:
286
-
287
-1) working directory -> index
288
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
289
-
290
-You update the index with information from the working directory with
291
-the gitlink:git-update-index[1] command. You
292
-generally update the index information by just specifying the filename
293
-you want to update, like so:
294
-
295
- git-update-index filename
296
-
297
-but to avoid common mistakes with filename globbing etc, the command
298
-will not normally add totally new entries or remove old entries,
299
-i.e. it will normally just update existing cache entries.
300
-
301
-To tell git that yes, you really do realize that certain files no
302
-longer exist, or that new files should be added, you
303
-should use the `--remove` and `--add` flags respectively.
304
-
305
-NOTE! A `--remove` flag does 'not' mean that subsequent filenames will
306
-necessarily be removed: if the files still exist in your directory
307
-structure, the index will be updated with their new status, not
308
-removed. The only thing `--remove` means is that update-cache will be
309
-considering a removed file to be a valid thing, and if the file really
310
-does not exist any more, it will update the index accordingly.
311
-
312
-As a special case, you can also do `git-update-index --refresh`, which
313
-will refresh the "stat" information of each index to match the current
314
-stat information. It will 'not' update the object status itself, and
315
-it will only update the fields that are used to quickly test whether
316
-an object still matches its old backing store object.
317
-
318
-2) index -> object database
319
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
320
-
321
-You write your current index file to a "tree" object with the program
322
-
323
- git-write-tree
324
-
325
-that doesn't come with any options - it will just write out the
326
-current index into the set of tree objects that describe that state,
327
-and it will return the name of the resulting top-level tree. You can
328
-use that tree to re-generate the index at any time by going in the
329
-other direction:
330
-
331
-3) object database -> index
332
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
333
-
334
-You read a "tree" file from the object database, and use that to
335
-populate (and overwrite - don't do this if your index contains any
336
-unsaved state that you might want to restore later!) your current
337
-index. Normal operation is just
338
-
339
- git-read-tree <sha1 of tree>
340
-
341
-and your index file will now be equivalent to the tree that you saved
342
-earlier. However, that is only your 'index' file: your working
343
-directory contents have not been modified.
344
-
345
-4) index -> working directory
346
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
347
-
348
-You update your working directory from the index by "checking out"
349
-files. This is not a very common operation, since normally you'd just
350
-keep your files updated, and rather than write to your working
351
-directory, you'd tell the index files about the changes in your
352
-working directory (i.e. `git-update-index`).
353
-
354
-However, if you decide to jump to a new version, or check out somebody
355
-else's version, or just restore a previous tree, you'd populate your
356
-index file with read-tree, and then you need to check out the result
357
-with
358
-
359
- git-checkout-index filename
360
-
361
-or, if you want to check out all of the index, use `-a`.
362
-
363
-NOTE! git-checkout-index normally refuses to overwrite old files, so
364
-if you have an old version of the tree already checked out, you will
365
-need to use the "-f" flag ('before' the "-a" flag or the filename) to
366
-'force' the checkout.
367
-
368
-
369
-Finally, there are a few odds and ends which are not purely moving
370
-from one representation to the other:
371
-
372
-5) Tying it all together
373
-~~~~~~~~~~~~~~~~~~~~~~~~
374
-To commit a tree you have instantiated with "git-write-tree", you'd
375
-create a "commit" object that refers to that tree and the history
376
-behind it - most notably the "parent" commits that preceded it in
377
-history.
378
-
379
-Normally a "commit" has one parent: the previous state of the tree
380
-before a certain change was made. However, sometimes it can have two
381
-or more parent commits, in which case we call it a "merge", due to the
382
-fact that such a commit brings together ("merges") two or more
383
-previous states represented by other commits.
384
-
385
-In other words, while a "tree" represents a particular directory state
386
-of a working directory, a "commit" represents that state in "time",
387
-and explains how we got there.
388
-
389
-You create a commit object by giving it the tree that describes the
390
-state at the time of the commit, and a list of parents:
391
-
392
- git-commit-tree <tree> -p <parent> [-p <parent2> ..]
393
-
394
-and then giving the reason for the commit on stdin (either through
395
-redirection from a pipe or file, or by just typing it at the tty).
396
-
397
-git-commit-tree will return the name of the object that represents
398
-that commit, and you should save it away for later use. Normally,
399
-you'd commit a new `HEAD` state, and while git doesn't care where you
400
-save the note about that state, in practice we tend to just write the
401
-result to the file pointed at by `.git/HEAD`, so that we can always see
402
-what the last committed state was.
403
-
404
-Here is an ASCII art by Jon Loeliger that illustrates how
405
-various pieces fit together.
406
-
407
-------------
408
-
409
- commit-tree
410
- commit obj
411
- +----+
412
- | |
413
- | |
414
- V V
415
- +-----------+
416
- | Object DB |
417
- | Backing |
418
- | Store |
419
- +-----------+
420
- ^
421
- write-tree | |
422
- tree obj | |
423
- | | read-tree
424
- | | tree obj
425
- V
426
- +-----------+
427
- | Index |
428
- | "cache" |
429
- +-----------+
430
- update-index ^
431
- blob obj | |
432
- | |
433
- checkout-index -u | | checkout-index
434
- stat | | blob obj
435
- V
436
- +-----------+
437
- | Working |
438
- | Directory |
439
- +-----------+
440
-
441
-------------
442
-
443
-
444
-6) Examining the data
445
-~~~~~~~~~~~~~~~~~~~~~
446
-
447
-You can examine the data represented in the object database and the
448
-index with various helper tools. For every object, you can use
449
-gitlink:git-cat-file[1] to examine details about the
450
-object:
451
-
452
- git-cat-file -t <objectname>
453
-
454
-shows the type of the object, and once you have the type (which is
455
-usually implicit in where you find the object), you can use
456
-
457
- git-cat-file blob|tree|commit|tag <objectname>
458
-
459
-to show its contents. NOTE! Trees have binary content, and as a result
460
-there is a special helper for showing that content, called
461
-`git-ls-tree`, which turns the binary content into a more easily
462
-readable form.
463
-
464
-It's especially instructive to look at "commit" objects, since those
465
-tend to be small and fairly self-explanatory. In particular, if you
466
-follow the convention of having the top commit name in `.git/HEAD`,
467
-you can do
468
-
469
- git-cat-file commit HEAD
470
-
471
-to see what the top commit was.
472
-
473
-7) Merging multiple trees
474
-~~~~~~~~~~~~~~~~~~~~~~~~~
475
-
476
-Git helps you do a three-way merge, which you can expand to n-way by
477
-repeating the merge procedure arbitrary times until you finally
478
-"commit" the state. The normal situation is that you'd only do one
479
-three-way merge (two parents), and commit it, but if you like to, you
480
-can do multiple parents in one go.
481
-
482
-To do a three-way merge, you need the two sets of "commit" objects
483
-that you want to merge, use those to find the closest common parent (a
484
-third "commit" object), and then use those commit objects to find the
485
-state of the directory ("tree" object) at these points.
486
-
487
-To get the "base" for the merge, you first look up the common parent
488
-of two commits with
489
-
490
- git-merge-base <commit1> <commit2>
491
-
492
-which will return you the commit they are both based on. You should
493
-now look up the "tree" objects of those commits, which you can easily
494
-do with (for example)
495
-
496
- git-cat-file commit <commitname> | head -1
497
-
498
-since the tree object information is always the first line in a commit
499
-object.
500
-
501
-Once you know the three trees you are going to merge (the one
502
-"original" tree, aka the common case, and the two "result" trees, aka
503
-the branches you want to merge), you do a "merge" read into the
504
-index. This will complain if it has to throw away your old index contents, so you should
505
-make sure that you've committed those - in fact you would normally
506
-always do a merge against your last commit (which should thus match
507
-what you have in your current index anyway).
508
-
509
-To do the merge, do
510
-
511
- git-read-tree -m -u <origtree> <yourtree> <targettree>
512
-
513
-which will do all trivial merge operations for you directly in the
514
-index file, and you can just write the result out with
515
-`git-write-tree`.
516
-
517
-Historical note. We did not have `-u` facility when this
518
-section was first written, so we used to warn that
519
-the merge is done in the index file, not in your
520
-working tree, and your working tree will not match your
521
-index after this step.
522
-This is no longer true. The above command, thanks to `-u`
523
-option, updates your working tree with the merge results for
524
-paths that have been trivially merged.
525
-
526
-
527
-8) Merging multiple trees, continued
528
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
529
-
530
-Sadly, many merges aren't trivial. If there are files that have
531
-been added, moved or removed, or if both branches have modified the
532
-same file, you will be left with an index tree that contains "merge
533
-entries" in it. Such an index tree can 'NOT' be written out to a tree
534
-object, and you will have to resolve any such merge clashes using
535
-other tools before you can write out the result.
536
-
537
-You can examine such index state with `git-ls-files --unmerged`
538
-command. An example:
539
-
540
-------------------------------------------------
541
-$ git-read-tree -m $orig HEAD $target
542
-$ git-ls-files --unmerged
543
-100644 263414f423d0e4d70dae8fe53fa34614ff3e2860 1 hello.c
544
-100644 06fa6a24256dc7e560efa5687fa84b51f0263c3a 2 hello.c
545
-100644 cc44c73eb783565da5831b4d820c962954019b69 3 hello.c
546
-------------------------------------------------
547
-
548
-Each line of the `git-ls-files --unmerged` output begins with
549
-the blob mode bits, blob SHA1, 'stage number', and the
550
-filename. The 'stage number' is git's way to say which tree it
551
-came from: stage 1 corresponds to `$orig` tree, stage 2 `HEAD`
552
-tree, and stage3 `$target` tree.
553
-
554
-Earlier we said that trivial merges are done inside
555
-`git-read-tree -m`. For example, if the file did not change
556
-from `$orig` to `HEAD` nor `$target`, or if the file changed
557
-from `$orig` to `HEAD` and `$orig` to `$target` the same way,
558
-obviously the final outcome is what is in `HEAD`. What the
559
-above example shows is that file `hello.c` was changed from
560
-`$orig` to `HEAD` and `$orig` to `$target` in a different way.
561
-You could resolve this by running your favorite 3-way merge
562
-program, e.g. `diff3` or `merge`, on the blob objects from
563
-these three stages yourself, like this:
564
-
565
-------------------------------------------------
566
-$ git-cat-file blob 263414f... >hello.c~1
567
-$ git-cat-file blob 06fa6a2... >hello.c~2
568
-$ git-cat-file blob cc44c73... >hello.c~3
569
-$ merge hello.c~2 hello.c~1 hello.c~3
570
-------------------------------------------------
571
-
572
-This would leave the merge result in `hello.c~2` file, along
573
-with conflict markers if there are conflicts. After verifying
574
-the merge result makes sense, you can tell git what the final
575
-merge result for this file is by:
576
-
577
- mv -f hello.c~2 hello.c
578
- git-update-index hello.c
579
-
580
-When a path is in unmerged state, running `git-update-index` for
581
-that path tells git to mark the path resolved.
582
-
583
-The above is the description of a git merge at the lowest level,
584
-to help you understand what conceptually happens under the hood.
585
-In practice, nobody, not even git itself, uses three `git-cat-file`
586
-for this. There is `git-merge-index` program that extracts the
587
-stages to temporary files and calls a "merge" script on it:
588
-
589
- git-merge-index git-merge-one-file hello.c
590
-
591
-and that is what higher level `git merge -s resolve` is implemented
592
-with.