@matheusbsbs / matheus-bagliano / commits / 09eff7b0f7
Fork 0

user-manual: move packfile and dangling object discussion

The discussions of packfiles and dangling objects both belong in the object database section. Signed-off-by: J. Bruce Fields <bfields@citi.umich.edu>

J. Bruce Fields committed Sep 8, 2007 at 22:13 UTC 09eff7b0f7c1f55f8714f19c5d87bbd92ddee453
1 file changed +147 -148
Documentation/user-manual.txt
+147 -148
@@ -2948,6 +2948,153 @@ objects. (Note that gitlink:git-tag[1] can also be used to create
2948 "lightweight tags", which are not tag objects at all, but just simple
2949 references in .git/refs/tags/).
2950
2951 +[[pack-files]]
2952 +How git stores objects efficiently: pack files
2953 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2954 +
2955 +We've seen how git stores each object in a file named after the
2956 +object's SHA1 hash.
2957 +
2958 +Unfortunately this system becomes inefficient once a project has a
2959 +lot of objects. Try this on an old project:
2960 +
2961 +------------------------------------------------
2962 +$ git count-objects
2963 +6930 objects, 47620 kilobytes
2964 +------------------------------------------------
2965 +
2966 +The first number is the number of objects which are kept in
2967 +individual files. The second is the amount of space taken up by
2968 +those "loose" objects.
2969 +
2970 +You can save space and make git faster by moving these loose objects in
2971 +to a "pack file", which stores a group of objects in an efficient
2972 +compressed format; the details of how pack files are formatted can be
2973 +found in link:technical/pack-format.txt[technical/pack-format.txt].
2974 +
2975 +To put the loose objects into a pack, just run git repack:
2976 +
2977 +------------------------------------------------
2978 +$ git repack
2979 +Generating pack...
2980 +Done counting 6020 objects.
2981 +Deltifying 6020 objects.
2982 + 100% (6020/6020) done
2983 +Writing 6020 objects.
2984 + 100% (6020/6020) done
2985 +Total 6020, written 6020 (delta 4070), reused 0 (delta 0)
2986 +Pack pack-3e54ad29d5b2e05838c75df582c65257b8d08e1c created.
2987 +------------------------------------------------
2988 +
2989 +You can then run
2990 +
2991 +------------------------------------------------
2992 +$ git prune
2993 +------------------------------------------------
2994 +
2995 +to remove any of the "loose" objects that are now contained in the
2996 +pack. This will also remove any unreferenced objects (which may be
2997 +created when, for example, you use "git reset" to remove a commit).
2998 +You can verify that the loose objects are gone by looking at the
2999 +.git/objects directory or by running
3000 +
3001 +------------------------------------------------
3002 +$ git count-objects
3003 +0 objects, 0 kilobytes
3004 +------------------------------------------------
3005 +
3006 +Although the object files are gone, any commands that refer to those
3007 +objects will work exactly as they did before.
3008 +
3009 +The gitlink:git-gc[1] command performs packing, pruning, and more for
3010 +you, so is normally the only high-level command you need.
3011 +
3012 +[[dangling-objects]]
3013 +Dangling objects
3014 +~~~~~~~~~~~~~~~~
3015 +
3016 +The gitlink:git-fsck[1] command will sometimes complain about dangling
3017 +objects. They are not a problem.
3018 +
3019 +The most common cause of dangling objects is that you've rebased a
3020 +branch, or you have pulled from somebody else who rebased a branch--see
3021 +<<cleaning-up-history>>. In that case, the old head of the original
3022 +branch still exists, as does everything it pointed to. The branch
3023 +pointer itself just doesn't, since you replaced it with another one.
3024 +
3025 +There are also other situations that cause dangling objects. For
3026 +example, a "dangling blob" may arise because you did a "git add" of a
3027 +file, but then, before you actually committed it and made it part of the
3028 +bigger picture, you changed something else in that file and committed
3029 +that *updated* thing - the old state that you added originally ends up
3030 +not being pointed to by any commit or tree, so it's now a dangling blob
3031 +object.
3032 +
3033 +Similarly, when the "recursive" merge strategy runs, and finds that
3034 +there are criss-cross merges and thus more than one merge base (which is
3035 +fairly unusual, but it does happen), it will generate one temporary
3036 +midway tree (or possibly even more, if you had lots of criss-crossing
3037 +merges and more than two merge bases) as a temporary internal merge
3038 +base, and again, those are real objects, but the end result will not end
3039 +up pointing to them, so they end up "dangling" in your repository.
3040 +
3041 +Generally, dangling objects aren't anything to worry about. They can
3042 +even be very useful: if you screw something up, the dangling objects can
3043 +be how you recover your old tree (say, you did a rebase, and realized
3044 +that you really didn't want to - you can look at what dangling objects
3045 +you have, and decide to reset your head to some old dangling state).
3046 +
3047 +For commits, you can just use:
3048 +
3049 +------------------------------------------------
3050 +$ gitk <dangling-commit-sha-goes-here> --not --all
3051 +------------------------------------------------
3052 +
3053 +This asks for all the history reachable from the given commit but not
3054 +from any branch, tag, or other reference. If you decide it's something
3055 +you want, you can always create a new reference to it, e.g.,
3056 +
3057 +------------------------------------------------
3058 +$ git branch recovered-branch <dangling-commit-sha-goes-here>
3059 +------------------------------------------------
3060 +
3061 +For blobs and trees, you can't do the same, but you can still examine
3062 +them. You can just do
3063 +
3064 +------------------------------------------------
3065 +$ git show <dangling-blob/tree-sha-goes-here>
3066 +------------------------------------------------
3067 +
3068 +to show what the contents of the blob were (or, for a tree, basically
3069 +what the "ls" for that directory was), and that may give you some idea
3070 +of what the operation was that left that dangling object.
3071 +
3072 +Usually, dangling blobs and trees aren't very interesting. They're
3073 +almost always the result of either being a half-way mergebase (the blob
3074 +will often even have the conflict markers from a merge in it, if you
3075 +have had conflicting merges that you fixed up by hand), or simply
3076 +because you interrupted a "git fetch" with ^C or something like that,
3077 +leaving _some_ of the new objects in the object database, but just
3078 +dangling and useless.
3079 +
3080 +Anyway, once you are sure that you're not interested in any dangling
3081 +state, you can just prune all unreachable objects:
3082 +
3083 +------------------------------------------------
3084 +$ git prune
3085 +------------------------------------------------
3086 +
3087 +and they'll be gone. But you should only run "git prune" on a quiescent
3088 +repository - it's kind of like doing a filesystem fsck recovery: you
3089 +don't want to do that while the filesystem is mounted.
3090 +
3091 +(The same is true of "git-fsck" itself, btw - but since
3092 +git-fsck never actually *changes* the repository, it just reports
3093 +on what it found, git-fsck itself is never "dangerous" to run.
3094 +Running it while somebody is actually changing the repository can cause
3095 +confusing and scary messages, but it won't actually do anything bad. In
3096 +contrast, running "git prune" while somebody is actively changing the
3097 +repository is a *BAD* idea).
3098
3099 [[the-index]]
3100 The index
@@ -3385,154 +3532,6 @@ $ git-merge-index git-merge-one-file hello.c
3532
3533 and that is what higher level `git merge -s resolve` is implemented with.
3534
3388 -[[pack-files]]
3389 -How git stores objects efficiently: pack files
3390 -----------------------------------------------
3391 -
3392 -We've seen how git stores each object in a file named after the
3393 -object's SHA1 hash.
3394 -
3395 -Unfortunately this system becomes inefficient once a project has a
3396 -lot of objects. Try this on an old project:
3397 -
3398 -------------------------------------------------
3399 -$ git count-objects
3400 -6930 objects, 47620 kilobytes
3401 -------------------------------------------------
3402 -
3403 -The first number is the number of objects which are kept in
3404 -individual files. The second is the amount of space taken up by
3405 -those "loose" objects.
3406 -
3407 -You can save space and make git faster by moving these loose objects in
3408 -to a "pack file", which stores a group of objects in an efficient
3409 -compressed format; the details of how pack files are formatted can be
3410 -found in link:technical/pack-format.txt[technical/pack-format.txt].
3411 -
3412 -To put the loose objects into a pack, just run git repack:
3413 -
3414 -------------------------------------------------
3415 -$ git repack
3416 -Generating pack...
3417 -Done counting 6020 objects.
3418 -Deltifying 6020 objects.
3419 - 100% (6020/6020) done
3420 -Writing 6020 objects.
3421 - 100% (6020/6020) done
3422 -Total 6020, written 6020 (delta 4070), reused 0 (delta 0)
3423 -Pack pack-3e54ad29d5b2e05838c75df582c65257b8d08e1c created.
3424 -------------------------------------------------
3425 -
3426 -You can then run
3427 -
3428 -------------------------------------------------
3429 -$ git prune
3430 -------------------------------------------------
3431 -
3432 -to remove any of the "loose" objects that are now contained in the
3433 -pack. This will also remove any unreferenced objects (which may be
3434 -created when, for example, you use "git reset" to remove a commit).
3435 -You can verify that the loose objects are gone by looking at the
3436 -.git/objects directory or by running
3437 -
3438 -------------------------------------------------
3439 -$ git count-objects
3440 -0 objects, 0 kilobytes
3441 -------------------------------------------------
3442 -
3443 -Although the object files are gone, any commands that refer to those
3444 -objects will work exactly as they did before.
3445 -
3446 -The gitlink:git-gc[1] command performs packing, pruning, and more for
3447 -you, so is normally the only high-level command you need.
3448 -
3449 -[[dangling-objects]]
3450 -Dangling objects
3451 -----------------
3452 -
3453 -The gitlink:git-fsck[1] command will sometimes complain about dangling
3454 -objects. They are not a problem.
3455 -
3456 -The most common cause of dangling objects is that you've rebased a
3457 -branch, or you have pulled from somebody else who rebased a branch--see
3458 -<<cleaning-up-history>>. In that case, the old head of the original
3459 -branch still exists, as does everything it pointed to. The branch
3460 -pointer itself just doesn't, since you replaced it with another one.
3461 -
3462 -There are also other situations that cause dangling objects. For
3463 -example, a "dangling blob" may arise because you did a "git add" of a
3464 -file, but then, before you actually committed it and made it part of the
3465 -bigger picture, you changed something else in that file and committed
3466 -that *updated* thing - the old state that you added originally ends up
3467 -not being pointed to by any commit or tree, so it's now a dangling blob
3468 -object.
3469 -
3470 -Similarly, when the "recursive" merge strategy runs, and finds that
3471 -there are criss-cross merges and thus more than one merge base (which is
3472 -fairly unusual, but it does happen), it will generate one temporary
3473 -midway tree (or possibly even more, if you had lots of criss-crossing
3474 -merges and more than two merge bases) as a temporary internal merge
3475 -base, and again, those are real objects, but the end result will not end
3476 -up pointing to them, so they end up "dangling" in your repository.
3477 -
3478 -Generally, dangling objects aren't anything to worry about. They can
3479 -even be very useful: if you screw something up, the dangling objects can
3480 -be how you recover your old tree (say, you did a rebase, and realized
3481 -that you really didn't want to - you can look at what dangling objects
3482 -you have, and decide to reset your head to some old dangling state).
3483 -
3484 -For commits, you can just use:
3485 -
3486 -------------------------------------------------
3487 -$ gitk <dangling-commit-sha-goes-here> --not --all
3488 -------------------------------------------------
3489 -
3490 -This asks for all the history reachable from the given commit but not
3491 -from any branch, tag, or other reference. If you decide it's something
3492 -you want, you can always create a new reference to it, e.g.,
3493 -
3494 -------------------------------------------------
3495 -$ git branch recovered-branch <dangling-commit-sha-goes-here>
3496 -------------------------------------------------
3497 -
3498 -For blobs and trees, you can't do the same, but you can still examine
3499 -them. You can just do
3500 -
3501 -------------------------------------------------
3502 -$ git show <dangling-blob/tree-sha-goes-here>
3503 -------------------------------------------------
3504 -
3505 -to show what the contents of the blob were (or, for a tree, basically
3506 -what the "ls" for that directory was), and that may give you some idea
3507 -of what the operation was that left that dangling object.
3508 -
3509 -Usually, dangling blobs and trees aren't very interesting. They're
3510 -almost always the result of either being a half-way mergebase (the blob
3511 -will often even have the conflict markers from a merge in it, if you
3512 -have had conflicting merges that you fixed up by hand), or simply
3513 -because you interrupted a "git fetch" with ^C or something like that,
3514 -leaving _some_ of the new objects in the object database, but just
3515 -dangling and useless.
3516 -
3517 -Anyway, once you are sure that you're not interested in any dangling
3518 -state, you can just prune all unreachable objects:
3519 -
3520 -------------------------------------------------
3521 -$ git prune
3522 -------------------------------------------------
3523 -
3524 -and they'll be gone. But you should only run "git prune" on a quiescent
3525 -repository - it's kind of like doing a filesystem fsck recovery: you
3526 -don't want to do that while the filesystem is mounted.
3527 -
3528 -(The same is true of "git-fsck" itself, btw - but since
3529 -git-fsck never actually *changes* the repository, it just reports
3530 -on what it found, git-fsck itself is never "dangerous" to run.
3531 -Running it while somebody is actually changing the repository can cause
3532 -confusing and scary messages, but it won't actually do anything bad. In
3533 -contrast, running "git prune" while somebody is actively changing the
3534 -repository is a *BAD* idea).
3535 -
3535 [[hacking-git]]
3536 Hacking git
3537 ===========