| 1 | git-fast-export(1) |
| 2 | ================== |
| 3 | |
| 4 | NAME |
| 5 | ---- |
| 6 | git-fast-export - Git data exporter |
| 7 | |
| 8 | |
| 9 | SYNOPSIS |
| 10 | -------- |
| 11 | [verse] |
| 12 | 'git fast-export' [<options>] | 'git fast-import' |
| 13 | |
| 14 | DESCRIPTION |
| 15 | ----------- |
| 16 | This program dumps the given revisions in a form suitable to be piped |
| 17 | into 'git fast-import'. |
| 18 | |
| 19 | You can use it as a human-readable bundle replacement (see |
| 20 | linkgit:git-bundle[1]), or as a format that can be edited before being |
| 21 | fed to 'git fast-import' in order to do history rewrites (an ability |
| 22 | relied on by tools like 'git filter-repo'). |
| 23 | |
| 24 | OPTIONS |
| 25 | ------- |
| 26 | --progress=<n>:: |
| 27 | Insert 'progress' statements every <n> objects, to be shown by |
| 28 | 'git fast-import' during import. |
| 29 | |
| 30 | --signed-tags=(verbatim|warn-verbatim|warn-strip|strip|abort):: |
| 31 | Specify how to handle signed tags. Since any transformation |
| 32 | after the export (or during the export, such as excluding |
| 33 | revisions) can change the hashes being signed, the signatures |
| 34 | may become invalid. |
| 35 | + |
| 36 | When asking to 'abort' (which is the default), this program will die |
| 37 | when encountering a signed tag. With 'strip', the tags will silently |
| 38 | be made unsigned, with 'warn-strip' they will be made unsigned but a |
| 39 | warning will be displayed, with 'verbatim', they will be silently |
| 40 | exported and with 'warn-verbatim' (or 'warn', a deprecated synonym), |
| 41 | they will be exported, but you will see a warning. 'verbatim' and |
| 42 | 'warn-verbatim' should only be used if you know that no transformation |
| 43 | affecting tags or any commit in their history will be performed by you |
| 44 | or by fast-export or fast-import, or if you do not care that the |
| 45 | resulting tag will have an invalid signature. |
| 46 | |
| 47 | --signed-commits=(verbatim|warn-verbatim|warn-strip|strip|abort):: |
| 48 | Specify how to handle signed commits. Behaves exactly as |
| 49 | '--signed-tags', but for commits. Default is 'strip', which |
| 50 | is the same as how earlier versions of this command without |
| 51 | this option behaved. |
| 52 | + |
| 53 | When exported, a signature starts with: |
| 54 | + |
| 55 | gpgsig <git-hash-algo> <signature-format> |
| 56 | + |
| 57 | where <git-hash-algo> is the Git object hash so either "sha1" or |
| 58 | "sha256", and <signature-format> is the signature type, so "openpgp", |
| 59 | "x509", "ssh" or "unknown". |
| 60 | + |
| 61 | For example, an OpenPGP signature on a SHA-1 commit starts with |
| 62 | `gpgsig sha1 openpgp`, while an SSH signature on a SHA-256 commit |
| 63 | starts with `gpgsig sha256 ssh`. |
| 64 | + |
| 65 | While all the signatures of a commit are exported, an importer may |
| 66 | choose to accept only some of them. For example |
| 67 | linkgit:git-fast-import[1] currently stores at most one signature per |
| 68 | Git hash algorithm in each commit. |
| 69 | + |
| 70 | NOTE: This is highly experimental and the format of the data stream may |
| 71 | change in the future without compatibility guarantees. |
| 72 | |
| 73 | --tag-of-filtered-object=(abort|drop|rewrite):: |
| 74 | Specify how to handle tags whose tagged object is filtered out. |
| 75 | Since revisions and files to export can be limited by path, |
| 76 | tagged objects may be filtered completely. |
| 77 | + |
| 78 | When asking to 'abort' (which is the default), this program will die |
| 79 | when encountering such a tag. With 'drop' it will omit such tags from |
| 80 | the output. With 'rewrite', if the tagged object is a commit, it will |
| 81 | rewrite the tag to tag an ancestor commit (via parent rewriting; see |
| 82 | linkgit:git-rev-list[1]). |
| 83 | |
| 84 | -M:: |
| 85 | -C:: |
| 86 | Perform move and/or copy detection, as described in the |
| 87 | linkgit:git-diff[1] manual page, and use it to generate |
| 88 | rename and copy commands in the output dump. |
| 89 | + |
| 90 | Note that earlier versions of this command did not complain and |
| 91 | produced incorrect results if you gave these options. |
| 92 | |
| 93 | --export-marks=<file>:: |
| 94 | Dumps the internal marks table to <file> when complete. |
| 95 | Marks are written one per line as `:markid SHA-1`. Only marks |
| 96 | for revisions are dumped; marks for blobs are ignored. |
| 97 | Backends can use this file to validate imports after they |
| 98 | have been completed, or to save the marks table across |
| 99 | incremental runs. As <file> is only opened and truncated |
| 100 | at completion, the same path can also be safely given to |
| 101 | --import-marks. |
| 102 | The file will not be written if no new object has been |
| 103 | marked/exported. |
| 104 | |
| 105 | --import-marks=<file>:: |
| 106 | Before processing any input, load the marks specified in |
| 107 | <file>. The input file must exist, must be readable, and |
| 108 | must use the same format as produced by --export-marks. |
| 109 | |
| 110 | --mark-tags:: |
| 111 | In addition to labelling blobs and commits with mark ids, also |
| 112 | label tags. This is useful in conjunction with |
| 113 | `--export-marks` and `--import-marks`, and is also useful (and |
| 114 | necessary) for exporting of nested tags. It does not hurt |
| 115 | other cases and would be the default, but many fast-import |
| 116 | frontends are not prepared to accept tags with mark |
| 117 | identifiers. |
| 118 | + |
| 119 | Any commits (or tags) that have already been marked will not be |
| 120 | exported again. If the backend uses a similar --import-marks file, |
| 121 | this allows for incremental bidirectional exporting of the repository |
| 122 | by keeping the marks the same across runs. |
| 123 | |
| 124 | --fake-missing-tagger:: |
| 125 | Some old repositories have tags without a tagger. The |
| 126 | fast-import protocol was pretty strict about that, and did not |
| 127 | allow that. So fake a tagger to be able to fast-import the |
| 128 | output. |
| 129 | |
| 130 | --use-done-feature:: |
| 131 | Start the stream with a 'feature done' stanza, and terminate |
| 132 | it with a 'done' command. |
| 133 | |
| 134 | --no-data:: |
| 135 | Skip output of blob objects and instead refer to blobs via |
| 136 | their original SHA-1 hash. This is useful when rewriting the |
| 137 | directory structure or history of a repository without |
| 138 | touching the contents of individual files. Note that the |
| 139 | resulting stream can only be used by a repository which |
| 140 | already contains the necessary objects. |
| 141 | |
| 142 | --full-tree:: |
| 143 | This option will cause fast-export to issue a "deleteall" |
| 144 | directive for each commit followed by a full list of all files |
| 145 | in the commit (as opposed to just listing the files which are |
| 146 | different from the commit's first parent). |
| 147 | |
| 148 | --anonymize:: |
| 149 | Anonymize the contents of the repository while still retaining |
| 150 | the shape of the history and stored tree. See the section on |
| 151 | `ANONYMIZING` below. |
| 152 | |
| 153 | --anonymize-map=<from>[:<to>]:: |
| 154 | Convert token `<from>` to `<to>` in the anonymized output. If |
| 155 | `<to>` is omitted, map `<from>` to itself (i.e., do not |
| 156 | anonymize it). See the section on `ANONYMIZING` below. |
| 157 | |
| 158 | --reference-excluded-parents:: |
| 159 | By default, running a command such as `git fast-export |
| 160 | master~5..master` will not include the commit master{tilde}5 |
| 161 | and will make master{tilde}4 no longer have master{tilde}5 as |
| 162 | a parent (though both the old master{tilde}4 and new |
| 163 | master{tilde}4 will have all the same files). Use |
| 164 | --reference-excluded-parents to instead have the stream |
| 165 | refer to commits in the excluded range of history by their |
| 166 | sha1sum. Note that the resulting stream can only be used by a |
| 167 | repository which already contains the necessary parent |
| 168 | commits. |
| 169 | |
| 170 | --show-original-ids:: |
| 171 | Add an extra directive to the output for commits and blobs, |
| 172 | `original-oid <SHA1SUM>`. While such directives will likely be |
| 173 | ignored by importers such as git-fast-import, it may be useful |
| 174 | for intermediary filters (e.g. for rewriting commit messages |
| 175 | which refer to older commits, or for stripping blobs by id). |
| 176 | |
| 177 | --reencode=(yes|no|abort):: |
| 178 | Specify how to handle `encoding` header in commit objects. When |
| 179 | asking to 'abort' (which is the default), this program will die |
| 180 | when encountering such a commit object. With 'yes', the commit |
| 181 | message will be re-encoded into UTF-8. With 'no', the original |
| 182 | encoding will be preserved. |
| 183 | |
| 184 | --refspec:: |
| 185 | Apply the specified refspec to each ref exported. Multiple of them can |
| 186 | be specified. |
| 187 | |
| 188 | [<git-rev-list-args>...]:: |
| 189 | A list of arguments, acceptable to 'git rev-parse' and |
| 190 | 'git rev-list', that specifies the specific objects and references |
| 191 | to export. For example, `master~10..master` causes the |
| 192 | current master reference to be exported along with all objects |
| 193 | added since its 10th ancestor commit and (unless the |
| 194 | --reference-excluded-parents option is specified) all files |
| 195 | common to master{tilde}9 and master{tilde}10. |
| 196 | |
| 197 | EXAMPLES |
| 198 | -------- |
| 199 | |
| 200 | ------------------------------------------------------------------- |
| 201 | $ git fast-export --all | (cd /empty/repository && git fast-import) |
| 202 | ------------------------------------------------------------------- |
| 203 | |
| 204 | This will export the whole repository and import it into the existing |
| 205 | empty repository. Except for reencoding commits that are not in |
| 206 | UTF-8, it would be a one-to-one mirror. |
| 207 | |
| 208 | ----------------------------------------------------- |
| 209 | $ git fast-export master~5..master | |
| 210 | sed "s|refs/heads/master|refs/heads/other|" | |
| 211 | git fast-import |
| 212 | ----------------------------------------------------- |
| 213 | |
| 214 | This makes a new branch called 'other' from 'master~5..master' |
| 215 | (i.e. if 'master' has linear history, it will take the last 5 commits). |
| 216 | |
| 217 | Note that this assumes that none of the blobs and commit messages |
| 218 | referenced by that revision range contains the string |
| 219 | 'refs/heads/master'. |
| 220 | |
| 221 | |
| 222 | ANONYMIZING |
| 223 | ----------- |
| 224 | |
| 225 | If the `--anonymize` option is given, git will attempt to remove all |
| 226 | identifying information from the repository while still retaining enough |
| 227 | of the original tree and history patterns to reproduce some bugs. The |
| 228 | goal is that a git bug which is found on a private repository will |
| 229 | persist in the anonymized repository, and the latter can be shared with |
| 230 | git developers to help solve the bug. |
| 231 | |
| 232 | With this option, git will replace all refnames, paths, blob contents, |
| 233 | commit and tag messages, names, and email addresses in the output with |
| 234 | anonymized data. Two instances of the same string will be replaced |
| 235 | equivalently (e.g., two commits with the same author will have the same |
| 236 | anonymized author in the output, but bear no resemblance to the original |
| 237 | author string). The relationship between commits, branches, and tags is |
| 238 | retained, as well as the commit timestamps (but the commit messages and |
| 239 | refnames bear no resemblance to the originals). The relative makeup of |
| 240 | the tree is retained (e.g., if you have a root tree with 10 files and 3 |
| 241 | trees, so will the output), but their names and the contents of the |
| 242 | files will be replaced. |
| 243 | |
| 244 | If you think you have found a git bug, you can start by exporting an |
| 245 | anonymized stream of the whole repository: |
| 246 | |
| 247 | --------------------------------------------------- |
| 248 | $ git fast-export --anonymize --all >anon-stream |
| 249 | --------------------------------------------------- |
| 250 | |
| 251 | Then confirm that the bug persists in a repository created from that |
| 252 | stream (many bugs will not, as they really do depend on the exact |
| 253 | repository contents): |
| 254 | |
| 255 | --------------------------------------------------- |
| 256 | $ git init anon-repo |
| 257 | $ cd anon-repo |
| 258 | $ git fast-import <../anon-stream |
| 259 | $ ... test your bug ... |
| 260 | --------------------------------------------------- |
| 261 | |
| 262 | If the anonymized repository shows the bug, it may be worth sharing |
| 263 | `anon-stream` along with a regular bug report. Note that the anonymized |
| 264 | stream compresses very well, so gzipping it is encouraged. If you want |
| 265 | to examine the stream to see that it does not contain any private data, |
| 266 | you can peruse it directly before sending. You may also want to try: |
| 267 | |
| 268 | --------------------------------------------------- |
| 269 | $ perl -pe 's/\d+/X/g' <anon-stream | sort -u | less |
| 270 | --------------------------------------------------- |
| 271 | |
| 272 | which shows all of the unique lines (with numbers converted to "X", to |
| 273 | collapse "User 0", "User 1", etc into "User X"). This produces a much |
| 274 | smaller output, and it is usually easy to quickly confirm that there is |
| 275 | no private data in the stream. |
| 276 | |
| 277 | Reproducing some bugs may require referencing particular commits or |
| 278 | paths, which becomes challenging after refnames and paths have been |
| 279 | anonymized. You can ask for a particular token to be left as-is or |
| 280 | mapped to a new value. For example, if you have a bug which reproduces |
| 281 | with `git rev-list sensitive -- secret.c`, you can run: |
| 282 | |
| 283 | --------------------------------------------------- |
| 284 | $ git fast-export --anonymize --all \ |
| 285 | --anonymize-map=sensitive:foo \ |
| 286 | --anonymize-map=secret.c:bar.c \ |
| 287 | >stream |
| 288 | --------------------------------------------------- |
| 289 | |
| 290 | After importing the stream, you can then run `git rev-list foo -- bar.c` |
| 291 | in the anonymized repository. |
| 292 | |
| 293 | Note that paths and refnames are split into tokens at slash boundaries. |
| 294 | The command above would anonymize `subdir/secret.c` as something like |
| 295 | `path123/bar.c`; you could then search for `bar.c` in the anonymized |
| 296 | repository to determine the final pathname. |
| 297 | |
| 298 | To make referencing the final pathname simpler, you can map each path |
| 299 | component; so if you also anonymize `subdir` to `publicdir`, then the |
| 300 | final pathname would be `publicdir/bar.c`. |
| 301 | |
| 302 | LIMITATIONS |
| 303 | ----------- |
| 304 | |
| 305 | Since 'git fast-import' cannot tag trees, you will not be |
| 306 | able to export the linux.git repository completely, as it contains |
| 307 | a tag referencing a tree instead of a commit. |
| 308 | |
| 309 | SEE ALSO |
| 310 | -------- |
| 311 | linkgit:git-fast-import[1] |
| 312 | |
| 313 | GIT |
| 314 | --- |
| 315 | Part of the linkgit:git[1] suite |