@matheusbsbs / matheus-bagliano / commits / 6f6826c52b
Fork 0

Add git-filter-branch

This script is derived from Pasky's cg-admin-rewritehist. In fact, it _is_ the same script, minimally adapted to work without cogito. It _should_ be able to perform the same tasks, even if only relying on core-git programs. All the work is Pasky's, just the adaption is mine. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Hopefully-signed-off-by: Petr "cogito master" Baudis <pasky@suse.cz> Signed-off-by: Junio C Hamano <junkio@cox.net>

Johannes Schindelin committed Jun 3, 2007 at 01:31 UTC 6f6826c52bb751450e2bfa28f07c817dfa5802d6
3 files changed +479 -1
Makefile
+2 -1
@@ -209,7 +209,8 @@ SCRIPT_SH = \
209 git-am.sh \
210 git-merge.sh git-merge-stupid.sh git-merge-octopus.sh \
211 git-merge-resolve.sh git-merge-ours.sh \
212 - git-lost-found.sh git-quiltimport.sh git-submodule.sh
212 + git-lost-found.sh git-quiltimport.sh git-submodule.sh \
213 + git-filter-branch.sh
214
215 SCRIPT_PERL = \
216 git-add--interactive.perl \
git-filter-branch.sh new
+430
@@ -0,0 +1,430 @@
1 +#!/bin/sh
2 +#
3 +# Rewrite revision history
4 +# Copyright (c) Petr Baudis, 2006
5 +# Minimal changes to "port" it to core-git (c) Johannes Schindelin, 2007
6 +#
7 +# Lets you rewrite GIT revision history by creating a new branch from
8 +# your current branch by applying custom filters on each revision.
9 +# Those filters can modify each tree (e.g. removing a file or running
10 +# a perl rewrite on all files) or information about each commit.
11 +# Otherwise, all information (including original commit times or merge
12 +# information) will be preserved.
13 +#
14 +# The command takes the new branch name as a mandatory argument and
15 +# the filters as optional arguments. If you specify no filters, the
16 +# commits will be recommitted without any changes, which would normally
17 +# have no effect and result with the new branch pointing to the same
18 +# branch as your current branch. (Nevertheless, this may be useful in
19 +# the future for compensating for some Git bugs or such, therefore
20 +# such a usage is permitted.)
21 +#
22 +# WARNING! The rewritten history will have different ids for all the
23 +# objects and will not converge with the original branch. You will not
24 +# be able to easily push and distribute the rewritten branch. Please do
25 +# not use this command if you do not know the full implications, and
26 +# avoid using it anyway - do not do what a simple single commit on top
27 +# of the current version would fix.
28 +#
29 +# Always verify that the rewritten version is correct before disposing
30 +# the original branch.
31 +#
32 +# Note that since this operation is extensively I/O expensive, it might
33 +# be a good idea to do it off-disk, e.g. on tmpfs. Reportedly the speedup
34 +# is very noticeable.
35 +#
36 +# OPTIONS
37 +# -------
38 +# -d TEMPDIR:: The path to the temporary tree used for rewriting
39 +# When applying a tree filter, the command needs to temporary
40 +# checkout the tree to some directory, which may consume
41 +# considerable space in case of large projects. By default it
42 +# does this in the '.git-rewrite/' directory but you can override
43 +# that choice by this parameter.
44 +#
45 +# -r STARTREV:: The commit id to start the rewrite at
46 +# Normally, the command will rewrite the entire history. If you
47 +# pass this argument, though, this will be the first commit it
48 +# will rewrite and keep the previous commits intact.
49 +#
50 +# -k KEEPREV:: A commit id until which _not_ to rewrite history
51 +# If you pass this argument, this commit and all of its
52 +# predecessors are kept intact.
53 +#
54 +# Filters
55 +# ~~~~~~~
56 +# The filters are applied in the order as listed below. The COMMAND
57 +# argument is always evaluated in shell using the 'eval' command.
58 +# The $GIT_COMMIT environment variable is permanently set to contain
59 +# the id of the commit being rewritten. The author/committer environment
60 +# variables are set before the first filter is run.
61 +#
62 +# A 'map' function is available that takes an "original sha1 id" argument
63 +# and outputs a "rewritten sha1 id" if the commit has been already
64 +# rewritten, fails otherwise; the 'map' function can return several
65 +# ids on separate lines if your commit filter emitted multiple commits
66 +# (see below).
67 +#
68 +# --env-filter COMMAND:: The filter for modifying environment
69 +# This is the filter for modifying the environment in which
70 +# the commit will be performed. Specifically, you might want
71 +# to rewrite the author/committer name/email/time environment
72 +# variables (see `git-commit` for details). Do not forget to
73 +# re-export the variables.
74 +#
75 +# --tree-filter COMMAND:: The filter for rewriting tree (and its contents)
76 +# This is the filter for rewriting the tree and its contents.
77 +# The COMMAND argument is evaluated in shell with the working
78 +# directory set to the root of the checked out tree. The new tree
79 +# is then used as-is (new files are auto-added, disappeared files
80 +# are auto-removed - .gitignore files nor any other ignore rules
81 +# HAVE NO EFFECT!).
82 +#
83 +# --index-filter COMMAND:: The filter for rewriting index
84 +# This is the filter for rewriting the Git's directory index.
85 +# It is similar to the tree filter but does not check out the
86 +# tree, which makes it much faster. However, you must use the
87 +# lowlevel Git index manipulation commands to do your work.
88 +#
89 +# --parent-filter COMMAND:: The filter for rewriting parents
90 +# This is the filter for rewriting the commit's parent list.
91 +# It will receive the parent string on stdin and shall output
92 +# the new parent string on stdout. The parent string is in
93 +# format accepted by `git-commit-tree`: empty for initial
94 +# commit, "-p parent" for a normal commit and "-p parent1
95 +# -p parent2 -p parent3 ..." for a merge commit.
96 +#
97 +# --msg-filter COMMAND:: The filter for rewriting commit message
98 +# This is the filter for rewriting the commit messages.
99 +# The COMMAND argument is evaluated in shell with the original
100 +# commit message on standard input; its standard output is
101 +# is used as the new commit message.
102 +#
103 +# --commit-filter COMMAND:: The filter for performing the commit
104 +# If this filter is passed, it will be called instead of the
105 +# `git-commit-tree` command, with those arguments:
106 +#
107 +# TREE_ID [-p PARENT_COMMIT_ID]...
108 +#
109 +# and the log message on stdin. The commit id is expected on
110 +# stdout. As a special extension, the commit filter may emit
111 +# multiple commit ids; in that case, all of them will be used
112 +# as parents instead of the original commit in further commits.
113 +#
114 +# --tag-name-filter COMMAND:: The filter for rewriting tag names.
115 +# If this filter is passed, it will be called for every tag ref
116 +# that points to a rewritten object (or to a tag object which
117 +# points to a rewritten object). The original tag name is passed
118 +# via standard input, and the new tag name is expected on standard
119 +# output.
120 +#
121 +# The original tags are not deleted, but can be overwritten;
122 +# use "--tag-name-filter=cat" to simply update the tags. In this
123 +# case, be very careful and make sure you have the old tags
124 +# backed up in case the conversion has run afoul.
125 +#
126 +# Note that there is currently no support for proper rewriting of
127 +# tag objects; in layman terms, if the tag has a message or signature
128 +# attached, the rewritten tag won't have it. Sorry. (It is by
129 +# definition impossible to preserve signatures at any rate, though.)
130 +#
131 +# EXAMPLE USAGE
132 +# -------------
133 +# Suppose you want to remove a file (containing confidential information
134 +# or copyright violation) from all commits:
135 +#
136 +# git-filter-branch --tree-filter 'rm filename' newbranch
137 +#
138 +# A significantly faster version:
139 +#
140 +# git-filter-branch --index-filter 'git-update-index --remove filename' newbranch
141 +#
142 +# Now, you will get the rewritten history saved in the branch 'newbranch'
143 +# (your current branch is left untouched).
144 +#
145 +# To "etch-graft" a commit to the revision history (set a commit to be
146 +# the parent of the current initial commit and propagate that):
147 +#
148 +# git-filter-branch --parent-filter sed\ 's/^$/-p graftcommitid/' newbranch
149 +#
150 +# (if the parent string is empty - therefore we are dealing with the
151 +# initial commit - add graftcommit as a parent). Note that this assumes
152 +# history with a single root (that is, no git-merge without common ancestors
153 +# happened). If this is not the case, use:
154 +#
155 +# git-filter-branch --parent-filter 'cat; [ "$GIT_COMMIT" = "COMMIT" ] && echo "-p GRAFTCOMMIT"' newbranch
156 +#
157 +# To remove commits authored by "Darl McBribe" from the history:
158 +#
159 +# git-filter-branch --commit-filter 'if [ "$GIT_AUTHOR_NAME" = "Darl McBribe" ]; then shift; while [ -n "$1" ]; do shift; echo "$1"; shift; done; else git-commit-tree "$@"; fi' newbranch
160 +#
161 +# (the shift magic first throws away the tree id and then the -p
162 +# parameters). Note that this handles merges properly! In case Darl
163 +# committed a merge between P1 and P2, it will be propagated properly
164 +# and all children of the merge will become merge commits with P1,P2
165 +# as their parents instead of the merge commit.
166 +#
167 +# To restrict rewriting to only part of the history, use -r or -k or both.
168 +# Consider this history:
169 +#
170 +# D--E--F--G--H
171 +# / /
172 +# A--B-----C
173 +#
174 +# To rewrite only commits F,G,H, use:
175 +#
176 +# git-filter-branch -r F ...
177 +#
178 +# To rewrite commits E,F,G,H, use one of these:
179 +#
180 +# git-filter-branch -r E -k C ...
181 +# git-filter-branch -k D -k C ...
182 +
183 +# Testsuite: TODO
184 +
185 +set -e
186 +
187 +USAGE="git-filter-branch [-d TEMPDIR] [-r STARTREV]... [-k KEEPREV]... [-s SRCBRANCH] [FILTERS] DESTBRANCH"
188 +. git-sh-setup
189 +
190 +map()
191 +{
192 + [ -r "$workdir/../map/$1" ] || return 1
193 + cat "$workdir/../map/$1"
194 +}
195 +
196 +# When piped a commit, output a script to set the ident of either
197 +# "author" or "committer
198 +
199 +set_ident () {
200 + lid="$(echo "$1" | tr "A-Z" "a-z")"
201 + uid="$(echo "$1" | tr "a-z" "A-Z")"
202 + pick_id_script='
203 + /^'$lid' /{
204 + s/'\''/'\''\\'\'\''/g
205 + h
206 + s/^'$lid' \([^<]*\) <[^>]*> .*$/\1/
207 + s/'\''/'\''\'\'\''/g
208 + s/.*/export GIT_'$uid'_NAME='\''&'\''/p
209 +
210 + g
211 + s/^'$lid' [^<]* <\([^>]*\)> .*$/\1/
212 + s/'\''/'\''\'\'\''/g
213 + s/.*/export GIT_'$uid'_EMAIL='\''&'\''/p
214 +
215 + g
216 + s/^'$lid' [^<]* <[^>]*> \(.*\)$/\1/
217 + s/'\''/'\''\'\'\''/g
218 + s/.*/export GIT_'$uid'_DATE='\''&'\''/p
219 +
220 + q
221 + }
222 + '
223 +
224 + LANG=C LC_ALL=C sed -ne "$pick_id_script"
225 + # Ensure non-empty id name.
226 + echo "[ -n \"\$GIT_${uid}_NAME\" ] || export GIT_${uid}_NAME=\"\${GIT_${uid}_EMAIL%%@*}\""
227 +}
228 +
229 +# list all parent's object names for a given commit
230 +get_parents () {
231 + git-rev-list -1 --parents "$1" | sed "s/^[0-9a-f]*//"
232 +}
233 +
234 +tempdir=.git-rewrite
235 +unchanged=" "
236 +filter_env=
237 +filter_tree=
238 +filter_index=
239 +filter_parent=
240 +filter_msg=cat
241 +filter_commit='git-commit-tree "$@"'
242 +filter_tag_name=
243 +srcbranch=HEAD
244 +while case "$#" in 0) usage;; esac
245 +do
246 + case "$1" in
247 + --)
248 + shift
249 + break
250 + ;;
251 + -*)
252 + ;;
253 + *)
254 + break;
255 + esac
256 +
257 + # all switches take one argument
258 + ARG="$1"
259 + case "$#" in 1) usage ;; esac
260 + shift
261 + OPTARG="$1"
262 + shift
263 +
264 + case "$ARG" in
265 + -d)
266 + tempdir="$OPTARG"
267 + ;;
268 + -r)
269 + unchanged="$(get_parents "$OPTARG") $unchanged"
270 + ;;
271 + -k)
272 + unchanged="$(git-rev-parse "$OPTARG"^{commit}) $unchanged"
273 + ;;
274 + --env-filter)
275 + filter_env="$OPTARG"
276 + ;;
277 + --tree-filter)
278 + filter_tree="$OPTARG"
279 + ;;
280 + --index-filter)
281 + filter_index="$OPTARG"
282 + ;;
283 + --parent-filter)
284 + filter_parent="$OPTARG"
285 + ;;
286 + --msg-filter)
287 + filter_msg="$OPTARG"
288 + ;;
289 + --commit-filter)
290 + filter_commit="$OPTARG"
291 + ;;
292 + --tag-name-filter)
293 + filter_tag_name="$OPTARG"
294 + ;;
295 + -s)
296 + srcbranch="$OPTARG"
297 + ;;
298 + *)
299 + usage
300 + ;;
301 + esac
302 +done
303 +
304 +dstbranch="$1"
305 +test -n "$dstbranch" || die "missing branch name"
306 +git-show-ref "refs/heads/$dstbranch" 2> /dev/null &&
307 + die "branch $dstbranch already exists"
308 +
309 +test ! -e "$tempdir" || die "$tempdir already exists, please remove it"
310 +mkdir -p "$tempdir/t"
311 +cd "$tempdir/t"
312 +workdir="$(pwd)"
313 +
314 +case "$GIT_DIR" in
315 +/*)
316 + ;;
317 +*)
318 + export GIT_DIR="$(pwd)/../../$GIT_DIR"
319 + ;;
320 +esac
321 +
322 +export GIT_INDEX_FILE="$(pwd)/../index"
323 +git-read-tree # seed the index file
324 +
325 +ret=0
326 +
327 +
328 +mkdir ../map # map old->new commit ids for rewriting parents
329 +
330 +# seed with identity mappings for the parents where we start off
331 +for commit in $unchanged; do
332 + echo $commit > ../map/$commit
333 +done
334 +
335 +git-rev-list --reverse --topo-order $srcbranch --not $unchanged >../revs
336 +commits=$(cat ../revs | wc -l | tr -d " ")
337 +
338 +test $commits -eq 0 && die "Found nothing to rewrite"
339 +
340 +i=0
341 +while read commit; do
342 + i=$((i+1))
343 + printf "$commit ($i/$commits) "
344 +
345 + git-read-tree -i -m $commit
346 +
347 + export GIT_COMMIT=$commit
348 + git-cat-file commit "$commit" >../commit
349 +
350 + eval "$(set_ident AUTHOR <../commit)"
351 + eval "$(set_ident COMMITTER <../commit)"
352 + eval "$filter_env"
353 +
354 + if [ "$filter_tree" ]; then
355 + git-checkout-index -f -u -a
356 + # files that $commit removed are now still in the working tree;
357 + # remove them, else they would be added again
358 + git-ls-files -z --others | xargs -0 rm -f
359 + eval "$filter_tree"
360 + git-diff-index -r $commit | cut -f 2- | tr '\n' '\0' | \
361 + xargs -0 git-update-index --add --replace --remove
362 + git-ls-files -z --others | \
363 + xargs -0 git-update-index --add --replace --remove
364 + fi
365 +
366 + eval "$filter_index"
367 +
368 + parentstr=
369 + for parent in $(get_parents $commit); do
370 + if [ -r "../map/$parent" ]; then
371 + for reparent in $(cat "../map/$parent"); do
372 + parentstr="$parentstr -p $reparent"
373 + done
374 + else
375 + die "assertion failed: parent $parent for commit $commit not found in rewritten ones"
376 + fi
377 + done
378 + if [ "$filter_parent" ]; then
379 + parentstr="$(echo "$parentstr" | eval "$filter_parent")"
380 + fi
381 +
382 + sed -e '1,/^$/d' <../commit | \
383 + eval "$filter_msg" | \
384 + sh -c "$filter_commit" git-commit-tree $(git-write-tree) $parentstr | \
385 + tee ../map/$commit
386 +done <../revs
387 +
388 +git-update-ref refs/heads/"$dstbranch" $(head -n 1 ../map/$(tail -n 1 ../revs))
389 +if [ "$(cat ../map/$(tail -n 1 ../revs) | wc -l)" -gt 1 ]; then
390 + echo "WARNING: Your commit filter caused the head commit to expand to several rewritten commits. Only the first such commit was recorded as the current $dstbranch head but you will need to resolve the situation now (probably by manually merging the other commits). These are all the commits:" >&2
391 + sed 's/^/ /' ../map/$(tail -n 1 ../revs) >&2
392 + ret=1
393 +fi
394 +
395 +if [ "$filter_tag_name" ]; then
396 + git-for-each-ref --format='%(objectname) %(objecttype) %(refname)' refs/tags |
397 + while read sha1 type ref; do
398 + ref="${ref#refs/tags/}"
399 + # XXX: Rewrite tagged trees as well?
400 + if [ "$type" != "commit" -a "$type" != "tag" ]; then
401 + continue;
402 + fi
403 +
404 + if [ "$type" = "tag" ]; then
405 + # Dereference to a commit
406 + sha1t="$sha1"
407 + sha1="$(git-rev-parse "$sha1"^{commit} 2>/dev/null)" || continue
408 + fi
409 +
410 + [ -f "../map/$sha1" ] || continue
411 + new_sha1="$(cat "../map/$sha1")"
412 + export GIT_COMMIT="$sha1"
413 + new_ref="$(echo "$ref" | eval "$filter_tag_name")"
414 +
415 + echo "$ref -> $new_ref ($sha1 -> $new_sha1)"
416 +
417 + if [ "$type" = "tag" ]; then
418 + # Warn that we are not rewriting the tag object itself.
419 + warn "unreferencing tag object $sha1t"
420 + fi
421 +
422 + git-update-ref "refs/tags/$new_ref" "$new_sha1"
423 + done
424 +fi
425 +
426 +cd ../..
427 +rm -rf "$tempdir"
428 +echo "Rewritten history saved to the $dstbranch branch"
429 +
430 +exit $ret
t/t7003-filter-branch.sh new
+47
@@ -0,0 +1,47 @@
1 +#!/bin/sh
2 +
3 +test_description='git-filter-branch'
4 +. ./test-lib.sh
5 +
6 +make_commit () {
7 + lower=$(echo $1 | tr A-Z a-z)
8 + echo $lower > $lower
9 + git add $lower
10 + git commit -m $1
11 + git tag $1
12 +}
13 +
14 +test_expect_success 'setup' '
15 + make_commit A
16 + make_commit B
17 + git checkout -b branch B
18 + make_commit D
19 + make_commit E
20 + git checkout master
21 + make_commit C
22 + git checkout branch
23 + git merge C
24 + git tag F
25 + make_commit G
26 + make_commit H
27 +'
28 +
29 +H=$(git-rev-parse H)
30 +
31 +test_expect_success 'rewrite identically' '
32 + git-filter-branch H2
33 +'
34 +
35 +test_expect_success 'result is really identical' '
36 + test $H = $(git-rev-parse H2)
37 +'
38 +
39 +test_expect_success 'rewrite, renaming a specific file' '
40 + git-filter-branch --tree-filter "mv d doh || :" H3
41 +'
42 +
43 +test_expect_success 'test that the file was renamed' '
44 + test d = $(git show H3:doh)
45 +'
46 +
47 +test_done