@matheusbsbs / matheus-bagliano / commits / dfd05e38f0
Fork 0

filter-branch: Big syntax change; support rewriting multiple refs

We used to take the first non-option argument as the name for the new branch. This syntax is not extensible to support rewriting more than just HEAD. Instead, we now have the following syntax: git filter-branch [<filter options>...] [<rev-list options>] All positive refs given in <rev-list options> are rewritten. Yes, in-place. If a ref was changed, the original head is stored in refs/original/$ref now, for your inspecting pleasure, in addition to the reflogs (since it is easier to inspect "git show-ref | grep original" than to inspect all the reflogs). This commit also adds the --force option to remove .git-rewrite/ and all refs from refs/original/ before filtering. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Johannes Schindelin committed Jul 23, 2007 at 18:34 UTC dfd05e38f07f76505158399cc433b0b1870a769a
3 files changed +182 -60
Documentation/git-filter-branch.txt
+28 -23
@@ -12,7 +12,7 @@ SYNOPSIS
12 [--index-filter <command>] [--parent-filter <command>]
13 [--msg-filter <command>] [--commit-filter <command>]
14 [--tag-name-filter <command>] [--subdirectory-filter <directory>]
15 - [-d <directory>] <new-branch-name> [<rev-list options>...]
15 + [-d <directory>] [-f | --force] [<rev-list options>...]
16
17 DESCRIPTION
18 -----------
@@ -26,10 +26,9 @@ information) will be preserved.
26 The command takes the new branch name as a mandatory argument and
27 the filters as optional arguments. If you specify no filters, the
28 commits will be recommitted without any changes, which would normally
29 -have no effect and result in the new branch pointing to the same
30 -branch as your current branch. Nevertheless, this may be useful in
31 -the future for compensating for some git bugs or such, therefore
32 -such a usage is permitted.
29 +have no effect. Nevertheless, this may be useful in the future for
30 +compensating for some git bugs or such, therefore such a usage is
31 +permitted.
32
33 *WARNING*! The rewritten history will have different object names for all
34 the objects and will not converge with the original branch. You will not
@@ -38,8 +37,9 @@ original branch. Please do not use this command if you do not know the
37 full implications, and avoid using it anyway, if a simple single commit
38 would suffice to fix your problem.
39
41 -Always verify that the rewritten version is correct before disposing
42 -the original branch.
40 +Always verify that the rewritten version is correct: The original refs,
41 +if different from the rewritten ones, will be stored in the namespace
42 +'refs/original/'.
43
44 Note that since this operation is extensively I/O expensive, it might
45 be a good idea to redirect the temporary directory off-disk, e.g. on
@@ -142,6 +142,11 @@ definition impossible to preserve signatures at any rate.)
142 does this in the '.git-rewrite/' directory but you can override
143 that choice by this parameter.
144
145 +-f\|--force::
146 + `git filter-branch` refuses to start with an existing temporary
147 + directory or when there are already refs starting with
148 + 'refs/original/', unless forced.
149 +
150 <rev-list-options>::
151 When options are given after the new branch name, they will
152 be passed to gitlink:git-rev-list[1]. Only commits in the resulting
@@ -156,14 +161,14 @@ Suppose you want to remove a file (containing confidential information
161 or copyright violation) from all commits:
162
163 -------------------------------------------------------
159 -git filter-branch --tree-filter 'rm filename' newbranch
164 +git filter-branch --tree-filter 'rm filename' HEAD
165 -------------------------------------------------------
166
167 A significantly faster version:
168
164 --------------------------------------------------------------------------------
165 -git filter-branch --index-filter 'git update-index --remove filename' newbranch
166 --------------------------------------------------------------------------------
169 +--------------------------------------------------------------------------
170 +git filter-branch --index-filter 'git update-index --remove filename' HEAD
171 +--------------------------------------------------------------------------
172
173 Now, you will get the rewritten history saved in the branch 'newbranch'
174 (your current branch is left untouched).
@@ -172,25 +177,25 @@ To set a commit (which typically is at the tip of another
177 history) to be the parent of the current initial commit, in
178 order to paste the other history behind the current history:
179
175 -------------------------------------------------------------------------
176 -git filter-branch --parent-filter 'sed "s/^\$/-p <graft-id>/"' newbranch
177 -------------------------------------------------------------------------
180 +-------------------------------------------------------------------
181 +git filter-branch --parent-filter 'sed "s/^\$/-p <graft-id>/"' HEAD
182 +-------------------------------------------------------------------
183
184 (if the parent string is empty - therefore we are dealing with the
185 initial commit - add graftcommit as a parent). Note that this assumes
186 history with a single root (that is, no merge without common ancestors
187 happened). If this is not the case, use:
188
184 --------------------------------------------------------------------------------
189 +--------------------------------------------------------------------------
190 git filter-branch --parent-filter \
186 - 'cat; test $GIT_COMMIT = <commit-id> && echo "-p <graft-id>"' newbranch
187 --------------------------------------------------------------------------------
191 + 'cat; test $GIT_COMMIT = <commit-id> && echo "-p <graft-id>"' HEAD
192 +--------------------------------------------------------------------------
193
194 or even simpler:
195
196 -----------------------------------------------
197 echo "$commit-id $graft-id" >> .git/info/grafts
193 -git filter-branch newbranch $graft-id..
198 +git filter-branch $graft-id..HEAD
199 -----------------------------------------------
200
201 To remove commits authored by "Darl McBribe" from the history:
@@ -208,7 +213,7 @@ git filter-branch --commit-filter '
213 done;
214 else
215 git commit-tree "$@";
211 - fi' newbranch
216 + fi' HEAD
217 ------------------------------------------------------------------------------
218
219 The shift magic first throws away the tree id and then the -p
@@ -238,14 +243,14 @@ A--B-----C
243 To rewrite only commits D,E,F,G,H, but leave A, B and C alone, use:
244
245 --------------------------------
241 -git filter-branch ... new-H C..H
246 +git filter-branch ... C..H
247 --------------------------------
248
249 To rewrite commits E,F,G,H, use one of these:
250
251 ----------------------------------------
247 -git filter-branch ... new-H C..H --not D
248 -git filter-branch ... new-H D..H --not C
252 +git filter-branch ... C..H --not D
253 +git filter-branch ... D..H --not C
254 ----------------------------------------
255
256 To move the whole tree into a subdirectory, or remove it from there:
@@ -255,7 +260,7 @@ git filter-branch --index-filter \
260 'git ls-files -s | sed "s-\t-&newsubdir/-" |
261 GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
262 git update-index --index-info &&
258 - mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' directorymoved
263 + mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' HEAD
264 ---------------------------------------------------------------
265
266
git-filter-branch.sh
+128 -22
@@ -78,6 +78,8 @@ filter_msg=cat
78 filter_commit='git commit-tree "$@"'
79 filter_tag_name=
80 filter_subdir=
81 +orig_namespace=refs/original/
82 +force=
83 while case "$#" in 0) usage;; esac
84 do
85 case "$1" in
@@ -85,6 +87,11 @@ do
87 shift
88 break
89 ;;
90 + --force|-f)
91 + shift
92 + force=t
93 + continue
94 + ;;
95 -*)
96 ;;
97 *)
@@ -126,24 +133,43 @@ do
133 --subdirectory-filter)
134 filter_subdir="$OPTARG"
135 ;;
136 + --original)
137 + orig_namespace="$OPTARG"
138 + ;;
139 *)
140 usage
141 ;;
142 esac
143 done
144
135 -dstbranch="$1"
136 -shift
137 -test -n "$dstbranch" || die "missing branch name"
138 -git show-ref "refs/heads/$dstbranch" 2> /dev/null &&
139 - die "branch $dstbranch already exists"
140 -
141 -test ! -e "$tempdir" || die "$tempdir already exists, please remove it"
145 +case "$force" in
146 +t)
147 + rm -rf "$tempdir"
148 +;;
149 +'')
150 + test -d "$tempdir" &&
151 + die "$tempdir already exists, please remove it"
152 +esac
153 mkdir -p "$tempdir/t" &&
154 +tempdir="$(cd "$tempdir"; pwd)" &&
155 cd "$tempdir/t" &&
156 workdir="$(pwd)" ||
157 die ""
158
159 +# Make sure refs/original is empty
160 +git for-each-ref > "$tempdir"/backup-refs
161 +while read sha1 type name
162 +do
163 + case "$force,$name" in
164 + ,$orig_namespace*)
165 + die "Namespace $orig_namespace not empty"
166 + ;;
167 + t,$orig_namespace*)
168 + git update-ref -d "$name" $sha1
169 + ;;
170 + esac
171 +done < "$tempdir"/backup-refs
172 +
173 case "$GIT_DIR" in
174 /*)
175 ;;
@@ -153,6 +179,29 @@ case "$GIT_DIR" in
179 esac
180 export GIT_DIR GIT_WORK_TREE=.
181
182 +# These refs should be updated if their heads were rewritten
183 +
184 +git rev-parse --revs-only --symbolic "$@" |
185 +while read ref
186 +do
187 + # normalize ref
188 + case "$ref" in
189 + HEAD)
190 + ref="$(git symbolic-ref "$ref")"
191 + ;;
192 + refs/*)
193 + ;;
194 + *)
195 + ref="$(git for-each-ref --format='%(refname)' |
196 + grep /"$ref")"
197 + esac
198 +
199 + git check-ref-format "$ref" && echo "$ref"
200 +done > "$tempdir"/heads
201 +
202 +test -s "$tempdir"/heads ||
203 + die "Which ref do you want to rewrite?"
204 +
205 export GIT_INDEX_FILE="$(pwd)/../index"
206 git read-tree || die "Could not seed the index"
207
@@ -174,6 +223,8 @@ commits=$(wc -l <../revs | tr -d " ")
223
224 test $commits -eq 0 && die "Found nothing to rewrite"
225
226 +# Rewrite the commits
227 +
228 i=0
229 while read commit parents; do
230 i=$(($i+1))
@@ -234,22 +285,75 @@ while read commit parents; do
285 $(git write-tree) $parentstr < ../message > ../map/$commit
286 done <../revs
287
237 -src_head=$(tail -n 1 ../revs | sed -e 's/ .*//')
238 -target_head=$(head -n 1 ../map/$src_head)
239 -case "$target_head" in
240 -'')
241 - echo Nothing rewritten
288 +# In case of a subdirectory filter, it is possible that a specified head
289 +# is not in the set of rewritten commits, because it was pruned by the
290 +# revision walker. Fix it by mapping these heads to the next rewritten
291 +# ancestor(s), i.e. the boundaries in the set of rewritten commits.
292 +
293 +# NEEDSWORK: we should sort the unmapped refs topologically first
294 +while read ref
295 +do
296 + sha1=$(git rev-parse "$ref"^0)
297 + test -f "$workdir"/../map/$sha1 && continue
298 + # Assign the boundarie(s) in the set of rewritten commits
299 + # as the replacement commit(s).
300 + # (This would look a bit nicer if --not --stdin worked.)
301 + for p in $((cd "$workdir"/../map; ls | sed "s/^/^/") |
302 + git rev-list $ref --boundary --stdin |
303 + sed -n "s/^-//p")
304 + do
305 + map $p >> "$workdir"/../map/$sha1
306 + done
307 +done < "$tempdir"/heads
308 +
309 +# Finally update the refs
310 +
311 +_x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
312 +_x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
313 +count=0
314 +echo
315 +while read ref
316 +do
317 + # avoid rewriting a ref twice
318 + test -f "$orig_namespace$ref" && continue
319 +
320 + sha1=$(git rev-parse "$ref"^0)
321 + rewritten=$(map $sha1)
322 +
323 + test $sha1 = "$rewritten" &&
324 + warn "WARNING: Ref '$ref' is unchanged" &&
325 + continue
326 +
327 + case "$rewritten" in
328 + '')
329 + echo "Ref '$ref' was deleted"
330 + git update-ref -m "filter-branch: delete" -d "$ref" $sha1 ||
331 + die "Could not delete $ref"
332 ;;
243 -*)
244 - git update-ref refs/heads/"$dstbranch" $target_head ||
245 - die "Could not update $dstbranch with $target_head"
246 - if [ $(wc -l <../map/$src_head) -gt 1 ]; then
247 - echo "WARNING: Your commit filter caused the head commit to expand to several rewritten commits. Only the first such commit was recorded as the current $dstbranch head but you will need to resolve the situation now (probably by manually merging the other commits). These are all the commits:" >&2
248 - sed 's/^/ /' ../map/$src_head >&2
249 - ret=1
250 - fi
333 + $_x40)
334 + echo "Ref '$ref' was rewritten"
335 + git update-ref -m "filter-branch: rewrite" \
336 + "$ref" $rewritten $sha1 ||
337 + die "Could not rewrite $ref"
338 ;;
252 -esac
339 + *)
340 + # NEEDSWORK: possibly add -Werror, making this an error
341 + warn "WARNING: '$ref' was rewritten into multiple commits:"
342 + warn "$rewritten"
343 + warn "WARNING: Ref '$ref' points to the first one now."
344 + rewritten=$(echo "$rewritten" | head -n 1)
345 + git update-ref -m "filter-branch: rewrite to first" \
346 + "$ref" $rewritten $sha1 ||
347 + die "Could not rewrite $ref"
348 + ;;
349 + esac
350 + git update-ref -m "filter-branch: backup" "$orig_namespace$ref" $sha1
351 + count=$(($count+1))
352 +done < "$tempdir"/heads
353 +
354 +# TODO: This should possibly go, with the semantics that all positive given
355 +# refs are updated, and their original heads stored in refs/original/
356 +# Filter tags
357
358 if [ "$filter_tag_name" ]; then
359 git for-each-ref --format='%(objectname) %(objecttype) %(refname)' refs/tags |
@@ -286,6 +390,8 @@ fi
390
391 cd ../..
392 rm -rf "$tempdir"
289 -printf "\nRewritten history saved to the $dstbranch branch\n"
393 +echo
394 +test $count -gt 0 && echo "These refs were rewritten:"
395 +git show-ref | grep ^"$orig_namespace"
396
397 exit $ret
t/t7003-filter-branch.sh
+26 -15
@@ -30,24 +30,24 @@ test_expect_success 'setup' '
30 H=$(git rev-parse H)
31
32 test_expect_success 'rewrite identically' '
33 - git-filter-branch H2
33 + git-filter-branch branch
34 '
35 -
35 test_expect_success 'result is really identical' '
37 - test $H = $(git rev-parse H2)
36 + test $H = $(git rev-parse HEAD)
37 '
38
39 test_expect_success 'rewrite, renaming a specific file' '
41 - git-filter-branch --tree-filter "mv d doh || :" H3
40 + git-filter-branch -f --tree-filter "mv d doh || :" HEAD
41 '
42
43 test_expect_success 'test that the file was renamed' '
45 - test d = $(git show H3:doh)
44 + test d = $(git show HEAD:doh)
45 '
46
48 -git tag oldD H3~4
47 +git tag oldD HEAD~4
48 test_expect_success 'rewrite one branch, keeping a side branch' '
50 - git-filter-branch --tree-filter "mv b boh || :" modD D..oldD
49 + git branch modD oldD &&
50 + git-filter-branch -f --tree-filter "mv b boh || :" D..modD
51 '
52
53 test_expect_success 'common ancestor is still common (unchanged)' '
@@ -69,7 +69,8 @@ test_expect_success 'filter subdirectory only' '
69 git rm a &&
70 test_tick &&
71 git commit -m "again not subdir" &&
72 - git-filter-branch --subdirectory-filter subdir sub
72 + git branch sub &&
73 + git-filter-branch -f --subdirectory-filter subdir refs/heads/sub
74 '
75
76 test_expect_success 'subdirectory filter result looks okay' '
@@ -89,7 +90,8 @@ test_expect_success 'setup and filter history that requires --full-history' '
90 test_tick &&
91 git commit -m "again subdir on master" &&
92 git merge branch &&
92 - git-filter-branch --subdirectory-filter subdir sub-master
93 + git branch sub-master &&
94 + git-filter-branch -f --subdirectory-filter subdir sub-master
95 '
96
97 test_expect_success 'subdirectory filter result looks okay' '
@@ -100,7 +102,8 @@ test_expect_success 'subdirectory filter result looks okay' '
102 '
103
104 test_expect_success 'use index-filter to move into a subdirectory' '
103 - git-filter-branch --index-filter \
105 + git branch directorymoved &&
106 + git-filter-branch -f --index-filter \
107 "git ls-files -s | sed \"s-\\t-&newsubdir/-\" |
108 GIT_INDEX_FILE=\$GIT_INDEX_FILE.new \
109 git update-index --index-info &&
@@ -108,9 +111,10 @@ test_expect_success 'use index-filter to move into a subdirectory' '
111 test -z "$(git diff HEAD directorymoved:newsubdir)"'
112
113 test_expect_success 'stops when msg filter fails' '
111 - ! git-filter-branch --msg-filter false nonono &&
112 - rm -rf .git-rewrite &&
113 - ! git rev-parse nonono
114 + old=$(git rev-parse HEAD) &&
115 + ! git-filter-branch -f --msg-filter false &&
116 + test $old = $(git rev-parse HEAD) &&
117 + rm -rf .git-rewrite
118 '
119
120 test_expect_success 'author information is preserved' '
@@ -118,7 +122,8 @@ test_expect_success 'author information is preserved' '
122 git add i &&
123 test_tick &&
124 GIT_AUTHOR_NAME="B V Uips" git commit -m bvuips &&
121 - git-filter-branch --msg-filter "cat; \
125 + git branch preserved-author &&
126 + git-filter-branch -f --msg-filter "cat; \
127 test \$GIT_COMMIT != $(git rev-parse master) || \
128 echo Hallo" \
129 preserved-author &&
@@ -129,7 +134,8 @@ test_expect_success "remove a certain author's commits" '
134 echo i > i &&
135 test_tick &&
136 git commit -m i i &&
132 - git-filter-branch --commit-filter "\
137 + git branch removed-author &&
138 + git-filter-branch -f --commit-filter "\
139 if [ \"\$GIT_AUTHOR_NAME\" = \"B V Uips\" ];\
140 then\
141 shift;\
@@ -148,4 +154,9 @@ test_expect_success "remove a certain author's commits" '
154 test 0 = $(git rev-list --author="B V Uips" removed-author | wc -l)
155 '
156
157 +test_expect_success 'barf on invalid name' '
158 + ! git filter-branch -f master xy-problem &&
159 + ! git filter-branch -f HEAD^
160 +'
161 +
162 test_done