write_index: optionally allow broken null sha1s

Commit 4337b58 (do not write null sha1s to on-disk index, 2012-07-28) added a safety check preventing git from writing null sha1s into the index. The intent was to catch errors in other parts of the code that might let such an entry slip into the index (or worse, a tree). Some existing repositories may have invalid trees that contain null sha1s already, though. Until 4337b58, a common way to clean this up would be to use git-filter-branch's index-filter to repair such broken entries. That now fails when filter-branch tries to write out the index. Introduce a GIT_ALLOW_NULL_SHA1 environment variable to relax this check and make it easier to recover from such a history. It is tempting to not involve filter-branch in this commit at all, and instead require the user to manually invoke GIT_ALLOW_NULL_SHA1=1 git filter-branch ... to perform an index-filter on a history with trees with null sha1s. That would be slightly safer, but requires some specialized knowledge from the user. So let's set the GIT_ALLOW_NULL_SHA1 variable automatically when checking out the to-be-filtered trees. Advice on using filter-branch to remove such entries already exists on places like stackoverflow, and this patch makes it Just Work again on recent versions of git. Further commands that touch the index will still notice and fail, unless they actually remove the broken entries. A filter-branch whose filters do not touch the index at all will not error out (since we complain of the null sha1 only on writing, not when making a tree out of the index), but this is acceptable, as we still print a loud warning, so the problem is unlikely to go unnoticed. Signed-off-by: Jeff King <peff@peff.net> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Jeff King committed Aug 27, 2013 at 16:41 UTC 83bd7437ca6acbee0db431fc8ec7cf823d9459ec
3 files changed +63 -4
git-filter-branch.sh
+3 -2
@@ -283,11 +283,12 @@ while read commit parents; do
283
284 case "$filter_subdir" in
285 "")
286 - git read-tree -i -m $commit
286 + GIT_ALLOW_NULL_SHA1=1 git read-tree -i -m $commit
287 ;;
288 *)
289 # The commit may not have the subdirectory at all
290 - err=$(git read-tree -i -m $commit:"$filter_subdir" 2>&1) || {
290 + err=$(GIT_ALLOW_NULL_SHA1=1 \
291 + git read-tree -i -m $commit:"$filter_subdir" 2>&1) || {
292 if ! git rev-parse -q --verify $commit:"$filter_subdir"
293 then
294 rm -f "$GIT_INDEX_FILE"
read-cache.c
+11 -2
@@ -1800,8 +1800,17 @@ int write_index(struct index_state *istate, int newfd)
1800 continue;
1801 if (!ce_uptodate(ce) && is_racy_timestamp(istate, ce))
1802 ce_smudge_racily_clean_entry(ce);
1803 - if (is_null_sha1(ce->sha1))
1804 - return error("cache entry has null sha1: %s", ce->name);
1803 + if (is_null_sha1(ce->sha1)) {
1804 + static const char msg[] = "cache entry has null sha1: %s";
1805 + static int allow = -1;
1806 +
1807 + if (allow < 0)
1808 + allow = git_env_bool("GIT_ALLOW_NULL_SHA1", 0);
1809 + if (allow)
1810 + warning(msg, ce->name);
1811 + else
1812 + return error(msg, ce->name);
1813 + }
1814 if (ce_write_entry(&c, newfd, ce, previous_name) < 0)
1815 return -1;
1816 }
t/t7009-filter-branch-null-sha1.sh new
+49
@@ -0,0 +1,49 @@
1 +#!/bin/sh
2 +
3 +test_description='filter-branch removal of trees with null sha1'
4 +. ./test-lib.sh
5 +
6 +test_expect_success 'setup: base commits' '
7 + test_commit one &&
8 + test_commit two &&
9 + test_commit three
10 +'
11 +
12 +test_expect_success 'setup: a commit with a bogus null sha1 in the tree' '
13 + {
14 + git ls-tree HEAD &&
15 + printf "160000 commit $_z40\\tbroken\\n"
16 + } >broken-tree
17 + echo "add broken entry" >msg &&
18 +
19 + tree=$(git mktree <broken-tree) &&
20 + test_tick &&
21 + commit=$(git commit-tree $tree -p HEAD <msg) &&
22 + git update-ref HEAD "$commit"
23 +'
24 +
25 +# we have to make one more commit on top removing the broken
26 +# entry, since otherwise our index does not match HEAD (and filter-branch will
27 +# complain). We could make the index match HEAD, but doing so would involve
28 +# writing a null sha1 into the index.
29 +test_expect_success 'setup: bring HEAD and index in sync' '
30 + test_tick &&
31 + git commit -a -m "back to normal"
32 +'
33 +
34 +test_expect_success 'filter commands are still checked' '
35 + test_must_fail git filter-branch \
36 + --force --prune-empty \
37 + --index-filter "git rm --cached --ignore-unmatch three.t"
38 +'
39 +
40 +test_expect_success 'removing the broken entry works' '
41 + echo three >expect &&
42 + git filter-branch \
43 + --force --prune-empty \
44 + --index-filter "git rm --cached --ignore-unmatch broken" &&
45 + git log -1 --format=%s >actual &&
46 + test_cmp expect actual
47 +'
48 +
49 +test_done