fast-(import|export): improve on commit signature output format
A recent commit, d9cb0e6ff8 (fast-export, fast-import: add support for signed-commits, 2025-03-10), added support for signed commits to fast-export and fast-import. When a signed commit is processed, fast-export can output either "gpgsig sha1" or "gpgsig sha256" depending on whether the signed commit uses the SHA-1 or SHA-256 Git object format. However, this implementation has a number of limitations: - the output format was not properly described in the documentation, - the output format is not very informative as it doesn't even say if the signature is an OpenPGP, an SSH, or an X509 signature, - the implementation doesn't support having both one signature on the SHA-1 object and one on the SHA-256 object. Let's improve on these limitations by improving fast-export and fast-import so that: - all the signatures are exported, - at most one signature on the SHA-1 object and one on the SHA-256 are imported, - if there is more than one signature on the SHA-1 object or on the SHA-256 object, fast-import emits a warning for each additional signature, - the output format is "gpgsig <git-hash-algo> <signature-format>", where <git-hash-algo> is the Git object format as before, and <signature-format> is the signature type ("openpgp", "x509", "ssh" or "unknown"), - the output is properly documented. About the output format: - <git-hash-algo> allows to know which representation of the commit was signed (the SHA-1 or the SHA-256 version) which helps with both signature verification and interoperability between repos with different hash functions, - <signature-format> helps tools that process the fast-export stream, so they don't have to parse the ASCII armor to identify the signature type. It could be even better to be able to import more than one signature on the SHA-1 object and on the SHA-256 object, but other parts of Git don't handle that well for now, so this is left for future improvements. Helped-by: brian m. carlson <sandals@crustytoothpaste.net> Helped-by: Elijah Newren <newren@gmail.com> Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Christian Couder committed
Jul 9, 2025 at 16:12 UTC
b5b3ddbe5c56c7ded95e7c47c985dc6d61f73ea0
7 files changed
+312
-44
Documentation/git-fast-export.adoc
+17
@@ -50,6 +50,23 @@ resulting tag will have an invalid signature.
50
is the same as how earlier versions of this command without
51
this option behaved.
52
+
53
+When exported, a signature starts with:
54
++
55
+gpgsig <git-hash-algo> <signature-format>
56
++
57
+where <git-hash-algo> is the Git object hash so either "sha1" or
58
+"sha256", and <signature-format> is the signature type, so "openpgp",
59
+"x509", "ssh" or "unknown".
60
++
61
+For example, an OpenPGP signature on a SHA-1 commit starts with
62
+`gpgsig sha1 openpgp`, while an SSH signature on a SHA-256 commit
63
+starts with `gpgsig sha256 ssh`.
64
++
65
+While all the signatures of a commit are exported, an importer may
66
+choose to accept only some of them. For example
67
+linkgit:git-fast-import[1] currently stores at most one signature per
68
+Git hash algorithm in each commit.
69
++
70
NOTE: This is highly experimental and the format of the data stream may
71
change in the future without compatibility guarantees.
72
Documentation/git-fast-import.adoc
+32
-6
@@ -445,7 +445,7 @@ one).
445
original-oid?
446
('author' (SP <name>)? SP LT <email> GT SP <when> LF)?
447
'committer' (SP <name>)? SP LT <email> GT SP <when> LF
448
- ('gpgsig' SP <alg> LF data)?
448
+ ('gpgsig' SP <algo> SP <format> LF data)?
449
('encoding' SP <encoding> LF)?
450
data
451
('from' SP <commit-ish> LF)?
@@ -518,13 +518,39 @@ their syntax.
518
^^^^^^^^
519
520
The optional `gpgsig` command is used to include a PGP/GPG signature
521
-that signs the commit data.
521
+or other cryptographic signature that signs the commit data.
522
523
-Here <alg> specifies which hashing algorithm is used for this
524
-signature, either `sha1` or `sha256`.
523
+....
524
+ 'gpgsig' SP <git-hash-algo> SP <signature-format> LF data
525
+....
526
+
527
+The `gpgsig` command takes two arguments:
528
+
529
+* `<git-hash-algo>` specifies which Git object format this signature
530
+ applies to, either `sha1` or `sha256`. This allows to know which
531
+ representation of the commit was signed (the SHA-1 or the SHA-256
532
+ version) which helps with both signature verification and
533
+ interoperability between repos with different hash functions.
534
+
535
+* `<signature-format>` specifies the type of signature, such as
536
+ `openpgp`, `x509`, `ssh`, or `unknown`. This is a convenience for
537
+ tools that process the stream, so they don't have to parse the ASCII
538
+ armor to identify the signature type.
539
+
540
+A commit may have at most one signature for the SHA-1 object format
541
+(stored in the "gpgsig" header) and one for the SHA-256 object format
542
+(stored in the "gpgsig-sha256" header).
543
+
544
+See below for a detailed description of the `data` command which
545
+contains the raw signature data.
546
+
547
+Signatures are not yet checked in the current implementation
548
+though. (Already setting the `extensions.compatObjectFormat`
549
+configuration option might help with verifying both SHA-1 and SHA-256
550
+object format signatures when it will be implemented.)
551
526
-NOTE: This is highly experimental and the format of the data stream may
527
-change in the future without compatibility guarantees.
552
+NOTE: This is highly experimental and the format of the `gpgsig`
553
+command may change in the future without compatibility guarantees.
554
555
`encoding`
556
^^^^^^^^^^
builtin/fast-export.c
+48
-14
@@ -29,6 +29,7 @@
29
#include "quote.h"
30
#include "remote.h"
31
#include "blob.h"
32
+#include "gpg-interface.h"
33
34
static const char *const fast_export_usage[] = {
35
N_("git fast-export [<rev-list-opts>]"),
@@ -652,6 +653,38 @@ static const char *find_commit_multiline_header(const char *msg,
653
return strbuf_detach(&val, NULL);
654
}
655
656
+static void print_signature(const char *signature, const char *object_hash)
657
+{
658
+ if (!signature)
659
+ return;
660
+
661
+ printf("gpgsig %s %s\ndata %u\n%s\n",
662
+ object_hash,
663
+ get_signature_format(signature),
664
+ (unsigned)strlen(signature),
665
+ signature);
666
+}
667
+
668
+static const char *append_signatures_for_header(struct string_list *signatures,
669
+ const char *pos,
670
+ const char *header,
671
+ const char *object_hash)
672
+{
673
+ const char *signature;
674
+ const char *start = pos;
675
+ const char *end = pos;
676
+
677
+ while ((signature = find_commit_multiline_header(start + 1,
678
+ header,
679
+ &end))) {
680
+ string_list_append(signatures, signature)->util = (void *)object_hash;
681
+ free((char *)signature);
682
+ start = end;
683
+ }
684
+
685
+ return end;
686
+}
687
+
688
static void handle_commit(struct commit *commit, struct rev_info *rev,
689
struct string_list *paths_of_changed_objects)
690
{
@@ -660,7 +693,7 @@ static void handle_commit(struct commit *commit, struct rev_info *rev,
693
const char *author, *author_end, *committer, *committer_end;
694
const char *encoding = NULL;
695
size_t encoding_len;
663
- const char *signature_alg = NULL, *signature = NULL;
696
+ struct string_list signatures = STRING_LIST_INIT_DUP;
697
const char *message;
698
char *reencoded = NULL;
699
struct commit_list *p;
@@ -700,10 +733,11 @@ static void handle_commit(struct commit *commit, struct rev_info *rev,
733
}
734
735
if (*commit_buffer_cursor == '\n') {
703
- if ((signature = find_commit_multiline_header(commit_buffer_cursor + 1, "gpgsig", &commit_buffer_cursor)))
704
- signature_alg = "sha1";
705
- else if ((signature = find_commit_multiline_header(commit_buffer_cursor + 1, "gpgsig-sha256", &commit_buffer_cursor)))
706
- signature_alg = "sha256";
736
+ const char *after_sha1 = append_signatures_for_header(&signatures, commit_buffer_cursor,
737
+ "gpgsig", "sha1");
738
+ const char *after_sha256 = append_signatures_for_header(&signatures, commit_buffer_cursor,
739
+ "gpgsig-sha256", "sha256");
740
+ commit_buffer_cursor = (after_sha1 > after_sha256) ? after_sha1 : after_sha256;
741
}
742
743
message = strstr(commit_buffer_cursor, "\n\n");
@@ -769,30 +803,30 @@ static void handle_commit(struct commit *commit, struct rev_info *rev,
803
printf("%.*s\n%.*s\n",
804
(int)(author_end - author), author,
805
(int)(committer_end - committer), committer);
772
- if (signature) {
806
+ if (signatures.nr) {
807
switch (signed_commit_mode) {
808
case SIGN_ABORT:
809
die("encountered signed commit %s; use "
810
"--signed-commits=<mode> to handle it",
811
oid_to_hex(&commit->object.oid));
812
case SIGN_WARN_VERBATIM:
779
- warning("exporting signed commit %s",
780
- oid_to_hex(&commit->object.oid));
813
+ warning("exporting %"PRIuMAX" signature(s) for commit %s",
814
+ (uintmax_t)signatures.nr, oid_to_hex(&commit->object.oid));
815
/* fallthru */
816
case SIGN_VERBATIM:
783
- printf("gpgsig %s\ndata %u\n%s",
784
- signature_alg,
785
- (unsigned)strlen(signature),
786
- signature);
817
+ for (size_t i = 0; i < signatures.nr; i++) {
818
+ struct string_list_item *item = &signatures.items[i];
819
+ print_signature(item->string, item->util);
820
+ }
821
break;
822
case SIGN_WARN_STRIP:
789
- warning("stripping signature from commit %s",
823
+ warning("stripping signature(s) from commit %s",
824
oid_to_hex(&commit->object.oid));
825
/* fallthru */
826
case SIGN_STRIP:
827
break;
828
}
795
- free((char *)signature);
829
+ string_list_clear(&signatures, 0);
830
}
831
if (!reencoded && encoding)
832
printf("encoding %.*s\n", (int)encoding_len, encoding);
builtin/fast-import.c
+91
-22
@@ -29,6 +29,7 @@
29
#include "commit-reach.h"
30
#include "khash.h"
31
#include "date.h"
32
+#include "gpg-interface.h"
33
34
#define PACK_ID_BITS 16
35
#define MAX_PACK_ID ((1<<PACK_ID_BITS)-1)
@@ -2718,15 +2719,82 @@ static struct hash_list *parse_merge(unsigned int *count)
2719
return list;
2720
}
2721
2722
+struct signature_data {
2723
+ char *hash_algo; /* "sha1" or "sha256" */
2724
+ char *sig_format; /* "openpgp", "x509", "ssh", or "unknown" */
2725
+ struct strbuf data; /* The actual signature data */
2726
+};
2727
+
2728
+static void parse_one_signature(struct signature_data *sig, const char *v)
2729
+{
2730
+ char *args = xstrdup(v); /* Will be freed when sig->hash_algo is freed */
2731
+ char *space = strchr(args, ' ');
2732
+
2733
+ if (!space)
2734
+ die("Expected gpgsig format: 'gpgsig <hash-algo> <signature-format>', "
2735
+ "got 'gpgsig %s'", args);
2736
+ *space = '\0';
2737
+
2738
+ sig->hash_algo = args;
2739
+ sig->sig_format = space + 1;
2740
+
2741
+ /* Validate hash algorithm */
2742
+ if (strcmp(sig->hash_algo, "sha1") &&
2743
+ strcmp(sig->hash_algo, "sha256"))
2744
+ die("Unknown git hash algorithm in gpgsig: '%s'", sig->hash_algo);
2745
+
2746
+ /* Validate signature format */
2747
+ if (!valid_signature_format(sig->sig_format))
2748
+ die("Invalid signature format in gpgsig: '%s'", sig->sig_format);
2749
+ if (!strcmp(sig->sig_format, "unknown"))
2750
+ warning("'unknown' signature format in gpgsig");
2751
+
2752
+ /* Read signature data */
2753
+ read_next_command();
2754
+ parse_data(&sig->data, 0, NULL);
2755
+}
2756
+
2757
+static void add_gpgsig_to_commit(struct strbuf *commit_data,
2758
+ const char *header,
2759
+ struct signature_data *sig)
2760
+{
2761
+ struct string_list siglines = STRING_LIST_INIT_NODUP;
2762
+
2763
+ if (!sig->hash_algo)
2764
+ return;
2765
+
2766
+ strbuf_addstr(commit_data, header);
2767
+ string_list_split_in_place(&siglines, sig->data.buf, "\n", -1);
2768
+ strbuf_add_separated_string_list(commit_data, "\n ", &siglines);
2769
+ strbuf_addch(commit_data, '\n');
2770
+ string_list_clear(&siglines, 1);
2771
+ strbuf_release(&sig->data);
2772
+ free(sig->hash_algo);
2773
+}
2774
+
2775
+static void store_signature(struct signature_data *stored_sig,
2776
+ struct signature_data *new_sig,
2777
+ const char *hash_type)
2778
+{
2779
+ if (stored_sig->hash_algo) {
2780
+ warning("multiple %s signatures found, "
2781
+ "ignoring additional signature",
2782
+ hash_type);
2783
+ strbuf_release(&new_sig->data);
2784
+ free(new_sig->hash_algo);
2785
+ } else {
2786
+ *stored_sig = *new_sig;
2787
+ }
2788
+}
2789
+
2790
static void parse_new_commit(const char *arg)
2791
{
2723
- static struct strbuf sig = STRBUF_INIT;
2792
static struct strbuf msg = STRBUF_INIT;
2725
- struct string_list siglines = STRING_LIST_INIT_NODUP;
2793
+ struct signature_data sig_sha1 = { NULL, NULL, STRBUF_INIT };
2794
+ struct signature_data sig_sha256 = { NULL, NULL, STRBUF_INIT };
2795
struct branch *b;
2796
char *author = NULL;
2797
char *committer = NULL;
2729
- char *sig_alg = NULL;
2798
char *encoding = NULL;
2799
struct hash_list *merge_list = NULL;
2800
unsigned int merge_count;
@@ -2750,13 +2818,23 @@ static void parse_new_commit(const char *arg)
2818
}
2819
if (!committer)
2820
die("Expected committer but didn't get one");
2753
- if (skip_prefix(command_buf.buf, "gpgsig ", &v)) {
2754
- sig_alg = xstrdup(v);
2755
- read_next_command();
2756
- parse_data(&sig, 0, NULL);
2821
+
2822
+ /* Process signatures (up to 2: one "sha1" and one "sha256") */
2823
+ while (skip_prefix(command_buf.buf, "gpgsig ", &v)) {
2824
+ struct signature_data sig = { NULL, NULL, STRBUF_INIT };
2825
+
2826
+ parse_one_signature(&sig, v);
2827
+
2828
+ if (!strcmp(sig.hash_algo, "sha1"))
2829
+ store_signature(&sig_sha1, &sig, "SHA-1");
2830
+ else if (!strcmp(sig.hash_algo, "sha256"))
2831
+ store_signature(&sig_sha256, &sig, "SHA-256");
2832
+ else
2833
+ BUG("parse_one_signature() returned unknown hash algo");
2834
+
2835
read_next_command();
2758
- } else
2759
- strbuf_setlen(&sig, 0);
2836
+ }
2837
+
2838
if (skip_prefix(command_buf.buf, "encoding ", &v)) {
2839
encoding = xstrdup(v);
2840
read_next_command();
@@ -2830,23 +2908,14 @@ static void parse_new_commit(const char *arg)
2908
strbuf_addf(&new_data,
2909
"encoding %s\n",
2910
encoding);
2833
- if (sig_alg) {
2834
- if (!strcmp(sig_alg, "sha1"))
2835
- strbuf_addstr(&new_data, "gpgsig ");
2836
- else if (!strcmp(sig_alg, "sha256"))
2837
- strbuf_addstr(&new_data, "gpgsig-sha256 ");
2838
- else
2839
- die("Expected gpgsig algorithm sha1 or sha256, got %s", sig_alg);
2840
- string_list_split_in_place(&siglines, sig.buf, "\n", -1);
2841
- strbuf_add_separated_string_list(&new_data, "\n ", &siglines);
2842
- strbuf_addch(&new_data, '\n');
2843
- }
2911
+
2912
+ add_gpgsig_to_commit(&new_data, "gpgsig ", &sig_sha1);
2913
+ add_gpgsig_to_commit(&new_data, "gpgsig-sha256 ", &sig_sha256);
2914
+
2915
strbuf_addch(&new_data, '\n');
2916
strbuf_addbuf(&new_data, &msg);
2846
- string_list_clear(&siglines, 1);
2917
free(author);
2918
free(committer);
2849
- free(sig_alg);
2919
free(encoding);
2920
2921
if (!store_object(OBJ_COMMIT, &new_data, NULL, &b->oid, next_mark))
gpg-interface.c
+12
@@ -144,6 +144,18 @@ static struct gpg_format *get_format_by_sig(const char *sig)
144
return NULL;
145
}
146
147
+const char *get_signature_format(const char *buf)
148
+{
149
+ struct gpg_format *format = get_format_by_sig(buf);
150
+ return format ? format->name : "unknown";
151
+}
152
+
153
+int valid_signature_format(const char *format)
154
+{
155
+ return (!!get_format_by_name(format) ||
156
+ !strcmp(format, "unknown"));
157
+}
158
+
159
void signature_check_clear(struct signature_check *sigc)
160
{
161
FREE_AND_NULL(sigc->payload);
gpg-interface.h
+12
@@ -47,6 +47,18 @@ struct signature_check {
47
48
void signature_check_clear(struct signature_check *sigc);
49
50
+/*
51
+ * Return the format of the signature (like "openpgp", "x509", "ssh"
52
+ * or "unknown").
53
+ */
54
+const char *get_signature_format(const char *buf);
55
+
56
+/*
57
+ * Is the signature format valid (like "openpgp", "x509", "ssh" or
58
+ * "unknown")
59
+ */
60
+int valid_signature_format(const char *format);
61
+
62
/*
63
* Look at a GPG signed tag object. If such a signature exists, store it in
64
* signature and the signed content in payload. Return 1 if a signature was
t/t9350-fast-export.sh
+100
-2
@@ -314,7 +314,7 @@ test_expect_success GPG 'signed-commits=abort' '
314
test_expect_success GPG 'signed-commits=verbatim' '
315
316
git fast-export --signed-commits=verbatim --reencode=no commit-signing >output &&
317
- grep "^gpgsig sha" output &&
317
+ test_grep -E "^gpgsig $GIT_DEFAULT_HASH openpgp" output &&
318
grep "encoding ISO-8859-1" output &&
319
(
320
cd new &&
@@ -328,7 +328,7 @@ test_expect_success GPG 'signed-commits=verbatim' '
328
test_expect_success GPG 'signed-commits=warn-verbatim' '
329
330
git fast-export --signed-commits=warn-verbatim --reencode=no commit-signing >output 2>err &&
331
- grep "^gpgsig sha" output &&
331
+ test_grep -E "^gpgsig $GIT_DEFAULT_HASH openpgp" output &&
332
grep "encoding ISO-8859-1" output &&
333
test -s err &&
334
(
@@ -369,6 +369,62 @@ test_expect_success GPG 'signed-commits=warn-strip' '
369
370
'
371
372
+test_expect_success GPGSM 'setup X.509 signed commit' '
373
+
374
+ git checkout -b x509-signing main &&
375
+ test_config gpg.format x509 &&
376
+ test_config user.signingkey $GIT_COMMITTER_EMAIL &&
377
+ echo "X.509 content" >file &&
378
+ git add file &&
379
+ git commit -S -m "X.509 signed commit" &&
380
+ X509_COMMIT=$(git rev-parse HEAD) &&
381
+ git checkout main
382
+
383
+'
384
+
385
+test_expect_success GPGSM 'round-trip X.509 signed commit' '
386
+
387
+ git fast-export --signed-commits=verbatim x509-signing >output &&
388
+ test_grep -E "^gpgsig $GIT_DEFAULT_HASH x509" output &&
389
+ (
390
+ cd new &&
391
+ git fast-import &&
392
+ git cat-file commit refs/heads/x509-signing >actual &&
393
+ grep "^gpgsig" actual &&
394
+ IMPORTED=$(git rev-parse refs/heads/x509-signing) &&
395
+ test $X509_COMMIT = $IMPORTED
396
+ ) <output
397
+
398
+'
399
+
400
+test_expect_success GPGSSH 'setup SSH signed commit' '
401
+
402
+ git checkout -b ssh-signing main &&
403
+ test_config gpg.format ssh &&
404
+ test_config user.signingkey "${GPGSSH_KEY_PRIMARY}" &&
405
+ echo "SSH content" >file &&
406
+ git add file &&
407
+ git commit -S -m "SSH signed commit" &&
408
+ SSH_COMMIT=$(git rev-parse HEAD) &&
409
+ git checkout main
410
+
411
+'
412
+
413
+test_expect_success GPGSSH 'round-trip SSH signed commit' '
414
+
415
+ git fast-export --signed-commits=verbatim ssh-signing >output &&
416
+ test_grep -E "^gpgsig $GIT_DEFAULT_HASH ssh" output &&
417
+ (
418
+ cd new &&
419
+ git fast-import &&
420
+ git cat-file commit refs/heads/ssh-signing >actual &&
421
+ grep "^gpgsig" actual &&
422
+ IMPORTED=$(git rev-parse refs/heads/ssh-signing) &&
423
+ test $SSH_COMMIT = $IMPORTED
424
+ ) <output
425
+
426
+'
427
+
428
test_expect_success 'setup submodule' '
429
430
test_config_global protocol.file.allow always &&
@@ -905,4 +961,46 @@ test_expect_success 'fast-export handles --end-of-options' '
961
test_cmp expect actual
962
'
963
964
+test_expect_success GPG 'setup a commit with dual signatures on its SHA-1 and SHA-256 formats' '
965
+ # Create a signed SHA-256 commit
966
+ git init --object-format=sha256 explicit-sha256 &&
967
+ git -C explicit-sha256 config extensions.compatObjectFormat sha1 &&
968
+ git -C explicit-sha256 checkout -b dual-signed &&
969
+ test_commit -C explicit-sha256 A &&
970
+ echo B >explicit-sha256/B &&
971
+ git -C explicit-sha256 add B &&
972
+ test_tick &&
973
+ git -C explicit-sha256 commit -S -m "signed" B &&
974
+ SHA256_B=$(git -C explicit-sha256 rev-parse dual-signed) &&
975
+
976
+ # Create the corresponding SHA-1 commit
977
+ SHA1_B=$(git -C explicit-sha256 rev-parse --output-object-format=sha1 dual-signed) &&
978
+
979
+ # Check that the resulting SHA-1 commit has both signatures
980
+ echo $SHA1_B | git -C explicit-sha256 cat-file --batch >out &&
981
+ test_grep -E "^gpgsig " out &&
982
+ test_grep -E "^gpgsig-sha256 " out
983
+'
984
+
985
+test_expect_success GPG 'export and import of doubly signed commit' '
986
+ git -C explicit-sha256 fast-export --signed-commits=verbatim dual-signed >output &&
987
+ test_grep -E "^gpgsig sha1 openpgp" output &&
988
+ test_grep -E "^gpgsig sha256 openpgp" output &&
989
+
990
+ (
991
+ cd new &&
992
+ git fast-import &&
993
+ git cat-file commit refs/heads/dual-signed >actual &&
994
+ test_grep -E "^gpgsig " actual &&
995
+ test_grep -E "^gpgsig-sha256 " actual &&
996
+ IMPORTED=$(git rev-parse refs/heads/dual-signed) &&
997
+ if test "$GIT_DEFAULT_HASH" = "sha1"
998
+ then
999
+ test $SHA1_B = $IMPORTED
1000
+ else
1001
+ test $SHA256_B = $IMPORTED
1002
+ fi
1003
+ ) <output
1004
+'
1005
+
1006
test_done