@cryptotaxi247 / kubo / commits / f63a997c3

IPLD Prime In IPFS: Target Merge Branch (#7976)

* feat: switch to using go-ipld-prime for codecs, path resolution, and the `dag put/get` commands * fix: `dag put/get` not roundtripping due to an extra new line being added (https://github.com/ipfs/go-ipfs/issues/3503) More detailed information is in the CHANGELOG.md file. Very high level: * IPLD codecs (and their plugins) must use go-ipld-prime * Added support for the dag-json codec * `dag get/put` use IPLD codec names from the multicodec table * `dag get` defaults to dag-json output instead of json, but may output with other codecs * Data model pathing can be achieved using the /ipld prefix. For example, you can use `/ipld/QmFoo/Links/0/Hash` to traverse through a DagPB node * With `dag get/put` the DagPB field names have been changed to match the ones in the protobuf listed in the specification Co-authored-by: hannahhoward <hannah@hannahhoward.net> Co-authored-by: Daniel Martí <mvdan@mvdan.cc> Co-authored-by: acruikshank <acruikshank@example.com> Co-authored-by: Steven Allen <steven@stebalien.com> Co-authored-by: Will Scott <will.scott@protocol.ai> Co-authored-by: Will Scott <will@cypherpunk.email> Co-authored-by: Rod Vagg <rod@vagg.org> Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com> Co-authored-by: Eric Myhre <hash@exultant.us>

Hannah Howard committed Aug 17, 2021 at 10:32 UTC f63a997c353a6d14311c5558a09c3d509bb909d2
19 files changed +511 -222
CHANGELOG.md
+23
@@ -1,5 +1,28 @@
1 # go-ipfs changelog
2
3 +## v0.10.0 TBD
4 +
5 +**IPLD Levels Up**
6 +
7 +The handling of data serialization as well as many aspects of DAG traversal and pathing have been migrated from older libraries, including [go-merkledag](https://github.com/ipfs/go-merkledag) and [go-ipld-format](https://github.com/ipfs/go-ipld-format) to the new **[go-ipld-prime](https://github.com/ipld/go-ipld-prime)** library and its components. This allows us to use many of the newer tools afforded by go-ipld-prime, stricter and more uniform codec implementations, support for additional (pluggable) codecs, and some minor performance improvements.
8 +
9 +This is significant refactor of a core component that touches many parts of IPFS, and does come with some **breaking changes**:
10 +
11 +* **IPLD plugins**:
12 + * The `PluginIPLD` interface has been changed to utilize go-ipld-prime. There is a demonstration of the change in the [bundled git plugin](./plugin/plugins/git/).
13 +* **The semantics of `dag put` and `dag get` change**:
14 + * `dag get` now takes the `format` option which accepts a multicodec name used to encode the output. By default this is `dag-json`. Users may notice differences from the previously plain Go JSON output, particularly where bytes are concerned which are now encoded using a form similar to CIDs: `{"/":{"bytes":"unpadded-base64-bytes"}}` rather than the previously Go-specific plain padded base64 string. See the [dag-json specification](https://ipld.io/specs/codecs/dag-json/spec/) for an explanation of these forms.
15 + * `dag get` no longer prints an additional new-line character at the end of the encoded block output. This means that the output as presented by `dag get` are the exact bytes of the requested node. A round-trip of such bytes back in through `dag put` using the same codec should result in the same CID.
16 + * `dag put` uses the `input-enc` option to specify the multicodec name of the format data is being provided in, and the `format` option to specify the multicodec multicodec name of the format the data should be stored in. These formerly defaulted to `json` and `cbor` respectively. They now default to `dag-json` and `dag-cbor` respectively but may be changed to any supported codec (bundled or loaded via plugin) by its [multicodec name](https://github.com/multiformats/multicodec/blob/master/table.csv).
17 + * The `json` and `cbor` multicodec names (as used by `input-enc` and `format` options) are now no longer aliases for `dag-json` and `dag-cbor` respectively. Instead, they now refer to their proper [multicodec](https://github.com/multiformats/multicodec/blob/master/table.csv) types. `cbor` refers to a plain CBOR format, which will not encode CIDs and does not have strict deterministic encoding rules. `json` is a plain JSON format, which also won't encode CIDs and will encode bytes in the Go-specific padded base64 string format rather than the dag-json method of byte encoding. See https://ipld.io/specs/codecs/ for more information on IPLD codecs.
18 +* The **dag-pb codec**, which is used to encode UnixFS data for IPFS, is now represented in a form via the `dag` API that mirrors the protobuf schema used to define the binary format and unifies the implementations and specification of dag-pb across the IPLD and IPFS stacks. Previously, additional layers of code within IPFS between protobuf serialization and UnixFS handling for file and directory data, obscured the forms that are described by the protobuf representation. Much of this code has now been replaced and there are fewer layers of transformation. This means that interacting with dag-pb data via the `dag` API will use different forms:
19 + * Previously, using `dag get` on a dag-pb block would present the block serialized as JSON as `{"data":"padded-base64-bytes","links":[{"Name":"foo","Size":100,"Cid":{"/":"Qm..."}},...]}`.
20 + * Using the dag-pb data model specification and the new default dag-json codec for output, this will now be serialized as: `{"Data":{"/":{"bytes":"unpadded-base64-bytes"}},"Links":[{"Name":"foo","Tsize":100,"Hash":{"/":"Qm..."}},...]}`. Aside from the change in byte formatting, most field names have changed: `data` → `Data`, `links` → `Links`, `Size` → `Tsize`, `Cid` → `Hash`. Note that this output can be changed now using the `--format` option to specify an alternative codec.
21 + * Using `dag put` and a `format` option of `dag-pb` now requires that the input conform to this dag-pb specified form. Previously, input using `{"data":"...","links":[...]}` was accepted, now it must be `{"Data":"...","Links":[...]}`.
22 + * Previously it was not possible to use paths to navigate to any of these properties of a dag-pb node, the only possible paths were named links, e.g. `dag get QmFoo/NamedLink` where `NamedLink` was one of the links whose name was `NamedLink`. This functionality remains the same, but by prefixing the path with `/ipld/` we enter data model pathing semantics and can `dag get /ipld/QmFoo/Links/0/Hash` to navigate to links or `/ipld/QmFoo/Data` to simply retrieve the data section of the node, for example.
23 + * See the [dag-pb specification](https://ipld.io/specs/codecs/dag-pb/) for details on the codec and its data model representation.
24 + * See this [detailed write-up](https://github.com/ipld/ipld/blob/master/design/tricky-choices/dag-pb-forms-impl-and-use.md) for further background on these changes.
25 +
26 ## v0.9.1 2021-07-20
27
28 This is a small bug fix release resolving the following issues:
core/commands/dag/dag.go
+6 -3
@@ -77,10 +77,10 @@ into an object of the specified format.
77 cmds.FileArg("object data", true, true, "The object to put").EnableStdin(),
78 },
79 Options: []cmds.Option{
80 - cmds.StringOption("format", "f", "Format that the object will be added as.").WithDefault("cbor"),
81 - cmds.StringOption("input-enc", "Format that the input object will be.").WithDefault("json"),
80 + cmds.StringOption("format", "f", "Format that the object will be added as.").WithDefault("dag-cbor"),
81 + cmds.StringOption("input-enc", "Format that the input object will be.").WithDefault("dag-json"),
82 cmds.BoolOption("pin", "Pin this object when adding."),
83 - cmds.StringOption("hash", "Hash function to use").WithDefault(""),
83 + cmds.StringOption("hash", "Hash function to use").WithDefault("sha2-256"),
84 },
85 Run: dagPut,
86 Type: OutputObject{},
@@ -108,6 +108,9 @@ format.
108 Arguments: []cmds.Argument{
109 cmds.StringArg("ref", true, false, "The object to get").EnableStdin(),
110 },
111 + Options: []cmds.Option{
112 + cmds.StringOption("format", "f", "Format that the object will be serialized as.").WithDefault("dag-json"),
113 + },
114 Run: dagGet,
115 }
116
core/commands/dag/get.go
+39 -6
@@ -1,11 +1,18 @@
1 package dagcmd
2
3 import (
4 - "strings"
4 + "fmt"
5 + "io"
6
7 "github.com/ipfs/go-ipfs/core/commands/cmdenv"
8 + ipldlegacy "github.com/ipfs/go-ipld-legacy"
9 "github.com/ipfs/interface-go-ipfs-core/path"
10
11 + "github.com/ipld/go-ipld-prime"
12 + "github.com/ipld/go-ipld-prime/multicodec"
13 + "github.com/ipld/go-ipld-prime/traversal"
14 + mc "github.com/multiformats/go-multicodec"
15 +
16 cmds "github.com/ipfs/go-ipfs-cmds"
17 )
18
@@ -15,6 +22,12 @@ func dagGet(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) e
22 return err
23 }
24
25 + format, _ := req.Options["format"].(string)
26 + var fCodec mc.Code
27 + if err := fCodec.Set(format); err != nil {
28 + return err
29 + }
30 +
31 rp, err := api.ResolvePath(req.Context, path.New(req.Arguments[0]))
32 if err != nil {
33 return err
@@ -25,14 +38,34 @@ func dagGet(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) e
38 return err
39 }
40
28 - var out interface{} = obj
41 + universal, ok := obj.(ipldlegacy.UniversalNode)
42 + if !ok {
43 + return fmt.Errorf("%T is not a valid IPLD node", obj)
44 + }
45 +
46 + finalNode := universal.(ipld.Node)
47 +
48 if len(rp.Remainder()) > 0 {
30 - rem := strings.Split(rp.Remainder(), "/")
31 - final, _, err := obj.Resolve(rem)
49 + remainderPath := ipld.ParsePath(rp.Remainder())
50 +
51 + finalNode, err = traversal.Get(finalNode, remainderPath)
52 if err != nil {
53 return err
54 }
35 - out = final
55 }
37 - return cmds.EmitOnce(res, &out)
56 +
57 + encoder, err := multicodec.LookupEncoder(uint64(fCodec))
58 + if err != nil {
59 + return fmt.Errorf("invalid encoding: %s - %s", format, err)
60 + }
61 +
62 + r, w := io.Pipe()
63 + go func() {
64 + defer w.Close()
65 + if err := encoder(finalNode, w); err != nil {
66 + _ = w.CloseWithError(err)
67 + }
68 + }()
69 +
70 + return res.Emit(r)
71 }
core/commands/dag/put.go
+64 -21
@@ -1,16 +1,28 @@
1 package dagcmd
2
3 import (
4 + "bytes"
5 "fmt"
5 - "math"
6
7 + blocks "github.com/ipfs/go-block-format"
8 + "github.com/ipfs/go-cid"
9 "github.com/ipfs/go-ipfs/core/commands/cmdenv"
8 - "github.com/ipfs/go-ipfs/core/coredag"
10 + ipldlegacy "github.com/ipfs/go-ipld-legacy"
11 + "github.com/ipld/go-ipld-prime/multicodec"
12 + basicnode "github.com/ipld/go-ipld-prime/node/basic"
13
14 cmds "github.com/ipfs/go-ipfs-cmds"
15 files "github.com/ipfs/go-ipfs-files"
16 ipld "github.com/ipfs/go-ipld-format"
13 - mh "github.com/multiformats/go-multihash"
17 + mc "github.com/multiformats/go-multicodec"
18 +
19 + // Expected minimal set of available format/ienc codecs.
20 + _ "github.com/ipld/go-codec-dagpb"
21 + _ "github.com/ipld/go-ipld-prime/codec/cbor"
22 + _ "github.com/ipld/go-ipld-prime/codec/dagcbor"
23 + _ "github.com/ipld/go-ipld-prime/codec/dagjson"
24 + _ "github.com/ipld/go-ipld-prime/codec/json"
25 + _ "github.com/ipld/go-ipld-prime/codec/raw"
26 )
27
28 func dagPut(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) error {
@@ -24,16 +36,33 @@ func dagPut(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) e
36 hash, _ := req.Options["hash"].(string)
37 dopin, _ := req.Options["pin"].(bool)
38
27 - // mhType tells inputParser which hash should be used. MaxUint64 means 'use
28 - // default hash' (sha256 for cbor, sha1 for git..)
29 - mhType := uint64(math.MaxUint64)
39 + var icodec mc.Code
40 + if err := icodec.Set(ienc); err != nil {
41 + return err
42 + }
43 + var fcodec mc.Code
44 + if err := fcodec.Set(format); err != nil {
45 + return err
46 + }
47 + var mhType mc.Code
48 + if err := mhType.Set(hash); err != nil {
49 + return err
50 + }
51
31 - if hash != "" {
32 - var ok bool
33 - mhType, ok = mh.Names[hash]
34 - if !ok {
35 - return fmt.Errorf("%s in not a valid multihash name", hash)
36 - }
52 + cidPrefix := cid.Prefix{
53 + Version: 1,
54 + Codec: uint64(fcodec),
55 + MhType: uint64(mhType),
56 + MhLength: -1,
57 + }
58 +
59 + decoder, err := multicodec.LookupDecoder(uint64(icodec))
60 + if err != nil {
61 + return err
62 + }
63 + encoder, err := multicodec.LookupEncoder(uint64(fcodec))
64 + if err != nil {
65 + return err
66 }
67
68 var adder ipld.NodeAdder = api.Dag()
@@ -48,22 +77,36 @@ func dagPut(req *cmds.Request, res cmds.ResponseEmitter, env cmds.Environment) e
77 if file == nil {
78 return fmt.Errorf("expected a regular file")
79 }
51 - nds, err := coredag.ParseInputs(ienc, format, file, mhType, -1)
80 +
81 + node := basicnode.Prototype.Any.NewBuilder()
82 + if err := decoder(node, file); err != nil {
83 + return err
84 + }
85 + n := node.Build()
86 +
87 + bd := bytes.NewBuffer([]byte{})
88 + if err := encoder(n, bd); err != nil {
89 + return err
90 + }
91 +
92 + blockCid, err := cidPrefix.Sum(bd.Bytes())
93 + if err != nil {
94 + return err
95 + }
96 + blk, err := blocks.NewBlockWithCid(bd.Bytes(), blockCid)
97 if err != nil {
98 return err
99 }
55 - if len(nds) == 0 {
56 - return fmt.Errorf("no node returned from ParseInputs")
100 + ln := ipldlegacy.LegacyNode{
101 + Block: blk,
102 + Node: n,
103 }
104
59 - for _, nd := range nds {
60 - err := b.Add(req.Context, nd)
61 - if err != nil {
62 - return err
63 - }
105 + if err := b.Add(req.Context, &ln); err != nil {
106 + return err
107 }
108
66 - cid := nds[0].Cid()
109 + cid := ln.Cid()
110 if err := res.Emit(&OutputObject{Cid: cid}); err != nil {
111 return err
112 }
core/core.go
+14 -13
@@ -17,6 +17,7 @@ import (
17 "github.com/ipfs/go-ipfs-pinner"
18
19 bserv "github.com/ipfs/go-blockservice"
20 + "github.com/ipfs/go-fetcher"
21 "github.com/ipfs/go-graphsync"
22 bstore "github.com/ipfs/go-ipfs-blockstore"
23 exchange "github.com/ipfs/go-ipfs-exchange-interface"
@@ -24,7 +25,6 @@ import (
25 ipld "github.com/ipfs/go-ipld-format"
26 logging "github.com/ipfs/go-log"
27 mfs "github.com/ipfs/go-mfs"
27 - resolver "github.com/ipfs/go-path/resolver"
28 goprocess "github.com/jbenet/goprocess"
29 connmgr "github.com/libp2p/go-libp2p-core/connmgr"
30 ic "github.com/libp2p/go-libp2p-core/crypto"
@@ -70,18 +70,19 @@ type IpfsNode struct {
70 PNetFingerprint libp2p.PNetFingerprint `optional:"true"` // fingerprint of private network
71
72 // Services
73 - Peerstore pstore.Peerstore `optional:"true"` // storage for other Peer instances
74 - Blockstore bstore.GCBlockstore // the block store (lower level)
75 - Filestore *filestore.Filestore `optional:"true"` // the filestore blockstore
76 - BaseBlocks node.BaseBlocks // the raw blockstore, no filestore wrapping
77 - GCLocker bstore.GCLocker // the locker used to protect the blockstore during gc
78 - Blocks bserv.BlockService // the block service, get/add blocks.
79 - DAG ipld.DAGService // the merkle dag service, get/add objects.
80 - Resolver *resolver.Resolver // the path resolution system
81 - Reporter *metrics.BandwidthCounter `optional:"true"`
82 - Discovery discovery.Service `optional:"true"`
83 - FilesRoot *mfs.Root
84 - RecordValidator record.Validator
73 + Peerstore pstore.Peerstore `optional:"true"` // storage for other Peer instances
74 + Blockstore bstore.GCBlockstore // the block store (lower level)
75 + Filestore *filestore.Filestore `optional:"true"` // the filestore blockstore
76 + BaseBlocks node.BaseBlocks // the raw blockstore, no filestore wrapping
77 + GCLocker bstore.GCLocker // the locker used to protect the blockstore during gc
78 + Blocks bserv.BlockService // the block service, get/add blocks.
79 + DAG ipld.DAGService // the merkle dag service, get/add objects.
80 + IPLDFetcherFactory fetcher.Factory `name:"ipldFetcher"` // fetcher that paths over the IPLD data model
81 + UnixFSFetcherFactory fetcher.Factory `name:"unixfsFetcher"` // fetcher that interprets UnixFS data
82 + Reporter *metrics.BandwidthCounter `optional:"true"`
83 + Discovery discovery.Service `optional:"true"`
84 + FilesRoot *mfs.Root
85 + RecordValidator record.Validator
86
87 // Online
88 PeerHost p2phost.Host `optional:"true"` // the network host (server+client)
core/coreapi/coreapi.go
+17 -13
@@ -19,11 +19,12 @@ import (
19 "fmt"
20
21 bserv "github.com/ipfs/go-blockservice"
22 - "github.com/ipfs/go-ipfs-blockstore"
23 - "github.com/ipfs/go-ipfs-exchange-interface"
22 + "github.com/ipfs/go-fetcher"
23 + blockstore "github.com/ipfs/go-ipfs-blockstore"
24 + exchange "github.com/ipfs/go-ipfs-exchange-interface"
25 offlinexch "github.com/ipfs/go-ipfs-exchange-offline"
25 - "github.com/ipfs/go-ipfs-pinner"
26 - "github.com/ipfs/go-ipfs-provider"
26 + pin "github.com/ipfs/go-ipfs-pinner"
27 + provider "github.com/ipfs/go-ipfs-provider"
28 offlineroute "github.com/ipfs/go-ipfs-routing/offline"
29 ipld "github.com/ipfs/go-ipld-format"
30 dag "github.com/ipfs/go-merkledag"
@@ -55,13 +56,14 @@ type CoreAPI struct {
56 baseBlocks blockstore.Blockstore
57 pinning pin.Pinner
58
58 - blocks bserv.BlockService
59 - dag ipld.DAGService
60 -
61 - peerstore pstore.Peerstore
62 - peerHost p2phost.Host
63 - recordValidator record.Validator
64 - exchange exchange.Interface
59 + blocks bserv.BlockService
60 + dag ipld.DAGService
61 + ipldFetcherFactory fetcher.Factory
62 + unixFSFetcherFactory fetcher.Factory
63 + peerstore pstore.Peerstore
64 + peerHost p2phost.Host
65 + recordValidator record.Validator
66 + exchange exchange.Interface
67
68 namesys namesys.NameSystem
69 routing routing.Routing
@@ -167,8 +169,10 @@ func (api *CoreAPI) WithOptions(opts ...options.ApiOption) (coreiface.CoreAPI, e
169 baseBlocks: n.BaseBlocks,
170 pinning: n.Pinning,
171
170 - blocks: n.Blocks,
171 - dag: n.DAG,
172 + blocks: n.Blocks,
173 + dag: n.DAG,
174 + ipldFetcherFactory: n.IPLDFetcherFactory,
175 + unixFSFetcherFactory: n.UnixFSFetcherFactory,
176
177 peerstore: n.Peerstore,
178 peerHost: n.PeerHost,
core/coreapi/path.go
+10 -14
@@ -8,10 +8,10 @@ import (
8 "github.com/ipfs/go-namesys/resolve"
9
10 "github.com/ipfs/go-cid"
11 + "github.com/ipfs/go-fetcher"
12 ipld "github.com/ipfs/go-ipld-format"
13 ipfspath "github.com/ipfs/go-path"
13 - "github.com/ipfs/go-path/resolver"
14 - uio "github.com/ipfs/go-unixfs/io"
14 + ipfspathresolver "github.com/ipfs/go-path/resolver"
15 coreiface "github.com/ipfs/interface-go-ipfs-core"
16 path "github.com/ipfs/interface-go-ipfs-core/path"
17 )
@@ -49,23 +49,19 @@ func (api *CoreAPI) ResolvePath(ctx context.Context, p path.Path) (path.Resolved
49 return nil, err
50 }
51
52 - var resolveOnce resolver.ResolveOnce
53 -
54 - switch ipath.Segments()[0] {
55 - case "ipfs":
56 - resolveOnce = uio.ResolveUnixfsOnce
57 - case "ipld":
58 - resolveOnce = resolver.ResolveSingle
59 - default:
52 + if ipath.Segments()[0] != "ipfs" && ipath.Segments()[0] != "ipld" {
53 return nil, fmt.Errorf("unsupported path namespace: %s", p.Namespace())
54 }
55
63 - r := &resolver.Resolver{
64 - DAG: api.dag,
65 - ResolveOnce: resolveOnce,
56 + var dataFetcher fetcher.Factory
57 + if ipath.Segments()[0] == "ipld" {
58 + dataFetcher = api.ipldFetcherFactory
59 + } else {
60 + dataFetcher = api.unixFSFetcherFactory
61 }
62 + resolver := ipfspathresolver.NewBasicResolver(dataFetcher)
63
68 - node, rest, err := r.ResolveToLastNode(ctx, ipath)
64 + node, rest, err := resolver.ResolveToLastNode(ctx, ipath)
65 if err != nil {
66 return nil, err
67 }
core/node/core.go
+27 -1
@@ -3,12 +3,13 @@ package node
3 import (
4 "context"
5 "fmt"
6 -
6 "github.com/ipfs/go-bitswap"
7 "github.com/ipfs/go-bitswap/network"
8 "github.com/ipfs/go-blockservice"
9 "github.com/ipfs/go-cid"
10 "github.com/ipfs/go-datastore"
11 + "github.com/ipfs/go-fetcher"
12 + bsfetcher "github.com/ipfs/go-fetcher/impl/blockservice"
13 "github.com/ipfs/go-filestore"
14 blockstore "github.com/ipfs/go-ipfs-blockstore"
15 exchange "github.com/ipfs/go-ipfs-exchange-interface"
@@ -18,6 +19,11 @@ import (
19 "github.com/ipfs/go-merkledag"
20 "github.com/ipfs/go-mfs"
21 "github.com/ipfs/go-unixfs"
22 + "github.com/ipfs/go-unixfsnode"
23 + dagpb "github.com/ipld/go-codec-dagpb"
24 + "github.com/ipld/go-ipld-prime"
25 + basicnode "github.com/ipld/go-ipld-prime/node/basic"
26 + "github.com/ipld/go-ipld-prime/schema"
27 "github.com/libp2p/go-libp2p-core/host"
28 "github.com/libp2p/go-libp2p-core/routing"
29 "go.uber.org/fx"
@@ -80,6 +86,26 @@ func (s *syncDagService) Session(ctx context.Context) format.NodeGetter {
86 return merkledag.NewSession(ctx, s.DAGService)
87 }
88
89 +type fetchersOut struct {
90 + fx.Out
91 + IPLDFetcher fetcher.Factory `name:"ipldFetcher"`
92 + UnixfsFetcher fetcher.Factory `name:"unixfsFetcher"`
93 +}
94 +
95 +// FetcherConfig returns a fetcher config that can build new fetcher instances
96 +func FetcherConfig(bs blockservice.BlockService) fetchersOut {
97 + ipldFetcher := bsfetcher.NewFetcherConfig(bs)
98 + ipldFetcher.PrototypeChooser = dagpb.AddSupportToChooser(func(lnk ipld.Link, lnkCtx ipld.LinkContext) (ipld.NodePrototype, error) {
99 + if tlnkNd, ok := lnkCtx.LinkNode.(schema.TypedLinkNode); ok {
100 + return tlnkNd.LinkTargetNodePrototype(), nil
101 + }
102 + return basicnode.Prototype.Any, nil
103 + })
104 +
105 + unixFSFetcher := ipldFetcher.WithReifier(unixfsnode.Reify)
106 + return fetchersOut{IPLDFetcher: ipldFetcher, UnixfsFetcher: unixFSFetcher}
107 +}
108 +
109 // Dag creates new DAGService
110 func Dag(bs blockservice.BlockService) format.DAGService {
111 return merkledag.NewDAGService(bs)
core/node/groups.go
+1 -2
@@ -18,7 +18,6 @@ import (
18
19 offline "github.com/ipfs/go-ipfs-exchange-offline"
20 offroute "github.com/ipfs/go-ipfs-routing/offline"
21 - "github.com/ipfs/go-path/resolver"
21 uio "github.com/ipfs/go-unixfs/io"
22 "go.uber.org/fx"
23 )
@@ -294,7 +293,7 @@ func Offline(cfg *config.Config) fx.Option {
293 var Core = fx.Options(
294 fx.Provide(BlockService),
295 fx.Provide(Dag),
297 - fx.Provide(resolver.NewBasicResolver),
296 + fx.Provide(FetcherConfig),
297 fx.Provide(Pinning),
298 fx.Provide(Files),
299 )
core/node/provider.go
+8 -3
@@ -5,12 +5,12 @@ import (
5 "fmt"
6 "time"
7
8 + "github.com/ipfs/go-fetcher"
9 "github.com/ipfs/go-ipfs-pinner"
10 "github.com/ipfs/go-ipfs-provider"
11 "github.com/ipfs/go-ipfs-provider/batched"
12 q "github.com/ipfs/go-ipfs-provider/queue"
13 "github.com/ipfs/go-ipfs-provider/simple"
13 - ipld "github.com/ipfs/go-ipld-format"
14 "github.com/libp2p/go-libp2p-core/routing"
15 "github.com/multiformats/go-multihash"
16 "go.uber.org/fx"
@@ -172,7 +172,12 @@ func SimpleProviders(reprovideStrategy string, reprovideInterval string) fx.Opti
172 }
173
174 func pinnedProviderStrategy(onlyRoots bool) interface{} {
175 - return func(pinner pin.Pinner, dag ipld.DAGService) simple.KeyChanFunc {
176 - return simple.NewPinnedProvider(onlyRoots, pinner, dag)
175 + type input struct {
176 + fx.In
177 + Pinner pin.Pinner
178 + IPLDFetcher fetcher.Factory `name:"ipldFetcher"`
179 + }
180 + return func(in input) simple.KeyChanFunc {
181 + return simple.NewPinnedProvider(onlyRoots, in.Pinner, in.IPLDFetcher)
182 }
183 }
fuse/readonly/readonly_unix.go
+33 -10
@@ -10,16 +10,18 @@ import (
10 "os"
11 "syscall"
12
13 + fuse "bazil.org/fuse"
14 + fs "bazil.org/fuse/fs"
15 + "github.com/ipfs/go-cid"
16 core "github.com/ipfs/go-ipfs/core"
17 + ipld "github.com/ipfs/go-ipld-format"
18 + logging "github.com/ipfs/go-log"
19 mdag "github.com/ipfs/go-merkledag"
20 path "github.com/ipfs/go-path"
21 + "github.com/ipfs/go-path/resolver"
22 ft "github.com/ipfs/go-unixfs"
23 uio "github.com/ipfs/go-unixfs/io"
18 -
19 - fuse "bazil.org/fuse"
20 - fs "bazil.org/fuse/fs"
21 - ipld "github.com/ipfs/go-ipld-format"
22 - logging "github.com/ipfs/go-log"
24 + cidlink "github.com/ipld/go-ipld-prime/linking/cid"
25 )
26
27 var log = logging.Logger("fuse/ipfs")
@@ -65,20 +67,41 @@ func (s *Root) Lookup(ctx context.Context, name string) (fs.Node, error) {
67 return nil, fuse.ENOENT
68 }
69
68 - nd, err := s.Ipfs.Resolver.ResolvePath(ctx, p)
70 + nd, ndLnk, err := resolver.NewBasicResolver(s.Ipfs.UnixFSFetcherFactory).ResolvePath(ctx, p)
71 if err != nil {
72 // todo: make this error more versatile.
73 return nil, fuse.ENOENT
74 }
75
74 - switch nd := nd.(type) {
75 - case *mdag.ProtoNode, *mdag.RawNode:
76 - return &Node{Ipfs: s.Ipfs, Nd: nd}, nil
76 + cidLnk, ok := ndLnk.(cidlink.Link)
77 + if !ok {
78 + log.Debugf("non-cidlink returned from ResolvePath: %v", ndLnk)
79 + return nil, fuse.ENOENT
80 + }
81 +
82 + // convert ipld-prime node to universal node
83 + blk, err := s.Ipfs.Blockstore.Get(cidLnk.Cid)
84 + if err != nil {
85 + log.Debugf("fuse failed to retrieve block: %v: %s", cidLnk, err)
86 + return nil, fuse.ENOENT
87 + }
88 +
89 + var fnd ipld.Node
90 + switch cidLnk.Cid.Prefix().Codec {
91 + case cid.DagProtobuf:
92 + fnd, err = mdag.ProtoNodeConverter(blk, nd)
93 + case cid.Raw:
94 + fnd, err = mdag.RawNodeConverter(blk, nd)
95 default:
78 - log.Error("fuse node was not a protobuf node")
96 + log.Error("fuse node was not a supported type")
97 return nil, fuse.ENOTSUP
98 }
99 + if err != nil {
100 + log.Error("could not convert protobuf or raw node")
101 + return nil, fuse.ENOENT
102 + }
103
104 + return &Node{Ipfs: s.Ipfs, Nd: fnd}, nil
105 }
106
107 // ReadDirAll reads a particular directory. Disallowed for root.
go.mod
+14 -8
@@ -14,7 +14,7 @@ require (
14 github.com/hashicorp/go-multierror v1.1.1
15 github.com/ipfs/go-bitswap v0.3.4
16 github.com/ipfs/go-block-format v0.0.3
17 - github.com/ipfs/go-blockservice v0.1.4
17 + github.com/ipfs/go-blockservice v0.1.7
18 github.com/ipfs/go-cid v0.0.7
19 github.com/ipfs/go-cidutil v0.0.2
20 github.com/ipfs/go-datastore v0.4.5
@@ -23,6 +23,7 @@ require (
23 github.com/ipfs/go-ds-flatfs v0.4.5
24 github.com/ipfs/go-ds-leveldb v0.4.2
25 github.com/ipfs/go-ds-measure v0.1.0
26 + github.com/ipfs/go-fetcher v1.5.0
27 github.com/ipfs/go-filestore v0.0.3
28 github.com/ipfs/go-fs-lock v0.0.7
29 github.com/ipfs/go-graphsync v0.8.0
@@ -36,26 +37,30 @@ require (
37 github.com/ipfs/go-ipfs-keystore v0.0.2
38 github.com/ipfs/go-ipfs-pinner v0.1.2
39 github.com/ipfs/go-ipfs-posinfo v0.0.1
39 - github.com/ipfs/go-ipfs-provider v0.5.1
40 + github.com/ipfs/go-ipfs-provider v0.6.1
41 github.com/ipfs/go-ipfs-routing v0.1.0
42 github.com/ipfs/go-ipfs-util v0.0.2
43 github.com/ipfs/go-ipld-cbor v0.0.5
44 github.com/ipfs/go-ipld-format v0.2.0
44 - github.com/ipfs/go-ipld-git v0.0.4
45 + github.com/ipfs/go-ipld-git v0.1.0
46 + github.com/ipfs/go-ipld-legacy v0.1.0
47 github.com/ipfs/go-ipns v0.1.0
48 github.com/ipfs/go-log v1.0.5
47 - github.com/ipfs/go-merkledag v0.3.2
49 + github.com/ipfs/go-merkledag v0.4.0
50 github.com/ipfs/go-metrics-interface v0.0.1
51 github.com/ipfs/go-metrics-prometheus v0.0.2
52 github.com/ipfs/go-mfs v0.1.2
53 github.com/ipfs/go-namesys v0.3.0
52 - github.com/ipfs/go-path v0.0.9
54 + github.com/ipfs/go-path v0.1.1
55 github.com/ipfs/go-pinning-service-http-client v0.1.0
56 github.com/ipfs/go-unixfs v0.2.5
57 + github.com/ipfs/go-unixfsnode v1.1.2
58 github.com/ipfs/go-verifcid v0.0.1
56 - github.com/ipfs/interface-go-ipfs-core v0.4.0
59 + github.com/ipfs/interface-go-ipfs-core v0.5.1
60 github.com/ipfs/tar-utils v0.0.1
61 github.com/ipld/go-car v0.3.1
62 + github.com/ipld/go-codec-dagpb v1.3.0
63 + github.com/ipld/go-ipld-prime v0.11.0
64 github.com/jbenet/go-random v0.0.0-20190219211222-123a90aedc0c
65 github.com/jbenet/go-temp-err-catcher v0.1.0
66 github.com/jbenet/goprocess v0.1.4
@@ -90,6 +95,7 @@ require (
95 github.com/multiformats/go-multiaddr v0.3.3
96 github.com/multiformats/go-multiaddr-dns v0.3.1
97 github.com/multiformats/go-multibase v0.0.3
98 + github.com/multiformats/go-multicodec v0.3.0
99 github.com/multiformats/go-multihash v0.0.15
100 github.com/opentracing/opentracing-go v1.2.0
101 github.com/pkg/errors v0.9.1
@@ -101,10 +107,10 @@ require (
107 go.opencensus.io v0.23.0
108 go.uber.org/fx v1.13.1
109 go.uber.org/zap v1.19.0
104 - golang.org/x/crypto v0.0.0-20210506145944-38f3c27a63bf
110 + golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a
111 golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5 // indirect
112 golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
107 - golang.org/x/sys v0.0.0-20210511113859-b0526f3d8744
113 + golang.org/x/sys v0.0.0-20210514084401-e8d321eab015
114 )
115
116 go 1.15
go.sum
+37 -24
@@ -356,10 +356,10 @@ github.com/ipfs/go-block-format v0.0.3/go.mod h1:4LmD4ZUw0mhO+JSKdpWwrzATiEfM7WW
356 github.com/ipfs/go-blockservice v0.0.7/go.mod h1:EOfb9k/Y878ZTRY/CH0x5+ATtaipfbRhbvNSdgc/7So=
357 github.com/ipfs/go-blockservice v0.1.0/go.mod h1:hzmMScl1kXHg3M2BjTymbVPjv627N7sYcvYaKbop39M=
358 github.com/ipfs/go-blockservice v0.1.1/go.mod h1:t+411r7psEUhLueM8C7aPA7cxCclv4O3VsUVxt9kz2I=
359 -github.com/ipfs/go-blockservice v0.1.2/go.mod h1:t+411r7psEUhLueM8C7aPA7cxCclv4O3VsUVxt9kz2I=
359 github.com/ipfs/go-blockservice v0.1.3/go.mod h1:OTZhFpkgY48kNzbgyvcexW9cHrpjBYIjSR0KoDOFOLU=
361 -github.com/ipfs/go-blockservice v0.1.4 h1:Vq+MlsH8000KbbUciRyYMEw/NNP8UAGmcqKi4uWmFGA=
360 github.com/ipfs/go-blockservice v0.1.4/go.mod h1:OTZhFpkgY48kNzbgyvcexW9cHrpjBYIjSR0KoDOFOLU=
361 +github.com/ipfs/go-blockservice v0.1.7 h1:yVe9te0M7ow8i+PPkx03YFSpxqzXx594d6h+34D6qMg=
362 +github.com/ipfs/go-blockservice v0.1.7/go.mod h1:GmS+BAt4hrwBKkzE11AFDQUrnvqjwFatGS2MY7wOjEM=
363 github.com/ipfs/go-cid v0.0.1/go.mod h1:GHWU/WuQdMPmIosc4Yn1bcCT7dSeX4lBafM7iqUPQvM=
364 github.com/ipfs/go-cid v0.0.2/go.mod h1:GHWU/WuQdMPmIosc4Yn1bcCT7dSeX4lBafM7iqUPQvM=
365 github.com/ipfs/go-cid v0.0.3/go.mod h1:GHWU/WuQdMPmIosc4Yn1bcCT7dSeX4lBafM7iqUPQvM=
@@ -400,6 +400,8 @@ github.com/ipfs/go-ds-leveldb v0.4.2 h1:QmQoAJ9WkPMUfBLnu1sBVy0xWWlJPg0m4kRAiJL9
400 github.com/ipfs/go-ds-leveldb v0.4.2/go.mod h1:jpbku/YqBSsBc1qgME8BkWS4AxzF2cEu1Ii2r79Hh9s=
401 github.com/ipfs/go-ds-measure v0.1.0 h1:vE4TyY4aeLeVgnnPBC5QzKIjKrqzha0NCujTfgvVbVQ=
402 github.com/ipfs/go-ds-measure v0.1.0/go.mod h1:1nDiFrhLlwArTME1Ees2XaBOl49OoCgd2A3f8EchMSY=
403 +github.com/ipfs/go-fetcher v1.5.0 h1:oreKTKBzja3S09rSmoZlA3KGVlRiUbJ1pQjtB4K6y3w=
404 +github.com/ipfs/go-fetcher v1.5.0/go.mod h1:5pDZ0393oRF/fHiLmtFZtpMNBQfHOYNPtryWedVuSWE=
405 github.com/ipfs/go-filestore v0.0.3 h1:MhZ1jT5K3NewZwim6rS/akcJLm1xM+r6nz6foeB9EwE=
406 github.com/ipfs/go-filestore v0.0.3/go.mod h1:dvXRykFzyyXN2CdNlRGzDAkXMDPyI+D7JE066SiKLSE=
407 github.com/ipfs/go-fs-lock v0.0.7 h1:6BR3dajORFrFTkb5EpCUFIAypsoxpGpDSVUdFwzgL9U=
@@ -442,8 +444,8 @@ github.com/ipfs/go-ipfs-posinfo v0.0.1/go.mod h1:SwyeVP+jCwiDu0C313l/8jg6ZxM0qqt
444 github.com/ipfs/go-ipfs-pq v0.0.1/go.mod h1:LWIqQpqfRG3fNc5XsnIhz/wQ2XXGyugQwls7BgUmUfY=
445 github.com/ipfs/go-ipfs-pq v0.0.2 h1:e1vOOW6MuOwG2lqxcLA+wEn93i/9laCY8sXAw76jFOY=
446 github.com/ipfs/go-ipfs-pq v0.0.2/go.mod h1:LWIqQpqfRG3fNc5XsnIhz/wQ2XXGyugQwls7BgUmUfY=
445 -github.com/ipfs/go-ipfs-provider v0.5.1 h1:kZj72jzWLtGcorlwnMvBL6y6KJk6klO2Kb8QSeqEB0o=
446 -github.com/ipfs/go-ipfs-provider v0.5.1/go.mod h1:fem6HKSru7n35Ljap6kowWdJrUzvcWJW01uhAkqNnzo=
447 +github.com/ipfs/go-ipfs-provider v0.6.1 h1:4VenAH1J5XH+/mgM2Y7p+QN3wlk7CInMDG8rsT2CGW4=
448 +github.com/ipfs/go-ipfs-provider v0.6.1/go.mod h1:I4Cig3InhftbRJohph76Qy/P2uKEZILNGiKvDJmmC28=
449 github.com/ipfs/go-ipfs-routing v0.0.1/go.mod h1:k76lf20iKFxQTjcJokbPM9iBXVXVZhcOwc360N4nuKs=
450 github.com/ipfs/go-ipfs-routing v0.1.0 h1:gAJTT1cEeeLj6/DlLX6t+NxD9fQe2ymTO6qWRDI/HQQ=
451 github.com/ipfs/go-ipfs-routing v0.1.0/go.mod h1:hYoUkJLyAUKhF58tysKpids8RNDPO42BVMgK5dNsoqY=
@@ -459,8 +461,10 @@ github.com/ipfs/go-ipld-format v0.0.1/go.mod h1:kyJtbkDALmFHv3QR6et67i35QzO3S0dC
461 github.com/ipfs/go-ipld-format v0.0.2/go.mod h1:4B6+FM2u9OJ9zCV+kSbgFAZlOrv1Hqbf0INGQgiKf9k=
462 github.com/ipfs/go-ipld-format v0.2.0 h1:xGlJKkArkmBvowr+GMCX0FEZtkro71K1AwiKnL37mwA=
463 github.com/ipfs/go-ipld-format v0.2.0/go.mod h1:3l3C1uKoadTPbeNfrDi+xMInYKlx2Cvg1BuydPSdzQs=
462 -github.com/ipfs/go-ipld-git v0.0.4 h1:fQv2Alq72g6mH+heDWQ9Awu5FQYc3hcCUVtzuWj/Mno=
463 -github.com/ipfs/go-ipld-git v0.0.4/go.mod h1:RuvMXa9qtJpDbqngyICCU/d+cmLFXxLsbIclmD0Lcr0=
464 +github.com/ipfs/go-ipld-git v0.1.0 h1:+HUPKbvOPicyF5J4gdRS3nvQRBgvYVGyvuAigiISkzc=
465 +github.com/ipfs/go-ipld-git v0.1.0/go.mod h1:pfIxVWClpLmyDHDWI7qTva4/cdOQyVzC+0jzl5/n1Sk=
466 +github.com/ipfs/go-ipld-legacy v0.1.0 h1:wxkkc4k8cnvIGIjPO0waJCe7SHEyFgl+yQdafdjGrpA=
467 +github.com/ipfs/go-ipld-legacy v0.1.0/go.mod h1:86f5P/srAmh9GcIcWQR9lfFLZPrIyyXQeVlOWeeWEuI=
468 github.com/ipfs/go-ipns v0.0.2/go.mod h1:WChil4e0/m9cIINWLxZe1Jtf77oz5L05rO2ei/uKJ5U=
469 github.com/ipfs/go-ipns v0.1.0 h1:jk03sneWwh+/bSHrFjRfE38xHDnTzqCsx2wsJ8ipukM=
470 github.com/ipfs/go-ipns v0.1.0/go.mod h1:3IbsuPkR6eAGcnx+E7j6HpOSbSQJPZ6zlRj+NK3jPxQ=
@@ -484,8 +488,9 @@ github.com/ipfs/go-merkledag v0.1.0/go.mod h1:SQiXrtSts3KGNmgOzMICy5c0POOpUNQLvB
488 github.com/ipfs/go-merkledag v0.2.3/go.mod h1:SQiXrtSts3KGNmgOzMICy5c0POOpUNQLvB3ClKnBAlk=
489 github.com/ipfs/go-merkledag v0.3.0/go.mod h1:4pymaZLhSLNVuiCITYrpViD6vmfZ/Ws4n/L9tfNv3S4=
490 github.com/ipfs/go-merkledag v0.3.1/go.mod h1:fvkZNNZixVW6cKSZ/JfLlON5OlgTXNdRLz0p6QG/I2M=
487 -github.com/ipfs/go-merkledag v0.3.2 h1:MRqj40QkrWkvPswXs4EfSslhZ4RVPRbxwX11js0t1xY=
491 github.com/ipfs/go-merkledag v0.3.2/go.mod h1:fvkZNNZixVW6cKSZ/JfLlON5OlgTXNdRLz0p6QG/I2M=
492 +github.com/ipfs/go-merkledag v0.4.0 h1:ixNu/5MJSaT/Qs073T0/HsWKwnOoBgqSq1g+GaJIen0=
493 +github.com/ipfs/go-merkledag v0.4.0/go.mod h1:XshXBkhyeS63YNGisLL1uDSfuTyrQIxVUOg3ojR5MOE=
494 github.com/ipfs/go-metrics-interface v0.0.1 h1:j+cpbjYvu4R8zbleSs36gvB7jR+wsL2fGD6n0jO4kdg=
495 github.com/ipfs/go-metrics-interface v0.0.1/go.mod h1:6s6euYU4zowdslK0GKHmqaIZ3j/b/tL7HTWtJ4VPgWY=
496 github.com/ipfs/go-metrics-prometheus v0.0.2 h1:9i2iljLg12S78OhC6UAiXi176xvQGiZaGVF1CUVdE+s=
@@ -495,8 +500,9 @@ github.com/ipfs/go-mfs v0.1.2/go.mod h1:T1QBiZPEpkPLzDqEJLNnbK55BVKVlNi2a+gVm4di
500 github.com/ipfs/go-namesys v0.3.0 h1:6lytKWj1rG0Ot6J0nTHvFw+06q1a6n7DLA2CbSGmZco=
501 github.com/ipfs/go-namesys v0.3.0/go.mod h1:/BL4xk8LP5Lq82AmaRKyxZv/eYRlumNiU9SZUe1Hlps=
502 github.com/ipfs/go-path v0.0.7/go.mod h1:6KTKmeRnBXgqrTvzFrPV3CamxcgvXX/4z79tfAd2Sno=
498 -github.com/ipfs/go-path v0.0.9 h1:BIi831cNED8YnIlIKo9y1SI3u+E+FwQQD+rIIw8PwFA=
503 github.com/ipfs/go-path v0.0.9/go.mod h1:VpDkSBKQ9EFQOUgi54Tq/O/tGi8n1RfYNks13M3DEs8=
504 +github.com/ipfs/go-path v0.1.1 h1:0rfiI0IoNTYUyQN0ifz2zQBR6mZhOKv7qW5Jjx/4fG8=
505 +github.com/ipfs/go-path v0.1.1/go.mod h1:vC8q4AKOtrjJz2NnllIrmr2ZbGlF5fW2OKKyhV9ggb0=
506 github.com/ipfs/go-peertaskqueue v0.0.4/go.mod h1:03H8fhyeMfKNFWqzYEVyMbcPUeYrqP1MX6Kd+aN+rMQ=
507 github.com/ipfs/go-peertaskqueue v0.1.0/go.mod h1:Jmk3IyCcfl1W3jTW3YpghSwSEC6IJ3Vzz/jUmWw8Z0U=
508 github.com/ipfs/go-peertaskqueue v0.1.1/go.mod h1:Jmk3IyCcfl1W3jTW3YpghSwSEC6IJ3Vzz/jUmWw8Z0U=
@@ -508,19 +514,24 @@ github.com/ipfs/go-unixfs v0.1.0/go.mod h1:lysk5ELhOso8+Fed9U1QTGey2ocsfaZ18h0NC
514 github.com/ipfs/go-unixfs v0.2.4/go.mod h1:SUdisfUjNoSDzzhGVxvCL9QO/nKdwXdr+gbMUdqcbYw=
515 github.com/ipfs/go-unixfs v0.2.5 h1:irj/WzIcgTBay48mSMUYDbKlIzIocXWcuUUsi5qOMOE=
516 github.com/ipfs/go-unixfs v0.2.5/go.mod h1:SUdisfUjNoSDzzhGVxvCL9QO/nKdwXdr+gbMUdqcbYw=
517 +github.com/ipfs/go-unixfsnode v1.1.2 h1:aTsCdhwU0F4dMShMwYGroAj4v4EzSONLdoENebvTRb0=
518 +github.com/ipfs/go-unixfsnode v1.1.2/go.mod h1:5dcE2x03pyjHk4JjamXmunTMzz+VUtqvPwZjIEkfV6s=
519 github.com/ipfs/go-verifcid v0.0.1 h1:m2HI7zIuR5TFyQ1b79Da5N9dnnCP1vcu2QqawmWlK2E=
520 github.com/ipfs/go-verifcid v0.0.1/go.mod h1:5Hrva5KBeIog4A+UpqlaIU+DEstipcJYQQZc0g37pY0=
513 -github.com/ipfs/interface-go-ipfs-core v0.4.0 h1:+mUiamyHIwedqP8ZgbCIwpy40oX7QcXUbo4CZOeJVJg=
521 github.com/ipfs/interface-go-ipfs-core v0.4.0/go.mod h1:UJBcU6iNennuI05amq3FQ7g0JHUkibHFAfhfUIy927o=
522 +github.com/ipfs/interface-go-ipfs-core v0.5.1 h1:1KMM7RkjUD8W5fSoRsa9xR6ZMzeL8fLHOUM1UEW9Y4M=
523 +github.com/ipfs/interface-go-ipfs-core v0.5.1/go.mod h1:lNBJrdXHtWS46evMPBdWtDQMDsrKcGbxCOGoKLkztOE=
524 github.com/ipfs/tar-utils v0.0.1 h1:8Na0KBD6GddGyXwU4rXNtVTE24iuZws8mENJQPLG7W4=
525 github.com/ipfs/tar-utils v0.0.1/go.mod h1:ACflm9wXvV9w0eMJt6yYXxS2zuIV+yXGNwbuq1bhLeE=
526 github.com/ipld/go-car v0.3.1 h1:WT+3cdmXlvmWOlGxk9webhj4auGO5QvgqC2vCCkFRXs=
527 github.com/ipld/go-car v0.3.1/go.mod h1:dPkEWeAK8KaVvH5TahaCs6Mncpd4lDMpkbs0/SPzuVs=
519 -github.com/ipld/go-codec-dagpb v1.2.0 h1:2umV7ud8HBMkRuJgd8gXw95cLhwmcYrihS3cQEy9zpI=
528 github.com/ipld/go-codec-dagpb v1.2.0/go.mod h1:6nBN7X7h8EOsEejZGqC7tej5drsdBAXbMHyBT+Fne5s=
529 +github.com/ipld/go-codec-dagpb v1.3.0 h1:czTcaoAuNNyIYWs6Qe01DJ+sEX7B+1Z0LcXjSatMGe8=
530 +github.com/ipld/go-codec-dagpb v1.3.0/go.mod h1:ga4JTU3abYApDC3pZ00BC2RSvC3qfBb9MSJkMLSwnhA=
531 github.com/ipld/go-ipld-prime v0.9.0/go.mod h1:KvBLMr4PX1gWptgkzRjVZCrLmSGcZCb/jioOQwCqZN8=
522 -github.com/ipld/go-ipld-prime v0.9.1-0.20210324083106-dc342a9917db h1:kFwGn8rXa/Z31ev1OFNQsYeNKNCdifnTPl/NvPy5L38=
532 github.com/ipld/go-ipld-prime v0.9.1-0.20210324083106-dc342a9917db/go.mod h1:KvBLMr4PX1gWptgkzRjVZCrLmSGcZCb/jioOQwCqZN8=
533 +github.com/ipld/go-ipld-prime v0.11.0 h1:jD/b/22R7CSL+F9xNffcexs+wO0Ji/TfwXO/TWck+70=
534 +github.com/ipld/go-ipld-prime v0.11.0/go.mod h1:+WIAkokurHmZ/KwzDOMUuoeJgaRQktHtEaLglS3ZeV8=
535 github.com/jackpal/gateway v1.0.5/go.mod h1:lTpwd4ACLXmpyiCTRtfiNyVnUmqT9RivzCDQetPfnjA=
536 github.com/jackpal/go-nat-pmp v1.0.1/go.mod h1:QPH045xvCAeXUZOxsnwmrtiCoxIr9eob+4orBN1SBKc=
537 github.com/jackpal/go-nat-pmp v1.0.2 h1:KzKSgb7qkJvOUTqYl9/Hg/me3pWgBmERKrTGD7BdWus=
@@ -565,8 +576,9 @@ github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+o
576 github.com/kkdai/bstream v0.0.0-20161212061736-f391b8402d23/go.mod h1:J+Gs4SYgM6CZQHDETBtE9HaSEkGmuNXF86RwHhHUvq4=
577 github.com/klauspost/compress v1.11.7 h1:0hzRabrMN4tSTvMfnL3SCv1ZGeAP23ynzodBgaHeMeg=
578 github.com/klauspost/compress v1.11.7/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
568 -github.com/klauspost/cpuid/v2 v2.0.4 h1:g0I61F2K2DjRHz1cnxlkNSBIaePVoJIjjnHui8QHbiw=
579 github.com/klauspost/cpuid/v2 v2.0.4/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
580 +github.com/klauspost/cpuid/v2 v2.0.6 h1:dQ5ueTiftKxp0gyjKSx5+8BtPWkyQbd95m8Gys/RarI=
581 +github.com/klauspost/cpuid/v2 v2.0.6/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
582 github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
583 github.com/konsorten/go-windows-terminal-sequences v1.0.3 h1:CE8S1cTafDpPvMhIxNJKvHsGVBgn1xWYf1NbHQhywc8=
584 github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
@@ -595,8 +607,6 @@ github.com/libp2p/go-conn-security-multistream v0.0.2/go.mod h1:nc9vud7inQ+d6SO0
607 github.com/libp2p/go-conn-security-multistream v0.1.0/go.mod h1:aw6eD7LOsHEX7+2hJkDxw1MteijaVcI+/eP2/x3J1xc=
608 github.com/libp2p/go-conn-security-multistream v0.2.0/go.mod h1:hZN4MjlNetKD3Rq5Jb/P5ohUnFLNzEAR4DLSzpn2QLU=
609 github.com/libp2p/go-conn-security-multistream v0.2.1 h1:ft6/POSK7F+vl/2qzegnHDaXFU0iWB4yVTYrioC6Zy0=
598 -github.com/libp2p/go-conn-security-multistream v0.2.1 h1:ft6/POSK7F+vl/2qzegnHDaXFU0iWB4yVTYrioC6Zy0=
599 -github.com/libp2p/go-conn-security-multistream v0.2.1/go.mod h1:cR1d8gA0Hr59Fj6NhaTpFhJZrjSYuNmhpT2r25zYR70=
610 github.com/libp2p/go-conn-security-multistream v0.2.1/go.mod h1:cR1d8gA0Hr59Fj6NhaTpFhJZrjSYuNmhpT2r25zYR70=
611 github.com/libp2p/go-doh-resolver v0.3.1 h1:1wbVGkB4Tdj4WEvjAuYknOPyt4vSSDn9thnj13pKPaY=
612 github.com/libp2p/go-doh-resolver v0.3.1/go.mod h1:y5go1ZppAq9N2eppbX0xON01CyPBeUg2yS6BTssssog=
@@ -616,6 +626,7 @@ github.com/libp2p/go-libp2p v0.7.4/go.mod h1:oXsBlTLF1q7pxr+9w6lqzS1ILpyHsaBPniV
626 github.com/libp2p/go-libp2p v0.8.1/go.mod h1:QRNH9pwdbEBpx5DTJYg+qxcVaDMAz3Ee/qDKwXujH5o=
627 github.com/libp2p/go-libp2p v0.12.0/go.mod h1:FpHZrfC1q7nA8jitvdjKBDF31hguaC676g/nT9PgQM0=
628 github.com/libp2p/go-libp2p v0.13.0/go.mod h1:pM0beYdACRfHO1WcJlp65WXyG2A6NqYM+t2DTVAJxMo=
629 +github.com/libp2p/go-libp2p v0.14.0/go.mod h1:dsQrWLAoIn+GkHPN/U+yypizkHiB9tnv79Os+kSgQ4Q=
630 github.com/libp2p/go-libp2p v0.14.4 h1:QCJE+jGyqxWdrSPuS4jByXCzosgaIg4SJTLCRplJ53w=
631 github.com/libp2p/go-libp2p v0.14.4/go.mod h1:EIRU0Of4J5S8rkockZM7eJp2S0UrCyi55m2kJVru3rM=
632 github.com/libp2p/go-libp2p-asn-util v0.0.0-20200825225859-85005c6cf052 h1:BM7aaOF7RpmNn9+9g6uTjGJ0cTzWr5j9i9IKeun2M8U=
@@ -805,6 +816,7 @@ github.com/libp2p/go-libp2p-yamux v0.2.8/go.mod h1:/t6tDqeuZf0INZMTgd0WxIRbtK2Ez
816 github.com/libp2p/go-libp2p-yamux v0.4.0/go.mod h1:+DWDjtFMzoAwYLVkNZftoucn7PelNoy5nm3tZ3/Zw30=
817 github.com/libp2p/go-libp2p-yamux v0.5.0/go.mod h1:AyR8k5EzyM2QN9Bbdg6X1SkVVuqLwTGf0L4DFq9g6po=
818 github.com/libp2p/go-libp2p-yamux v0.5.1/go.mod h1:dowuvDu8CRWmr0iqySMiSxK+W0iL5cMVO9S94Y6gkv4=
819 +github.com/libp2p/go-libp2p-yamux v0.5.3/go.mod h1:Vy3TMonBAfTMXHWopsMc8iX/XGRYrRlpUaMzaeuHV/s=
820 github.com/libp2p/go-libp2p-yamux v0.5.4 h1:/UOPtT/6DHPtr3TtKXBHa6g0Le0szYuI33Xc/Xpd7fQ=
821 github.com/libp2p/go-libp2p-yamux v0.5.4/go.mod h1:tfrXbyaTqqSU654GTvK3ocnSZL3BuHoeTSqhcel1wsE=
822 github.com/libp2p/go-maddr-filter v0.0.1/go.mod h1:6eT12kSQMA9x2pvFQa+xesMKUBlj9VImZbj3B9FBH/Q=
@@ -886,6 +898,7 @@ github.com/libp2p/go-yamux v1.4.0/go.mod h1:fr7aVgmdNGJK+N1g+b6DW6VxzbRCjCOejR/h
898 github.com/libp2p/go-yamux v1.4.1 h1:P1Fe9vF4th5JOxxgQvfbOHkrGqIZniTLf+ddhZp8YTI=
899 github.com/libp2p/go-yamux v1.4.1/go.mod h1:fr7aVgmdNGJK+N1g+b6DW6VxzbRCjCOejR/hkmpooHE=
900 github.com/libp2p/go-yamux/v2 v2.0.0/go.mod h1:NVWira5+sVUIU6tu1JWvaRn1dRnG+cawOJiflsAM+7U=
901 +github.com/libp2p/go-yamux/v2 v2.1.1/go.mod h1:3So6P6TV6r75R9jiBpiIKgU/66lOarCZjqROGxzPpPQ=
902 github.com/libp2p/go-yamux/v2 v2.2.0 h1:RwtpYZ2/wVviZ5+3pjC8qdQ4TKnrak0/E01N1UWoAFU=
903 github.com/libp2p/go-yamux/v2 v2.2.0/go.mod h1:3So6P6TV6r75R9jiBpiIKgU/66lOarCZjqROGxzPpPQ=
904 github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20190605223551-bc2310a04743/go.mod h1:qklhhLq1aX+mtWk9cPHPzaBjWImj5ULL6C7HFJtXQMM=
@@ -991,8 +1004,6 @@ github.com/multiformats/go-multiaddr-dns v0.0.2/go.mod h1:9kWcqw/Pj6FwxAwW38n/94
1004 github.com/multiformats/go-multiaddr-dns v0.2.0/go.mod h1:TJ5pr5bBO7Y1B18djPuRsVkduhQH2YqYSbxWJzYGdK0=
1005 github.com/multiformats/go-multiaddr-dns v0.3.0/go.mod h1:mNzQ4eTGDg0ll1N9jKPOUogZPoJ30W8a7zk66FQPpdQ=
1006 github.com/multiformats/go-multiaddr-dns v0.3.1 h1:QgQgR+LQVt3NPTjbrLLpsaT2ufAA2y0Mkk+QRVJbW3A=
994 -github.com/multiformats/go-multiaddr-dns v0.3.1 h1:QgQgR+LQVt3NPTjbrLLpsaT2ufAA2y0Mkk+QRVJbW3A=
995 -github.com/multiformats/go-multiaddr-dns v0.3.1/go.mod h1:G/245BRQ6FJGmryJCrOuTdB37AMA5AMOVuO6NY3JwTk=
1007 github.com/multiformats/go-multiaddr-dns v0.3.1/go.mod h1:G/245BRQ6FJGmryJCrOuTdB37AMA5AMOVuO6NY3JwTk=
1008 github.com/multiformats/go-multiaddr-fmt v0.0.1/go.mod h1:aBYjqL4T/7j4Qx+R73XSv/8JsgnRFlf0w2KGLCmXl3Q=
1009 github.com/multiformats/go-multiaddr-fmt v0.1.0 h1:WLEFClPycPkp4fnIzoFoV9FVd49/eQsuaL3/CWe167E=
@@ -1009,8 +1020,9 @@ github.com/multiformats/go-multiaddr-net v0.2.0/go.mod h1:gGdH3UXny6U3cKKYCvpXI5
1020 github.com/multiformats/go-multibase v0.0.1/go.mod h1:bja2MqRZ3ggyXtZSEDKpl0uO/gviWFaSteVbWT51qgs=
1021 github.com/multiformats/go-multibase v0.0.3 h1:l/B6bJDQjvQ5G52jw4QGSYeOTZoAwIO77RblWplfIqk=
1022 github.com/multiformats/go-multibase v0.0.3/go.mod h1:5+1R4eQrT3PkYZ24C3W2Ue2tPwIdYQD509ZjSb5y9Oc=
1012 -github.com/multiformats/go-multicodec v0.2.0 h1:MUzKZWxOFagwLLtlx96pub9zwDQAbMAf1k9fXOdc3so=
1023 github.com/multiformats/go-multicodec v0.2.0/go.mod h1:/y4YVwkfMyry5kFbMTbLJKErhycTIftytRV+llXdyS4=
1024 +github.com/multiformats/go-multicodec v0.3.0 h1:tstDwfIjiHbnIjeM5Lp+pMrSeN+LCMsEwOrkPmWm03A=
1025 +github.com/multiformats/go-multicodec v0.3.0/go.mod h1:qGGaQmioCDh+TeFOnxrbU0DaIPw8yFgAZgFG0V7p1qQ=
1026 github.com/multiformats/go-multihash v0.0.1/go.mod h1:w/5tugSrLEbWqlcgJabL3oHFKTwfvkofsjW2Qa1ct4U=
1027 github.com/multiformats/go-multihash v0.0.5/go.mod h1:lt/HCbqlQwlPBz7lv0sQCdtfcMtlJvakRUn/0Ual8po=
1028 github.com/multiformats/go-multihash v0.0.8/go.mod h1:YSLudS+Pi8NHE7o6tb3D8vrpKa63epEDmG8nTduyAew=
@@ -1026,7 +1038,6 @@ github.com/multiformats/go-multistream v0.1.0/go.mod h1:fJTiDfXJVmItycydCnNx4+wS
1038 github.com/multiformats/go-multistream v0.1.1/go.mod h1:KmHZ40hzVxiaiwlj3MEbYgK9JFk2/9UktWZAF54Du38=
1039 github.com/multiformats/go-multistream v0.2.0/go.mod h1:5GZPQZbkWOLOn3J2y4Y99vVW7vOfsAflxARk3x14o6k=
1040 github.com/multiformats/go-multistream v0.2.1/go.mod h1:5GZPQZbkWOLOn3J2y4Y99vVW7vOfsAflxARk3x14o6k=
1029 -github.com/multiformats/go-multistream v0.2.1/go.mod h1:5GZPQZbkWOLOn3J2y4Y99vVW7vOfsAflxARk3x14o6k=
1041 github.com/multiformats/go-multistream v0.2.2 h1:TCYu1BHTDr1F/Qm75qwYISQdzGcRdC21nFgQW7l7GBo=
1042 github.com/multiformats/go-multistream v0.2.2/go.mod h1:UIcnm7Zuo8HKG+HkWgfQsGL+/MIEhyTqbODbIUwSXKs=
1043 github.com/multiformats/go-varint v0.0.1/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE=
@@ -1299,8 +1310,9 @@ go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/
1310 go.uber.org/multierr v1.3.0/go.mod h1:VgVr7evmIr6uPjLBxg28wmKNXyqE9akIJ5XnfpiKl+4=
1311 go.uber.org/multierr v1.4.0/go.mod h1:VgVr7evmIr6uPjLBxg28wmKNXyqE9akIJ5XnfpiKl+4=
1312 go.uber.org/multierr v1.5.0/go.mod h1:FeouvMocqHpRaaGuG9EjoKcStLC43Zu/fmqdUMPcKYU=
1302 -go.uber.org/multierr v1.6.0 h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4=
1313 go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=
1314 +go.uber.org/multierr v1.7.0 h1:zaiO/rmgFjbmCXdSYJWQcdvOCsthmdaHfr3Gm2Kx4Ec=
1315 +go.uber.org/multierr v1.7.0/go.mod h1:7EAYxJLBy9rStEaz58O2t4Uvip6FSURkq8/ppBp95ak=
1316 go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee/go.mod h1:vJERXedbb3MVM5f9Ejo0C68/HhF8uaILCdgjnY+goOA=
1317 go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
1318 go.uber.org/zap v1.13.0/go.mod h1:zwrFLgMcdUuIBviXEYEH1YKNaOBnKXsx2IPda5bBwHM=
@@ -1340,10 +1352,10 @@ golang.org/x/crypto v0.0.0-20200602180216-279210d13fed/go.mod h1:LzIPMQfyMNhhGPh
1352 golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
1353 golang.org/x/crypto v0.0.0-20200820211705-5c72a883971a/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
1354 golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
1343 -golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
1355 golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
1345 -golang.org/x/crypto v0.0.0-20210506145944-38f3c27a63bf h1:B2n+Zi5QeYRDAEodEu72OS36gmTWjgpXr2+cWcBW90o=
1356 golang.org/x/crypto v0.0.0-20210506145944-38f3c27a63bf/go.mod h1:P+XmwS30IXTQdn5tA2iutPOUgjI07+tq3H3K9MVA1s8=
1357 +golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a h1:kr2P4QFmQr29mSLA43kwrOcgcReGTfbE9N577tCTuBc=
1358 +golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a/go.mod h1:P+XmwS30IXTQdn5tA2iutPOUgjI07+tq3H3K9MVA1s8=
1359 golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
1360 golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
1361 golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -1499,7 +1511,6 @@ golang.org/x/sys v0.0.0-20201214210602-f9fddec55a1e/go.mod h1:h1NjWce9XRLGQEsW7w
1511 golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
1512 golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
1513 golang.org/x/sys v0.0.0-20210303074136-134d130e1a04/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
1502 -golang.org/x/sys v0.0.0-20210303074136-134d130e1a04/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
1514 golang.org/x/sys v0.0.0-20210309074719-68d13333faf2/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
1515 golang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
1516 golang.org/x/sys v0.0.0-20210317225723-c4fcb01b228e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -1507,8 +1518,9 @@ golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7w
1518 golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
1519 golang.org/x/sys v0.0.0-20210426080607-c94f62235c83/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
1520 golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
1510 -golang.org/x/sys v0.0.0-20210511113859-b0526f3d8744 h1:yhBbb4IRs2HS9PPlAg6DMC6mUOKexJBNsLf4Z+6En1Q=
1521 golang.org/x/sys v0.0.0-20210511113859-b0526f3d8744/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
1522 +golang.org/x/sys v0.0.0-20210514084401-e8d321eab015 h1:hZR0X1kPW+nwyJ9xRxqZk1vx5RUObAPBdKVvXPDUH/E=
1523 +golang.org/x/sys v0.0.0-20210514084401-e8d321eab015/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
1524 golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
1525 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 h1:v+OssWQX+hTHEmOBgwxdZxK4zHq3yOs8F9J7mk0PY8E=
1526 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
@@ -1641,8 +1653,9 @@ google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpAD
1653 google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
1654 google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
1655 google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
1644 -google.golang.org/protobuf v1.26.0 h1:bxAC2xTBsZGibn2RTntX0oH50xLsqy1OxA9tTL3p/lk=
1656 google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
1657 +google.golang.org/protobuf v1.27.1 h1:SnqbnDw1V7RiZcXPx5MEeqPv2s79L9i7BJUlG/+RurQ=
1658 +google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
1659 gopkg.in/alecthomas/kingpin.v2 v2.2.6 h1:jMFz6MfLP0/4fUyZle81rXUoxOBFi19VUFKVDOQfozc=
1660 gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
1661 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
plugin/ipld.go
+3 -6
@@ -1,16 +1,13 @@
1 package plugin
2
3 import (
4 - "github.com/ipfs/go-ipfs/core/coredag"
5 -
6 - ipld "github.com/ipfs/go-ipld-format"
4 + multicodec "github.com/ipld/go-ipld-prime/multicodec"
5 )
6
7 // PluginIPLD is an interface that can be implemented to add handlers for
10 -// for different IPLD formats
8 +// for different IPLD codecs
9 type PluginIPLD interface {
10 Plugin
11
14 - RegisterBlockDecoders(dec ipld.BlockDecoder) error
15 - RegisterInputEncParsers(iec coredag.InputEncParsers) error
12 + Register(multicodec.Registry) error
13 }
plugin/loader/loader.go
+2 -7
@@ -10,14 +10,13 @@ import (
10
11 config "github.com/ipfs/go-ipfs-config"
12 cserialize "github.com/ipfs/go-ipfs-config/serialize"
13 + "github.com/ipld/go-ipld-prime/multicodec"
14
15 "github.com/ipfs/go-ipfs/core"
16 "github.com/ipfs/go-ipfs/core/coreapi"
16 - coredag "github.com/ipfs/go-ipfs/core/coredag"
17 plugin "github.com/ipfs/go-ipfs/plugin"
18 fsrepo "github.com/ipfs/go-ipfs/repo/fsrepo"
19
20 - ipld "github.com/ipfs/go-ipld-format"
20 logging "github.com/ipfs/go-log"
21 opentracing "github.com/opentracing/opentracing-go"
22 )
@@ -335,11 +334,7 @@ func injectDatastorePlugin(pl plugin.PluginDatastore) error {
334 }
335
336 func injectIPLDPlugin(pl plugin.PluginIPLD) error {
338 - err := pl.RegisterBlockDecoders(ipld.DefaultBlockDecoder)
339 - if err != nil {
340 - return err
341 - }
342 - return pl.RegisterInputEncParsers(coredag.DefaultInputEncParsers)
337 + return pl.Register(multicodec.DefaultRegistry)
338 }
339
340 func injectTracerPlugin(pl plugin.PluginTracer) error {
plugin/plugins/git/git.go
+13 -34
@@ -2,17 +2,15 @@ package git
2
3 import (
4 "compress/zlib"
5 - "fmt"
5 "io"
7 - "math"
6
9 - "github.com/ipfs/go-ipfs/core/coredag"
7 "github.com/ipfs/go-ipfs/plugin"
8
12 - "github.com/ipfs/go-cid"
13 - "github.com/ipfs/go-ipld-format"
9 + // Note that depending on this package registers it's multicodec encoder and decoder.
10 git "github.com/ipfs/go-ipld-git"
15 - mh "github.com/multiformats/go-multihash"
11 + "github.com/ipld/go-ipld-prime"
12 + "github.com/ipld/go-ipld-prime/multicodec"
13 + mc "github.com/multiformats/go-multicodec"
14 )
15
16 // Plugins is exported list of plugins that will be loaded
@@ -36,40 +34,21 @@ func (*gitPlugin) Init(_ *plugin.Environment) error {
34 return nil
35 }
36
39 -func (*gitPlugin) RegisterBlockDecoders(dec format.BlockDecoder) error {
40 - dec.Register(cid.GitRaw, git.DecodeBlock)
37 +func (*gitPlugin) Register(reg multicodec.Registry) error {
38 + // register a custom identifier in the reserved range for import of "zlib-encoded git objects."
39 + reg.RegisterDecoder(uint64(mc.ReservedStart+mc.GitRaw), decodeZlibGit)
40 + reg.RegisterEncoder(uint64(mc.GitRaw), git.Encode)
41 + reg.RegisterDecoder(uint64(mc.GitRaw), git.Decode)
42 return nil
43 }
44
44 -func (*gitPlugin) RegisterInputEncParsers(iec coredag.InputEncParsers) error {
45 - iec.AddParser("raw", "git", parseRawGit)
46 - iec.AddParser("zlib", "git", parseZlibGit)
47 - return nil
48 -}
49 -
50 -func parseRawGit(r io.Reader, mhType uint64, mhLen int) ([]format.Node, error) {
51 - if mhType != math.MaxUint64 && mhType != mh.SHA1 {
52 - return nil, fmt.Errorf("unsupported mhType %d", mhType)
53 - }
54 -
55 - if mhLen != -1 && mhLen != mh.DefaultLengths[mh.SHA1] {
56 - return nil, fmt.Errorf("invalid mhLen %d", mhLen)
57 - }
58 -
59 - nd, err := git.ParseObject(r)
60 - if err != nil {
61 - return nil, err
62 - }
63 -
64 - return []format.Node{nd}, nil
65 -}
66 -
67 -func parseZlibGit(r io.Reader, mhType uint64, mhLen int) ([]format.Node, error) {
45 +func decodeZlibGit(na ipld.NodeAssembler, r io.Reader) error {
46 rc, err := zlib.NewReader(r)
47 if err != nil {
70 - return nil, err
48 + return err
49 }
50
51 defer rc.Close()
74 - return parseRawGit(rc, mhType, mhLen)
52 +
53 + return git.Decode(na, rc)
54 }
test/sharness/t0053-dag.sh
+176 -43
@@ -21,45 +21,158 @@ test_expect_success "make a few test files" '
21 HASH4=$(ipfs add --pin=false -q file4)
22 '
23
24 -test_expect_success "make an ipld object in json" '
24 +test_expect_success "make an ipld object in dag-json" '
25 printf "{\"hello\":\"world\",\"cats\":[{\"/\":\"%s\"},{\"water\":{\"/\":\"%s\"}}],\"magic\":{\"/\":\"%s\"},\"sub\":{\"dict\":\"ionary\",\"beep\":[0,\"bop\"]}}" $HASH1 $HASH2 $HASH3 > ipld_object
26 '
27
28 +# This data is in https://github.com/ipld/codec-fixtures/tree/master/fixtures/dagpb_Data_some
29 +test_expect_success "make the same ipld object in dag-cbor, dag-json and dag-pb" '
30 + echo "omREYXRhRQABAgMEZUxpbmtzgA==" | base64 -d > ipld_object_dagcbor
31 + echo "CgUAAQIDBA==" | base64 -d > ipld_object_dagpb
32 + echo "{\"Data\":{\"/\":{\"bytes\":\"AAECAwQ\"}},\"Links\":[]}" > ipld_object_dagjson
33 +'
34 +
35 test_dag_cmd() {
29 - test_expect_success "can add an ipld object using protobuf" '
30 - IPLDHASH=$(cat ipld_object | ipfs dag put -f protobuf)
36 + # Working with a plain IPLD hello-world object that's dag-json and dag-cbor compatible
37 +
38 + test_expect_success "can add an ipld object using defaults (dag-json to dag-cbor)" '
39 + IPLDHASH=$(cat ipld_object | ipfs dag put)
40 '
41
33 - test_expect_success "output looks correct" '
34 - EXPHASH="QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n"
42 + test_expect_success "CID looks correct" '
43 + EXPHASH="bafyreiblwimnjbqcdoeafiobk6q27jcw64ew7n2fmmhdpldd63edmjecde"
44 test $EXPHASH = $IPLDHASH
45 '
46
38 - test_expect_success "can add an ipld object using protobuf and --cid=base=base32" '
39 - IPLDHASHb32=$(cat ipld_object | ipfs dag put -f protobuf --cid-base=base32)
47 + test_expect_success "can add an ipld object using dag-json to dag-json" '
48 + IPLDHASH=$(cat ipld_object | ipfs dag put --input-enc=dag-json -f dag-json)
49 '
50
42 - test_expect_success "output looks correct (does not upgrade to CIDv1)" '
43 - test $EXPHASH = $IPLDHASHb32
51 + test_expect_success "CID looks correct" '
52 + EXPHASH="baguqeera6gviseelmbzn2ugoddo5vulxlshqs3kw5ymgsb6w4cabnoh4ldpa"
53 + test $EXPHASH = $IPLDHASH
54 '
55
46 - test_expect_success "can add an ipld object" '
47 - IPLDHASH=$(cat ipld_object | ipfs dag put)
56 + test_expect_success "can add an ipld object using dag-json to dag-cbor" '
57 + IPLDHASH=$(cat ipld_object | ipfs dag put --input-enc=dag-json -f dag-cbor)
58 '
59
50 - test_expect_success "output looks correct" '
51 - EXPHASH="bafyreidjtjfmavdk7epvztob2m5vlm3pxp3gjmpyewro4qlbw5n4f4iz64"
60 + test_expect_success "CID looks correct" '
61 + EXPHASH="bafyreiblwimnjbqcdoeafiobk6q27jcw64ew7n2fmmhdpldd63edmjecde"
62 test $EXPHASH = $IPLDHASH
63 '
64
55 - test_expect_success "can add an ipld object using --cid-base=base32" '
56 - IPLDHASHb32=$(cat ipld_object | ipfs dag put --cid-base=base32)
65 + test_expect_success "can add an ipld object using cid-base=base58btc" '
66 + IPLDb58HASH=$(cat ipld_object | ipfs dag put -cid-base=base58btc)
67 '
68
59 - test_expect_success "output looks correct" '
60 - test $(ipfs cid base32 $EXPHASH) = $IPLDHASHb32
69 + test_expect_success "CID looks correct" '
70 + EXPHASH="zdpuAoN1XJ3GsrxEzMuCbRKZzRUVJekJUCbPVgCgE4D9yYqVi"
71 + test $EXPHASH = $IPLDb58HASH
72 + '
73 +
74 + # Same object, different forms
75 + # (1) dag-cbor input
76 +
77 + test_expect_success "can add a dag-cbor input block stored as dag-cbor" '
78 + IPLDCBORHASH=$(cat ipld_object_dagcbor | ipfs dag put --input-enc=dag-cbor -f dag-cbor)
79 + '
80 +
81 + test_expect_success "dag-cbor CID looks correct" '
82 + EXPHASH="bafyreieculsmrexh3ty5jentbvuku452o27mst4h2tq2rb2zntqhgcstji"
83 + test $EXPHASH = $IPLDCBORHASH
84 + '
85 +
86 + test_expect_success "can add a dag-cbor input block stored as dag-pb" '
87 + IPLDPBHASH=$(cat ipld_object_dagcbor | ipfs dag put --input-enc=dag-cbor -f dag-pb)
88 + '
89 +
90 + test_expect_success "dag-pb CID looks correct" '
91 + EXPHASH="bafybeibazl2z4vqp2tmwcfag6wirmtpnomxknqcgrauj7m2yisrz3qjbom"
92 + test $EXPHASH = $IPLDPBHASH
93 + '
94 +
95 + test_expect_success "can add a dag-cbor input block stored as dag-json" '
96 + IPLDJSONHASH=$(cat ipld_object_dagcbor | ipfs dag put --input-enc=dag-cbor -f dag-json)
97 + '
98 +
99 + test_expect_success "dag-json CID looks correct" '
100 + EXPHASH="baguqeerajwksxu3lxpomdwxvosl542zl3xknhjgxtq3277gafrhl6vdw5tcq"
101 + test $EXPHASH = $IPLDJSONHASH
102 + '
103 +
104 + # (2) dag-json input
105 +
106 + test_expect_success "can add a dag-json input block stored as dag-cbor" '
107 + IPLDCBORHASH=$(cat ipld_object_dagjson | ipfs dag put --input-enc=dag-json -f dag-cbor)
108 + '
109 +
110 + test_expect_success "dag-cbor CID looks correct" '
111 + EXPHASH="bafyreieculsmrexh3ty5jentbvuku452o27mst4h2tq2rb2zntqhgcstji"
112 + test $EXPHASH = $IPLDCBORHASH
113 + '
114 +
115 + test_expect_success "can add a dag-json input block stored as dag-pb" '
116 + IPLDPBHASH=$(cat ipld_object_dagjson | ipfs dag put --input-enc=dag-json -f dag-pb)
117 + '
118 +
119 + test_expect_success "dag-pb CID looks correct" '
120 + EXPHASH="bafybeibazl2z4vqp2tmwcfag6wirmtpnomxknqcgrauj7m2yisrz3qjbom"
121 + test $EXPHASH = $IPLDPBHASH
122 + '
123 +
124 + test_expect_success "can add a dag-json input block stored as dag-json" '
125 + IPLDJSONHASH=$(cat ipld_object_dagjson | ipfs dag put --input-enc=dag-json -f dag-json)
126 '
127
128 + test_expect_success "dag-json CID looks correct" '
129 + EXPHASH="baguqeerajwksxu3lxpomdwxvosl542zl3xknhjgxtq3277gafrhl6vdw5tcq"
130 + test $EXPHASH = $IPLDJSONHASH
131 + '
132 +
133 + # (3) dag-pb input
134 +
135 + test_expect_success "can add a dag-pb input block stored as dag-cbor" '
136 + IPLDCBORHASH=$(cat ipld_object_dagpb | ipfs dag put --input-enc=dag-pb -f dag-cbor)
137 + '
138 +
139 + test_expect_success "dag-cbor CID looks correct" '
140 + EXPHASH="bafyreieculsmrexh3ty5jentbvuku452o27mst4h2tq2rb2zntqhgcstji"
141 + test $EXPHASH = $IPLDCBORHASH
142 + '
143 +
144 + test_expect_success "can add a dag-pb input block stored as dag-pb" '
145 + IPLDPBHASH=$(cat ipld_object_dagpb | ipfs dag put --input-enc=dag-pb -f dag-pb)
146 + '
147 +
148 + test_expect_success "dag-pb CID looks correct" '
149 + EXPHASH="bafybeibazl2z4vqp2tmwcfag6wirmtpnomxknqcgrauj7m2yisrz3qjbom"
150 + test $EXPHASH = $IPLDPBHASH
151 + '
152 +
153 + test_expect_success "can add a dag-pb input block stored as dag-json" '
154 + IPLDJSONHASH=$(cat ipld_object_dagpb | ipfs dag put --input-enc=dag-pb -f dag-json)
155 + '
156 +
157 + test_expect_success "dag-json CID looks correct" '
158 + EXPHASH="baguqeerajwksxu3lxpomdwxvosl542zl3xknhjgxtq3277gafrhl6vdw5tcq"
159 + test $EXPHASH = $IPLDJSONHASH
160 + '
161 +
162 + test_expect_success "can get dag-cbor, dag-json, dag-pb blocks as dag-json" '
163 + ipfs dag get $IPLDCBORHASH >& dag-get-cbor &&
164 + ipfs dag get $IPLDJSONHASH >& dag-get-json &&
165 + ipfs dag get $IPLDPBHASH >& dag-get-pb
166 + '
167 +
168 + # this doesn't tell us if they are correct, we test that better below
169 + test_expect_success "outputs are the same" '
170 + test_cmp dag-get-cbor dag-get-json &&
171 + test_cmp dag-get-cbor dag-get-pb
172 + '
173 +
174 + # Traversals using the original hello-world object
175 +
176 test_expect_success "various path traversals work" '
177 ipfs cat $IPLDHASH/cats/0 > out1 &&
178 ipfs cat $IPLDHASH/cats/1/water > out2 &&
@@ -81,25 +194,46 @@ test_dag_cmd() {
194 '
195
196 test_expect_success "sub-objects look right" '
84 - echo "\"world\"" > sub1_exp &&
197 + echo -n "\"world\"" > sub1_exp &&
198 test_cmp sub1_exp sub1 &&
86 - echo "{\"beep\":[0,\"bop\"],\"dict\":\"ionary\"}" > sub2_exp &&
199 + echo -n "{\"beep\":[0,\"bop\"],\"dict\":\"ionary\"}" > sub2_exp &&
200 test_cmp sub2_exp sub2 &&
88 - echo "[0,\"bop\"]" > sub3_exp &&
201 + echo -n "[0,\"bop\"]" > sub3_exp &&
202 test_cmp sub3_exp sub3 &&
90 - echo "0" > sub4_exp &&
203 + echo -n "0" > sub4_exp &&
204 test_cmp sub4_exp sub4 &&
92 - echo "\"bop\"" > sub5_exp &&
205 + echo -n "\"bop\"" > sub5_exp &&
206 test_cmp sub5_exp sub5
207 '
208
96 - test_expect_success "can pin cbor object" '
97 - ipfs pin add $EXPHASH
209 + test_expect_success "traversals using /ipld/ work" '
210 + ipfs dag get /ipld/$IPLDPBHASH/Data > ipld_path_Data_actual
211 + '
212 +
213 + test_expect_success "retrieved node looks right" '
214 + echo -n "{\"/\":{\"bytes\":\"AAECAwQ\"}}" > ipld_path_Data_expected
215 + test_cmp ipld_path_Data_actual ipld_path_Data_expected
216 + '
217 +
218 + test_expect_success "can pin ipld object" '
219 + ipfs pin add $IPLDHASH
220 + '
221 +
222 + test_expect_success "can pin dag-pb object" '
223 + ipfs pin add $IPLDPBHASH
224 + '
225 +
226 + test_expect_success "can pin dag-cbor object" '
227 + ipfs pin add $IPLDCBORHASH
228 + '
229 +
230 + test_expect_success "can pin dag-json object" '
231 + ipfs pin add $IPLDJSONHASH
232 '
233
234 test_expect_success "after gc, objects still accessible" '
235 ipfs repo gc > /dev/null &&
102 - ipfs refs -r --timeout=2s $EXPHASH > /dev/null
236 + ipfs refs -r --timeout=2s $IPLDJSONHASH > /dev/null
237 '
238
239 test_expect_success "can get object" '
@@ -111,7 +245,7 @@ test_dag_cmd() {
245 '
246
247 test_expect_success "retrieved object hashes back correctly" '
114 - IPLDHASH2=$(cat ipld_obj_out | ipfs dag put) &&
248 + IPLDHASH2=$(cat ipld_obj_out | ipfs dag put --input-enc=dag-json -f dag-cbor) &&
249 test "$IPLDHASH" = "$IPLDHASH2"
250 '
251
@@ -124,7 +258,7 @@ test_dag_cmd() {
258 '
259
260 test_expect_success "output looks correct" '
127 - echo "{\"data\":\"CAISB2Zvb2JhcgoYBw==\",\"links\":[]}" > dag_get_pb_exp &&
261 + echo -n "{\"Data\":{\"/\":{\"bytes\":\"CAISB2Zvb2JhcgoYBw\"}},\"Links\":[]}" > dag_get_pb_exp &&
262 test_cmp dag_get_pb_exp dag_get_pb_out
263 '
264
@@ -133,24 +267,23 @@ test_dag_cmd() {
267 '
268
269 test_expect_success "output looks correct" '
136 - echo "{\"data\":\"CAISBGZvbwoYBA==\",\"links\":[]}" > cat_exp &&
270 + echo -n "{\"Data\":{\"/\":{\"bytes\":\"CAISBGZvbwoYBA\"}},\"Links\":[]}" > cat_exp &&
271 test_cmp cat_exp cat_out
272 '
273
140 - test_expect_success "non-canonical cbor input is normalized" '
141 - HASH=$(cat ../t0053-dag-data/non-canon.cbor | ipfs dag put --format=cbor --input-enc=raw) &&
274 + test_expect_success "non-canonical dag-cbor input is normalized" '
275 + HASH=$(cat ../t0053-dag-data/non-canon.cbor | ipfs dag put --format=dag-cbor --input-enc=dag-cbor) &&
276 test $HASH = "bafyreiawx7ona7oa2ptcoh6vwq4q6bmd7x2ibtkykld327bgb7t73ayrqm" ||
277 test_fsh echo $HASH
278 '
279
146 - test_expect_success "non-canonical cbor input is normalized with input-enc cbor" '
147 - HASH=$(cat ../t0053-dag-data/non-canon.cbor | ipfs dag put --format=cbor --input-enc=cbor) &&
148 - test $HASH = "bafyreiawx7ona7oa2ptcoh6vwq4q6bmd7x2ibtkykld327bgb7t73ayrqm" ||
149 - test_fsh echo $HASH
280 + test_expect_success "cbor input can be fetched" '
281 + EXPARR=$(ipfs dag get $HASH/arr)
282 + test $EXPARR = "[]"
283 '
284
285 test_expect_success "add an ipld with pin" '
153 - PINHASH=$(printf {\"foo\":\"bar\"} | ipfs dag put --pin=true)
286 + PINHASH=$(printf {\"foo\":\"bar\"} | ipfs dag put --input-enc=dag-json --pin=true)
287 '
288
289 test_expect_success "after gc, objects still accessible" '
@@ -158,23 +291,23 @@ test_dag_cmd() {
291 ipfs refs -r --timeout=2s $PINHASH > /dev/null
292 '
293
161 - test_expect_success "can add an ipld object with sha3 hash" '
162 - IPLDHASH=$(cat ipld_object | ipfs dag put --hash sha3)
294 + test_expect_success "can add an ipld object with sha3-512 hash" '
295 + IPLDHASH=$(cat ipld_object | ipfs dag put --hash sha3-512)
296 '
297
298 test_expect_success "output looks correct" '
166 - EXPHASH="bafyriqgae54zjl3bjebmbat2rjem4ewj6vni6jxohandmvk3bibfgv3sioyeidppsghvulryxats43br3b7afa6jy77x6gqzqaicer6ljicck"
299 + EXPHASH="bafyriqforjma7y7akqz7nhuu73r6liggj5zhkbfiqgicywe3fgkna2ijlhod2af3ue7doj56tlzt5hh6iu5esafc4msr3sd53jol5m2o25ucy"
300 test $EXPHASH = $IPLDHASH
301 '
302
303 test_expect_success "prepare dag-pb object" '
304 echo foo > test_file &&
172 - HASH=$(ipfs add -wq test_file | tail -n1)
305 + HASH=$(ipfs add -wq test_file | tail -n1 | ipfs cid base32)
306 '
307
308 test_expect_success "dag put with json dag-pb works" '
309 ipfs dag get $HASH > pbjson &&
177 - cat pbjson | ipfs dag put --format=dag-pb --input-enc=json > dag_put_out
310 + cat pbjson | ipfs dag put --format=dag-pb --input-enc=dag-json > dag_put_out
311 '
312
313 test_expect_success "dag put with dag-pb works output looks good" '
@@ -184,7 +317,7 @@ test_dag_cmd() {
317
318 test_expect_success "dag put with raw dag-pb works" '
319 ipfs block get $HASH > pbraw &&
187 - cat pbraw | ipfs dag put --format=dag-pb --input-enc=raw > dag_put_out
320 + cat pbraw | ipfs dag put --format=dag-pb --input-enc=dag-pb > dag_put_out
321 '
322
323 test_expect_success "dag put with dag-pb works output looks good" '
@@ -271,10 +404,10 @@ test_dag_cmd() {
404
405 test_expect_success "dag stat of simple IPLD object" '
406 ipfs dag stat $NESTED_HASH > actual_stat_inner_ipld_obj &&
274 - echo "Size: 15, NumBlocks: 1" > exp_stat_inner_ipld_obj &&
407 + echo "Size: 8, NumBlocks: 1" > exp_stat_inner_ipld_obj &&
408 test_cmp exp_stat_inner_ipld_obj actual_stat_inner_ipld_obj &&
409 ipfs dag stat $HASH > actual_stat_ipld_obj &&
277 - echo "Size: 61, NumBlocks: 2" > exp_stat_ipld_obj &&
410 + echo "Size: 54, NumBlocks: 2" > exp_stat_ipld_obj &&
411 test_cmp exp_stat_ipld_obj actual_stat_ipld_obj
412 '
413
test/sharness/t0055-dag-put-json-new-line.sh
+4 -9
@@ -8,14 +8,14 @@ test_init_ipfs
8
9 test_expect_success 'create test JSON files' '
10 WANT_JSON="{\"data\":1234}"
11 - WANT_HASH="bafyreidcqah6v3py3ujdc3ris22rjlfntaw7ajrus2f2477kpnizulaoea"
11 + WANT_HASH="bafyreidbm2zncsc3j25zn7lofgd4woeh6eygdy73thfosuni2rwr3bhcvu"
12 printf "${WANT_JSON}\n" > with_newline.json &&
13 printf "${WANT_JSON}" > without_newline.json
14 '
15
16 test_expect_success 'puts as CBOR work' '
17 - GOT_HASH_WITHOUT_NEWLINE="$(cat without_newline.json | ipfs dag put -f cbor)"
18 - GOT_HASH_WITH_NEWLINE="$(cat with_newline.json | ipfs dag put -f cbor)"
17 + GOT_HASH_WITHOUT_NEWLINE="$(cat without_newline.json | ipfs dag put -f dag-cbor)"
18 + GOT_HASH_WITH_NEWLINE="$(cat with_newline.json | ipfs dag put -f dag-cbor)"
19 '
20
21 test_expect_success 'put hashes with or without newline are equal' '
@@ -27,12 +27,7 @@ test_expect_success 'hashes are of expected value' '
27 test "${WANT_HASH}" = "${GOT_HASH_WITHOUT_NEWLINE}"
28 '
29
30 -# Retrieval must not contain a new-line regardless of input JSON, because
31 -# objects are put using the stable CBOR format.
32 -# despite this, dag retrieval returns JSON with new-line.
33 -# Expect failure until fixed, as per:
34 -# - https://github.com/ipfs/go-ipfs/issues/3503#issuecomment-877295280
35 -test_expect_failure "retrieval by hash does not have new line" '
30 +test_expect_success "retrieval by hash does not have new line" '
31 ipfs dag get "${WANT_HASH}" > got.json
32 test_cmp without_newline.json got.json
33 '
test/sharness/t0280-plugin-git.sh
+20 -5
@@ -17,25 +17,40 @@ test_expect_success "prepare test data" '
17
18 test_dag_git() {
19 test_expect_success "add objects via dag put" '
20 - find objects -type f -exec ipfs dag put --format=git --input-enc=zlib {} \; -exec echo \; > hashes
20 + find objects -type f -exec ipfs dag put --format=git-raw --input-enc=0x300078 --hash=sha1 {} \; -exec echo -n \; > hashes
21 '
22
23 test_expect_success "successfully get added objects" '
24 cat hashes | xargs -I {} ipfs dag get -- {} > /dev/null
25 '
26
27 + test_expect_success "dag get works" '
28 + echo -n "{\"message\":\"Some version\n\",\"object\":{\"/\":\"baf4bcfeq6c2mspupcvftgevza56h7rmozose6wi\"},\"tag\":\"v1\",\"tagger\":{\"date\":\"1497302532\",\"email\":\"johndoe@example.com\",\"name\":\"John Doe\",\"timezone\":\"+0200\"},\"type\":\"commit\"}" > tag_expected &&
29 + ipfs dag get baf4bcfhzi72pcj5cc4ocz7igcduubuu7aa3cddi > tag_actual
30 + '
31 +
32 + test_expect_success "outputs look correct" '
33 + test_cmp tag_expected tag_actual
34 + '
35 +
36 test_expect_success "path traversals work" '
28 - echo \"YmxvYiA3ACcsLnB5Zgo=\" > file1 &&
29 - ipfs dag get baf4bcfhzi72pcj5cc4ocz7igcduubuu7aa3cddi/object/parents/0/tree/dir2/hash/f3/hash > out1
37 + echo -n "{\"date\":\"1497302532\",\"email\":\"johndoe@example.com\",\"name\":\"John Doe\",\"timezone\":\"+0200\"}" > author_expected &&
38 + echo -n "{\"/\":{\"bytes\":\"YmxvYiAxMgBIZWxsbyB3b3JsZAo\"}}" > file1_expected &&
39 + echo -n "{\"/\":{\"bytes\":\"YmxvYiA3ACcsLnB5Zgo\"}}" > file2_expected &&
40 + ipfs dag get baf4bcfhzi72pcj5cc4ocz7igcduubuu7aa3cddi/object/author > author_actual &&
41 + ipfs dag get baf4bcfhzi72pcj5cc4ocz7igcduubuu7aa3cddi/object/tree/file/hash > file1_actual &&
42 + ipfs dag get baf4bcfhzi72pcj5cc4ocz7igcduubuu7aa3cddi/object/parents/0/tree/dir2/hash/f3/hash > file2_actual
43 '
44
45 test_expect_success "outputs look correct" '
33 - test_cmp file1 out1
46 + test_cmp author_expected author_actual &&
47 + test_cmp file1_expected file1_actual &&
48 + test_cmp file2_expected file2_actual
49 '
50 }
51
52 # should work offline
38 -#test_dag_git
53 +test_dag_git
54
55 # should work online
56 test_launch_ipfs_daemon