feat(cli/rpc/add): fast provide of root CID (#11046)
* feat: fast provide * Check error from provideRoot * do not provide if nil router * fix(commands): prevent panic from typed nil DHTClient interface Fixes panic when ipfsNode.DHTClient is a non-nil interface containing a nil pointer value (typed nil). This happened when Routing.Type=delegated or when using HTTP-only routing without DHT. The panic occurred because: - Go interfaces can be non-nil while containing nil pointer values - Simple `if DHTClient == nil` checks pass, but calling methods panics - Example: `(*ddht.DHT)(nil)` stored in interface passes nil check Solution: - Add HasActiveDHTClient() method to check both interface and concrete value - Update all 7 call sites to use proper check before DHT operations - Rename provideRoot → provideCIDSync for clarity - Add structured logging with "fast-provide" prefix for easier filtering - Add tests covering nil cases and valid DHT configurations Fixes: https://github.com/ipfs/kubo/pull/11046#issuecomment-3525313349 * feat(add): split fast-provide into two flags for async/sync control Renames --fast-provide to --fast-provide-root and adds --fast-provide-wait to give users control over synchronous vs asynchronous providing behavior. Changes: - --fast-provide-root (default: true): enables immediate root CID providing - --fast-provide-wait (default: false): controls whether to block until complete - Default behavior: async provide (fast, non-blocking) - Opt-in: --fast-provide-wait for guaranteed discoverability (slower, blocking) - Can disable with --fast-provide-root=false to rely on background reproviding Implementation: - Async mode: launches goroutine with detached context for fire-and-forget - Added 10 second timeout to prevent hanging on network issues - Timeout aligns with other kubo operations (ping, DNS resolve, p2p) - Sufficient for DHT with sweep provider or accelerated client - Sync mode: blocks on provideCIDSync until completion (uses req.Context) - Improved structured logging with "fast-provide-root:" prefix - Removed redundant "root CID" from messages (already in prefix) - Clear async/sync distinction in log messages - Added FAST PROVIDE OPTIMIZATION section to ipfs add --help explaining: - The problem: background queue takes time, content not immediately discoverable - The solution: extra immediate announcement of just the root CID - The benefit: peers can find content right away while queue handles rest - Usage: async by default, --fast-provide-wait for guaranteed completion Changelog: - Added highlight section for fast root CID providing feature - Updated TOC and overview - Included usage examples with clear comments explaining each mode - Emphasized this is extra announcement independent of background queue The feature works best with sweep provider and accelerated DHT client where provide operations are significantly faster. * fix(add): respect Provide config in fast-provide-root fast-provide-root should honor the same config settings as the regular provide system: - skip when Provide.Enabled is false - skip when Provide.DHT.Interval is 0 - respect Provide.Strategy (all/pinned/roots/mfs/combinations) This ensures fast-provide only runs when appropriate based on user configuration and the nature of the content being added (pinned vs unpinned, added to MFS or not). * Update core/commands/add.go --------- Co-authored-by: gammazero <11790789+gammazero@users.noreply.github.com> Co-authored-by: Marcin Rataj <lidel@lidel.org>
Guillaume Michel committed
Nov 14, 2025 at 20:08 UTC
d56fe3a0261debbb7c37b93806e6345bae49667b
10 files changed
+386
-14
core/commands/add.go
+115
-8
@@ -1,6 +1,7 @@
1
package commands
2
3
import (
4
+ "context"
5
"errors"
6
"fmt"
7
"io"
@@ -8,6 +9,7 @@ import (
9
gopath "path"
10
"strconv"
11
"strings"
12
+ "time"
13
14
"github.com/ipfs/kubo/config"
15
"github.com/ipfs/kubo/core/commands/cmdenv"
@@ -61,20 +63,50 @@ const (
63
inlineLimitOptionName = "inline-limit"
64
toFilesOptionName = "to-files"
65
64
- preserveModeOptionName = "preserve-mode"
65
- preserveMtimeOptionName = "preserve-mtime"
66
- modeOptionName = "mode"
67
- mtimeOptionName = "mtime"
68
- mtimeNsecsOptionName = "mtime-nsecs"
66
+ preserveModeOptionName = "preserve-mode"
67
+ preserveMtimeOptionName = "preserve-mtime"
68
+ modeOptionName = "mode"
69
+ mtimeOptionName = "mtime"
70
+ mtimeNsecsOptionName = "mtime-nsecs"
71
+ fastProvideRootOptionName = "fast-provide-root"
72
+ fastProvideWaitOptionName = "fast-provide-wait"
73
)
74
71
-const adderOutChanSize = 8
75
+const (
76
+ adderOutChanSize = 8
77
+
78
+ // fastProvideTimeout is the maximum time allowed for async fast-provide operations.
79
+ // Prevents hanging on network issues when providing root CID in background.
80
+ // 10 seconds is sufficient for DHT operations with sweep provider or accelerated client.
81
+ fastProvideTimeout = 10 * time.Second
82
+)
83
84
var AddCmd = &cmds.Command{
85
Helptext: cmds.HelpText{
86
Tagline: "Add a file or directory to IPFS.",
87
ShortDescription: `
88
Adds the content of <path> to IPFS. Use -r to add directories (recursively).
89
+
90
+FAST PROVIDE OPTIMIZATION:
91
+
92
+When you add content to IPFS, it gets queued for announcement on the DHT.
93
+The background queue can take some time to process, meaning other peers
94
+won't find your content immediately after 'ipfs add' completes.
95
+
96
+To make sharing faster, 'ipfs add' does an extra immediate announcement
97
+of just the root CID to the DHT. This lets other peers start discovering
98
+your content right away, while the regular background queue still handles
99
+announcing all the blocks later.
100
+
101
+By default, this extra announcement runs in the background without slowing
102
+down the command. If you need to be certain the root CID is discoverable
103
+before the command returns (for example, sharing a link immediately),
104
+use --fast-provide-wait to wait for the announcement to complete.
105
+Use --fast-provide-root=false to skip this optimization and rely only on
106
+the background queue (controlled by Provide.Strategy and Provide.DHT.Interval).
107
+
108
+This works best with the sweep provider and accelerated DHT client.
109
+Automatically skipped when DHT is not available.
110
`,
111
LongDescription: `
112
Adds the content of <path> to IPFS. Use -r to add directories.
@@ -213,6 +245,8 @@ https://github.com/ipfs/kubo/blob/master/docs/config.md#import
245
cmds.UintOption(modeOptionName, "Custom POSIX file mode to store in created UnixFS entries. WARNING: experimental, forces dag-pb for root block, disables raw-leaves"),
246
cmds.Int64Option(mtimeOptionName, "Custom POSIX modification time to store in created UnixFS entries (seconds before or after the Unix Epoch). WARNING: experimental, forces dag-pb for root block, disables raw-leaves"),
247
cmds.UintOption(mtimeNsecsOptionName, "Custom POSIX modification time (optional time fraction in nanoseconds)"),
248
+ cmds.BoolOption(fastProvideRootOptionName, "Immediately provide root CID to DHT for fast content discovery. When disabled, root CID is queued for background providing instead.").WithDefault(true),
249
+ cmds.BoolOption(fastProvideWaitOptionName, "Wait for fast-provide-root to complete before returning. Ensures root CID is discoverable when command finishes.").WithDefault(false),
250
},
251
PreRun: func(req *cmds.Request, env cmds.Environment) error {
252
quiet, _ := req.Options[quietOptionName].(bool)
@@ -283,6 +317,8 @@ https://github.com/ipfs/kubo/blob/master/docs/config.md#import
317
mode, _ := req.Options[modeOptionName].(uint)
318
mtime, _ := req.Options[mtimeOptionName].(int64)
319
mtimeNsecs, _ := req.Options[mtimeNsecsOptionName].(uint)
320
+ fastProvideRoot, _ := req.Options[fastProvideRootOptionName].(bool)
321
+ fastProvideWait, _ := req.Options[fastProvideWaitOptionName].(bool)
322
323
if chunker == "" {
324
chunker = cfg.Import.UnixFSChunker.WithDefault(config.DefaultUnixFSChunker)
@@ -421,11 +457,12 @@ https://github.com/ipfs/kubo/blob/master/docs/config.md#import
457
}
458
var added int
459
var fileAddedToMFS bool
460
+ var lastRootCid path.ImmutablePath // Track the root CID for fast-provide
461
addit := toadd.Entries()
462
for addit.Next() {
463
_, dir := addit.Node().(files.Directory)
464
errCh := make(chan error, 1)
428
- events := make(chan interface{}, adderOutChanSize)
465
+ events := make(chan any, adderOutChanSize)
466
opts[len(opts)-1] = options.Unixfs.Events(events)
467
468
go func() {
@@ -437,6 +474,9 @@ https://github.com/ipfs/kubo/blob/master/docs/config.md#import
474
return
475
}
476
477
+ // Store the root CID for potential fast-provide operation
478
+ lastRootCid = pathAdded
479
+
480
// creating MFS pointers when optional --to-files is set
481
if toFilesSet {
482
if addit.Name() == "" {
@@ -560,12 +600,79 @@ https://github.com/ipfs/kubo/blob/master/docs/config.md#import
600
return fmt.Errorf("expected a file argument")
601
}
602
603
+ // Apply fast-provide-root if the flag is enabled
604
+ if fastProvideRoot && (lastRootCid != path.ImmutablePath{}) {
605
+ cfg, err := ipfsNode.Repo.Config()
606
+ if err != nil {
607
+ return err
608
+ }
609
+
610
+ // Parse the provide strategy to check if we should provide based on pin/MFS status
611
+ strategyStr := cfg.Provide.Strategy.WithDefault(config.DefaultProvideStrategy)
612
+ strategy := config.ParseProvideStrategy(strategyStr)
613
+
614
+ // Determine if we should provide based on strategy
615
+ shouldProvide := false
616
+ if strategy == config.ProvideStrategyAll {
617
+ // 'all' strategy: always provide
618
+ shouldProvide = true
619
+ } else {
620
+ // For combined strategies (pinned+mfs), check each component
621
+ if strategy&config.ProvideStrategyPinned != 0 && dopin {
622
+ shouldProvide = true
623
+ } else if strategy&config.ProvideStrategyRoots != 0 && dopin {
624
+ shouldProvide = true
625
+ } else if strategy&config.ProvideStrategyMFS != 0 && toFilesSet {
626
+ shouldProvide = true
627
+ }
628
+ }
629
+
630
+ switch {
631
+ case !cfg.Provide.Enabled.WithDefault(config.DefaultProvideEnabled):
632
+ log.Debugw("fast-provide-root: skipped", "reason", "Provide.Enabled is false")
633
+ case cfg.Provide.DHT.Interval.WithDefault(config.DefaultProvideDHTInterval) == 0:
634
+ log.Debugw("fast-provide-root: skipped", "reason", "Provide.DHT.Interval is 0")
635
+ case !shouldProvide:
636
+ log.Debugw("fast-provide-root: skipped", "reason", "strategy does not match content", "strategy", strategyStr, "pinned", dopin, "to-files", toFilesSet)
637
+ case !ipfsNode.HasActiveDHTClient():
638
+ log.Debugw("fast-provide-root: skipped", "reason", "DHT not available")
639
+ default:
640
+ rootCid := lastRootCid.RootCid()
641
+
642
+ if fastProvideWait {
643
+ // Synchronous mode: block until provide completes
644
+ log.Debugw("fast-provide-root: providing synchronously", "cid", rootCid)
645
+ if err := provideCIDSync(req.Context, ipfsNode.DHTClient, rootCid); err != nil {
646
+ log.Warnw("fast-provide-root: sync provide failed", "cid", rootCid, "error", err)
647
+ } else {
648
+ log.Debugw("fast-provide-root: sync provide completed", "cid", rootCid)
649
+ }
650
+ } else {
651
+ // Asynchronous mode (default): fire-and-forget, don't block
652
+ log.Debugw("fast-provide-root: providing asynchronously", "cid", rootCid)
653
+ go func() {
654
+ // Use detached context with timeout to prevent hanging on network issues
655
+ ctx, cancel := context.WithTimeout(context.Background(), fastProvideTimeout)
656
+ defer cancel()
657
+ if err := provideCIDSync(ctx, ipfsNode.DHTClient, rootCid); err != nil {
658
+ log.Warnw("fast-provide-root: async provide failed", "cid", rootCid, "error", err)
659
+ } else {
660
+ log.Debugw("fast-provide-root: async provide completed", "cid", rootCid)
661
+ }
662
+ }()
663
+ }
664
+ }
665
+ } else if fastProvideWait && !fastProvideRoot {
666
+ // Log that wait flag is ignored when provide-root is disabled
667
+ log.Debugw("fast-provide-root: wait flag ignored", "reason", "fast-provide-root disabled")
668
+ }
669
+
670
return nil
671
},
672
PostRun: cmds.PostRunMap{
673
cmds.CLI: func(res cmds.Response, re cmds.ResponseEmitter) error {
674
sizeChan := make(chan int64, 1)
568
- outChan := make(chan interface{})
675
+ outChan := make(chan any)
676
req := res.Request()
677
678
// Could be slow.
core/commands/dht.go
+2
-2
@@ -56,7 +56,7 @@ var queryDhtCmd = &cmds.Command{
56
return err
57
}
58
59
- if nd.DHTClient == nil {
59
+ if !nd.HasActiveDHTClient() {
60
return ErrNotDHT
61
}
62
@@ -70,7 +70,7 @@ var queryDhtCmd = &cmds.Command{
70
ctx, events := routing.RegisterForQueryEvents(ctx)
71
72
client := nd.DHTClient
73
- if client == nd.DHT {
73
+ if nd.DHT != nil && client == nd.DHT {
74
client = nd.DHT.WAN
75
if !nd.DHT.WANActive() {
76
client = nd.DHT.LAN
core/commands/provide.go
+19
@@ -1,6 +1,7 @@
1
package commands
2
3
import (
4
+ "context"
5
"errors"
6
"fmt"
7
"io"
@@ -11,6 +12,7 @@ import (
12
13
humanize "github.com/dustin/go-humanize"
14
boxoprovider "github.com/ipfs/boxo/provider"
15
+ cid "github.com/ipfs/go-cid"
16
cmds "github.com/ipfs/go-ipfs-cmds"
17
"github.com/ipfs/kubo/core/commands/cmdenv"
18
"github.com/libp2p/go-libp2p-kad-dht/fullrt"
@@ -18,6 +20,7 @@ import (
20
"github.com/libp2p/go-libp2p-kad-dht/provider/buffered"
21
"github.com/libp2p/go-libp2p-kad-dht/provider/dual"
22
"github.com/libp2p/go-libp2p-kad-dht/provider/stats"
23
+ routing "github.com/libp2p/go-libp2p/core/routing"
24
"github.com/probe-lab/go-libdht/kad/key"
25
"golang.org/x/exp/constraints"
26
)
@@ -575,3 +578,19 @@ func humanInt[T constraints.Integer](val T) string {
578
func humanFull(val float64, decimals int) string {
579
return humanize.CommafWithDigits(val, decimals)
580
}
581
+
582
+// provideCIDSync performs a synchronous/blocking provide operation to announce
583
+// the given CID to the DHT.
584
+//
585
+// - If the accelerated DHT client is used, a DHT lookup isn't needed, we
586
+// directly allocate provider records to closest peers.
587
+// - If Provide.DHT.SweepEnabled=true or OptimisticProvide=true, we make an
588
+// optimistic provide call.
589
+// - Else we make a standard provide call (much slower).
590
+//
591
+// IMPORTANT: The caller MUST verify DHT availability using HasActiveDHTClient()
592
+// before calling this function. Calling with a nil or invalid router will cause
593
+// a panic - this is the caller's responsibility to prevent.
594
+func provideCIDSync(ctx context.Context, router routing.Routing, c cid.Cid) error {
595
+ return router.Provide(ctx, c, true)
596
+}
core/commands/routing.go
+15
@@ -211,6 +211,10 @@ var provideRefRoutingCmd = &cmds.Command{
211
ctx, events := routing.RegisterForQueryEvents(ctx)
212
213
var provideErr error
214
+ // TODO: not sure if necessary to call StartProviding for `ipfs routing
215
+ // provide <cid>`, since either cid is already being provided, or it will
216
+ // be garbage collected and not reprovided anyway. So we may simply stick
217
+ // with a single (optimistic) provide, and skip StartProviding call.
218
go func() {
219
defer cancel()
220
if rec {
@@ -226,6 +230,16 @@ var provideRefRoutingCmd = &cmds.Command{
230
}
231
}()
232
233
+ if nd.HasActiveDHTClient() {
234
+ // If node has a DHT client, provide immediately the supplied cids before
235
+ // returning.
236
+ for _, c := range cids {
237
+ if err = provideCIDSync(req.Context, nd.DHTClient, c); err != nil {
238
+ return fmt.Errorf("error providing cid: %w", err)
239
+ }
240
+ }
241
+ }
242
+
243
for e := range events {
244
if err := res.Emit(e); err != nil {
245
return err
@@ -300,6 +314,7 @@ func provideCids(prov node.DHTProvider, cids []cid.Cid) error {
314
for i, c := range cids {
315
mhs[i] = c.Hash()
316
}
317
+ // providing happens asynchronously
318
return prov.StartProviding(true, mhs...)
319
}
320
core/commands/stat_dht.go
+2
-1
@@ -75,7 +75,8 @@ This interface is not stable and may change from release to release.
75
var dht *dht.IpfsDHT
76
77
var separateClient bool
78
- if nd.DHTClient != nd.DHT {
78
+ // Check if using separate DHT client (e.g., accelerated DHT)
79
+ if nd.HasActiveDHTClient() && nd.DHTClient != nd.DHT {
80
separateClient = true
81
}
82
core/commands/version.go
+1
-1
@@ -255,7 +255,7 @@ func DetectNewKuboVersion(nd *core.IpfsNode, minPercent int64) (VersionCheckOutp
255
}
256
257
// Amino DHT client keeps information about previously seen peers
258
- if nd.DHTClient != nd.DHT && nd.DHTClient != nil {
258
+ if nd.HasActiveDHTClient() && nd.DHTClient != nd.DHT {
259
client, ok := nd.DHTClient.(*fullrt.FullRT)
260
if !ok {
261
return VersionCheckOutput{}, errors.New("could not perform version check due to missing or incompatible DHT configuration")
core/core.go
+38
@@ -30,9 +30,11 @@ import (
30
ipld "github.com/ipfs/go-ipld-format"
31
logging "github.com/ipfs/go-log/v2"
32
ddht "github.com/libp2p/go-libp2p-kad-dht/dual"
33
+ "github.com/libp2p/go-libp2p-kad-dht/fullrt"
34
pubsub "github.com/libp2p/go-libp2p-pubsub"
35
psrouter "github.com/libp2p/go-libp2p-pubsub-router"
36
record "github.com/libp2p/go-libp2p-record"
37
+ routinghelpers "github.com/libp2p/go-libp2p-routing-helpers"
38
connmgr "github.com/libp2p/go-libp2p/core/connmgr"
39
ic "github.com/libp2p/go-libp2p/core/crypto"
40
p2phost "github.com/libp2p/go-libp2p/core/host"
@@ -143,6 +145,42 @@ func (n *IpfsNode) Close() error {
145
return n.stop()
146
}
147
148
+// HasActiveDHTClient checks if the node's DHT client is active and usable for DHT operations.
149
+//
150
+// Returns false for:
151
+// - nil DHTClient
152
+// - typed nil pointers (e.g., (*ddht.DHT)(nil))
153
+// - no-op routers (routinghelpers.Null)
154
+//
155
+// Note: This method only checks for known DHT client types (ddht.DHT, fullrt.FullRT).
156
+// Custom routing.Routing implementations are not explicitly validated.
157
+//
158
+// This method prevents the "typed nil interface" bug where an interface contains
159
+// a nil pointer of a concrete type, which passes nil checks but panics when methods
160
+// are called.
161
+func (n *IpfsNode) HasActiveDHTClient() bool {
162
+ if n.DHTClient == nil {
163
+ return false
164
+ }
165
+
166
+ // Check for no-op router (Routing.Type=none)
167
+ if _, ok := n.DHTClient.(routinghelpers.Null); ok {
168
+ return false
169
+ }
170
+
171
+ // Check for typed nil *ddht.DHT (common when Routing.Type=delegated or HTTP-only)
172
+ if d, ok := n.DHTClient.(*ddht.DHT); ok && d == nil {
173
+ return false
174
+ }
175
+
176
+ // Check for typed nil *fullrt.FullRT (accelerated DHT client)
177
+ if f, ok := n.DHTClient.(*fullrt.FullRT); ok && f == nil {
178
+ return false
179
+ }
180
+
181
+ return true
182
+}
183
+
184
// Context returns the IpfsNode context
185
func (n *IpfsNode) Context() context.Context {
186
if n.ctx == nil {
core/core_test.go
+161
@@ -1,15 +1,28 @@
1
package core
2
3
import (
4
+ "os"
5
+ "path/filepath"
6
"testing"
7
8
context "context"
9
10
"github.com/ipfs/kubo/repo"
11
12
+ "github.com/ipfs/boxo/filestore"
13
+ "github.com/ipfs/boxo/keystore"
14
datastore "github.com/ipfs/go-datastore"
15
syncds "github.com/ipfs/go-datastore/sync"
16
config "github.com/ipfs/kubo/config"
17
+ "github.com/ipfs/kubo/core/node/libp2p"
18
+ golib "github.com/libp2p/go-libp2p"
19
+ ddht "github.com/libp2p/go-libp2p-kad-dht/dual"
20
+ "github.com/libp2p/go-libp2p-kad-dht/fullrt"
21
+ routinghelpers "github.com/libp2p/go-libp2p-routing-helpers"
22
+ "github.com/libp2p/go-libp2p/core/host"
23
+ "github.com/libp2p/go-libp2p/core/peer"
24
+ pstore "github.com/libp2p/go-libp2p/core/peerstore"
25
+ mocknet "github.com/libp2p/go-libp2p/p2p/net/mock"
26
)
27
28
func TestInitialization(t *testing.T) {
@@ -65,3 +78,151 @@ var testIdentity = config.Identity{
78
PeerID: "QmNgdzLieYi8tgfo2WfTUzNVH5hQK9oAYGVf6dxN12NrHt",
79
PrivKey: "CAASrRIwggkpAgEAAoICAQCwt67GTUQ8nlJhks6CgbLKOx7F5tl1r9zF4m3TUrG3Pe8h64vi+ILDRFd7QJxaJ/n8ux9RUDoxLjzftL4uTdtv5UXl2vaufCc/C0bhCRvDhuWPhVsD75/DZPbwLsepxocwVWTyq7/ZHsCfuWdoh/KNczfy+Gn33gVQbHCnip/uhTVxT7ARTiv8Qa3d7qmmxsR+1zdL/IRO0mic/iojcb3Oc/PRnYBTiAZFbZdUEit/99tnfSjMDg02wRayZaT5ikxa6gBTMZ16Yvienq7RwSELzMQq2jFA4i/TdiGhS9uKywltiN2LrNDBcQJSN02pK12DKoiIy+wuOCRgs2NTQEhU2sXCk091v7giTTOpFX2ij9ghmiRfoSiBFPJA5RGwiH6ansCHtWKY1K8BS5UORM0o3dYk87mTnKbCsdz4bYnGtOWafujYwzueGx8r+IWiys80IPQKDeehnLW6RgoyjszKgL/2XTyP54xMLSW+Qb3BPgDcPaPO0hmop1hW9upStxKsefW2A2d46Ds4HEpJEry7PkS5M4gKL/zCKHuxuXVk14+fZQ1rstMuvKjrekpAC2aVIKMI9VRA3awtnje8HImQMdj+r+bPmv0N8rTTr3eS4J8Yl7k12i95LLfK+fWnmUh22oTNzkRlaiERQrUDyE4XNCtJc0xs1oe1yXGqazCIAQIDAQABAoICAQCk1N/ftahlRmOfAXk//8wNl7FvdJD3le6+YSKBj0uWmN1ZbUSQk64chr12iGCOM2WY180xYjy1LOS44PTXaeW5bEiTSnb3b3SH+HPHaWCNM2EiSogHltYVQjKW+3tfH39vlOdQ9uQ+l9Gh6iTLOqsCRyszpYPqIBwi1NMLY2Ej8PpVU7ftnFWouHZ9YKS7nAEiMoowhTu/7cCIVwZlAy3AySTuKxPMVj9LORqC32PVvBHZaMPJ+X1Xyijqg6aq39WyoztkXg3+Xxx5j5eOrK6vO/Lp6ZUxaQilHDXoJkKEJjgIBDZpluss08UPfOgiWAGkW+L4fgUxY0qDLDAEMhyEBAn6KOKVL1JhGTX6GjhWziI94bddSpHKYOEIDzUy4H8BXnKhtnyQV6ELS65C2hj9D0IMBTj7edCF1poJy0QfdK0cuXgMvxHLeUO5uc2YWfbNosvKxqygB9rToy4b22YvNwsZUXsTY6Jt+p9V2OgXSKfB5VPeRbjTJL6xqvvUJpQytmII/C9JmSDUtCbYceHj6X9jgigLk20VV6nWHqCTj3utXD6NPAjoycVpLKDlnWEgfVELDIk0gobxUqqSm3jTPEKRPJgxkgPxbwxYumtw++1UY2y35w3WRDc2xYPaWKBCQeZy+mL6ByXp9bWlNvxS3Knb6oZp36/ovGnf2pGvdQKCAQEAyKpipz2lIUySDyE0avVWAmQb2tWGKXALPohzj7AwkcfEg2GuwoC6GyVE2sTJD1HRazIjOKn3yQORg2uOPeG7sx7EKHxSxCKDrbPawkvLCq8JYSy9TLvhqKUVVGYPqMBzu2POSLEA81QXas+aYjKOFWA2Zrjq26zV9ey3+6Lc6WULePgRQybU8+RHJc6fdjUCCfUxgOrUO2IQOuTJ+FsDpVnrMUGlokmWn23OjL4qTL9wGDnWGUs2pjSzNbj3qA0d8iqaiMUyHX/D/VS0wpeT1osNBSm8suvSibYBn+7wbIApbwXUxZaxMv2OHGz3empae4ckvNZs7r8wsI9UwFt8mwKCAQEA4XK6gZkv9t+3YCcSPw2ensLvL/xU7i2bkC9tfTGdjnQfzZXIf5KNdVuj/SerOl2S1s45NMs3ysJbADwRb4ahElD/V71nGzV8fpFTitC20ro9fuX4J0+twmBolHqeH9pmeGTjAeL1rvt6vxs4FkeG/yNft7GdXpXTtEGaObn8Mt0tPY+aB3UnKrnCQoQAlPyGHFrVRX0UEcp6wyyNGhJCNKeNOvqCHTFObhbhO+KWpWSN0MkVHnqaIBnIn1Te8FtvP/iTwXGnKc0YXJUG6+LM6LmOguW6tg8ZqiQeYyyR+e9eCFH4csLzkrTl1GxCxwEsoSLIMm7UDcjttW6tYEghkwKCAQEAmeCO5lCPYImnN5Lu71ZTLmI2OgmjaANTnBBnDbi+hgv61gUCToUIMejSdDCTPfwv61P3TmyIZs0luPGxkiKYHTNqmOE9Vspgz8Mr7fLRMNApESuNvloVIY32XVImj/GEzh4rAfM6F15U1sN8T/EUo6+0B/Glp+9R49QzAfRSE2g48/rGwgf1JVHYfVWFUtAzUA+GdqWdOixo5cCsYJbqpNHfWVZN/bUQnBFIYwUwysnC29D+LUdQEQQ4qOm+gFAOtrWU62zMkXJ4iLt8Ify6kbrvsRXgbhQIzzGS7WH9XDarj0eZciuslr15TLMC1Azadf+cXHLR9gMHA13mT9vYIQKCAQA/DjGv8cKCkAvf7s2hqROGYAs6Jp8yhrsN1tYOwAPLRhtnCs+rLrg17M2vDptLlcRuI/vIElamdTmylRpjUQpX7yObzLO73nfVhpwRJVMdGU394iBIDncQ+JoHfUwgqJskbUM40dvZdyjbrqc/Q/4z+hbZb+oN/GXb8sVKBATPzSDMKQ/xqgisYIw+wmDPStnPsHAaIWOtni47zIgilJzD0WEk78/YjmPbUrboYvWziK5JiRRJFA1rkQqV1c0M+OXixIm+/yS8AksgCeaHr0WUieGcJtjT9uE8vyFop5ykhRiNxy9wGaq6i7IEecsrkd6DqxDHWkwhFuO1bSE83q/VAoIBAEA+RX1i/SUi08p71ggUi9WFMqXmzELp1L3hiEjOc2AklHk2rPxsaTh9+G95BvjhP7fRa/Yga+yDtYuyjO99nedStdNNSg03aPXILl9gs3r2dPiQKUEXZJ3FrH6tkils/8BlpOIRfbkszrdZIKTO9GCdLWQ30dQITDACs8zV/1GFGrHFrqnnMe/NpIFHWNZJ0/WZMi8wgWO6Ik8jHEpQtVXRiXLqy7U6hk170pa4GHOzvftfPElOZZjy9qn7KjdAQqy6spIrAE94OEL+fBgbHQZGLpuTlj6w6YGbMtPU8uo7sXKoc6WOCb68JWft3tejGLDa1946HAWqVM9B/UcneNc=",
80
}
81
+
82
+// mockHostOption creates a HostOption that uses the provided mocknet.
83
+// Inlined to avoid import cycle with core/mock package.
84
+func mockHostOption(mn mocknet.Mocknet) libp2p.HostOption {
85
+ return func(id peer.ID, ps pstore.Peerstore, opts ...golib.Option) (host.Host, error) {
86
+ var cfg golib.Config
87
+ if err := cfg.Apply(opts...); err != nil {
88
+ return nil, err
89
+ }
90
+
91
+ // The mocknet does not use the provided libp2p.Option. This options include
92
+ // the listening addresses we want our peer listening on. Therefore, we have
93
+ // to manually parse the configuration and add them here.
94
+ ps.AddAddrs(id, cfg.ListenAddrs, pstore.PermanentAddrTTL)
95
+ return mn.AddPeerWithPeerstore(id, ps)
96
+ }
97
+}
98
+
99
+func TestHasActiveDHTClient(t *testing.T) {
100
+ // Test 1: nil DHTClient
101
+ t.Run("nil DHTClient", func(t *testing.T) {
102
+ node := &IpfsNode{
103
+ DHTClient: nil,
104
+ }
105
+ if node.HasActiveDHTClient() {
106
+ t.Error("Expected false for nil DHTClient")
107
+ }
108
+ })
109
+
110
+ // Test 2: Typed nil *ddht.DHT (common case when Routing.Type=delegated)
111
+ t.Run("typed nil ddht.DHT", func(t *testing.T) {
112
+ node := &IpfsNode{
113
+ DHTClient: (*ddht.DHT)(nil),
114
+ }
115
+ if node.HasActiveDHTClient() {
116
+ t.Error("Expected false for typed nil *ddht.DHT")
117
+ }
118
+ })
119
+
120
+ // Test 3: Typed nil *fullrt.FullRT (accelerated DHT client)
121
+ t.Run("typed nil fullrt.FullRT", func(t *testing.T) {
122
+ node := &IpfsNode{
123
+ DHTClient: (*fullrt.FullRT)(nil),
124
+ }
125
+ if node.HasActiveDHTClient() {
126
+ t.Error("Expected false for typed nil *fullrt.FullRT")
127
+ }
128
+ })
129
+
130
+ // Test 4: routinghelpers.Null no-op router (Routing.Type=none)
131
+ t.Run("routinghelpers.Null", func(t *testing.T) {
132
+ node := &IpfsNode{
133
+ DHTClient: routinghelpers.Null{},
134
+ }
135
+ if node.HasActiveDHTClient() {
136
+ t.Error("Expected false for routinghelpers.Null")
137
+ }
138
+ })
139
+
140
+ // Test 5: Valid standard dual DHT (Routing.Type=auto/dht/dhtclient)
141
+ t.Run("valid standard dual DHT", func(t *testing.T) {
142
+ ctx := context.Background()
143
+ mn := mocknet.New()
144
+ defer mn.Close()
145
+
146
+ ds := syncds.MutexWrap(datastore.NewMapDatastore())
147
+ c := config.Config{}
148
+ c.Identity = testIdentity
149
+ c.Addresses.Swarm = []string{"/ip4/0.0.0.0/tcp/4001"}
150
+
151
+ r := &repo.Mock{
152
+ C: c,
153
+ D: ds,
154
+ K: keystore.NewMemKeystore(),
155
+ F: filestore.NewFileManager(ds, filepath.Dir(os.TempDir())),
156
+ }
157
+
158
+ node, err := NewNode(ctx, &BuildCfg{
159
+ Routing: libp2p.DHTServerOption,
160
+ Repo: r,
161
+ Host: mockHostOption(mn),
162
+ Online: true,
163
+ })
164
+ if err != nil {
165
+ t.Fatalf("Failed to create node with DHT: %v", err)
166
+ }
167
+ defer node.Close()
168
+
169
+ // First verify test setup created the expected DHT type
170
+ if node.DHTClient == nil {
171
+ t.Fatalf("Test setup failed: DHTClient is nil")
172
+ }
173
+
174
+ if _, ok := node.DHTClient.(*ddht.DHT); !ok {
175
+ t.Fatalf("Test setup failed: expected DHTClient to be *ddht.DHT, got %T", node.DHTClient)
176
+ }
177
+
178
+ // Now verify HasActiveDHTClient() correctly identifies it as active
179
+ if !node.HasActiveDHTClient() {
180
+ t.Error("Expected true for valid dual DHT client")
181
+ }
182
+ })
183
+
184
+ // Test 6: Valid accelerated DHT client (Routing.Type=autoclient)
185
+ t.Run("valid accelerated DHT client", func(t *testing.T) {
186
+ ctx := context.Background()
187
+ mn := mocknet.New()
188
+ defer mn.Close()
189
+
190
+ ds := syncds.MutexWrap(datastore.NewMapDatastore())
191
+ c := config.Config{}
192
+ c.Identity = testIdentity
193
+ c.Addresses.Swarm = []string{"/ip4/0.0.0.0/tcp/4001"}
194
+ c.Routing.AcceleratedDHTClient = config.True
195
+
196
+ r := &repo.Mock{
197
+ C: c,
198
+ D: ds,
199
+ K: keystore.NewMemKeystore(),
200
+ F: filestore.NewFileManager(ds, filepath.Dir(os.TempDir())),
201
+ }
202
+
203
+ node, err := NewNode(ctx, &BuildCfg{
204
+ Routing: libp2p.DHTOption,
205
+ Repo: r,
206
+ Host: mockHostOption(mn),
207
+ Online: true,
208
+ })
209
+ if err != nil {
210
+ t.Fatalf("Failed to create node with accelerated DHT: %v", err)
211
+ }
212
+ defer node.Close()
213
+
214
+ // First verify test setup created the expected accelerated DHT type
215
+ if node.DHTClient == nil {
216
+ t.Fatalf("Test setup failed: DHTClient is nil")
217
+ }
218
+
219
+ if _, ok := node.DHTClient.(*fullrt.FullRT); !ok {
220
+ t.Fatalf("Test setup failed: expected DHTClient to be *fullrt.FullRT, got %T", node.DHTClient)
221
+ }
222
+
223
+ // Now verify HasActiveDHTClient() correctly identifies it as active
224
+ if !node.HasActiveDHTClient() {
225
+ t.Error("Expected true for valid accelerated DHT client")
226
+ }
227
+ })
228
+}
core/node/libp2p/host.go
+13
-1
@@ -55,12 +55,24 @@ func Host(mctx helpers.MetricsCtx, lc fx.Lifecycle, params P2PHostIn) (out P2PHo
55
return out, err
56
}
57
58
+ // Optimistic provide is enabled either via dedicated expierimental flag, or when DHT Provide Sweep is enabled.
59
+ // When DHT Provide Sweep is enabled, all provide operations go through the
60
+ // `SweepingProvider`, hence the provides don't use the optimistic provide
61
+ // logic. Provides use `SweepingProvider.StartProviding()` and not
62
+ // `IpfsDHT.Provide()`, which is where the optimistic provide logic is
63
+ // implemented. However, `IpfsDHT.Provide()` is used to quickly provide roots
64
+ // when user manually adds content with the `--fast-provide` flag enabled. In
65
+ // this case we want to use optimistic provide logic to quickly announce the
66
+ // content to the network. This should be the only use case of
67
+ // `IpfsDHT.Provide()` when DHT Provide Sweep is enabled.
68
+ optimisticProvide := cfg.Experimental.OptimisticProvide || cfg.Provide.DHT.SweepEnabled.WithDefault(config.DefaultProvideDHTSweepEnabled)
69
+
70
routingOptArgs := RoutingOptionArgs{
71
Ctx: ctx,
72
Datastore: params.Repo.Datastore(),
73
Validator: params.Validator,
74
BootstrapPeers: bootstrappers,
63
- OptimisticProvide: cfg.Experimental.OptimisticProvide,
75
+ OptimisticProvide: optimisticProvide,
76
OptimisticProvideJobsPoolSize: cfg.Experimental.OptimisticProvideJobsPoolSize,
77
LoopbackAddressesOnLanDHT: cfg.Routing.LoopbackAddressesOnLanDHT.WithDefault(config.DefaultLoopbackAddressesOnLanDHT),
78
}
docs/changelogs/v0.39.md
+20
-1
@@ -11,6 +11,7 @@ This release was brought to you by the [Shipyard](https://ipshipyard.com/) team.
11
- [Overview](#overview)
12
- [🔦 Highlights](#-highlights)
13
- [🎯 Amino DHT Sweep provider is now the default](#-amino-dht-sweep-provider-is-now-the-default)
14
+ - [⚡ Fast root CID providing for immediate content discovery](#-fast-root-cid-providing-for-immediate-content-discovery)
15
- [📊 Detailed statistics for Sweep provider with `ipfs provide stat`](#-detailed-statistics-for-sweep-provider-with-ipfs-provide-stat)
16
- [⏯️ Provider resume cycle for improved reproviding reliability](#provider-resume-cycle-for-improved-reproviding-reliability)
17
- [🔔 Sweep provider slow reprovide warnings](#-sweep-provider-slow-reprovide-warnings)
@@ -25,7 +26,7 @@ This release was brought to you by the [Shipyard](https://ipshipyard.com/) team.
26
27
### Overview
28
28
-Kubo 0.39.0 graduates the experimental sweep provider to default, bringing efficient content announcement to all nodes. This release adds detailed provider statistics, automatic state persistence for reliable reproviding after restarts, and proactive monitoring alerts for identifying issues early. It also includes important fixes for UPnP port forwarding, RISC-V prebuilt binaries, and finalizes the deprecation of the legacy go-ipfs name.
29
+Kubo 0.39.0 graduates the experimental sweep provider to default, bringing efficient content announcement to all nodes. This release adds fast root CID providing for immediate content discovery via `ipfs add`, detailed provider statistics, automatic state persistence for reliable reproviding after restarts, and proactive monitoring alerts for identifying issues early. It also includes important fixes for UPnP port forwarding, RISC-V prebuilt binaries, and finalizes the deprecation of the legacy go-ipfs name.
30
31
### 🔦 Highlights
32
@@ -50,6 +51,24 @@ The Amino DHT Sweep provider system, introduced as experimental in v0.38, is now
51
52
For background on the sweep provider design and motivations, see [`Provide.DHT.SweepEnabled`](https://github.com/ipfs/kubo/blob/master/docs/config.md#providedhtsweepenabled) and [ipshipyard.com#8](https://github.com/ipshipyard/ipshipyard.com/pull/8).
53
54
+#### ⚡ Fast root CID providing for immediate content discovery
55
+
56
+When you add content to IPFS, it normally gets queued for announcement on the DHT. This background queue can take time to process, meaning other peers won't find your content immediately after `ipfs add` completes.
57
+
58
+To make sharing faster, `ipfs add` now does an extra immediate announcement of just the root CID to the DHT (controlled by the new `--fast-provide-root` flag, enabled by default). This lets other peers start discovering your content right away, while the regular background queue still handles announcing all the blocks later.
59
+
60
+By default, this extra announcement runs in the background without slowing down the command. For use cases requiring guaranteed discoverability before the command returns (for example, sharing a link immediately), use `--fast-provide-wait` to block until the announcement completes.
61
+
62
+**Usage examples:**
63
+
64
+```bash
65
+ipfs add file.txt # Root CID provided immediately in background, independent of queue (default)
66
+ipfs add file.txt --fast-provide-wait # Blocks until root CID announcement completes (slower, guaranteed)
67
+ipfs add file.txt --fast-provide-root=false # Skip immediate announcement, use background queue only
68
+```
69
+
70
+This optimization works best with the sweep provider and accelerated DHT client, where provide operations are significantly faster than traditional DHT providing. The feature is automatically skipped when DHT is unavailable (e.g., `Routing.Type=none` or delegated-only configurations).
71
+
72
#### 📊 Detailed statistics for Sweep provider with `ipfs provide stat`
73
74
The Sweep provider system now exposes detailed statistics through `ipfs provide stat`, helping you monitor provider health and troubleshoot issues.