docs: added experimental features documentation on the Accelerated DHT Client and batched providing system
Adin Schmahmann committed
May 13, 2021 at 01:05 UTC
a67cab1c60bc3165d7492355863b6dd847c4fb14
1 file changed
+54
docs/experimental-features.md
+54
@@ -26,6 +26,7 @@ the above issue.
26
- [Strategic Providing](#strategic-providing)
27
- [Graphsync](#graphsync)
28
- [Noise](#noise)
29
+- [Accelerated DHT Client](#accelerated-dht-client)
30
31
---
32
@@ -547,3 +548,56 @@ ipfs config --json Experimental.GraphsyncEnabled true
548
Stable, enabled by default
549
550
[Noise](https://github.com/libp2p/specs/tree/master/noise) libp2p transport based on the [Noise Protocol Framework](https://noiseprotocol.org/noise.html). While TLS remains the default transport in go-ipfs, Noise is easier to implement and is thus the "interop" transport between IPFS and libp2p implementations.
551
+
552
+## Accelerated DHT Client
553
+
554
+### In Version
555
+
556
+0.9.0
557
+
558
+### State
559
+
560
+Experimental, default-disabled.
561
+
562
+Utilizes an alternative DHT client that searches for and maintains more information about the network
563
+in exchange for being more performant.
564
+
565
+When it is enabled:
566
+- DHT operations should complete much faster than with it disabled
567
+- A batching reprovider system will be enabled which takes advantage of some properties of the experimental client to
568
+ very efficiently put provider records into the network
569
+- The standard DHT client (and server if enabled) are run alongside the alternative client
570
+- The operations `ipfs stats dht` and `ipfs stats provide` will have different outputs
571
+ - `ipfs stats provide` only works when the accelerated DHT client is enabled and shows various statistics regarding
572
+ the provider/reprovider system
573
+ - `ipfs stats dht` will default to showing information about the new client
574
+
575
+**Caveats:**
576
+1. Running the experimental client likely will result in more resource consumption (connections, RAM, CPU, bandwidth)
577
+ - Users that are limited in the number of parallel connections their machines/networks can perform will likely suffer
578
+ - Currently, the resource usage is not smooth as the client crawls the network in rounds and reproviding is similarly
579
+ done in rounds
580
+ - Users who previously had a lot of content but were unable to advertise it on the network will see an increase in
581
+ egress bandwidth as their nodes start to advertise all of their CIDs into the network. If you have lots of data
582
+ entering your node that you don't want to advertise consider using [Reprovider Strategies](config.md#reproviderstrategy)
583
+ to reduce the number of CIDs that you are reproviding. Similarly, if you are running a node that deals mostly with
584
+ short-lived temporary data (e.g. you use a separate node for ingesting data then for storing and serving it) then
585
+ you may benefit from using [Strategic Providing](#strategic-providing) to prevent advertising of data that you
586
+ ultimately will not have.
587
+2. Currently, the DHT is not usable for queries for the first 5-10 minutes of operation as the routing table is being
588
+prepared. This means operations like searching the DHT for particular peers or content will not work
589
+ - You can see if the DHT has been initially populated by running `ipfs stats dht`
590
+3. Currently, the accelerated DHT client is not compatible with LAN-based DHTs and will not perform operations against
591
+them
592
+
593
+### How to enable
594
+
595
+```
596
+ipfs config --json Experimental.AcceleratedDHTClient true
597
+```
598
+
599
+### Road to being a real feature
600
+
601
+- [ ] Needs more people to use and report on how well it works
602
+- [ ] Should be usable for queries (even if slower/less efficient) shortly after startup
603
+- [ ] Should be usable with non-WAN DHTs
\ No newline at end of file