docs: add optimistic provide feature description
Dennis Trautwein committed
Apr 6, 2023 at 16:28 UTC
cb2e109e4d5882f4fc5e5e5a981460c961525b20
1 file changed
+74
docs/experimental-features.md
+74
@@ -27,6 +27,7 @@ the above issue.
27
- [Graphsync](#graphsync)
28
- [Noise](#noise)
29
- [Accelerated DHT Client](#accelerated-dht-client)
30
+- [Optimistic Provide](#optimistic-provide)
31
32
---
33
@@ -597,3 +598,76 @@ ipfs config --json Experimental.AcceleratedDHTClient true
598
- [ ] Needs more people to use and report on how well it works
599
- [ ] Should be usable for queries (even if slower/less efficient) shortly after startup
600
- [ ] Should be usable with non-WAN DHTs
601
+
602
+## Optimistic Provide
603
+
604
+### In Version
605
+
606
+0.20.0
607
+
608
+### State
609
+
610
+Experimental, disabled by default.
611
+
612
+When the DHT client tries to store a provider in the DHT, it typically searches for the 20 peers that are closest to the
613
+target key. However, this process can be time-consuming, as the search terminates only after no closer peers are found
614
+among the three currently (during the query) known closest ones. In cases where these closest peers are slow to respond
615
+(which often happens if they are located at the edge of the DHT network), the query gets blocked by the slowest peer.
616
+
617
+To address this issue, the `OptimisticProvide` feature can be enabled. This feature allows the client to estimate the
618
+network size and determine how close a peer _likely_ needs to be to the target key to be within the 20 closest peers.
619
+While searching for the closest peers in the DHT, the client will _optimistically_ store the provider record with peers
620
+and abort the query completely when the set of currently known 20 closest peers are also _likely_ the actual 20 closest
621
+ones. This heuristic approach can significantly speed up the process, resulting in a speed improvement of 2x to >10x.
622
+
623
+When it is enabled:
624
+
625
+- DHT provide operations should complete much faster than with it disabled
626
+- This can be tested with commands such as `ipfs routing provide`
627
+
628
+**Tradeoffs**
629
+
630
+There are now the classic client, the accelerated DHT client, and optimistic provide that improve the provider process.
631
+There are different trade-offs with all of them. The accelerated DHT client is still faster to provide large amounts
632
+of provider records at the cost of high resource requirements. Optimistic provide doesn't have the high resource
633
+requirements but might not choose optimal peers and is not as fast as the accelerated client, but still much faster
634
+than the classic client.
635
+
636
+**Caveats:**
637
+
638
+1. Providing optimistically requires a current network size estimation. This estimation is calculated through routing
639
+ table refresh queries and is only available after the daemon has been running for some time. If there is no network
640
+ size estimation available the client will transparently fall back to the classic approach.
641
+2. The chosen peers to store the provider records might not be the actual closest ones. Measurements showed that this
642
+ is not a problem.
643
+3. The optimistic provide process returns already after 15 out of the 20 provider records were stored with peers. The
644
+ reasoning here is that one out of the remaining 5 peers are very likely to time out and delay the whole process. To
645
+ limit the number of in-flight async requests there is the second `OptimisticProvideJobsPoolSize` setting. Currently,
646
+ this is set to 60. This means that at most 60 parallel background requests are allowed to be in-flight. If this
647
+ limit is exceeded optimistic provide will block until all 20 provider records are written. This is still 2x faster
648
+ than the classic approach but not as fast as returning early which yields >10x speed-ups.
649
+4. Since the in-flight background requests are likely to time out, they are not consuming many resources and the job
650
+ pool size could probably be much higher.
651
+
652
+For more information, see:
653
+
654
+- Project doc: https://protocollabs.notion.site/Optimistic-Provide-2c79745820fa45649d48de038516b814
655
+- go-libp2p-kad-dht: https://github.com/libp2p/go-libp2p-kad-dht/pull/783
656
+
657
+### Configuring
658
+To enable:
659
+
660
+```
661
+ipfs config --json Experimental.OptimisticProvide true
662
+```
663
+
664
+If you want to change the `OptimisticProvideJobsPoolSize` setting from its default of 60:
665
+
666
+```
667
+ipfs config --json Experimental.OptimisticProvideJobsPoolSize 120
668
+```
669
+
670
+### Road to being a real feature
671
+
672
+- [ ] Needs more people to use and report on how well it works
673
+- [ ] Should prove at least equivalent availability of provider records as the classic approach