docs: add sweeping provide worker count recommendation (#11001)
Add recommentation for worker count for the sweeping provide system for users with millions of CIDs. (cherry picked from commit cf8194a8d169fdc07eae3386a27bd3b7fc3f4864)
Guillaume Michel committed
Oct 1, 2025 at 16:01 UTC
48cb03c3f435e7b6a53f3db79747119fe0e7f932
1 file changed
+25
docs/config.md
+25
@@ -1633,12 +1633,14 @@ this limit in the configuration.
1633
**Why operations fail instead of auto-flushing:** Automatic flushing once the limit
1634
is reached was considered but rejected because it can lead to data corruption issues
1635
that are difficult to debug. When the system decides to flush without user knowledge, it can:
1636
+
1637
- Create partial states that violate user expectations about atomicity
1638
- Interfere with concurrent operations in unexpected ways
1639
- Make debugging and recovery much harder when issues occur
1640
1641
By failing explicitly, users maintain control over when their data is persisted,
1642
allowing them to:
1643
+
1644
- Batch related operations together before flushing
1645
- Handle errors predictably at natural transaction boundaries
1646
- Understand exactly when and why their data is written to disk
@@ -1647,6 +1649,7 @@ If you expect automatic flushing behavior, simply use the default `--flush=true`
1649
(or omit the flag entirely) instead of `--flush=false`.
1650
1651
**⚠️ WARNING:** Increasing this limit or disabling it (setting to 0) can lead to:
1652
+
1653
- **Out-of-memory errors (OOM)** - Each unflushed operation consumes memory
1654
- **Data loss** - If the daemon crashes before flushing, all unflushed changes are lost
1655
- **Degraded performance** - Large unflushed caches slow down MFS operations
@@ -2033,6 +2036,19 @@ connections this setting can generate.
2036
> At the same time, mind that raising this value too high may lead to increased load.
2037
> Proceed with caution, ensure proper hardware and networking are in place.
2038
2039
+> [!TIP]
2040
+> **When `SweepEnabled` is true:** Users providing millions of CIDs or more
2041
+> should increase the worker count accordingly. Underprovisioning can lead to
2042
+> slow provides (burst workers) and inability to keep up with content
2043
+> reproviding (periodic workers). For nodes with sufficient resources (CPU,
2044
+> bandwidth, number of connections), dedicating `1024` for [periodic
2045
+> workers](#providedhtdedicatedperiodicworkers) and `512` for [burst
2046
+> workers](#providedhtdedicatedburstworkers), and `2048` [max
2047
+> workers](#providedhtmaxworkers) should be adequate even for the largest
2048
+> users. The system will only use workers as needed - unused resources won't be
2049
+> consumed. Ensure you adjust the swarm [connection manager](#swarmconnmgr) and
2050
+> [resource manager](#swarmresourcemgr) configuration accordingly.
2051
+
2052
Default: `16`
2053
2054
Type: `optionalInteger` (non-negative; `0` means unlimited number of workers)
@@ -2098,6 +2114,11 @@ number of workers will be dedicated to the periodic region reprovide only. The s
2114
Any remaining workers (MaxWorkers - DedicatedPeriodicWorkers - DedicatedBurstWorkers)
2115
form a shared pool that can be used for either type of work as needed.
2116
2117
+> [!NOTE]
2118
+> If the provider system isn't able to keep up with reproviding all your
2119
+> content within the [Provide.DHT.Interval](#providedhtinterval), consider
2120
+> increasing this value.
2121
+
2122
Default: `2`
2123
2124
Type: `optionalInteger` (`0` means there are no dedicated workers, but the
@@ -2121,6 +2142,10 @@ number of workers will be dedicated to burst provides only. In addition to
2142
these, if there are available workers in the pool, they can also be used for
2143
burst provides.
2144
2145
+> [!NOTE]
2146
+> If CIDs aren't provided quickly enough to your taste, and you can afford more
2147
+> CPU and bandwidth, consider increasing this value.
2148
+
2149
Default: `1`
2150
2151
Type: `optionalInteger` (`0` means there are no dedicated workers, but the