966
967
### `Datastore.BloomFilterSize`
968
969
-A number representing the size in bytes of the blockstore's [bloom
970
-filter](https://en.wikipedia.org/wiki/Bloom_filter). A value of zero represents
971
-the feature is disabled.
972
-
973
-This site generates useful graphs for various bloom filter values:
974
-<https://hur.st/bloomfilter/?n=1e6&p=0.01&m=&k=7> You may use it to find a
975
-preferred optimal value, where `m` is `BloomFilterSize` in bits. Remember to
976
-convert the value `m` from bits, into bytes for use as `BloomFilterSize` in the
977
-config file. For example, for 1,000,000 blocks, expecting a 1% false-positive
978
-rate, you'd end up with a filter size of 9592955 bits, so for `BloomFilterSize`
979
-we'd want to use 1199120 bytes. As of writing, [7 hash
980
-functions](https://github.com/ipfs/go-ipfs-blockstore/blob/547442836ade055cc114b562a3cc193d4e57c884/caching.go#L22)
981
-are used, so the constant `k` is 7 in the formula.
982
-
983
-Enabling the BloomFilter can provide performance improvements specially when
984
-responding to many requests for inexistent blocks. It however requires a full
985
-sweep of all the datastore keys on daemon start. On very large datastores this
986
-can be a very taxing operation, particularly if the datastore does not support
987
-querying existing keys without reading their values at the same time (blocks).
969
+The size in **bytes** of the blockstore's [bloom filter](https://en.wikipedia.org/wiki/Bloom_filter).
970
+A value of `0` disables the feature.
971
+
972
+The bloom filter answers "does the blockstore *not* have this CID?" from RAM
973
+without touching the datastore. A negative answer is exact (no false
974
+negatives, so blocks are never falsely reported missing); a positive answer
975
+is probabilistic and falls through to the underlying blockstore for
976
+verification. The chance of a false "maybe present" is the filter's
977
+**false-positive rate (FPR)**. A false positive costs one wasted datastore
978
+lookup; it never causes data loss or incorrect retrieval. The lower the FPR,
979
+the more `Has()` calls the filter answers from RAM alone.
980
+
981
+This cache pays off most on nodes that field many requests for content they
982
+don't host: public gateways, mirrors, and peers asked to serve
983
+opportunistically-cached blocks.
984
+
985
+The complementary cache for the *positive* path (block exists, look up its
986
+size) is [`Datastore.BlockKeyCacheSize`](#datastoreblockkeycachesize).
987
+
988
+#### How kubo's bloom filter is sized
989
+
990
+Kubo wires the underlying [`ipfs/bbloom`](https://github.com/ipfs/bbloom)
991
+filter with `k=7` hash positions. Two kubo-specific behaviors matter for
992
+sizing:
993
+
994
+1. **Power-of-two bit-count rounding.** bbloom rounds the requested bit
995
+ count up to the next power of two, so a `BloomFilterSize` value that is
996
+ not itself a power of two in bits silently allocates more memory than
997
+ configured. For example, `BloomFilterSize: 1199120` (~1.14 MiB)
998
+ actually allocates a `16,777,216`-bit (= 2 MiB) filter internally. For
999
+ predictable behavior, pick `BloomFilterSize` values that are
1000
+ power-of-two byte counts: 1 MiB, 2 MiB, 4 MiB, ..., 256 MiB, 512 MiB,
1001
+ 1 GiB.
1002
+2. **Fixed `k=7`.** With seven hash positions, FPR for a filter of `m`
1003
+ bits and `n` inserted entries is `(1 - exp(-7n/m))^7`. To hit a
1004
+ target FPR, budget roughly ~1.8 bytes per entry at ~1% FPR, ~2.8
1005
+ bytes per entry at ~0.1% FPR, and ~4.2 bytes per entry at ~0.01%
1006
+ FPR. These figures already include the average ~1.5x penalty from
1007
+ the power-of-two rounding above; the worst case is ~2x.
1008
+
1009
+#### Reference sizing
1010
+
1011
+Power-of-two `BloomFilterSize` values for common blockset sizes, with the
1012
+FPR you can expect at the design point and at 2× growth:
1013
+
1014
+| Expected blocks (`n`) | `BloomFilterSize` | FPR at `n` | FPR at 2× `n` |
1015
+|---:|---:|---:|---:|
1016
+| 10,000,000 | `16777216` (16 MiB) | ~0.18% | ~5% |
1017
+| 25,000,000 | `33554432` (32 MiB) | ~0.58% | ~11% |
1018
+| 50,000,000 | `67108864` (64 MiB) | ~0.58% | ~11% |
1019
+| 100,000,000 | `134217728` (128 MiB) | ~0.58% | ~11% |
1020
+| 200,000,000 | `268435456` (256 MiB) | ~0.58% | ~11% |
1021
+
1022
+For a tighter FPR at the design point, step up to the next power of two.
1023
+
1024
+The [hur.st/bloomfilter](https://hur.st/bloomfilter/?n=10e6&p=0.01&m=&k=7)
1025
+calculator works as a reference for exploring `(n, p, m)` combinations
1026
+(remember kubo uses `k=7`); just keep in mind that the `m` it suggests
1027
+is the optimal-fit value, while bbloom rounds up to the next power of
1028
+two on top of that.
1029
+
1030
+#### Saturation as the repo grows
1031
+
1032
+A bloom filter is fixed-size after creation. As more CIDs are inserted
1033
+past its design `n`, the false-positive rate climbs steeply. Rough
1034
+behavior with a filter sized for ~0.6% FPR at its design point:
1035
+
1036
+- At `n`: ~0.6% FPR. Every "definitely not" reliably saves a datastore
1037
+ lookup.
1038
+- At ~`2 × n`: ~11% FPR. Most negatives still save lookups, but tail
1039
+ latency rises because each "maybe" still hits the datastore.
1040
+- At ~`4 × n`: ~58% FPR. Most "maybe" answers fall through. The filter
1041
+ is mostly paying CPU and RAM cost without short-circuiting much.
1042
+- At ~`8 × n` or more: above ~95% FPR. Effectively saturated. The
1043
+ filter answers "maybe" for nearly every CID and provides no benefit.
1044
+
1045
+Size for **expected steady-state, not today's count**, and re-tune after
1046
+crossing the design point. Bloom filters cannot grow in place; raising
1047
+`BloomFilterSize` and restarting the daemon rebuilds the filter from
1048
+scratch.
1049
+
1050
+#### Risks of an undersized filter
1051
+
1052
+A poorly-sized filter is **never a correctness issue**. Bloom filters
1053
+have no false negatives, so blocks are never falsely reported missing.
1054
+The risks are operational:
1055
+
1056
+- **Wasted RAM and CPU.** Every `Has()` still runs all seven hash
1057
+ positions. Once the filter saturates, those cycles return nothing.
1058
+- **Silent regression as the pinset grows.** A filter sized for last
1059
+ year's data can drift past saturation without warning; the
1060
+ negative-`Has` short-circuit benefit just quietly disappears.
1061
+- **Recurring startup tax.** The filter rebuilds on every daemon
1062
+ restart (see below). On slow disks this means minutes of
1063
+ `AllKeysChan` walking, paid in full even when the resulting filter
1064
+ is too small to help.
1065
+
1066
+Quick health check: divide `BloomFilterSize` by your current block count.
1067
+Below ~`1` byte/block the filter is past its design point; below
1068
+~`0.5` bytes/block it is effectively saturated.
1069
+
1070
+#### Startup cost
1071
+
1072
+The filter is not persisted across restarts. Every daemon start rebuilds it
1073
+by walking all datastore keys (`AllKeysChan`). On very large blockstores or
1074
+slow disks this can take many minutes, during which `Has()` falls through
1075
+to the datastore and the filter provides no benefit. Datastores that cannot
1076
+enumerate keys without reading values (block content) pay even more here;
1077
+flatfs and pebble both support keys-only iteration, so the rebuild cost
1078
+scales with the keyset, not data volume.
1079
1080
Default: `0` (disabled)
1081
1102
1103
### `Datastore.BlockKeyCacheSize`
1104
1014
-A number representing the maximum size in bytes of the blockstore's Two-Queue
1015
-cache, which caches block-cids and their block-sizes. Use `0` to disable.
1105
+The maximum **number of entries** held in the blockstore's Two-Queue cache. The
1106
+cache stores per-CID metadata (existence and block size) but never block
1107
+content. Use `0` to disable.
1108
1017
-This cache, once primed, can greatly speed up operations like `ipfs repo stat`
1018
-as there is no need to read full blocks to know their sizes. Size should be
1019
-adjusted depending on the number of CIDs on disk (`NumObjects in`ipfs repo stat`).
1109
+A cache hit answers `Has` and `GetSize` from RAM and skips the underlying
1110
+datastore lookup. This includes the per-block `os.Stat` flatfs does to learn a
1111
+block's size, which is the dominant cost on bitswap servers responding to peer
1112
+wantlists.
1113
1021
-Default: `65536` (64KiB)
1114
+The cache uses a [Two-Queue (2Q) replacement policy](https://pkg.go.dev/github.com/hashicorp/golang-lru/v2#TwoQueueCache):
1115
+an entry must be touched twice before it is promoted to the frequently-used
1116
+tier. A long one-shot scan (reprovider, GC, `ipfs repo verify`) therefore
1117
+does not evict the hot entries that bitswap repeatedly serves.
1118
1023
-Type: `optionalInteger` (non-negative, bytes)
1119
+#### Sizing
1120
+
1121
+Memory usage is roughly the entry count times the per-entry overhead, which
1122
+combines 2Q bookkeeping, the multihash key bytes, and the cached value. As a
1123
+rough estimate, budget ~200 bytes per entry, so `1048576` (1M entries) is on
1124
+the order of ~200 MB resident. The cache only needs to cover the **hot
1125
+working set** of CIDs (the ones repeatedly hit by inbound bitswap, gateway,
1126
+or DAG-resolution traffic), not the entire blockstore.
1127
+
1128
+The default of `65536` is sized for small dev/desktop nodes. Operators
1129
+running public gateways, pinning clusters, or any node serving non-trivial
1130
+bitswap traffic should size this against the active working set. See
1131
+[`Datastore.BloomFilterSize`](#datastorebloomfiltersize) for the
1132
+complementary negative-`Has()` short-circuit that pairs well with this cache.
1133
+
1134
+Default: `65536` (entries)
1135
+
1136
+Type: `optionalInteger` (non-negative, number of entries)
1137
1138
### `Datastore.Spec`
1139