Introduce buzhash chunker
It has the same properties as Rabin but is much faster. Benchmark results: ``` name speed Buzhash2/1K-4 1.71GB/s ± 4% Buzhash2/1M-4 1.15GB/s ± 2% Buzhash2/16M-4 892MB/s ± 1% Buzhash2/100M-4 904MB/s ± 2% Rabin/1K-4 13.8MB/s ± 3% Rabin/1M-4 171MB/s ± 3% Rabin/16M-4 182MB/s ± 4% Rabin/100M-4 182MB/s ± 3% Default/1K-4 1.74GB/s ± 4% Default/1M-4 3.22GB/s ± 2% Default/16M-4 3.88GB/s ± 2% Default/100M-4 4.21GB/s ± 6% ``` License: MIT Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
Jakub Sztandera committed
Oct 7, 2019 at 18:06 UTC
67f98377342a3edaf9c2decd68943c5590af5891
2 files changed
+3
-1
go.mod
+1
-1
@@ -28,7 +28,7 @@ require (
28
github.com/ipfs/go-filestore v0.0.2
29
github.com/ipfs/go-fs-lock v0.0.1
30
github.com/ipfs/go-ipfs-blockstore v0.1.0
31
- github.com/ipfs/go-ipfs-chunker v0.0.1
31
+ github.com/ipfs/go-ipfs-chunker v0.0.3
32
github.com/ipfs/go-ipfs-cmds v0.1.1
33
github.com/ipfs/go-ipfs-config v0.0.11
34
github.com/ipfs/go-ipfs-ds-help v0.0.1
go.sum
+2
@@ -183,6 +183,8 @@ github.com/ipfs/go-ipfs-blocksutil v0.0.1 h1:Eh/H4pc1hsvhzsQoMEP3Bke/aW5P5rVM1IW
183
github.com/ipfs/go-ipfs-blocksutil v0.0.1/go.mod h1:Yq4M86uIOmxmGPUHv/uI7uKqZNtLb449gwKqXjIsnRk=
184
github.com/ipfs/go-ipfs-chunker v0.0.1 h1:cHUUxKFQ99pozdahi+uSC/3Y6HeRpi9oTeUHbE27SEw=
185
github.com/ipfs/go-ipfs-chunker v0.0.1/go.mod h1:tWewYK0we3+rMbOh7pPFGDyypCtvGcBFymgY4rSDLAw=
186
+github.com/ipfs/go-ipfs-chunker v0.0.3 h1:UuXhKoxvxl/vGhie+WXFRZYCwpZjbKF2SzD1Tvxif1I=
187
+github.com/ipfs/go-ipfs-chunker v0.0.3/go.mod h1:RkGJorerOQNTDPgmX7HtJ5YzVQqaIYdzI/hrCHty5Kc=
188
github.com/ipfs/go-ipfs-cmds v0.1.1 h1:H9/BLf5rcsULHMj/x8gC0e5o+raYhqk1OQsfzbGMNM4=
189
github.com/ipfs/go-ipfs-cmds v0.1.1/go.mod h1:k1zMXcOLtljA9iAnZHddbH69yVm5+weRL0snmMD/rK0=
190
github.com/ipfs/go-ipfs-config v0.0.11 h1:5/4nas2CQXiKr2/MLxU24GDGTBvtstQIQezuk7ltOQQ=