@cryptotaxi247 / kubo / commits / 39455605f

CHANGELOG flesh out for 0.4.21-RC3

Steven Allen committed May 28, 2019 at 04:14 UTC 39455605f7470b10fe43f483f49e90a7b60a99a4
1 file changed +212 -12
CHANGELOG.md
+212 -12
@@ -1,28 +1,228 @@
1 # go-ipfs changelog
2
3 -## 0.4.21 XX-XX-XX
3 +## 0.4.21 2019-05-28
4 +
5 +We're happy to announce go-ipfs 0.4.21. This release has some critical bug fixes
6 +and a handful of new features so every user should upgrade.
7 +
8 +Key bug fixes:
9 +
10 +* Too many open file descriptors/too many peers
11 + ([#6237](https://github.com/ipfs/go-ipfs/issues/6237)).
12 +* Adding multiple files at the same time doesn't work
13 + ([#6254](https://github.com/ipfs/go-ipfs/pull/6255)).
14 +* CPU utilization spikes and then holds at 100%
15 + ([#5613](https://github.com/ipfs/go-ipfs/issues/5613)).
16 +
17 +Key features:
18 +
19 +* Experimental TLS1.3 support (to eventually replace secio).
20 +* OpenSSL support for SECIO handshakes (performance improvement).
21 +
22 +**IMPORTANT:** This release fixes a bug in our security transport that could
23 +potentially drop data from the channel. Note: This issue affects neither the
24 +privacy nor the integrity of the data with respect to a third-party attacker.
25 +Only the peer sending us data could trigger this bug.
26 +
27 +**ALL USERS MUST UPGRADE.** We intended to introduce a feature this release that,
28 +unfortunately, [reliably triggered this bug][secio-bug]. To avoid partitioning
29 +the network, we've decided to postpone this feature for a release or two.
30 +
31 +Specifically, we're going to provide a minimum _one month_ upgrade period. After
32 +that, we'll start testing the impact of deploying the proposed changes.
33 +
34 +If you're running the mainline go-ipfs, please upgrade ASAP. If you're building
35 +a separate app or working on a forked go-ipfs, make sure to upgrade
36 +github.com/libp2p/go-libp2p-secio to _at least_ v0.0.3.
37 +
38 +[secio-bug]: https://github.com/libp2p/go-libp2p/issues/644
39 +
40 +### Bug Fixes And Enhancements
41 +
42 +This release includes quite a number of critical bug fixes and
43 +performance/reliability enhancements.
44 +
45 +#### Error when adding multiple files
46 +
47 +The last release broke the simple command `ipfs add file1 file2`. It turns out
48 +we simply lacked a test case for this. Both of these issues (the bug and the
49 +lack of a test case) have now been fixed.
50 +
51 +#### SECIO
52 +
53 +As noted above, we've fixed a bug that could cause data to be dropped from a
54 +SECIO connection on read. Specifically, this happens when:
55 +
56 +1. The capacity of the read buffer is greater than the length.
57 +2. The remote peer sent more than the length but less than the capacity in a
58 + single secio "frame".
59 +
60 +In this case, we'd fill the read buffer to it's capacity instead of its length.
61 +
62 +#### Too many open files, too many peers, etc.
63 +
64 +Go-ipfs automatically closes the least useful connections when it accumulates
65 +too many connections. Unfortunately, some relayed connections were blocking in
66 +`Close()`, halting the entire process.
67 +
68 +#### Out of control CPU usage
69 +
70 +Many users noted out of control CPU usage this release. This turned out to be a
71 +long-standing issue with how the DHT handled provider records (records recording
72 +which peers have what content):
73 +
74 +1. It wasn't removing provider records for content until the set of providers
75 + completely emptied.
76 +2. It was loading every provider record into memory whenever we updated the set
77 + of providers.
78 +
79 +Combined, these two issues were trashing the provider record cache, forcing the
80 +DHT to repeatedly load and discard provider records.
81 +
82 +#### Improved Connection Management
83 +
84 +Go-ipfs uses a service called the "connection manager" to rank connections them
85 +by "usefulness" and close the least-useful ones when go-ipfs collects too many.
86 +
87 +TODO: Fixed race condition between handling new connections and tagging them.
88 +
89 +#### Improved Bitswap Connection Management
90 +
91 +Bitswap now uses the connection manager to mark all peers downloading blocks as
92 +important (while downloading). Previously, it only marked peers from which _it_
93 +was downloading blocks.
94 +
95 +#### Reduced Memory Usage
96 +
97 +The most noticeable memory reduction in this release comes from fixing connection
98 +closing. However, we've made a few additional improvements:
99 +
100 +* Bitswap's "work queue" no longer remembers every peer it has seen
101 + indefinitely.
102 +* The peerstore now interns protocol names.
103 +* The per-peer goroutine count has been reduced.
104 +* The DHT now wastes less memory on idle peers by pooling buffered writers and
105 + returning them to the pool when not actively using them.
106 +
107 +#### Improved Address Freshness
108 +
109 +TODO: Push new addresses when they change.
110 +
111 +#### Increased File Descriptor Limit
112 +
113 +The default file descriptor limit has been raised to 8192 (from 2048).
114 +Unfortunately, go-ipfs behaves poorly when it runs out of file descriptors and
115 +it uses a _lot_ of file descriptors.
116 +
117 +Luckily, most modern kernels can handle thousands of file descriptors without
118 +any difficulty.
119
120 ### Commands
121
7 -#### Fixed: `ipfs add` with multiple files
122 +This release brings no new commands but does introduce a few changes, bugfixes,
123 +and enhancements. This section is hardly complete but it lists the most
124 +noticeable changes.
125
9 -In version 0.4.20 a bug was introduced preventing `ipfs add fileA fileB` from working.
10 -This was fixed in ([ipfs/go-ipfs#6255](https://github.com/ipfs/go-ipfs/pull/6255)).
126 +Take note: this release also introduces a few breaking changes.
127
12 -### New: TLS 1.3 handshake
128 +#### [DEPRECATION] The URLStore Command Deprecated
129
14 -Go-libp2p has gained ability to use TLS 1.3 for connection encryption and authentication.
15 -Go-ipfs will accept TLS 1.3 handshakes but will still primarily use secio.
16 -If you want to help testing the new handshake, you can enable [this experiment](https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#tls-13-as-default-handshake-protocol).
130 +The experimental `ipfs urlstore` command is now deprecated. Please use `ipfs add
131 +--nocopy URL` instead.
132
18 -### New: Build system `GOCC` variable
133 +#### [BREAKING] The DHT Command Base64 Encodes Values
134
20 -Build system now uses `GOCC` environment variable allowing for use of multiple
21 -go versions during builds.
135 +When responding to an `ipfs dht get` command, the daemon now encodes the
136 +returned value using base64. The `ipfs` command will automatically decode this
137 +value before returning it to the user so this change should only affect those
138 +using the HTTP API directly.
139 +
140 +Unfortunately, this change was necessary as DHT records are arbitrary binary
141 +blobs which can't be directly stored in JSON strings.
142 +
143 +#### [BREAKING] Base32 Encoded v1 CIDs By Default
144 +
145 +Both js-ipfs and go-ipfs now encode CIDv1 CIDs using base32 by default, instead
146 +of base58. Unfortunately, base58 is case-sensitive and doesn't play well with
147 +browsers (see [#4143](https://github.com/ipfs/go-ipfs/issues/4143).
148 +
149 +#### Human Readable Numbers
150 +
151 +The `ipfs bitswap stat` and and `ipfs object stat` commands now support a
152 +`--humanize` flag that formats numbers with human-readable units (GiB, MiB,
153 +etc.).
154 +
155 +#### Improved Errors
156 +
157 +This release improves to types of errors:
158 +
159 +1. Commands that take paths/multiaddrs now include the path/multiaddr in the
160 + error message when it fails to parse.
161 +2. `ipfs swarm connect` now returns a detailed error describing which addresses
162 + were tried and why the dial failed.
163 +
164 +#### Ping Improvements
165 +
166 +The ping command has received some small improvements and fixes:
167 +
168 +1. It now exits with a non-zero exit status on failure.
169 +2. It no longer succeeds with zero successful pings if we have a zombie but
170 + non-functional connection to the peer being pinged
171 + ([#6298](https://github.com/ipfs/go-ipfs/issues/6298)).
172 +3. It now prints out the average latency when canceled with `^C` (like the unix
173 + `ping` command).
174 +
175 +### Features
176
177 +This release is primarily a bug fix release but it still includes two nice
178 +features from libp2p.
179
24 -(please point out what should be featured, doesn't have to be go-ipfs).
180 +#### Experimental TLS1.3 support
181
182 +Go-ipfs now has experimental TLS1.3 support. Currently, libp2p (IPFS's
183 +networking library) uses a custom TLS-like protocol we call SECIO. However, the
184 +conventional wisdom concerning custom security transports is "just don't" so we
185 +are working on replacing it with TLS1.3
186 +
187 +To choose this protocol by default, set the `Experimental.PreferTLS` config
188 +variable:
189 +
190 +```bash
191 +> ipfs config --bool Experimental.PreferTLS true
192 +```
193 +
194 +Why TLS1.3 and not X (noise, etc.)?
195 +
196 +1. Libp2p allows negotiating transports so there's no reason not to add noise
197 + support to libp2p as well.
198 +2. TLS has wide language support which should make implementing libp2p for new
199 + languages significantly simpler.
200 +
201 +#### OpenSSL Support
202 +
203 +Go-ipfs can now (optionally) be built with OpenSSL support for improved
204 +performance when establishing connections. This is primarily useful for nodes
205 +receiving multiple inbound connections per second.
206 +
207 +To enable openssl support, rebuild go-ipfs with:
208 +
209 +```bash
210 +> make build GOFLAGS=-tags=openssl
211 +```
212 +
213 +### CoreAPI
214 +
215 +The CoreAPI refactor is still underway and we've made significant progress
216 +towards a usable ipfs-as-a-library constructor. Specifically, we've integrated
217 +integrating the [fx](https://go.uber.org/fx) dependency injection system and are
218 +now working on cleaning up our initialization logic. This should make it easier
219 +to inject new services into a go-ipfs process without messing with the core
220 +internals.
221 +
222 +### Build: `GOCC` Environment Variable
223 +
224 +Build system now uses `GOCC` environment variable allowing for use of specific
225 +go versions during builds.
226
227 ### Changelog
228