feat: HTTP retrieval enabled by default (#10836)
Marcin Rataj committed
Jun 17, 2025 at 01:48 UTC
b45ed8a3d7449a3c9f1e4d4f4e4da623b2a66ff9
4 files changed
+34
-10
config/http_retrieval.go
+1
-1
@@ -12,7 +12,7 @@ type HTTPRetrieval struct {
12
}
13
14
const (
15
- DefaultHTTPRetrievalEnabled = false // opt-in for now, until we figure out https://github.com/ipfs/specs/issues/496
15
+ DefaultHTTPRetrievalEnabled = true
16
DefaultHTTPRetrievalNumWorkers = 16
17
DefaultHTTPRetrievalTLSInsecureSkipVerify = false // only for testing with self-signed HTTPS certs
18
DefaultHTTPRetrievalMaxBlockSize = "2MiB" // matching bitswap: https://specs.ipfs.tech/bitswap-protocol/#block-sizes
docs/changelogs/v0.36.md
+7
@@ -11,6 +11,7 @@ This release was brought to you by the [Shipyard](http://ipshipyard.com/) team.
11
12
- [Overview](#overview)
13
- [🔦 Highlights](#-highlights)
14
+ - [HTTP Retrieval client enabled by default](#http-retrieval-client-enabled-by-default)
15
- [Update go-log to v2](#update-go-log-to-v2)
16
- [AutoNATv2 Client](#autonatv2-client)
17
- [Smarter AutoTLS registration](#smarter-autotls-registration)
@@ -24,6 +25,12 @@ This release was brought to you by the [Shipyard](http://ipshipyard.com/) team.
25
26
### 🔦 Highlights
27
28
+#### HTTP Retrieval client enabled by default
29
+
30
+This release promotes the HTTP Retrieval client from an experimental feature to a standard feature that is enabled by default. When possible, Kubo will be retrieving blocks over plain HTTPS (HTTP/2) without any extra user configuration.
31
+
32
+See [`HTTPRetrieval`](https://github.com/ipfs/kubo/blob/master/docs/config.md#httpretrieval) for more details.
33
+
34
#### Update go-log to v2
35
36
go-log v2 has been out for quite a while now and it is time to deprecate v1.
docs/config.md
+10
-9
@@ -2686,25 +2686,26 @@ Type: `object`
2686
2687
### `HTTPRetrieval.Enabled`
2688
2689
-> [!CAUTION]
2690
-> This feature is **EXPERIMENTAL** and may change in future release. Enable with caution, and provide feedback via GitHub issues.
2691
-
2689
Controls whether HTTP-based block retrieval is enabled.
2690
2694
-When enabled, Kubo will be able to act on `/tls/http` (HTTP/2) providers ([Trustless HTTP Gateways](https://specs.ipfs.tech/http-gateways/trustless-gateway/)) returned by the [`Routing.DelegatedRouters`](#routingdelegatedrouters)
2691
+When enabled, Kubo will act on `/tls/http` (HTTP/2) providers ([Trustless HTTP Gateways](https://specs.ipfs.tech/http-gateways/trustless-gateway/)) returned by the [`Routing.DelegatedRouters`](#routingdelegatedrouters)
2692
to perform pure HTTP [block retrievals](https://specs.ipfs.tech/http-gateways/trustless-gateway/#block-responses-application-vnd-ipld-raw)
2696
-in addition to [Bitswap over Libp2p](#bitswap).
2693
+(`/ipfs/cid?format=raw`, `Accept: application/vnd.ipld.raw`)
2694
+alongside [Bitswap over Libp2p](#bitswap).
2695
2698
-HTTP requests for `application/vnd.ipld.raw` will be issued instead of Bitswap if a peer has a `/tls/http` multiaddr
2696
+HTTP requests for `application/vnd.ipld.raw` will be made instead of Bitswap when a peer has a `/tls/http` multiaddr
2697
and the HTTPS server returns HTTP 200 for the [probe path](https://specs.ipfs.tech/http-gateways/trustless-gateway/#dedicated-probe-paths).
2698
2699
> [!IMPORTANT]
2702
-> - Requires TLS and HTTP/2.
2700
+> This feature is relatively new. Please report any issues via [Github](https://github.com/ipfs/kubo/issues/new).
2701
+>
2702
+> Important notes:
2703
+> - TLS and HTTP/2 are required. For privacy reasons, and to maintain feature-parity with browsers, unencrypted `http://` providers are ignored and not used.
2704
> - This feature works in the same way as Bitswap: connected HTTP-peers receive optimistic block requests even for content that they are not announcing.
2704
-> - HTTP client does not follow redirects. Providers should keep announcements up to date.
2705
+> - For performance reasons, and to avoid loops, the HTTP client does not follow redirects. Providers should keep announcements up to date.
2706
> - IPFS ecosystem is working towards [supporting HTTP providers on Amino DHT](https://github.com/ipfs/specs/issues/496). Currently, HTTP providers are mostly limited to results from [`Routing.DelegatedRouters`](#routingdelegatedrouters) endpoints and requires `Routing.Type=auto|autoclient`.
2707
2707
-Default: `false`
2708
+Default: `true`
2709
2710
Type: `flag`
2711
test/sharness/t0119-prometheus-data/prometheus_metrics
+16
@@ -7,6 +7,22 @@ exchange_bitswap_wantlists_seconds_bucket
7
exchange_bitswap_wantlists_seconds_count
8
exchange_bitswap_wantlists_seconds_sum
9
exchange_bitswap_wantlists_total
10
+exchange_httpnet_request_duration_seconds_bucket
11
+exchange_httpnet_request_duration_seconds_count
12
+exchange_httpnet_request_duration_seconds_sum
13
+exchange_httpnet_request_sent_bytes
14
+exchange_httpnet_requests_body_failure
15
+exchange_httpnet_requests_failure
16
+exchange_httpnet_requests_in_flight
17
+exchange_httpnet_requests_total
18
+exchange_httpnet_response_bytes_bucket
19
+exchange_httpnet_response_bytes_count
20
+exchange_httpnet_response_bytes_sum
21
+exchange_httpnet_wantlists_items_total
22
+exchange_httpnet_wantlists_seconds_bucket
23
+exchange_httpnet_wantlists_seconds_count
24
+exchange_httpnet_wantlists_seconds_sum
25
+exchange_httpnet_wantlists_total
26
go_gc_duration_seconds
27
go_gc_duration_seconds_count
28
go_gc_duration_seconds_sum