fix/gateway: escape directory redirect url (#10649)
* fix/gateway: escape directory redirect URLs When a director gets redirected to a URL with a treailing slash, special chars in the directory name must be escaped in the redirect URL. This upgrades to a version of box that has that fix. Closes #10536 * Fix sharness test for new redirect URLs * Update to latest boxo * Use latest gateway-conformance (cherry picked from commit b7cc47aef5ee95ff1a45aedc76a176d5f6008a71)
Andrew Gillis committed
Jan 3, 2025 at 14:37 UTC
5e1bacc2a8d458495fe272a0283e9b043928fa00
9 files changed
+22
-16
.github/workflows/gateway-conformance.yml
+4
-4
@@ -41,7 +41,7 @@ jobs:
41
steps:
42
# 1. Download the gateway-conformance fixtures
43
- name: Download gateway-conformance fixtures
44
- uses: ipfs/gateway-conformance/.github/actions/extract-fixtures@v0.6
44
+ uses: ipfs/gateway-conformance/.github/actions/extract-fixtures@v0.7
45
with:
46
output: fixtures
47
@@ -94,7 +94,7 @@ jobs:
94
95
# 6. Run the gateway-conformance tests
96
- name: Run gateway-conformance tests
97
- uses: ipfs/gateway-conformance/.github/actions/test@v0.6
97
+ uses: ipfs/gateway-conformance/.github/actions/test@v0.7
98
with:
99
gateway-url: http://127.0.0.1:8080
100
subdomain-url: http://localhost:8080
@@ -128,7 +128,7 @@ jobs:
128
steps:
129
# 1. Download the gateway-conformance fixtures
130
- name: Download gateway-conformance fixtures
131
- uses: ipfs/gateway-conformance/.github/actions/extract-fixtures@v0.6
131
+ uses: ipfs/gateway-conformance/.github/actions/extract-fixtures@v0.7
132
with:
133
output: fixtures
134
@@ -201,7 +201,7 @@ jobs:
201
202
# 9. Run the gateway-conformance tests over libp2p
203
- name: Run gateway-conformance tests over libp2p
204
- uses: ipfs/gateway-conformance/.github/actions/test@v0.6
204
+ uses: ipfs/gateway-conformance/.github/actions/test@v0.7
205
with:
206
gateway-url: http://127.0.0.1:8092
207
args: --specs "trustless-gateway,-trustless-ipns-gateway" -skip 'TestGatewayCar/GET_response_for_application/vnd.ipld.car/Header_Content-Length'
docs/changelogs/v0.33.md
+6
@@ -15,6 +15,7 @@
15
- [New options for faster writes: `WriteThrough`, `BlockKeyCacheSize`, `BatchMaxNodes`, `BatchMaxSize`](#new-options-for-faster-writes-writethrough-blockkeycachesize-batchmaxnodes-batchmaxsize)
16
- [MFS stability with large number of writes](#mfs-stability-with-large-number-of-writes)
17
- [📦️ Important dependency updates](#-important-dependency-updates)
18
+ - [Escape Redirect URL for Directory](#escape-redirect-url-for-directory)
19
- [📝 Changelog](#-changelog)
20
- [👨👩👧👦 Contributors](#-contributors)
21
@@ -89,6 +90,11 @@ We have fixed a number of issues that were triggered by writing or copying many
90
- update `p2p-forge/client` to [v0.2.0](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.2.0) (incl. [v0.1.0](https://github.com/ipshipyard/p2p-forge/releases/tag/v0.1.0))
91
- update `ipfs-webui` to [v4.4.2](https://github.com/ipfs/ipfs-webui/releases/tag/v4.4.2) (incl. [v4.4.1](https://github.com/ipfs/ipfs-webui/releases/tag/v4.4.1))
92
93
+
94
+#### Escape Redirect URL for Directory
95
+
96
+When navigating to a subdirectory, served by the Kubo web server, a subdirectory without a trailing slash gets redirected to a URL with a trailing slash. If there are special characters such as "%" in the subdirectory name then these must be escaped in the redirect URL. Previously this was not being done and was preventing navigation to such subdirectories, requiring the user to manually add a trailing slash to the subdirectory URL. This is now fixed to handle the redirect to URLs with characters that must be escaped.
97
+
98
### 📝 Changelog
99
100
### 👨👩👧👦 Contributors
docs/examples/kubo-as-a-library/go.mod
+1
-1
@@ -7,7 +7,7 @@ go 1.23
7
replace github.com/ipfs/kubo => ./../../..
8
9
require (
10
- github.com/ipfs/boxo v0.26.0
10
+ github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac
11
github.com/ipfs/kubo v0.0.0-00010101000000-000000000000
12
github.com/libp2p/go-libp2p v0.38.1
13
github.com/multiformats/go-multiaddr v0.14.0
docs/examples/kubo-as-a-library/go.sum
+2
-2
@@ -304,8 +304,8 @@ github.com/ipfs-shipyard/nopfs/ipfs v0.25.0 h1:OqNqsGZPX8zh3eFMO8Lf8EHRRnSGBMqcd
304
github.com/ipfs-shipyard/nopfs/ipfs v0.25.0/go.mod h1:BxhUdtBgOXg1B+gAPEplkg/GpyTZY+kCMSfsJvvydqU=
305
github.com/ipfs/bbloom v0.0.4 h1:Gi+8EGJ2y5qiD5FbsbpX/TMNcJw8gSqr7eyjHa4Fhvs=
306
github.com/ipfs/bbloom v0.0.4/go.mod h1:cS9YprKXpoZ9lT0n/Mw/a6/aFV6DTjTLYHeA+gyqMG0=
307
-github.com/ipfs/boxo v0.26.0 h1:RRxEon7rJMy8ScVaTLncSZ5/nA6majYhRSbzc80snO8=
308
-github.com/ipfs/boxo v0.26.0/go.mod h1:iHyc9cjoF7/zoiKVY65d2fBWRhoS2zx4cMk8hKgqrac=
307
+github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac h1:RbM53YTWjs7nZAAiUV7GSIV7S8QOaq0YkGAuuNNGM7A=
308
+github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac/go.mod h1:iHyc9cjoF7/zoiKVY65d2fBWRhoS2zx4cMk8hKgqrac=
309
github.com/ipfs/go-bitfield v1.1.0 h1:fh7FIo8bSwaJEh6DdTWbCeZ1eqOaOkKFI74SCnsWbGA=
310
github.com/ipfs/go-bitfield v1.1.0/go.mod h1:paqf1wjq/D2BBmzfTVFlJQ9IlFOZpg422HL0HqsGWHU=
311
github.com/ipfs/go-bitswap v0.11.0 h1:j1WVvhDX1yhG32NTC9xfxnqycqYIlhzEzLXG/cU1HyQ=
go.mod
+1
-1
@@ -22,7 +22,7 @@ require (
22
github.com/hashicorp/go-version v1.7.0
23
github.com/ipfs-shipyard/nopfs v0.0.14
24
github.com/ipfs-shipyard/nopfs/ipfs v0.25.0
25
- github.com/ipfs/boxo v0.26.0
25
+ github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac
26
github.com/ipfs/go-block-format v0.2.0
27
github.com/ipfs/go-cid v0.4.1
28
github.com/ipfs/go-cidutil v0.1.0
go.sum
+2
-2
@@ -368,8 +368,8 @@ github.com/ipfs-shipyard/nopfs/ipfs v0.25.0 h1:OqNqsGZPX8zh3eFMO8Lf8EHRRnSGBMqcd
368
github.com/ipfs-shipyard/nopfs/ipfs v0.25.0/go.mod h1:BxhUdtBgOXg1B+gAPEplkg/GpyTZY+kCMSfsJvvydqU=
369
github.com/ipfs/bbloom v0.0.4 h1:Gi+8EGJ2y5qiD5FbsbpX/TMNcJw8gSqr7eyjHa4Fhvs=
370
github.com/ipfs/bbloom v0.0.4/go.mod h1:cS9YprKXpoZ9lT0n/Mw/a6/aFV6DTjTLYHeA+gyqMG0=
371
-github.com/ipfs/boxo v0.26.0 h1:RRxEon7rJMy8ScVaTLncSZ5/nA6majYhRSbzc80snO8=
372
-github.com/ipfs/boxo v0.26.0/go.mod h1:iHyc9cjoF7/zoiKVY65d2fBWRhoS2zx4cMk8hKgqrac=
371
+github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac h1:RbM53YTWjs7nZAAiUV7GSIV7S8QOaq0YkGAuuNNGM7A=
372
+github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac/go.mod h1:iHyc9cjoF7/zoiKVY65d2fBWRhoS2zx4cMk8hKgqrac=
373
github.com/ipfs/go-bitfield v1.1.0 h1:fh7FIo8bSwaJEh6DdTWbCeZ1eqOaOkKFI74SCnsWbGA=
374
github.com/ipfs/go-bitfield v1.1.0/go.mod h1:paqf1wjq/D2BBmzfTVFlJQ9IlFOZpg422HL0HqsGWHU=
375
github.com/ipfs/go-bitswap v0.11.0 h1:j1WVvhDX1yhG32NTC9xfxnqycqYIlhzEzLXG/cU1HyQ=
test/dependencies/go.mod
+1
-1
@@ -120,7 +120,7 @@ require (
120
github.com/huin/goupnp v1.3.0 // indirect
121
github.com/inconshreveable/mousetrap v1.1.0 // indirect
122
github.com/ipfs/bbloom v0.0.4 // indirect
123
- github.com/ipfs/boxo v0.26.0 // indirect
123
+ github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac // indirect
124
github.com/ipfs/go-block-format v0.2.0 // indirect
125
github.com/ipfs/go-cid v0.4.1 // indirect
126
github.com/ipfs/go-datastore v0.6.0 // indirect
test/dependencies/go.sum
+2
-2
@@ -324,8 +324,8 @@ github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2
324
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
325
github.com/ipfs/bbloom v0.0.4 h1:Gi+8EGJ2y5qiD5FbsbpX/TMNcJw8gSqr7eyjHa4Fhvs=
326
github.com/ipfs/bbloom v0.0.4/go.mod h1:cS9YprKXpoZ9lT0n/Mw/a6/aFV6DTjTLYHeA+gyqMG0=
327
-github.com/ipfs/boxo v0.26.0 h1:RRxEon7rJMy8ScVaTLncSZ5/nA6majYhRSbzc80snO8=
328
-github.com/ipfs/boxo v0.26.0/go.mod h1:iHyc9cjoF7/zoiKVY65d2fBWRhoS2zx4cMk8hKgqrac=
327
+github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac h1:RbM53YTWjs7nZAAiUV7GSIV7S8QOaq0YkGAuuNNGM7A=
328
+github.com/ipfs/boxo v0.26.1-0.20250103202834-5810fed5a3ac/go.mod h1:iHyc9cjoF7/zoiKVY65d2fBWRhoS2zx4cMk8hKgqrac=
329
github.com/ipfs/go-block-format v0.2.0 h1:ZqrkxBA2ICbDRbK8KJs/u0O3dlp6gmAuuXUJNiW1Ycs=
330
github.com/ipfs/go-block-format v0.2.0/go.mod h1:+jpL11nFx5A/SPpsoBn6Bzkra/zaArfSmsknbPMYgzM=
331
github.com/ipfs/go-cid v0.4.1 h1:A/T3qGvxi4kpKWWcPC/PgbvDA2bjVLO7n4UeVwnbs/s=
test/sharness/t0115-gateway-dir-listing.sh
+3
-3
@@ -40,7 +40,7 @@ test_expect_success "path gw: backlink on root CID should be hidden" '
40
test_expect_success "path gw: redirect dir listing to URL with trailing slash" '
41
curl -sD - http://127.0.0.1:$GWAY_PORT/ipfs/${DIR_CID}/ą/ę > list_response &&
42
test_should_contain "HTTP/1.1 301 Moved Permanently" list_response &&
43
- test_should_contain "Location: /ipfs/${DIR_CID}/%c4%85/%c4%99/" list_response
43
+ test_should_contain "Location: /ipfs/${DIR_CID}/%C4%85/%C4%99/" list_response
44
'
45
46
test_expect_success "path gw: Etag should be present" '
@@ -81,7 +81,7 @@ test_expect_success "subdomain gw: backlink on root CID should be hidden" '
81
test_expect_success "subdomain gw: redirect dir listing to URL with trailing slash" '
82
curl -sD - --resolve $DIR_HOSTNAME:$GWAY_PORT:127.0.0.1 http://$DIR_HOSTNAME:$GWAY_PORT/ą/ę > list_response &&
83
test_should_contain "HTTP/1.1 301 Moved Permanently" list_response &&
84
- test_should_contain "Location: /%c4%85/%c4%99/" list_response
84
+ test_should_contain "Location: /%C4%85/%C4%99/" list_response
85
'
86
87
test_expect_success "subdomain gw: Etag should be present" '
@@ -130,7 +130,7 @@ test_expect_success "dnslink gw: backlink on root CID should be hidden" '
130
test_expect_success "dnslink gw: redirect dir listing to URL with trailing slash" '
131
curl -sD - --resolve $DNSLINK_HOSTNAME:$GWAY_PORT:127.0.0.1 http://$DNSLINK_HOSTNAME:$GWAY_PORT/ą/ę > list_response &&
132
test_should_contain "HTTP/1.1 301 Moved Permanently" list_response &&
133
- test_should_contain "Location: /%c4%85/%c4%99/" list_response
133
+ test_should_contain "Location: /%C4%85/%C4%99/" list_response
134
'
135
136
test_expect_success "dnslink gw: Etag should be present" '