@cryptotaxi247 / kubo / commits / 33e73a6cb

fix(mfs): fix fsync deadlock, set attrs, disable default caching (#11255)

* fix(MFS): fix deadlock, attrs, caching * unmount ipns and mfs in mount tests; allow offline * set attrs Uid, Gid, and Valid for readonly and /ipns * doc: update changelog * fix(fuse): maximize kernel cache for immutable /ipfs paths /ipfs content is addressed by CID and never changes, so kernel attribute caching is safe and avoids unnecessary FUSE round-trips. Also sets uid/gid on Root.Attr for consistency. * docs: move FUSE changelog to v0.41 highlights * fix(fuse): make IPNS fsync a no-op Calling fsync on a file opened through /ipns deadlocks and eventually panics, taking down the entire IPNS mount. The Fsync handler called mfs.File.Flush(), which tries to open a second write descriptor on the same file. Only one write descriptor can exist at a time (desclock is exclusive), and the first one from Open is still held. The new one blocks forever waiting for the lock. After the FUSE timeout, Release tries to close the original descriptor and hits a nil pointer panic in DagModifier.Sync. Make Fsync a no-op, matching the MFS mount. Data gets flushed when the file is closed. Also improve the MFS Fsync comment to explain the same constraint. * fix(fuse): set uid/gid on IPNS symlinks The "local" symlink in /ipns showed uid=0 gid=0 (root) while directories and files showed the daemon's uid/gid. Set uid/gid and disable attr caching to match other mutable IPNS nodes. Also add TODO comments across all three FUSE mounts for using Mode and Mtime from UnixFS records when present, and for wiring IPNS record TTL into attr cache duration. * fix(fuse): return empty listing for empty directories IPNS Directory.ReadDirAll and readonly Node.ReadDirAll returned ENOENT when a directory had no children. An empty directory still exists, it just has nothing in it. Return an empty slice instead. MFS already handles this correctly. The readonly Root.ReadDirAll correctly returns EPERM (you can't list all of /ipfs). The IPNS Root.ReadDirAll always has entries (peer keys), so it was never affected. This matters for /ipfs because empty directories are valid content-addressed objects (e.g. QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn is a well-known CID of an empty UnixFS directory). * fix(fuse): always sync MFS writes to root on close The Sync flag was computed as `req.Flags|fuse.OpenSync > 0` (bitwise OR), which is always true because fuse.OpenSync is non-zero. Replace with an explicit `true` to match the IPNS mount and make the intent clear: FUSE writes must always propagate to the MFS root when the file is closed, regardless of whether the caller set O_SYNC. * docs: update FUSE changelog for new fixes * test(fuse): add empty directory listing tests Verify that listing an empty directory returns an empty result instead of an error, for all three FUSE mounts: - /mfs: empty root + empty subdirectory - /ipns: empty peer directory + empty subdirectory - /ipfs: empty UnixFS directory added to the DAG * test(fuse): add append and byte-at-a-time write tests for MFS IPNS had TestAppendFile and TestMultiWrite but MFS did not. Add matching tests to cover appending to an existing file and writing one byte at a time. * ci(fuse): add dedicated FUSE test job with auto-detection Add a fuse-tests CI job that installs fuse3, sets TEST_FUSE=1, and runs FUSE unit tests. Previously these tests were compiled out by the nofuse build tag (set when TEST_FUSE=0 in the unit-tests job). Introduce fuse/fusetest package with shared test helpers: - SkipUnlessFUSE: respects TEST_FUSE env var (0=skip, 1=run) with auto-detection fallback that checks for fusermount in PATH - MountError: fatals when TEST_FUSE=1 (CI expects FUSE to work), skips when auto-detecting (local dev without FUSE) Replace the old ci.NoFuse() (checked TEST_NO_FUSE, a dead env var nobody set) and per-file maybeSkipFuseTests wrappers. On Linux, bazil.org/fuse hardcodes "fusermount" but modern distros only ship "fusermount3". The CI job creates a symlink; the auto-detect gives a helpful skip message when only fusermount3 is found locally. * fix(fuse): handle EINTR on close in IPNS concurrent write test TestConcurrentWrites was flaky because Go's goroutine preemption signal (SIGURG) can interrupt the FUSE FLUSH inside close(), returning EINTR. The write itself already succeeded and the kernel will still send RELEASE to the daemon, so the data is safe. Replace os.WriteFile with explicit open/write/close so we can ignore EINTR on close while still catching real errors. * fix(fuse): resolve bare file CIDs on /ipfs mount Accessing a file by its CID at the /ipfs FUSE mount root returned ENOENT because ProtoNodeConverter cannot handle UnixFS file ADLs. Decode dag-pb blocks directly from bytes instead. Closes https://github.com/ipfs/kubo/issues/9044 * fix(fuse): fix same-directory rename on /mfs Renaming a file within the same MFS directory left the source behind. The directory's entry cache was re-synced before the old name was removed. Unlink the source before AddChild to match the working IPNS pattern. * test(fuse): add mixed dag-pb/raw directory test Covers the scenario from https://github.com/ipfs/kubo/issues/9044: a directory with both dag-pb and raw-leaf children read through the /ipfs FUSE mount. * test(fuse): remove redundant testing.Short() checks SkipUnlessFUSE(t) already handles skipping via TEST_FUSE. The testing.Short() guard was a second skip gate that served no purpose since FUSE tests only run under make test_fuse. * fix(fuse): get DAG node before unlinking source in rename Move GetNode() before Unlink() in both mfs and ipns Rename so that a GetNode() failure does not leave the source entry already removed. Also add FUSE test instructions to AGENTS.md. * ci: skip fuse3 install when fusermount exists Self-hosted runners persist state, so after the first run fuse3 and the symlink are already in place. Skip apt-get update and install entirely when fusermount is in PATH. --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> Co-authored-by: Marcin Rataj <lidel@lidel.org>

William Morriss committed Apr 2, 2026 at 10:52 UTC 33e73a6cb64d8b618aed4220f15da3e5a0525183
15 files changed +633 -153
.github/workflows/gotest.yml
+29
@@ -149,6 +149,35 @@ jobs:
149 run: cat test/cli/cli-tests.md >> $GITHUB_STEP_SUMMARY
150 if: failure() || success()
151
152 + # FUSE filesystem tests (require /dev/fuse and fusermount)
153 + fuse-tests:
154 + if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'
155 + runs-on: ${{ fromJSON(github.repository == 'ipfs/kubo' && '["self-hosted", "linux", "x64", "2xlarge"]' || '"ubuntu-latest"') }}
156 + timeout-minutes: 5
157 + env:
158 + GOTRACEBACK: single
159 + TEST_FUSE: 1
160 + defaults:
161 + run:
162 + shell: bash
163 + steps:
164 + - name: Check out Kubo
165 + uses: actions/checkout@v6
166 + - name: Set up Go
167 + uses: actions/setup-go@v6
168 + with:
169 + go-version-file: 'go.mod'
170 + - name: Install FUSE
171 + run: |
172 + if ! command -v fusermount &>/dev/null; then
173 + sudo apt-get update
174 + sudo apt-get install -y fuse3
175 + # bazil.org/fuse looks for "fusermount", fuse3 only ships "fusermount3"
176 + sudo ln -sf /usr/bin/fusermount3 /usr/local/bin/fusermount
177 + fi
178 + - name: Run FUSE tests
179 + run: make test_fuse
180 +
181 # Example tests (kubo-as-a-library)
182 example-tests:
183 if: github.repository == 'ipfs/kubo' || github.event_name == 'workflow_dispatch'
AGENTS.md
+13
@@ -93,6 +93,7 @@ The full test suite is composed of several targets:
93 | `make test_short` | fast subset (`test_go_fmt` + `test_unit`) |
94 | `make test_unit` | unit tests with coverage (excludes `test/cli`) |
95 | `make test_cli` | CLI integration tests (requires `make build` first) |
96 +| `make test_fuse` | FUSE filesystem tests (requires `/dev/fuse` and `fusermount` in PATH) |
97 | `make test_sharness` | legacy shell-based integration tests |
98 | `make test_go_fmt` | checks Go source formatting |
99 | `make -O test_go_lint` | runs `golangci-lint` |
@@ -121,6 +122,16 @@ export IPFS_PATH="$(mktemp -d)"
122
123 If you see "version (N) is lower than repos (M)", the `ipfs` binary in `PATH` is outdated. Rebuild with `make build` and verify `PATH`.
124
125 +### Running FUSE Tests
126 +
127 +FUSE tests require `/dev/fuse` and `fusermount` in `PATH`. On systems with only fuse3, create a symlink:
128 +
129 +```bash
130 +ln -s /usr/bin/fusermount3 /tmp/fusermount && PATH="/tmp:$PATH" make test_fuse
131 +```
132 +
133 +Set `TEST_FUSE=1` to make mount failures fatal (CI does this). Without it, tests auto-detect and skip when FUSE is unavailable.
134 +
135 ### Running Sharness Tests
136
137 Sharness tests are legacy shell-based tests. Run individual tests with a timeout:
@@ -144,8 +155,10 @@ pkill -f "ipfs daemon"
155 - all new integration tests go in `test/cli/`, not `test/sharness/`
156 - if a `test/sharness` test needs significant changes, remove it and add a replacement in `test/cli/`
157 - use [testify](https://github.com/stretchr/testify) for assertions (already a dependency)
158 +- use `t.Context()` instead of `context.Background()` in tests
159 - for Go 1.25+, use `testing/synctest` when testing concurrent code (goroutines, channels, timers)
160 - reuse existing `.car` fixtures in `test/cli/fixtures/` when possible; only add new fixtures when the test requires data not covered by existing ones
161 +- when writing tests that cover CIDv0 vs CIDv1, always set the CID version explicitly (never rely on defaults); if chunk size matters for the test, also set the chunker explicitly
162 - always re-run modified tests locally before submitting to confirm they pass
163 - avoid emojis in test names and test log output
164
Rules.mk
+1
@@ -138,6 +138,7 @@ help:
138 @echo ' test_short - Run fast tests (test_go_fmt, test_unit)'
139 @echo ' test_unit - Run unit tests with coverage (excludes test/cli)'
140 @echo ' test_cli - Run CLI integration tests (requires built binary)'
141 + @echo ' test_fuse - Run FUSE tests (requires /dev/fuse and fusermount)'
142 @echo ' test_go_fmt - Check Go source formatting'
143 @echo ' test_go_build - Build kubo for all platforms from .github/build-platforms.yml'
144 @echo ' test_go_lint - Run golangci-lint'
docs/changelogs/v0.41.md
+13
@@ -15,6 +15,7 @@ This release was brought to you by the [Shipyard](https://ipshipyard.com/) team.
15 - [🖥️ WebUI Improvements](#-webui-improvements)
16 - [🔧 Correct provider addresses for custom HTTP routing](#-correct-provider-addresses-for-custom-http-routing)
17 - [`ipfs object patch` validates UnixFS node types](#ipfs-object-patch-validates-unixfs-node-types)
18 + - [📂 FUSE Mount Fixes](#-fuse-mount-fixes)
19 - [📦️ Dependency updates](#-dependency-updates)
20 - [📝 Changelog](#-changelog)
21 - [👨‍👩‍👧‍👦 Contributors](#-contributors)
@@ -98,6 +99,18 @@ directory types correctly, including large sharded directories.
99
100 A `--allow-non-unixfs` flag is available on both `ipfs object patch` commands to bypass validation.
101
102 +#### 📂 FUSE Mount Fixes
103 +
104 +FUSE mounts (`/ipfs`, `/ipns`, `/mfs`) now work with editors like VIM that rely on `fsync` and expect standard file ownership. FUSE support is still experimental. If you run into problems, please report them at [kubo/issues](https://github.com/ipfs/kubo/issues).
105 +
106 +- **No more deadlocks on save.** Editors that call `fsync` after writing would hang indefinitely on `/mfs` and `/ipns` mounts. `Fsync` is now a no-op on both; data is flushed when the file descriptor is closed.
107 +- **Files are no longer owned by root.** Mounts now report the uid/gid of the daemon process, so access works without `allow_other`.
108 +- **Offline IPNS writes succeed.** Writing through `/ipns` FUSE mounts no longer fails when the node has no network. IPNS records are stored locally and published when connectivity returns.
109 +- **Smarter kernel caching.** Attribute caching is disabled for mutable mounts (`/ipns`, `/mfs`) to prevent stale reads after writes. Immutable `/ipfs` paths use long-lived caching since content addressed by CID never changes.
110 +- **Empty directories list correctly.** Listing an empty directory on `/ipfs` or `/ipns` no longer returns an error.
111 +- **Bare file CIDs work on `/ipfs`.** Accessing a file by its CID directly under the `/ipfs` mount (e.g. `/ipfs/<CID>`) no longer returns "not found". This was a [long-standing regression](https://github.com/ipfs/kubo/issues/9044) that only affected files; directories were not affected.
112 +- **Rename works on `/mfs`.** Renaming a file within the same directory no longer leaves the source behind.
113 +
114 #### 📦️ Dependency updates
115
116 - update `go-libp2p` to [v0.48.0](https://github.com/libp2p/go-libp2p/releases/tag/v0.48.0)
fuse/fusetest/detect.go new
+57
@@ -0,0 +1,57 @@
1 +//go:build !nofuse
2 +
3 +package fusetest
4 +
5 +import (
6 + "os"
7 + "os/exec"
8 + "runtime"
9 + "testing"
10 +)
11 +
12 +// fuseFlagFromEnv returns the value of TEST_FUSE if set, or empty string.
13 +// Also checks the legacy TEST_NO_FUSE for backwards compatibility.
14 +func fuseFlagFromEnv() string {
15 + if v := os.Getenv("TEST_FUSE"); v != "" {
16 + return v
17 + }
18 + // Legacy: TEST_NO_FUSE=1 is equivalent to TEST_FUSE=0
19 + if os.Getenv("TEST_NO_FUSE") == "1" {
20 + return "0"
21 + }
22 + return ""
23 +}
24 +
25 +// fuseAvailable checks whether FUSE is likely to work on this system
26 +// and skips with a helpful message if not.
27 +//
28 +// On Linux, bazil.org/fuse requires "fusermount" (not "fusermount3") in
29 +// PATH. Systems with only fuse3 installed need a symlink:
30 +//
31 +// sudo ln -s /usr/bin/fusermount3 /usr/local/bin/fusermount
32 +func fuseAvailable(t *testing.T) bool {
33 + t.Helper()
34 +
35 + switch runtime.GOOS {
36 + case "linux", "darwin", "freebsd", "netbsd", "openbsd":
37 + default:
38 + t.Skip("FUSE not supported on", runtime.GOOS)
39 + return false
40 + }
41 +
42 + if runtime.GOOS == "linux" {
43 + if _, err := exec.LookPath("fusermount"); err == nil {
44 + return true
45 + }
46 + if _, err := exec.LookPath("fusermount3"); err == nil {
47 + t.Skip("fusermount3 found but bazil.org/fuse needs \"fusermount\"; create a symlink: sudo ln -s /usr/bin/fusermount3 /usr/local/bin/fusermount")
48 + }
49 + t.Skip("fusermount not found in PATH")
50 + return false
51 + }
52 +
53 + if _, err := exec.LookPath("umount"); err != nil {
54 + t.Skip("umount not found in PATH")
55 + }
56 + return true
57 +}
fuse/fusetest/fusetest.go new
+42
@@ -0,0 +1,42 @@
1 +//go:build !nofuse
2 +
3 +// Package fusetest provides test helpers shared across FUSE test packages.
4 +package fusetest
5 +
6 +import (
7 + "testing"
8 +)
9 +
10 +// SkipUnlessFUSE skips the test when FUSE is not available.
11 +//
12 +// Decision order:
13 +// 1. TEST_FUSE=0 (or legacy TEST_NO_FUSE=1) → skip
14 +// 2. TEST_FUSE=1 → run (CI should set this after installing fuse3)
15 +// 3. Neither set → auto-detect based on platform and fusermount in PATH;
16 +// skip with a helpful message if not found
17 +func SkipUnlessFUSE(t *testing.T) {
18 + t.Helper()
19 +
20 + if v := fuseFlagFromEnv(); v != "" {
21 + if v == "0" {
22 + t.Skip("FUSE tests disabled (TEST_FUSE=0)")
23 + }
24 + return // TEST_FUSE=1, run unconditionally
25 + }
26 +
27 + fuseAvailable(t) // skips with a helpful message if not available
28 +}
29 +
30 +// MountError handles a FUSE mount error. When TEST_FUSE=1 (CI), a mount
31 +// failure is fatal because the environment is expected to have working FUSE.
32 +// When auto-detecting (no TEST_FUSE set), mount failures cause a skip.
33 +func MountError(t *testing.T, err error) {
34 + t.Helper()
35 + if err == nil {
36 + return
37 + }
38 + if fuseFlagFromEnv() == "1" {
39 + t.Fatal("FUSE mount failed (TEST_FUSE=1, expected FUSE to work):", err)
40 + }
41 + t.Skip("FUSE mount failed:", err)
42 +}
fuse/ipns/ipns_test.go
+85 -42
@@ -5,30 +5,24 @@ package ipns
5 import (
6 "bytes"
7 "context"
8 + "errors"
9 "fmt"
10 "io"
11 mrand "math/rand"
12 "os"
13 "sync"
14 + "syscall"
15 "testing"
16
15 - "bazil.org/fuse"
16 -
17 core "github.com/ipfs/kubo/core"
18 coreapi "github.com/ipfs/kubo/core/coreapi"
19
20 fstest "bazil.org/fuse/fs/fstestutil"
21 racedet "github.com/ipfs/go-detect-race"
22 "github.com/ipfs/go-test/random"
23 - ci "github.com/libp2p/go-libp2p-testing/ci"
23 + "github.com/ipfs/kubo/fuse/fusetest"
24 )
25
26 -func maybeSkipFuseTests(t *testing.T) {
27 - if ci.NoFuse() {
28 - t.Skip("Skipping FUSE tests")
29 - }
30 -}
31 -
26 func randBytes(size int) []byte {
27 b := make([]byte, size)
28 _, err := io.ReadFull(random.NewRand(), b)
@@ -55,8 +49,27 @@ func writeFileOrFail(t *testing.T, size int, path string) []byte {
49
50 func writeFile(size int, path string) ([]byte, error) {
51 data := randBytes(size)
58 - err := os.WriteFile(path, data, 0o666)
59 - return data, err
52 + // Same flags as os.WriteFile: write-only, create if missing, truncate if exists.
53 + // We open manually instead of using os.WriteFile so we can handle EINTR on close.
54 + f, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o666)
55 + if err != nil {
56 + return nil, err
57 + }
58 + _, err = f.Write(data)
59 + if err != nil {
60 + f.Close()
61 + return nil, err
62 + }
63 + // Go's goroutine preemption (SIGURG) can interrupt the FUSE FLUSH
64 + // inside close(), returning EINTR. This is not a data loss: the write
65 + // already succeeded and the kernel will still send RELEASE to the FUSE
66 + // daemon. Go intentionally does not retry close() on EINTR because the
67 + // fd is already closed on Linux and its state is undefined on other
68 + // platforms, making retry unsafe.
69 + if err := f.Close(); err != nil && !errors.Is(err, syscall.EINTR) {
70 + return nil, err
71 + }
72 + return data, nil
73 }
74
75 func verifyFile(t *testing.T, path string, wantData []byte) {
@@ -100,7 +113,7 @@ func (m *mountWrap) Close() error {
113
114 func setupIpnsTest(t *testing.T, node *core.IpfsNode) (*core.IpfsNode, *mountWrap) {
115 t.Helper()
103 - maybeSkipFuseTests(t)
116 + fusetest.SkipUnlessFUSE(t)
117
118 var err error
119 if node == nil {
@@ -125,12 +138,7 @@ func setupIpnsTest(t *testing.T, node *core.IpfsNode) (*core.IpfsNode, *mountWra
138 t.Fatal(err)
139 }
140 mnt, err := fstest.MountedT(t, fs, nil)
128 - if err == fuse.ErrOSXFUSENotFound {
129 - t.Skip(err)
130 - }
131 - if err != nil {
132 - t.Fatalf("error mounting at temporary directory: %v", err)
133 - }
141 + fusetest.MountError(t, err)
142
143 return node, &mountWrap{
144 Mount: mnt,
@@ -155,11 +163,37 @@ func TestIpnsLocalLink(t *testing.T) {
163 }
164 }
165
166 +// Test that empty directories can be listed without errors.
167 +func TestEmptyDirListing(t *testing.T) {
168 + nd, mnt := setupIpnsTest(t, nil)
169 + defer mnt.Close()
170 +
171 + // The peer's IPNS directory starts empty.
172 + peerDir := mnt.Dir + "/" + nd.Identity.String()
173 + entries, err := os.ReadDir(peerDir)
174 + if err != nil {
175 + t.Fatal(err)
176 + }
177 + if len(entries) != 0 {
178 + t.Fatalf("expected empty peer dir, got %d entries", len(entries))
179 + }
180 +
181 + // Create a subdirectory and list it while still empty.
182 + subdir := peerDir + "/emptydir"
183 + if err := os.Mkdir(subdir, os.ModeDir); err != nil {
184 + t.Fatal(err)
185 + }
186 + entries, err = os.ReadDir(subdir)
187 + if err != nil {
188 + t.Fatal(err)
189 + }
190 + if len(entries) != 0 {
191 + t.Fatalf("expected empty subdirectory, got %d entries", len(entries))
192 + }
193 +}
194 +
195 // Test writing a file and reading it back.
196 func TestIpnsBasicIO(t *testing.T) {
160 - if testing.Short() {
161 - t.SkipNow()
162 - }
197 nd, mnt := setupIpnsTest(t, nil)
198 defer closeMount(mnt)
199
@@ -186,11 +220,38 @@ func TestIpnsBasicIO(t *testing.T) {
220 }
221 }
222
223 +// Test renaming a file within the same IPNS directory.
224 +func TestRenameFile(t *testing.T) {
225 + nd, mnt := setupIpnsTest(t, nil)
226 + defer closeMount(mnt)
227 +
228 + peerDir := mnt.Dir + "/" + nd.Identity.String()
229 + src := peerDir + "/before.txt"
230 + dst := peerDir + "/after.txt"
231 +
232 + data := writeFileOrFail(t, 500, src)
233 +
234 + if err := os.Rename(src, dst); err != nil {
235 + t.Fatal(err)
236 + }
237 +
238 + // Source must be gone.
239 + if _, err := os.Stat(src); !os.IsNotExist(err) {
240 + t.Fatalf("source still exists after rename: %v", err)
241 + }
242 +
243 + // Destination must have the original content.
244 + got, err := os.ReadFile(dst)
245 + if err != nil {
246 + t.Fatal(err)
247 + }
248 + if !bytes.Equal(got, data) {
249 + t.Fatalf("content mismatch: got %d bytes, want %d", len(got), len(data))
250 + }
251 +}
252 +
253 // Test to make sure file changes persist over mounts of ipns.
254 func TestFilePersistence(t *testing.T) {
191 - if testing.Short() {
192 - t.SkipNow()
193 - }
255 node, mnt := setupIpnsTest(t, nil)
256
257 fname := "/local/atestfile"
@@ -251,9 +312,6 @@ func TestMultipleDirs(t *testing.T) {
312
313 // Test to make sure the filesystem reports file sizes correctly.
314 func TestFileSizeReporting(t *testing.T) {
254 - if testing.Short() {
255 - t.SkipNow()
256 - }
315 _, mnt := setupIpnsTest(t, nil)
316 defer mnt.Close()
317
@@ -272,9 +330,6 @@ func TestFileSizeReporting(t *testing.T) {
330
331 // Test to make sure you can't create multiple entries with the same name.
332 func TestDoubleEntryFailure(t *testing.T) {
275 - if testing.Short() {
276 - t.SkipNow()
277 - }
333 _, mnt := setupIpnsTest(t, nil)
334 defer mnt.Close()
335
@@ -291,9 +346,6 @@ func TestDoubleEntryFailure(t *testing.T) {
346 }
347
348 func TestAppendFile(t *testing.T) {
294 - if testing.Short() {
295 - t.SkipNow()
296 - }
349 _, mnt := setupIpnsTest(t, nil)
350 defer mnt.Close()
351
@@ -332,9 +384,6 @@ func TestAppendFile(t *testing.T) {
384 }
385
386 func TestConcurrentWrites(t *testing.T) {
335 - if testing.Short() {
336 - t.SkipNow()
337 - }
387 _, mnt := setupIpnsTest(t, nil)
388 defer mnt.Close()
389
@@ -381,9 +430,6 @@ func TestConcurrentWrites(t *testing.T) {
430 func TestFSThrash(t *testing.T) {
431 files := make(map[string][]byte)
432
384 - if testing.Short() {
385 - t.SkipNow()
386 - }
433 _, mnt := setupIpnsTest(t, nil)
434 defer mnt.Close()
435
@@ -464,9 +510,6 @@ func TestFSThrash(t *testing.T) {
510
511 // Test writing a medium sized file one byte at a time.
512 func TestMultiWrite(t *testing.T) {
467 - if testing.Short() {
468 - t.SkipNow()
469 - }
513
514 _, mnt := setupIpnsTest(t, nil)
515 defer mnt.Close()
fuse/ipns/ipns_unix.go
+27 -23
@@ -86,7 +86,7 @@ type Root struct {
86
87 func ipnsPubFunc(ipfs iface.CoreAPI, key iface.Key) mfs.PubFunc {
88 return func(ctx context.Context, c cid.Cid) error {
89 - _, err := ipfs.Name().Publish(ctx, path.FromCid(c), options.Name.Key(key.Name()))
89 + _, err := ipfs.Name().Publish(ctx, path.FromCid(c), options.Name.Key(key.Name()), options.Name.AllowOffline(true))
90 return err
91 }
92 }
@@ -153,7 +153,11 @@ func CreateRoot(ctx context.Context, ipfs iface.CoreAPI, keys map[string]iface.K
153 // Attr returns file attributes.
154 func (r *Root) Attr(ctx context.Context, a *fuse.Attr) error {
155 log.Debug("Root Attr")
156 + // TODO: wire TTL from IPNS record (capped at Ipns.MaxCacheTTL) here instead of 0
157 + a.Valid = 0
158 a.Mode = os.ModeDir | 0o111 // -rw+x
159 + a.Uid = uint32(os.Getuid())
160 + a.Gid = uint32(os.Getgid())
161 return nil
162 }
163
@@ -252,6 +256,9 @@ type File struct {
256 // Attr returns the attributes of a given node.
257 func (d *Directory) Attr(ctx context.Context, a *fuse.Attr) error {
258 log.Debug("Directory Attr")
259 + // TODO: wire TTL from IPNS record (capped at Ipns.MaxCacheTTL) here instead of 0
260 + a.Valid = 0
261 + // TODO: use Mode from UnixFS record if present
262 a.Mode = os.ModeDir | 0o555
263 a.Uid = uint32(os.Getuid())
264 a.Gid = uint32(os.Getgid())
@@ -261,11 +268,14 @@ func (d *Directory) Attr(ctx context.Context, a *fuse.Attr) error {
268 // Attr returns the attributes of a given node.
269 func (fi *FileNode) Attr(ctx context.Context, a *fuse.Attr) error {
270 log.Debug("File Attr")
271 + // TODO: wire TTL from IPNS record (capped at Ipns.MaxCacheTTL) here instead of 0
272 + a.Valid = 0
273 size, err := fi.fi.Size()
274 if err != nil {
275 // In this case, the dag node in question may not be unixfs
276 return fmt.Errorf("fuse/ipns: failed to get file.Size(): %s", err)
277 }
278 + // TODO: use Mode and Mtime from UnixFS record if present
279 a.Mode = os.FileMode(0o666)
280 a.Size = uint64(size)
281 a.Uid = uint32(os.Getuid())
@@ -313,10 +323,7 @@ func (d *Directory) ReadDirAll(ctx context.Context) ([]fuse.Dirent, error) {
323 entries[i] = dirent
324 }
325
316 - if len(entries) > 0 {
317 - return entries, nil
318 - }
319 - return nil, syscall.Errno(syscall.ENOENT)
326 + return entries, nil
327 }
328
329 func (fi *File) Read(ctx context.Context, req *fuse.ReadRequest, resp *fuse.ReadResponse) error {
@@ -381,20 +388,14 @@ func (fi *File) Setattr(ctx context.Context, req *fuse.SetattrRequest, resp *fus
388 return nil
389 }
390
384 -// Fsync flushes the content in the file to disk.
391 +// Fsync is a no-op. We can't flush here because mfs.File.Flush opens a new
392 +// write descriptor, which needs an exclusive lock (desclock) that the caller
393 +// already holds from Open. Attempting it deadlocks until the FUSE timeout,
394 +// then panics on Release. Data is flushed when the file is closed instead.
395 +// TODO: a proper fix needs changes in boxo/mfs to allow flushing from an
396 +// existing descriptor. Ideas welcome, but for now this is the best we can do.
397 func (fi *FileNode) Fsync(ctx context.Context, req *fuse.FsyncRequest) error {
386 - // This needs to perform a *full* flush because, in MFS, a write isn't
387 - // persisted until the root is updated.
388 - errs := make(chan error, 1)
389 - go func() {
390 - errs <- fi.fi.Flush()
391 - }()
392 - select {
393 - case err := <-errs:
394 - return err
395 - case <-ctx.Done():
396 - return ctx.Err()
397 - }
398 + return nil
399 }
400
401 func (fi *File) Forget() {
@@ -502,6 +503,14 @@ func (d *Directory) Rename(ctx context.Context, req *fuse.RenameRequest, newDir
503 return err
504 }
505
506 + nd, err := cur.GetNode()
507 + if err != nil {
508 + return err
509 + }
510 +
511 + // Unlink the source before adding to the destination. For
512 + // same-directory renames, this clears the old name from the
513 + // directory's entry cache before AddChild repopulates it.
514 err = d.dir.Unlink(req.OldName)
515 if err != nil {
516 return err
@@ -509,11 +518,6 @@ func (d *Directory) Rename(ctx context.Context, req *fuse.RenameRequest, newDir
518
519 switch newDir := newDir.(type) {
520 case *Directory:
512 - nd, err := cur.GetNode()
513 - if err != nil {
514 - return err
515 - }
516 -
521 err = newDir.dir.AddChild(req.NewName, nd)
522 if err != nil {
523 return err
fuse/ipns/link_unix.go
+4
@@ -16,7 +16,11 @@ type Link struct {
16
17 func (l *Link) Attr(ctx context.Context, a *fuse.Attr) error {
18 log.Debug("Link attr.")
19 + // TODO: wire TTL from IPNS record (capped at Ipns.MaxCacheTTL) here instead of 0
20 + a.Valid = 0
21 a.Mode = os.ModeSymlink | 0o555
22 + a.Uid = uint32(os.Getuid())
23 + a.Gid = uint32(os.Getgid())
24 return nil
25 }
26
fuse/mfs/mfs_test.go
+150 -10
@@ -19,14 +19,12 @@ import (
19 "bazil.org/fuse/fs/fstestutil"
20 "github.com/ipfs/kubo/core"
21 "github.com/ipfs/kubo/core/node"
22 - "github.com/libp2p/go-libp2p-testing/ci"
22 + "github.com/ipfs/kubo/fuse/fusetest"
23 )
24
25 // Create an Ipfs.Node, a filesystem and a mount point.
26 func setUp(t *testing.T, ipfs *core.IpfsNode) (fs.FS, *fstestutil.Mount) {
27 - if ci.NoFuse() {
28 - t.Skip("Skipping FUSE tests")
29 - }
27 + fusetest.SkipUnlessFUSE(t)
28
29 if ipfs == nil {
30 var err error
@@ -38,12 +36,7 @@ func setUp(t *testing.T, ipfs *core.IpfsNode) (fs.FS, *fstestutil.Mount) {
36
37 fs := NewFileSystem(ipfs)
38 mnt, err := fstestutil.MountedT(t, fs, nil)
41 - if err == fuse.ErrOSXFUSENotFound {
42 - t.Skip(err)
43 - }
44 - if err != nil {
45 - t.Fatal(err)
46 - }
39 + fusetest.MountError(t, err)
40
41 return fs, mnt
42 }
@@ -90,6 +83,34 @@ func TestReadWrite(t *testing.T) {
83 })
84 }
85
86 +// Test that empty directories can be listed without errors.
87 +func TestEmptyDirListing(t *testing.T) {
88 + _, mnt := setUp(t, nil)
89 + defer mnt.Close()
90 +
91 + // The MFS root starts empty.
92 + entries, err := os.ReadDir(mnt.Dir)
93 + if err != nil {
94 + t.Fatal(err)
95 + }
96 + if len(entries) != 0 {
97 + t.Fatalf("expected empty root, got %d entries", len(entries))
98 + }
99 +
100 + // Create a directory and list it while still empty.
101 + dir := mnt.Dir + "/emptydir"
102 + if err := os.Mkdir(dir, os.ModeDir); err != nil {
103 + t.Fatal(err)
104 + }
105 + entries, err = os.ReadDir(dir)
106 + if err != nil {
107 + t.Fatal(err)
108 + }
109 + if len(entries) != 0 {
110 + t.Fatalf("expected empty directory, got %d entries", len(entries))
111 + }
112 +}
113 +
114 // Test creating a directory.
115 func TestMkdir(t *testing.T) {
116 _, mnt := setUp(t, nil)
@@ -294,6 +315,125 @@ func TestConcurrentRW(t *testing.T) {
315 })
316 }
317
318 +// Test appending data to an existing file.
319 +func TestAppendFile(t *testing.T) {
320 + _, mnt := setUp(t, nil)
321 + defer mnt.Close()
322 +
323 + path := mnt.Dir + "/appendfile"
324 +
325 + initial := make([]byte, 1300)
326 + if _, err := rand.Read(initial); err != nil {
327 + t.Fatal(err)
328 + }
329 + if err := os.WriteFile(path, initial, 0o644); err != nil {
330 + t.Fatal(err)
331 + }
332 +
333 + fi, err := os.OpenFile(path, os.O_RDWR|os.O_APPEND, 0o644)
334 + if err != nil {
335 + t.Fatal(err)
336 + }
337 +
338 + extra := make([]byte, 500)
339 + if _, err := rand.Read(extra); err != nil {
340 + t.Fatal(err)
341 + }
342 +
343 + n, err := fi.Write(extra)
344 + if err != nil {
345 + t.Fatal(err)
346 + }
347 + if n != len(extra) {
348 + t.Fatalf("short write: %d != %d", n, len(extra))
349 + }
350 + if err := fi.Close(); err != nil {
351 + t.Fatal(err)
352 + }
353 +
354 + got, err := os.ReadFile(path)
355 + if err != nil {
356 + t.Fatal(err)
357 + }
358 + want := append(initial, extra...)
359 + if !bytes.Equal(got, want) {
360 + t.Fatalf("content mismatch: got %d bytes, want %d", len(got), len(want))
361 + }
362 +}
363 +
364 +// Test writing a file one byte at a time.
365 +func TestMultiWrite(t *testing.T) {
366 + _, mnt := setUp(t, nil)
367 + defer mnt.Close()
368 +
369 + path := mnt.Dir + "/multiwrite"
370 + fi, err := os.Create(path)
371 + if err != nil {
372 + t.Fatal(err)
373 + }
374 +
375 + data := make([]byte, 1001)
376 + if _, err := rand.Read(data); err != nil {
377 + t.Fatal(err)
378 + }
379 +
380 + for i := range data {
381 + n, err := fi.Write(data[i : i+1])
382 + if err != nil {
383 + t.Fatal(err)
384 + }
385 + if n != 1 {
386 + t.Fatal("short write")
387 + }
388 + }
389 + if err := fi.Close(); err != nil {
390 + t.Fatal(err)
391 + }
392 +
393 + got, err := os.ReadFile(path)
394 + if err != nil {
395 + t.Fatal(err)
396 + }
397 + if !bytes.Equal(got, data) {
398 + t.Fatal("content mismatch")
399 + }
400 +}
401 +
402 +// Test renaming a file within the same directory.
403 +func TestRenameFile(t *testing.T) {
404 + _, mnt := setUp(t, nil)
405 + defer mnt.Close()
406 +
407 + src := mnt.Dir + "/before.txt"
408 + dst := mnt.Dir + "/after.txt"
409 +
410 + data := make([]byte, 500)
411 + if _, err := rand.Read(data); err != nil {
412 + t.Fatal(err)
413 + }
414 + if err := os.WriteFile(src, data, 0o644); err != nil {
415 + t.Fatal(err)
416 + }
417 +
418 + if err := os.Rename(src, dst); err != nil {
419 + t.Fatal(err)
420 + }
421 +
422 + // Source must be gone.
423 + if _, err := os.Stat(src); !os.IsNotExist(err) {
424 + t.Fatalf("source still exists after rename: %v", err)
425 + }
426 +
427 + // Destination must have the original content.
428 + got, err := os.ReadFile(dst)
429 + if err != nil {
430 + t.Fatal(err)
431 + }
432 + if !bytes.Equal(got, data) {
433 + t.Fatalf("content mismatch: got %d bytes, want %d", len(got), len(data))
434 + }
435 +}
436 +
437 // Test ipfs_cid extended attribute
438 func TestMFSRootXattr(t *testing.T) {
439 ipfs, err := core.NewNode(context.Background(), &node.BuildCfg{})
fuse/mfs/mfs_unix.go
+31 -15
@@ -44,9 +44,13 @@ type Dir struct {
44
45 // Directory attributes (stat).
46 func (dir *Dir) Attr(ctx context.Context, attr *fuse.Attr) error {
47 + attr.Valid = 0
48 + // TODO: use Mode from UnixFS record if present
49 attr.Mode = mfsDirMode
50 attr.Size = dirSize * blockSize
51 attr.Blocks = dirSize
52 + attr.Uid = uint32(os.Getuid())
53 + attr.Gid = uint32(os.Getgid())
54 return nil
55 }
56
@@ -61,6 +65,8 @@ func (dir *Dir) Lookup(ctx context.Context, req *fuse.LookupRequest, resp *fuse.
65 return nil, err
66 }
67
68 + resp.EntryValid = 0
69 +
70 switch mfsNode.Type() {
71 case mfs.TDir:
72 result := Dir{
@@ -136,29 +142,29 @@ func (dir *Dir) Remove(ctx context.Context, req *fuse.RemoveRequest) error {
142
143 // Move (mv) an MFS file.
144 func (dir *Dir) Rename(ctx context.Context, req *fuse.RenameRequest, newDir fs.Node) error {
139 - file, err := dir.mfsDir.Child(req.OldName)
145 + child, err := dir.mfsDir.Child(req.OldName)
146 if err != nil {
147 return err
148 }
143 - node, err := file.GetNode()
149 +
150 + nd, err := child.GetNode()
151 if err != nil {
152 return err
153 }
147 - targetDir := newDir.(*Dir)
154
149 - // Remove file if exists
150 - err = targetDir.mfsDir.Unlink(req.NewName)
151 - if err != nil && err != os.ErrNotExist {
155 + // Unlink the source first. For same-directory renames, this clears
156 + // the old name from the directory's entry cache before AddChild
157 + // repopulates it with the new name. Without this ordering, Flush
158 + // would sync the stale cache entry back into the DAG.
159 + if err := dir.mfsDir.Unlink(req.OldName); err != nil {
160 return err
161 }
162
155 - err = targetDir.mfsDir.AddChild(req.NewName, node)
156 - if err != nil {
163 + targetDir := newDir.(*Dir)
164 + if err := targetDir.mfsDir.Unlink(req.NewName); err != nil && err != os.ErrNotExist {
165 return err
166 }
159 -
160 - err = dir.mfsDir.Unlink(req.OldName)
161 - if err != nil {
167 + if err := targetDir.mfsDir.AddChild(req.NewName, nd); err != nil {
168 return err
169 }
170
@@ -199,7 +205,7 @@ func (dir *Dir) Create(ctx context.Context, req *fuse.CreateRequest, resp *fuse.
205 flags := mfs.Flags{
206 Read: accessMode == fuse.OpenReadOnly || accessMode == fuse.OpenReadWrite,
207 Write: accessMode == fuse.OpenWriteOnly || accessMode == fuse.OpenReadWrite,
202 - Sync: req.Flags|fuse.OpenSync > 0,
208 + Sync: true, // FUSE writes must propagate to the MFS root on close
209 }
210
211 fd, err := mfsFile.Open(flags)
@@ -241,6 +247,8 @@ type File struct {
247
248 // File attributes.
249 func (file *File) Attr(ctx context.Context, attr *fuse.Attr) error {
250 + attr.Valid = 0
251 +
252 size, _ := file.mfsFile.Size()
253
254 attr.Size = uint64(size)
@@ -253,7 +261,10 @@ func (file *File) Attr(ctx context.Context, attr *fuse.Attr) error {
261 mtime, _ := file.mfsFile.ModTime()
262 attr.Mtime = mtime
263
264 + // TODO: use Mode from UnixFS record if present
265 attr.Mode = mfsFileMode
266 + attr.Uid = uint32(os.Getuid())
267 + attr.Gid = uint32(os.Getgid())
268 return nil
269 }
270
@@ -263,7 +274,7 @@ func (file *File) Open(ctx context.Context, req *fuse.OpenRequest, resp *fuse.Op
274 flags := mfs.Flags{
275 Read: accessMode == fuse.OpenReadOnly || accessMode == fuse.OpenReadWrite,
276 Write: accessMode == fuse.OpenWriteOnly || accessMode == fuse.OpenReadWrite,
266 - Sync: req.Flags|fuse.OpenSync > 0,
277 + Sync: true, // FUSE writes must propagate to the MFS root on close
278 }
279 fd, err := file.mfsFile.Open(flags)
280 if err != nil {
@@ -281,9 +292,14 @@ func (file *File) Open(ctx context.Context, req *fuse.OpenRequest, resp *fuse.Op
292 }, nil
293 }
294
284 -// Sync the file's contents to MFS.
295 +// Fsync is a no-op. We can't flush here because mfs.File.Flush opens a new
296 +// write descriptor, which needs an exclusive lock (desclock) that the caller
297 +// already holds from Open. Attempting it deadlocks until the FUSE timeout.
298 +// Data is flushed when the file is closed instead.
299 +// TODO: a proper fix needs changes in boxo/mfs to allow flushing from an
300 +// existing descriptor. Ideas welcome, but for now this is the best we can do.
301 func (file *File) Fsync(ctx context.Context, req *fuse.FsyncRequest) error {
286 - return file.mfsFile.Sync()
302 + return nil
303 }
304
305 // List file xattr.
fuse/node/mount_test.go
+15 -23
@@ -5,25 +5,15 @@ package node
5 import (
6 "context"
7 "os"
8 - "strings"
8 "testing"
9 "time"
10
12 - "bazil.org/fuse"
13 -
11 core "github.com/ipfs/kubo/core"
12 + "github.com/ipfs/kubo/fuse/fusetest"
13 ipns "github.com/ipfs/kubo/fuse/ipns"
14 mount "github.com/ipfs/kubo/fuse/mount"
17 -
18 - ci "github.com/libp2p/go-libp2p-testing/ci"
15 )
16
21 -func maybeSkipFuseTests(t *testing.T) {
22 - if ci.NoFuse() {
23 - t.Skip("Skipping FUSE tests")
24 - }
25 -}
26 -
17 func mkdir(t *testing.T, path string) {
18 err := os.Mkdir(path, os.ModeDir|os.ModePerm)
19 if err != nil {
@@ -33,12 +23,9 @@ func mkdir(t *testing.T, path string) {
23
24 // Test externally unmounting, then trying to unmount in code.
25 func TestExternalUnmount(t *testing.T) {
36 - if testing.Short() {
37 - t.SkipNow()
38 - }
26
27 // TODO: needed?
41 - maybeSkipFuseTests(t)
28 + fusetest.SkipUnlessFUSE(t)
29
30 node, err := core.NewNode(context.Background(), &core.BuildCfg{})
31 if err != nil {
@@ -61,15 +48,20 @@ func TestExternalUnmount(t *testing.T) {
48 mkdir(t, mfsDir)
49
50 err = Mount(node, ipfsDir, ipnsDir, mfsDir)
64 - if err != nil {
65 - if strings.Contains(err.Error(), "unable to check fuse version") || err == fuse.ErrOSXFUSENotFound {
66 - t.Skip(err)
67 - }
68 - }
51 + fusetest.MountError(t, err)
52
70 - if err != nil {
71 - t.Fatalf("error mounting: %v", err)
72 - }
53 + t.Cleanup(func() {
54 + if node.Mounts.Mfs != nil && node.Mounts.Mfs.IsActive() {
55 + if err := node.Mounts.Mfs.Unmount(); err != nil {
56 + t.Fatal(err)
57 + }
58 + }
59 + if node.Mounts.Ipns != nil && node.Mounts.Ipns.IsActive() {
60 + if err := node.Mounts.Ipns.Unmount(); err != nil {
61 + t.Fatal(err)
62 + }
63 + }
64 + })
65
66 // Run shell command to externally unmount the directory
67 cmd, err := mount.UnmountCmd(ipfsDir)
fuse/readonly/ipfs_test.go
+135 -26
@@ -15,8 +15,6 @@ import (
15 "sync"
16 "testing"
17
18 - "bazil.org/fuse"
19 -
18 core "github.com/ipfs/kubo/core"
19 coreapi "github.com/ipfs/kubo/core/coreapi"
20 coremock "github.com/ipfs/kubo/core/mock"
@@ -30,15 +28,10 @@ import (
28 "github.com/ipfs/boxo/path"
29 ipld "github.com/ipfs/go-ipld-format"
30 "github.com/ipfs/go-test/random"
33 - ci "github.com/libp2p/go-libp2p-testing/ci"
31 + options "github.com/ipfs/kubo/core/coreiface/options"
32 + "github.com/ipfs/kubo/fuse/fusetest"
33 )
34
36 -func maybeSkipFuseTests(t *testing.T) {
37 - if ci.NoFuse() {
38 - t.Skip("Skipping FUSE tests")
39 - }
40 -}
41 -
35 func randObj(t *testing.T, nd *core.IpfsNode, size int64) (ipld.Node, []byte) {
36 buf := make([]byte, size)
37 _, err := io.ReadFull(random.NewRand(), buf)
@@ -56,7 +49,7 @@ func randObj(t *testing.T, nd *core.IpfsNode, size int64) (ipld.Node, []byte) {
49
50 func setupIpfsTest(t *testing.T, node *core.IpfsNode) (*core.IpfsNode, *fstest.Mount) {
51 t.Helper()
59 - maybeSkipFuseTests(t)
52 + fusetest.SkipUnlessFUSE(t)
53
54 var err error
55 if node == nil {
@@ -68,21 +61,146 @@ func setupIpfsTest(t *testing.T, node *core.IpfsNode) (*core.IpfsNode, *fstest.M
61
62 fs := NewFileSystem(node)
63 mnt, err := fstest.MountedT(t, fs, nil)
71 - if err == fuse.ErrOSXFUSENotFound {
72 - t.Skip(err)
64 + fusetest.MountError(t, err)
65 +
66 + return node, mnt
67 +}
68 +
69 +// Test that an empty directory can be listed without errors.
70 +func TestEmptyDirListing(t *testing.T) {
71 + nd, mnt := setupIpfsTest(t, nil)
72 + defer mnt.Close()
73 +
74 + // Create an empty UnixFS directory and add it to the DAG.
75 + db, err := uio.NewDirectory(nd.DAG)
76 + if err != nil {
77 + t.Fatal(err)
78 }
79 + emptyDir, err := db.GetNode()
80 if err != nil {
75 - t.Fatalf("error mounting temporary directory: %v", err)
81 + t.Fatal(err)
82 + }
83 + if err := nd.DAG.Add(nd.Context(), emptyDir); err != nil {
84 + t.Fatal(err)
85 }
86
78 - return node, mnt
87 + // List it via FUSE.
88 + dirPath := gopath.Join(mnt.Dir, emptyDir.Cid().String())
89 + entries, err := os.ReadDir(dirPath)
90 + if err != nil {
91 + t.Fatal(err)
92 + }
93 + if len(entries) != 0 {
94 + t.Fatalf("expected empty directory, got %d entries", len(entries))
95 + }
96 +}
97 +
98 +// Test that a bare file CID can be read at the /ipfs mount root.
99 +func TestBareFileCID(t *testing.T) {
100 + nd, mnt := setupIpfsTest(t, nil)
101 + defer mnt.Close()
102 +
103 + api, err := coreapi.NewCoreAPI(nd)
104 + if err != nil {
105 + t.Fatal(err)
106 + }
107 +
108 + content := []byte("bare file CID test content")
109 +
110 + t.Run("CIDv0", func(t *testing.T) {
111 + resolved, err := api.Unixfs().Add(t.Context(),
112 + files.NewBytesFile(content),
113 + options.Unixfs.CidVersion(0),
114 + options.Unixfs.RawLeaves(false))
115 + if err != nil {
116 + t.Fatal(err)
117 + }
118 + cidStr := resolved.RootCid().String()
119 + got, err := os.ReadFile(gopath.Join(mnt.Dir, cidStr))
120 + if err != nil {
121 + t.Fatalf("read %s via FUSE: %v", cidStr, err)
122 + }
123 + if !bytes.Equal(got, content) {
124 + t.Fatalf("content mismatch: got %d bytes, want %d", len(got), len(content))
125 + }
126 + })
127 +
128 + t.Run("CIDv1", func(t *testing.T) {
129 + resolved, err := api.Unixfs().Add(t.Context(),
130 + files.NewBytesFile(content),
131 + options.Unixfs.CidVersion(1),
132 + options.Unixfs.RawLeaves(true))
133 + if err != nil {
134 + t.Fatal(err)
135 + }
136 + cidStr := resolved.RootCid().String()
137 + got, err := os.ReadFile(gopath.Join(mnt.Dir, cidStr))
138 + if err != nil {
139 + t.Fatalf("read %s via FUSE: %v", cidStr, err)
140 + }
141 + if !bytes.Equal(got, content) {
142 + t.Fatalf("content mismatch: got %d bytes, want %d", len(got), len(content))
143 + }
144 + })
145 +}
146 +
147 +// Test reading a directory that contains both dag-pb and raw-leaf children.
148 +// This is the typical layout produced by `ipfs add --raw-leaves`: the
149 +// directory node is dag-pb, while file leaves are raw blocks.
150 +func TestMixedDAGDirectory(t *testing.T) {
151 + nd, mnt := setupIpfsTest(t, nil)
152 + defer mnt.Close()
153 +
154 + api, err := coreapi.NewCoreAPI(nd)
155 + if err != nil {
156 + t.Fatal(err)
157 + }
158 +
159 + fileA := []byte("file in dag-pb leaf")
160 + fileB := []byte("file in raw leaf")
161 +
162 + dir := files.NewMapDirectory(map[string]files.Node{
163 + "dagpb.txt": files.NewBytesFile(fileA),
164 + "raw.txt": files.NewBytesFile(fileB),
165 + })
166 +
167 + // CIDv1 with raw leaves: directory is dag-pb, file leaves are raw.
168 + resolved, err := api.Unixfs().Add(t.Context(), dir,
169 + options.Unixfs.CidVersion(1),
170 + options.Unixfs.RawLeaves(true))
171 + if err != nil {
172 + t.Fatal(err)
173 + }
174 +
175 + dirPath := gopath.Join(mnt.Dir, resolved.RootCid().String())
176 +
177 + entries, err := os.ReadDir(dirPath)
178 + if err != nil {
179 + t.Fatal(err)
180 + }
181 + if len(entries) != 2 {
182 + t.Fatalf("expected 2 entries, got %d", len(entries))
183 + }
184 +
185 + for _, tc := range []struct {
186 + name string
187 + want []byte
188 + }{
189 + {"dagpb.txt", fileA},
190 + {"raw.txt", fileB},
191 + } {
192 + got, err := os.ReadFile(gopath.Join(dirPath, tc.name))
193 + if err != nil {
194 + t.Fatalf("read %s: %v", tc.name, err)
195 + }
196 + if !bytes.Equal(got, tc.want) {
197 + t.Fatalf("%s: content mismatch: got %d bytes, want %d", tc.name, len(got), len(tc.want))
198 + }
199 + }
200 }
201
202 // Test writing an object and reading it back through fuse.
203 func TestIpfsBasicRead(t *testing.T) {
83 - if testing.Short() {
84 - t.SkipNow()
85 - }
204 nd, mnt := setupIpfsTest(t, nil)
205 defer mnt.Close()
206
@@ -123,9 +241,6 @@ func getPaths(t *testing.T, ipfs *core.IpfsNode, name string, n *dag.ProtoNode)
241
242 // Perform a large number of concurrent reads to stress the system.
243 func TestIpfsStressRead(t *testing.T) {
126 - if testing.Short() {
127 - t.SkipNow()
128 - }
244 nd, mnt := setupIpfsTest(t, nil)
245 defer mnt.Close()
246
@@ -235,9 +350,6 @@ func TestIpfsStressRead(t *testing.T) {
350
351 // Test writing a file and reading it back.
352 func TestIpfsBasicDirRead(t *testing.T) {
238 - if testing.Short() {
239 - t.SkipNow()
240 - }
353 nd, mnt := setupIpfsTest(t, nil)
354 defer mnt.Close()
355
@@ -289,9 +401,6 @@ func TestIpfsBasicDirRead(t *testing.T) {
401
402 // Test to make sure the filesystem reports file sizes correctly.
403 func TestFileSizeReporting(t *testing.T) {
292 - if testing.Short() {
293 - t.SkipNow()
294 - }
404 nd, mnt := setupIpfsTest(t, nil)
405 defer mnt.Close()
406
fuse/readonly/readonly_unix.go
+22 -14
@@ -8,6 +8,7 @@ import (
8 "io"
9 "os"
10 "syscall"
11 + "time"
12
13 fuse "bazil.org/fuse"
14 fs "bazil.org/fuse/fs"
@@ -19,12 +20,15 @@ import (
20 ipld "github.com/ipfs/go-ipld-format"
21 logging "github.com/ipfs/go-log/v2"
22 core "github.com/ipfs/kubo/core"
22 - ipldprime "github.com/ipld/go-ipld-prime"
23 cidlink "github.com/ipld/go-ipld-prime/linking/cid"
24 )
25
26 var log = logging.Logger("fuse/ipfs")
27
28 +// /ipfs paths are immutable (content-addressed by CID), so the kernel
29 +// can cache attributes and directory entries for as long as it wants.
30 +const immutableAttrCacheTime = 365 * 24 * time.Hour
31 +
32 // FileSystem is the readonly IPFS Fuse Filesystem.
33 type FileSystem struct {
34 Ipfs *core.IpfsNode
@@ -47,7 +51,10 @@ type Root struct {
51
52 // Attr returns file attributes.
53 func (*Root) Attr(ctx context.Context, a *fuse.Attr) error {
54 + a.Valid = immutableAttrCacheTime
55 a.Mode = os.ModeDir | 0o111 // -rw+x
56 + a.Uid = uint32(os.Getuid())
57 + a.Gid = uint32(os.Getgid())
58 return nil
59 }
60
@@ -84,7 +91,8 @@ func (s *Root) Lookup(ctx context.Context, name string) (fs.Node, error) {
91 return nil, syscall.Errno(syscall.ENOENT)
92 }
93
87 - // convert ipld-prime node to universal node
94 + // Build a legacy ipld.Node from the raw block so the rest of the
95 + // FUSE code (Attr, ReadDirAll, Read) can work with it.
96 blk, err := s.Ipfs.Blockstore.Get(ctx, cidLnk.Cid)
97 if err != nil {
98 log.Debugf("fuse failed to retrieve block: %v: %s", cidLnk, err)
@@ -94,13 +102,12 @@ func (s *Root) Lookup(ctx context.Context, name string) (fs.Node, error) {
102 var fnd ipld.Node
103 switch cidLnk.Cid.Prefix().Codec {
104 case cid.DagProtobuf:
97 - adl, ok := nd.(ipldprime.ADL)
98 - if ok {
99 - substrate := adl.Substrate()
100 - fnd, err = mdag.ProtoNodeConverter(blk, substrate)
101 - } else {
102 - fnd, err = mdag.ProtoNodeConverter(blk, nd)
103 - }
105 + // Decode directly from block bytes. UnixFS files are
106 + // represented as ADLs in ipld-prime, and their substrate
107 + // type is not a dagpb.PBNode, so ProtoNodeConverter fails
108 + // for them. Decoding from bytes works for all dag-pb blocks
109 + // (files, directories, HAMT shards, symlinks, raw).
110 + fnd, err = mdag.DecodeProtobuf(blk.RawData())
111 case cid.Raw:
112 fnd, err = mdag.RawNodeConverter(blk, nd)
113 default:
@@ -108,7 +115,7 @@ func (s *Root) Lookup(ctx context.Context, name string) (fs.Node, error) {
115 return nil, syscall.Errno(syscall.ENOTSUP)
116 }
117 if err != nil {
111 - log.Errorf("could not convert protobuf or raw node: %s", err)
118 + log.Errorf("could not decode block as protobuf or raw node: %s", err)
119 return nil, syscall.Errno(syscall.ENOENT)
120 }
121
@@ -140,8 +147,12 @@ func (s *Node) loadData() error {
147 }
148
149 // Attr returns the attributes of a given node.
150 +// TODO: use Mode and Mtime from UnixFS record if present
151 func (s *Node) Attr(ctx context.Context, a *fuse.Attr) error {
152 log.Debug("Node attr")
153 + a.Valid = immutableAttrCacheTime
154 + a.Uid = uint32(os.Getuid())
155 + a.Gid = uint32(os.Getgid())
156 if rawnd, ok := s.Nd.(*mdag.RawNode); ok {
157 a.Mode = 0o444
158 a.Size = uint64(len(rawnd.RawData()))
@@ -247,10 +258,7 @@ func (s *Node) ReadDirAll(ctx context.Context) ([]fuse.Dirent, error) {
258 return nil, err
259 }
260
250 - if len(entries) > 0 {
251 - return entries, nil
252 - }
253 - return nil, syscall.Errno(syscall.ENOENT)
261 + return entries, nil
262 }
263
264 func (s *Node) Getxattr(ctx context.Context, req *fuse.GetxattrRequest, resp *fuse.GetxattrResponse) error {
mk/golang.mk
+9
@@ -69,6 +69,15 @@ test_cli: cmd/ipfs/ipfs test/bin/gotestsum $$(DEPS_GO)
69 PATH="$(CURDIR)/cmd/ipfs:$(CURDIR)/test/bin:$$PATH" gotestsum $(GOTESTSUM_NOCOLOR) --jsonfile test/cli/cli-tests.json -- -v -timeout=$(TEST_CLI_TIMEOUT) ./test/cli/... ./test/integration/... ./client/rpc/...
70 .PHONY: test_cli
71
72 +# FUSE unit tests (requires /dev/fuse and fusermount in PATH)
73 +# Set TEST_FUSE=1 to make mount failures fatal instead of skipping
74 +TEST_FUSE_TIMEOUT ?= 5m
75 +test_fuse: test/bin/gotestsum $$(DEPS_GO)
76 + mkdir -p test/fuse
77 + rm -f test/fuse/fuse-tests.json
78 + TEST_FUSE=1 gotestsum $(GOTESTSUM_NOCOLOR) --jsonfile test/fuse/fuse-tests.json -- -v -timeout=$(TEST_FUSE_TIMEOUT) ./fuse/...
79 +.PHONY: test_fuse
80 +
81 # Example tests (docs/examples/kubo-as-a-library)
82 # Tests against both published and current kubo versions
83 # Uses timeout to ensure CI gets output before job-level timeout kills everything