@cryptotaxi247 / kubo / commits / f4d6253ae

docs: add developer guide for local development workflow (#11128)

addresses https://discuss.ipfs.tech/t/19933 - add docs/developer-guide.md with prerequisites, build, test, and troubleshooting - link from README.md, docs/README.md, and CONTRIBUTING.md - document test suite differences (unit vs e2e, test/cli vs test/sharness) - include tips for running specific tests during development

Marcin Rataj committed Jan 7, 2026 at 17:09 UTC f4d6253ae201a032d6d992adfcd3bba7ea1d6658

4 files changed +326 -45

CONTRIBUTING.md

+6 -2

@@ -1,6 +1,10 @@

1 -IPFS as a project, including go-ipfs and all of its modules, follows the [standard IPFS Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md).

1 +# Contributing to Kubo

3 -We also adhere to the [GO IPFS Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING_GO.md) which provide additional information of how to collaborate and contribute in the Go implementation of IPFS.

3 +**For development setup, building, and testing, see the [Developer Guide](docs/developer-guide.md).**

4 +

5 +IPFS as a project, including Kubo and all of its modules, follows the [standard IPFS Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md).

6 +

7 +We also adhere to the [Go IPFS Community contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING_GO.md) which provide additional information on how to collaborate and contribute to the Go implementation of IPFS.

9 We appreciate your time and attention for going over these. Please open an issue on ipfs/community if you have any questions.

README.md

+1 -40

@@ -98,11 +98,6 @@ Before opening an issue, consider using one of the following locations to ensure

98 - [Troubleshooting](#troubleshooting-1)

99 - [Packages](#packages)

100 - [Development](#development)

101 - - [Map of Implemented Subsystems](#map-of-implemented-subsystems)

102 - - [CLI, HTTP-API, Architecture Diagram](#cli-http-api-architecture-diagram)

103 - - [Testing](#testing)

104 - - [Development Dependencies](#development-dependencies)

105 - - [Developer Notes](#developer-notes)

101 - [Maintainer Info](#maintainer-info)

102 - [Contributing](#contributing)

103 - [License](#license)

@@ -450,41 +445,7 @@ See [IPFS in GO](https://docs.ipfs.tech/reference/go/api/) documentation.

445

446 ## Development

447

453 -Some places to get you started on the codebase:

454 -

455 -- Main file: [./cmd/ipfs/main.go](https://github.com/ipfs/kubo/blob/master/cmd/ipfs/main.go)

456 -- CLI Commands: [./core/commands/](https://github.com/ipfs/kubo/tree/master/core/commands)

457 -- Bitswap (the data trading engine): [go-bitswap](https://github.com/ipfs/go-bitswap)

458 -- libp2p

459 - - libp2p: https://github.com/libp2p/go-libp2p

460 - - DHT: https://github.com/libp2p/go-libp2p-kad-dht

461 -- [IPFS : The `Add` command demystified](https://github.com/ipfs/kubo/tree/master/docs/add-code-flow.md)

462 -

463 -### Map of Implemented Subsystems

464 -**WIP**: This is a high-level architecture diagram of the various sub-systems of this specific implementation. To be updated with how they interact. Anyone who has suggestions is welcome to comment [here](https://docs.google.com/drawings/d/1OVpBT2q-NtSJqlPX3buvjYhOnWfdzb85YEsM_njesME/edit) on how we can improve this!

465 -<img src="https://docs.google.com/drawings/d/e/2PACX-1vS_n1FvSu6mdmSirkBrIIEib2gqhgtatD9awaP2_WdrGN4zTNeg620XQd9P95WT-IvognSxIIdCM5uE/pub?w=1446&h=1036">

466 -

467 -### CLI, HTTP-API, Architecture Diagram

468 -

469 -![](./docs/cli-http-api-core-diagram.png)

470 -

471 -> [Origin](https://github.com/ipfs/pm/pull/678#discussion_r210410924)

472 -

473 -Description: Dotted means "likely going away". The "Legacy" parts are thin wrappers around some commands to translate between the new system and the old system. The grayed-out parts on the "daemon" diagram are there to show that the code is all the same, it's just that we turn some pieces on and some pieces off depending on whether we're running on the client or the server.

474 -

475 -### Testing

476 -

477 -```

478 -make test

479 -```

480 -

481 -### Development Dependencies

482 -

483 -If you make changes to the protocol buffers, you will need to install the [protoc compiler](https://github.com/google/protobuf).

484 -

485 -### Developer Notes

486 -

487 -Find more documentation for developers on [docs](./docs)

448 +See the [Developer Guide](docs/developer-guide.md) for build instructions, testing, architecture, and contribution workflow.

449

450 ## Maintainer Info

451

docs/README.md

+3 -3

@@ -14,9 +14,9 @@ Otherwise, check out the following guides to using and developing IPFS:

15 ## Developing `kubo`

17 -- First, please read the Contributing Guidelines [for IPFS projects](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md) and then the Contributing Guidelines for [Go code specifically](https://github.com/ipfs/community/blob/master/CONTRIBUTING_GO.md)

18 -- Building on…

19 - - [Windows](windows.md)

17 +- **[Developer Guide](developer-guide.md)** - prerequisites, build, test, and contribute

18 +- Contributing Guidelines [for IPFS projects](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md) and for [Go code specifically](https://github.com/ipfs/community/blob/master/CONTRIBUTING_GO.md)

19 +- Building on [Windows](windows.md)

20 - [Performance Debugging Guidelines](debug-guide.md)

21 - [Release Checklist](releases.md)

docs/developer-guide.md new

+316

@@ -0,0 +1,316 @@

1 +# Developer Guide

2 +

3 +By the end of this guide, you will be able to:

4 +

5 +- Build Kubo from source

6 +- Run the test suites

7 +- Make and verify code changes

8 +

9 +This guide covers the local development workflow. For user documentation, see [docs.ipfs.tech](https://docs.ipfs.tech/).

10 +

11 +## Table of Contents

12 +

13 +- [Prerequisites](#prerequisites)

14 +- [Quick Start](#quick-start)

15 +- [Building](#building)

16 +- [Running Tests](#running-tests)

17 +- [Running the Linter](#running-the-linter)

18 +- [Common Development Tasks](#common-development-tasks)

19 +- [Code Organization](#code-organization)

20 +- [Architecture](#architecture)

21 +- [Troubleshooting](#troubleshooting)

22 +- [Development Dependencies](#development-dependencies)

23 +- [Further Reading](#further-reading)

24 +

25 +## Prerequisites

26 +

27 +Before you begin, ensure you have:

28 +

29 +- **Go** - see `go.mod` for the minimum required version

30 +- **Git**

31 +- **GNU Make**

32 +- **GCC** (optional) - required for CGO (Go's C interop); without it, build with `CGO_ENABLED=0`

33 +

34 +## Quick Start

35 +

36 +```bash

37 +git clone https://github.com/ipfs/kubo.git

38 +cd kubo

39 +make build

40 +./cmd/ipfs/ipfs version

41 +```

42 +

43 +You should see output like:

44 +

45 +```

46 +ipfs version 0.34.0-dev

47 +```

48 +

49 +The binary is built to `cmd/ipfs/ipfs`. To install it system-wide:

50 +

51 +```bash

52 +make install

53 +```

54 +

55 +This installs the binary to `$GOPATH/bin`.

56 +

57 +## Building

58 +

59 +| Command | Description |

60 +|---------|-------------|

61 +| `make build` | build the `ipfs` binary to `cmd/ipfs/ipfs` |

62 +| `make install` | install to `$GOPATH/bin` |

63 +| `make nofuse` | build without FUSE support |

64 +| `make build CGO_ENABLED=0` | build without CGO (no C compiler needed) |

65 +

66 +For Windows-specific instructions, see [windows.md](windows.md).

67 +

68 +## Running Tests

69 +

70 +Kubo has two types of tests:

71 +

72 +- **Unit tests** - test individual packages in isolation. Fast and don't require a running daemon.

73 +- **End-to-end tests** - spawn real `ipfs` nodes, run actual CLI commands, and test the full system. Slower but catch integration issues.

74 +

75 +Note that `go test ./...` runs both unit and end-to-end tests. Use `make test` to run all tests. CI runs unit and end-to-end tests in separate jobs for faster feedback.

76 +

77 +<!-- TODO: uncomment when https://github.com/ipfs/kubo/pull/11113 is merged

78 +| Command | What it runs |

79 +|---------|--------------|

80 +| `make test_unit` | unit tests only (excludes `test/cli`) |

81 +| `make test_cli` | CLI end-to-end tests only (requires `make build` first) |

82 +| `make test_sharness` | sharness end-to-end tests only |

83 +| `make test` | all tests (unit + CLI + sharness) |

84 +-->

85 +

86 +For end-to-end tests, Kubo has two suites:

87 +

88 +- **`test/cli`** - modern Go-based test harness that spawns real `ipfs` nodes and runs actual CLI commands. All new tests should be added here.

89 +- **`test/sharness`** - legacy bash-based tests. We are slowly migrating these to `test/cli`.

90 +

91 +When modifying tests: cosmetic changes to `test/sharness` are fine, but if significant rewrites are needed, remove the outdated sharness test and add a modern one to `test/cli` instead.

92 +

93 +### Before Running Tests

94 +

95 +**Environment requirements**: some legacy tests expect default ports (8080, 5001, 4001) to be free and no mDNS (local network discovery) Kubo service on the LAN. Tests may fail if you have a local Kubo instance running. Before running the full test suite, stop any running `ipfs daemon`.

96 +

97 +Two critical setup steps:

98 +

99 +1. **Rebuild after code changes**: if you modify any `.go` files outside of `test/`, you must run `make build` before running integration tests.

100 +

101 +2. **Set environment variables**: integration tests use the `ipfs` binary from `PATH` and need an isolated `IPFS_PATH`. Run these commands from the repository root:

102 +

103 +```bash

104 +export PATH="$PWD/cmd/ipfs:$PATH"

105 +export IPFS_PATH="$(mktemp -d)"

106 +```

107 +

108 +### Unit Tests

109 +

110 +```bash

111 +go test ./...

112 +```

113 +

114 +### CLI Integration Tests (`test/cli`)

115 +

116 +These are Go-based integration tests that invoke the `ipfs` CLI.

117 +

118 +Instead of running the entire test suite, you can run a specific test to get faster feedback during development.

119 +

120 +Run a specific test (recommended during development):

121 +

122 +```bash

123 +go test ./test/cli/... -run TestAdd -v

124 +```

125 +

126 +Run all CLI tests:

127 +

128 +```bash

129 +go test ./test/cli/...

130 +```

131 +

132 +Run a specific test:

133 +

134 +```bash

135 +go test ./test/cli/... -run TestAdd

136 +```

137 +

138 +Run with verbose output:

139 +

140 +```bash

141 +go test ./test/cli/... -v

142 +```

143 +

144 +**Common error**: "version (16) is lower than repos (17)" means your `PATH` points to an old binary. Check `which ipfs` and rebuild with `make build`.

145 +

146 +### Sharness Tests (`test/sharness`)

147 +

148 +Shell-based integration tests using [sharness](https://github.com/chriscool/sharness) (a portable shell testing framework).

149 +

150 +```bash

151 +cd test/sharness

152 +```

153 +

154 +Run a specific test:

155 +

156 +```bash

157 +timeout 60s ./t0080-repo.sh

158 +```

159 +

160 +Run with verbose output (this disables automatic cleanup):

161 +

162 +```bash

163 +./t0080-repo.sh -v

164 +```

165 +

166 +**Cleanup**: the `-v` flag disables automatic cleanup. Before re-running tests, kill any dangling `ipfs daemon` processes:

167 +

168 +```bash

169 +pkill -f "ipfs daemon"

170 +```

171 +

172 +### Full Test Suite

173 +

174 +```bash

175 +make test # run all tests

176 +make test_short # run shorter test suite

177 +```

178 +

179 +## Running the Linter

180 +

181 +Run the linter using the Makefile target (not `golangci-lint` directly):

182 +

183 +```bash

184 +make -O test_go_lint

185 +```

186 +

187 +## Common Development Tasks

188 +

189 +### Modifying CLI Commands

190 +

191 +After editing help text in `core/commands/`, verify the output width:

192 +

193 +```bash

194 +go test ./test/cli/... -run TestCommandDocsWidth

195 +```

196 +

197 +### Updating Dependencies

198 +

199 +Use the Makefile target (not `go mod tidy` directly):

200 +

201 +```bash

202 +make mod_tidy

203 +```

204 +

205 +### Editing the Changelog

206 +

207 +When modifying `docs/changelogs/`:

208 +

209 +- update the Table of Contents when adding sections

210 +- add user-facing changes to the Highlights section (the Changelog section is auto-generated)

211 +

212 +### Running the Daemon

213 +

214 +Always run the daemon with a timeout or shut it down promptly.

215 +

216 +With timeout:

217 +

218 +```bash

219 +timeout 60s ipfs daemon

220 +```

221 +

222 +Or shut down via API:

223 +

224 +```bash

225 +ipfs shutdown

226 +```

227 +

228 +For multi-step experiments, store `IPFS_PATH` in a file to ensure consistency.

229 +

230 +## Code Organization

231 +

232 +| Directory | Description |

233 +|-----------|-------------|

234 +| `cmd/ipfs/` | CLI entry point and binary |

235 +| `core/` | core IPFS node implementation |

236 +| `core/commands/` | CLI command definitions |

237 +| `core/coreapi/` | Go API implementation |

238 +| `client/rpc/` | HTTP RPC client |

239 +| `plugin/` | plugin system |

240 +| `repo/` | repository management |

241 +| `test/cli/` | Go-based CLI integration tests |

242 +| `test/sharness/` | legacy shell-based integration tests |

243 +| `docs/` | documentation |

244 +

245 +Key external dependencies:

246 +

247 +- [go-libp2p](https://github.com/libp2p/go-libp2p) - networking stack

248 +- [go-libp2p-kad-dht](https://github.com/libp2p/go-libp2p-kad-dht) - distributed hash table

249 +- [boxo](https://github.com/ipfs/boxo) - IPFS SDK (including Bitswap, the data exchange engine)

250 +

251 +For a deep dive into how code flows through Kubo, see [The `Add` command demystified](add-code-flow.md).

252 +

253 +## Architecture

254 +

255 +**Map of Implemented Subsystems** ([editable source](https://docs.google.com/drawings/d/1OVpBT2q-NtSJqlPX3buvjYhOnWfdzb85YEsM_njesME/edit)):

256 +

257 +<img src="https://docs.google.com/drawings/d/e/2PACX-1vS_n1FvSu6mdmSirkBrIIEib2gqhgtatD9awaP2_WdrGN4zTNeg620XQd9P95WT-IvognSxIIdCM5uE/pub?w=1446&h=1036">

258 +

259 +**CLI, HTTP-API, Core Diagram**:

260 +

261 +![](./cli-http-api-core-diagram.png)

262 +

263 +## Troubleshooting

264 +

265 +### "version (N) is lower than repos (M)" Error

266 +

267 +This means the `ipfs` binary in your `PATH` is older than expected.

268 +

269 +Check which binary is being used:

270 +

271 +```bash

272 +which ipfs

273 +```

274 +

275 +Rebuild and verify PATH:

276 +

277 +```bash

278 +make build

279 +export PATH="$PWD/cmd/ipfs:$PATH"

280 +./cmd/ipfs/ipfs version

281 +```

282 +

283 +### FUSE Issues

284 +

285 +If you don't need FUSE support, build without it:

286 +

287 +```bash

288 +make nofuse

289 +```

290 +

291 +Or set the `TEST_FUSE=0` environment variable when running tests.

292 +

293 +### Build Fails with "No such file: stdlib.h"

294 +

295 +You're missing a C compiler. Either install GCC or build without CGO:

296 +

297 +```bash

298 +make build CGO_ENABLED=0

299 +```

300 +

301 +## Development Dependencies

302 +

303 +If you make changes to the protocol buffers, you will need to install the [protoc compiler](https://github.com/google/protobuf).

304 +

305 +## Further Reading

306 +

307 +- [The `Add` command demystified](add-code-flow.md) - deep dive into code flow

308 +- [Configuration reference](config.md)

309 +- [Performance debugging](debug-guide.md)

310 +- [Experimental features](experimental-features.md)

311 +- [Release process](releases.md)

312 +- [Contributing guidelines](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)

313 +

314 +## Source Code

315 +

316 +The complete source code is at [github.com/ipfs/kubo](https://github.com/ipfs/kubo).