@cryptotaxi247 / kubo / commits / 9539b4d8b

docs: cleanup broken links and outdated content (#11100)

- docs/README.md: restructure to surface 20+ previously undiscoverable docs - docs/README.md: fix broken github-issue-guide.md link (file was removed) - docs/add-code-flow.md: rewrite with current code flow and mermaid diagrams - docs/customizing.md, docs/gateway.md: use specs.ipfs.tech URLs - README.md: fix orphan #nix anchor, use go.dev links, link to contributors graph - remove stale docs/AUTHORS and docs/generate-authors.sh (last updated 2016)

Marcin Rataj committed Jan 30, 2026 at 19:38 UTC 9539b4d8b8509031a45c243fecbec6a4234aef17

5 files changed +215 -213

docs/AUTHORS deleted

-112

@@ -1,112 +0,0 @@

1 -# This file lists all individuals having contributed content to the repository.

2 -# For how it is generated, see `docs/generate-authors.sh`.

3 -

4 -Aaron Hill <aa1ronham@gmail.com>

5 -Adam Gashlin <agashlin@gmail.com>

6 -Adrian Ulrich <adrian@blinkenlights.ch>

7 -Alex <alexgbahm@gmail.com>

8 -anarcat <anarcat@users.noreply.github.com>

9 -Andres Buritica <andres@thelinuxkid.com>

10 -Andrew Chin <achin@eminence32.net>

11 -Andy Leap <andyleap@gmail.com>

12 -Artem Andreenko <mio@volmy.com>

13 -Baptiste Jonglez <baptiste--git@jonglez.org>

14 -Brendan Benshoof <brendan@glidr.net>

15 -Brendan Mc <Bren2010@users.noreply.github.com>

16 -Brian Tiger Chow <brian.holderchow@gmail.com>

17 -Caio Alonso <caio@caioalonso.com>

18 -Carlos Cobo <toqueteos@gmail.com>

19 -Cayman Nava <caymannava@gmail.com>

20 -Chas Leichner <chas@chas.io>

21 -Chris Grimmett <xtoast@gmail.com>

22 -Chris P <sahib@online.de>

23 -Chris Sasarak <chris.sasarak@gmail.com>

24 -Christian Couder <chriscool@tuxfamily.org>

25 -Christian Kniep <christian@qnib.org>

26 -Christopher Sasarak <chris.sasarak@gmail.com>

27 -David <github@kattfest.se>

28 -David Braun <David.Braun@Toptal.com>

29 -David Dias <daviddias.p@gmail.com>

30 -David Wagner <wagdav@gmail.com>

31 -dignifiedquire <dignifiedquire@gmail.com>

32 -Dominic Della Valle <DDVpublic@Gmail.com>

33 -Dominic Tarr <dominic.tarr@gmail.com>

34 -drathir <drathir87@gmail.com>

35 -Dylan Powers <dylan.kyle.powers@gmail.com>

36 -Emery Hemingway <emery@vfemail.net>

37 -epitron <chris@ill-logic.com>

38 -Ethan Buchman <ethan@coinculture.info>

39 -Etienne Laurin <etienne@atnnn.com>

40 -Forrest Weston <fweston@eecs.wsu.edu>

41 -Francesco Canessa <makevoid@gmail.com>

42 -gatesvp <gatesvp@gmail.com>

43 -Giuseppe Bertone <bertone.giuseppe@gmail.com>

44 -Harlan T Wood <harlantwood@users.noreply.github.com>

45 -Hector Sanjuan <code@hector.link>

46 -Henry <cryptix@riseup.net>

47 -Ho-Sheng Hsiao <talktohosh@gmail.com>

48 -Jakub Sztandera <kubuxu@protonmail.ch>

49 -Jason Carver <jacarver@linkedin.com>

50 -Jonathan Dahan <jonathan@jonathan.is>

51 -Juan Batiz-Benet <juan@benet.ai>

52 -Karthik Bala <drmelonhead@gmail.com>

53 -Kevin Atkinson <k@kevina.org>

54 -Kevin Wallace <kevin@pentabarf.net>

55 -klauspost <klauspost@gmail.com>

56 -Knut Ahlers <knut@ahlers.me>

57 -Konstantin Koroviev <kkoroviev@gmail.com>

58 -kpcyrd <git@rxv.cc>

59 -Kristoffer Ström <kristoffer@rymdkoloni.se>

60 -Lars Gierth <larsg@systemli.org>

61 -llSourcell <sirajravel@gmail.com>

62 -Marcin Janczyk <marcinjanczyk@gmail.com>

63 -Marcin Rataj <lidel@lidel.org>

64 -Markus Amalthea Magnuson <markus.magnuson@gmail.com>

65 -michael <pfista@gmail.com>

66 -Michael Lovci <michaeltlovci@gmail.com>

67 -Michael Muré <mure.michael@gmail.com>

68 -Michael Pfister <pfista@gmail.com>

69 -Mildred Ki'Lya <mildred-pub.git@mildred.fr>

70 -Muneeb Ali <muneeb@ali.vc>

71 -Nick Hamann <nick@wabbo.org>

72 -palkeo <contact@palkeo.com>

73 -Patrick Connolly <patrick.c.connolly@gmail.com>

74 -Pavol Rusnak <stick@gk2.sk>

75 -Peter Borzov <peter@sowingo.com>

76 -Philip Nelson <me@pnelson.ca>

77 -Quinn Slack <sqs@sourcegraph.com>

78 -ReadmeCritic <frankensteinbot@gmail.com>

79 -rht <rhtbot@gmail.com>

80 -Richard Littauer <richard.littauer@gmail.com>

81 -Robert Carlsen <rwcarlsen@gmail.com>

82 -Roerick Sweeney <sroerick@gmail.com>

83 -Sean Lang <slang800@gmail.com>

84 -SH <github@hertenberger.bayern>

85 -Shanti Bouchez-Mongardé <shanti-pub.git@mildred.fr>

86 -Shaun Bruce <shaun.m.bruce@gmail.com>

87 -Simon Kirkby <tigger@interthingy.com>

88 -Siraj Ravel <jason.ravel@cbsinteractive.com>

89 -Siva Chandran <siva.chandran@realimage.com>

90 -slothbag <slothbag>

91 -sroerick <sroerick@gmail.com>

92 -Stephan Seidt <evilhackerdude@gmail.com>

93 -Stephen Sugden <me@stephensugden.com>

94 -Stephen Whitmore <stephen.whitmore@gmail.com>

95 -Steven Allen <steven@stebalien.com>

96 -Tarnay Kálmán <kalmisoft@gmail.com>

97 -theswitch <theswitch@users.noreply.github.com>

98 -Thomas Gardner <tmg@fastmail.com>

99 -Tim Groeneveld <tim@timg.ws>

100 -Tommi Virtanen <tv@eagain.net>

101 -Tonis Tiigi <tonistiigi@gmail.com>

102 -Tor Arne Vestbø <torarnv@gmail.com>

103 -Travis Person <travis.person@gmail.com>

104 -verokarhu <andreas.metsala@gmail.com>

105 -Vijayee Kulkaa <vijayee.kulkaa@.husmail.com>

106 -Vitor Baptista <vitor@vitorbaptista.com>

107 -vitzli <vitzli@gmail.com>

108 -W. Trevor King <wking@tremily.us>

109 -Whyrusleeping <why@ipfs.io>

110 -wzhd <dev@wzhd.org>

111 -Yuval Langer <yuval.langer@gmail.com>

112 -ᴍᴀᴛᴛ ʙᴇʟʟ <mappum@gmail.com>

docs/README.md

+41 -22

@@ -1,39 +1,58 @@

1 # Developer Documentation and Guides

3 -If you are looking for User Documentation & Guides, please visit [docs.ipfs.tech](https://docs.ipfs.tech/) or check [General Documentation](#general-documentation).

3 +If you're looking for User Documentation & Guides, visit [docs.ipfs.tech](https://docs.ipfs.tech/).

5 -If you’re experiencing an issue with IPFS, **please follow [our issue guide](github-issue-guide.md) when filing an issue!**

5 +If you're experiencing an issue with IPFS, please [file an issue](https://github.com/ipfs/kubo/issues/new/choose) in this repository.

7 -Otherwise, check out the following guides to using and developing IPFS:

8 -

9 -## General Documentation

7 +## Configuration

9 - [Configuration reference](config.md)

12 - - [Datastore configuration](datastores.md)

13 - - [Experimental features](experimental-features.md)

10 + - [Datastore configuration](datastores.md)

11 + - [Experimental features](experimental-features.md)

12 +- [Environment variables](environment-variables.md)

13 +

14 +## Running Kubo

15 +

16 +- [Gateway configuration](gateway.md)

17 +- [Delegated routing](delegated-routing.md)

18 +- [Content blocking](content-blocking.md) (for public node operators)

19 +- [libp2p resource management](libp2p-resource-management.md)

20 +- [Mounting IPFS with FUSE](fuse.md)

21 +

22 +## Metrics & Monitoring

23 +

24 +- [Prometheus metrics](metrics.md)

25 +- [Telemetry plugin](telemetry.md)

26 +- [Provider statistics](provide-stats.md)

27 +- [Performance debugging](debug-guide.md)

15 -## Developing `kubo`

29 +## Development

31 - **[Developer Guide](developer-guide.md)** - prerequisites, build, test, and contribute

32 - Contributing Guidelines [for IPFS projects](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md) and for [Go code specifically](https://github.com/ipfs/community/blob/master/CONTRIBUTING_GO.md)

19 -- Building on [Windows](windows.md)

20 -- [Performance Debugging Guidelines](debug-guide.md)

21 -- [Release Checklist](releases.md)

33 +- [Building on Windows](windows.md)

34 +- [Customizing Kubo](customizing.md)

35 +- [Installing plugins](plugins.md)

36 +- [Release checklist](releases.md)

38 ## Guides

25 -- [How to Implement an API Client](implement-api-bindings.md)

26 -- [Connecting with Websockets](transports.md) — if you want `js-ipfs` nodes in web browsers to connect to your `kubo` node, you will need to turn on websocket support in your `kubo` node.

40 +- [Transferring files over IPFS](file-transfer.md)

41 +- [How to implement an API client](implement-api-bindings.md)

42 +- [HTTP/RPC clients](http-rpc-clients.md)

43 +- [Websocket transports](transports.md)

44 +- [Command completion](command-completion.md)

28 -## Advanced User Guides

46 +## Production

30 -- [Transferring a File Over IPFS](file-transfer.md)

31 -- [Installing command completion](command-completion.md)

32 -- [Mounting IPFS with FUSE](fuse.md)

33 -- [Installing plugins](plugins.md)

34 -- [Setting up an IPFS Gateway](https://github.com/ipfs/kubo/blob/master/docs/gateway.md)

48 +- [Reverse proxy setup](production/reverse-proxy.md)

49 +

50 +## Specifications

51 +

52 +- [Repository structure](specifications/repository.md)

53 +- [Filesystem datastore](specifications/repository_fs.md)

54 +- [Keystore](specifications/keystore.md)

36 -## Other

56 +## Examples

38 -- [Thanks to all our contributors ❤️](AUTHORS) (We use the `generate-authors.sh` script to regenerate this list.)

39 -- [How to file a GitHub Issue](github-issue-guide.md)

58 +- [Kubo as a library](examples/kubo-as-a-library/README.md)

docs/add-code-flow.md

+173 -66

@@ -1,102 +1,209 @@

1 -# IPFS : The `Add` command demystified

1 +# How `ipfs add` Works

3 -The goal of this document is to capture the code flow for adding a file (see the `coreapi` package) using the IPFS CLI, in the process exploring some data structures and packages like `ipld.Node` (aka `dagnode`), `FSNode`, `MFS`, etc.

3 +This document explains what happens when you run `ipfs add` to import files into IPFS. Understanding this flow helps when debugging, optimizing imports, or building applications on top of IPFS.

5 -## Concepts

6 -- [Files](https://github.com/ipfs/docs/issues/133)

5 +- [The Big Picture](#the-big-picture)

6 +- [Try It Yourself](#try-it-yourself)

7 +- [Step by Step](#step-by-step)

8 + - [Step 1: Chunking](#step-1-chunking)

9 + - [Step 2: Building the DAG](#step-2-building-the-dag)

10 + - [Step 3: Storing Blocks](#step-3-storing-blocks)

11 + - [Step 4: Pinning](#step-4-pinning)

12 + - [Alternative: Organizing with MFS](#alternative-organizing-with-mfs)

13 +- [Options](#options)

14 +- [UnixFS Format](#unixfs-format)

15 +- [Code Architecture](#code-architecture)

16 + - [Key Files](#key-files)

17 + - [The Adder](#the-adder)

18 +- [Further Reading](#further-reading)

8 ----

20 +## The Big Picture

10 -**Try this yourself**

11 ->

12 -> ```

13 -> # Convert a file to the IPFS format.

14 -> echo "Hello World" > new-file

15 -> ipfs add new-file

16 -> added QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u new-file

17 -> 12 B / 12 B [=========================================================] 100.00%

18 ->

19 -> # Add a file to the MFS.

20 -> NEW_FILE_HASH=$(ipfs add new-file -Q)

21 -> ipfs files cp /ipfs/$NEW_FILE_HASH /new-file

22 ->

23 -> # Get information from the file in MFS.

24 -> ipfs files stat /new-file

25 -> # QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u

26 -> # Size: 12

27 -> # CumulativeSize: 20

28 -> # ChildBlocks: 0

29 -> # Type: file

30 ->

31 -> # Retrieve the contents.

32 -> ipfs files read /new-file

33 -> # Hello World

34 -> ```

22 +When you add a file to IPFS, three main things happen:

36 -## Code Flow

24 +1. **Chunking** - The file is split into smaller pieces

25 +2. **DAG Building** - Those pieces are organized into a tree structure (a [Merkle DAG](https://docs.ipfs.tech/concepts/merkle-dag/))

26 +3. **Pinning** - The root of the tree is pinned so it persists in your local node

38 -**[`UnixfsAPI.Add()`](https://github.com/ipfs/go-ipfs/blob/v0.4.18/core/coreapi/unixfs.go#L31)** - *Entrypoint into the `Unixfs` package*

28 +The result is a Content Identifier (CID) - a hash that uniquely identifies your content and can be used to retrieve it from anywhere in the IPFS network.

40 -The `UnixfsAPI.Add()` acts on the input data or files, to build a _merkledag_ node (in essence it is the entire tree represented by the root node) and adds it to the _blockstore_.

41 -Within the function, a new `Adder` is created with the configured `Blockstore` and __DAG service__`.

30 +```mermaid

31 +flowchart LR

32 + A["Your File (bytes)"] --> B["Chunker (split data)"]

33 + B --> C["DAG Builder (tree)"]

34 + C --> D["CID (hash)"]

35 +```

43 -- **[`adder.AddAllAndPin(files)`](https://github.com/ipfs/go-ipfs/blob/v0.4.18/core/coreunix/add.go#L403)** - *Entrypoint to the `Add` logic*

44 - encapsulates a lot of the underlying functionality that will be investigated in the following sections.

37 +## Try It Yourself

46 - Our focus will be on the simplest case, a single file, handled by `Adder.addFile(file files.File)`.

39 +```bash

40 +# Add a simple file

41 +echo "Hello World" > hello.txt

42 +ipfs add hello.txt

43 +# added QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u hello.txt

48 - - **[`adder.addFile(file files.File)`](https://github.com/ipfs/go-ipfs/blob/v0.4.18/core/coreunix/add.go#L450)** - *Create the _DAG_ and add to `MFS`*

45 +# See what's inside

46 +ipfs cat QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u

47 +# Hello World

50 - The `addFile(file)` method takes the data and converts it into a __DAG__ tree and adds the root of the tree into the `MFS`.

49 +# View the DAG structure

50 +ipfs dag get QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u

51 +```

52 - https://github.com/ipfs/go-ipfs/blob/v0.4.18/core/coreunix/add.go#L508-L521

53 +## Step by Step

54 - There are two main methods to focus on -

55 +### Step 1: Chunking

56 - 1. **[`adder.add(io.Reader)`](https://github.com/ipfs/go-ipfs/blob/v0.4.18/core/coreunix/add.go#L115)** - *Create and return the **root** __DAG__ node*

57 +Big files are split into chunks because:

58 - This method converts the input data (`io.Reader`) to a __DAG__ tree, by splitting the data into _chunks_ using the `Chunker` and organizing them into a __DAG__ (with a *trickle* or *balanced* layout. See [balanced](https://github.com/ipfs/go-unixfs/blob/6b769632e7eb8fe8f302e3f96bf5569232e7a3ee/importer/balanced/builder.go) for more info).

59 +- Large files need to be broken down for efficient transfer

60 +- Identical chunks across files are stored only once (deduplication)

61 +- You can fetch parts of a file without downloading the whole thing

60 - The method returns the **root** `ipld.Node` of the __DAG__.

63 +**Chunking strategies** (set with `--chunker`):

62 - 2. **[`adder.addNode(ipld.Node, path)`](https://github.com/ipfs/go-ipfs/blob/v0.4.18/core/coreunix/add.go#L366)** - *Add **root** __DAG__ node to the `MFS`*

65 +| Strategy | Description | Best For |

66 +|----------|-------------|----------|

67 +| `size-N` | Fixed size chunks | General use |

68 +| `rabin` | Content-defined chunks using rolling hash | Deduplication across similar files |

69 +| `buzhash` | Alternative content-defined chunking | Similar to rabin |

64 - Now that we have the **root** node of the `DAG`, this needs to be added to the `MFS` file system.

65 - Fetch (or create, if doesn't already exist) the `MFS` **root** using `mfsRoot()`.

71 +See `ipfs add --help` for current defaults, or [Import](config.md#import) for making them permanent.

67 - > NOTE: The `MFS` **root** is an ephemeral root, created and destroyed solely for the `add` functionality.

73 +Content-defined chunking (rabin/buzhash) finds natural boundaries in the data. This means if you edit the middle of a file, only the changed chunks need to be re-stored - the rest can be deduplicated.

69 - Assuming the directory already exists in the MFS file system, (if it doesn't exist it will be created using `mfs.Mkdir()`), the **root** __DAG__ node is added to the `MFS` File system using the `mfs.PutNode()` function.

75 +### Step 2: Building the DAG

71 - - **[MFS] [`PutNode(mfs.Root, path, ipld.Node)`](https://github.com/ipfs/go-mfs/blob/v0.1.18/ops.go#L86)** - *Insert node at path into given `MFS`*

77 +Each chunk becomes a leaf node in a tree. If a file has many chunks, intermediate nodes group them together. This creates a Merkle DAG (Directed Acyclic Graph) where:

73 - The `path` param is used to determine the `MFS Directory`, which is first looked up in the `MFS` using `lookupDir()` function. This is followed by adding the **root** __DAG__ node (`ipld.Node`) into this `Directory` using `directory.AddChild()` method.

79 +- Each node is identified by a hash of its contents

80 +- Parent nodes contain links (hashes) to their children

81 +- The root node's hash becomes the file's CID

75 - - **[MFS] Add Child To `UnixFS`**

76 - - **[`directory.AddChild(filename, ipld.Node)`](https://github.com/ipfs/go-mfs/blob/v0.1.18/dir.go#L350)** - *Add **root** __DAG__ node under this directory*

83 +**Layout strategies**:

78 - Within this method the node is added to the `Directory`'s __DAG service__ using the `dserv.Add()` method, followed by adding the **root** __DAG__ node with the given name, in the `directory.addUnixFSChild(directory.child{name, ipld.Node})` method.

85 +**Balanced layout** (default):

80 - - **[MFS] [`directory.addUnixFSChild(child)`](https://github.com/ipfs/go-mfs/blob/v0.1.18/dir.go#L375)** - *Add child to inner UnixFS Directory*

87 +```mermaid

88 +graph TD

89 + Root --> Node1[Node]

90 + Root --> Node2[Node]

91 + Node1 --> Leaf1[Leaf]

92 + Node1 --> Leaf2[Leaf]

93 + Node2 --> Leaf3[Leaf]

94 +```

82 - The node is then added as a child to the inner `UnixFS` directory using the `(BasicDirectory).AddChild()` method.

96 +All leaves at similar depth. Good for random access - you can jump to any part of the file efficiently.

84 - > NOTE: This is not to be confused with the `directory.AddChild(filename, ipld.Node)`, as this operates on the `UnixFS` `BasicDirectory` object.

98 +**Trickle layout** (`--trickle`):

86 - - **[UnixFS] [`(BasicDirectory).AddChild(ctx, name, ipld.Node)`](https://github.com/ipfs/go-unixfs/blob/v1.1.16/io/directory.go#L137)** - *Add child to `BasicDirectory`*

100 +```mermaid

101 +graph TD

102 + Root --> Leaf1[Leaf]

103 + Root --> Node1[Node]

104 + Root --> Node2[Node]

105 + Node1 --> Leaf2[Leaf]

106 + Node2 --> Leaf3[Leaf]

107 +```

108

88 - > IMPORTANT: It should be noted that the `BasicDirectory` object uses the `ProtoNode` type object which is an implementation of the `ipld.Node` interface, seen and used throughout this document. Ideally the `ipld.Node` should always be used, unless we need access to specific functions from `ProtoNode` (like `Copy()`) that are not available in the interface.

109 +Leaves added progressively. Good for streaming - you can start reading before the whole file is added.

110

90 - This method first attempts to remove any old links (`ProtoNode.RemoveNodeLink(name)`) to the `ProtoNode` prior to adding a link to the newly added `ipld.Node`, using `ProtoNode.AddNodeLink(name, ipld.Node)`.

111 +### Step 3: Storing Blocks

112

92 - - **[Merkledag] [`AddNodeLink()`](https://github.com/ipfs/go-merkledag/blob/v1.1.15/node.go#L99)**

113 +As the DAG is built, each node is stored in the blockstore:

114

94 - The `AddNodeLink()` method is where an `ipld.Link` is created with the `ipld.Node`'s `CID` and size in the `ipld.MakeLink(ipld.Node)` method, and is then appended to the `ProtoNode`'s links in the `ProtoNode.AddRawLink(name)` method.

115 +- **Normal mode**: Data is copied into IPFS's internal storage (`~/.ipfs/blocks/`)

116 +- **Filestore mode** (`--nocopy`): Only references to the original file are stored (saves disk space but the original file must remain in place)

117

96 - - **[`adder.Finalize()`](https://github.com/ipfs/go-ipfs/blob/v0.4.18/core/coreunix/add.go#L200)** - *Fetch and return the __DAG__ **root** from the `MFS` and `UnixFS` directory*

118 +### Step 4: Pinning

119

98 - The `Finalize` method returns the `ipld.Node` from the `UnixFS` `Directory`.

120 +By default, added content is pinned (`ipfs add --pin=true`). This tells your IPFS node to keep this data - without pinning, content may eventually be removed to free up space.

121

100 - - **[`adder.PinRoot()`](https://github.com/ipfs/go-ipfs/blob/v0.4.18/core/coreunix/add.go#L171)** - *Pin all files under the `MFS` **root***

122 +### Alternative: Organizing with MFS

123

102 - The whole process ends with `PinRoot` recursively pinning all the files under the `MFS` **root**

124 +Instead of pinning, you can use the [Mutable File System (MFS)](https://docs.ipfs.tech/concepts/file-systems/#mutable-file-system-mfs) to organize content using familiar paths like `/photos/vacation.jpg` instead of raw CIDs:

125 +

126 +```bash

127 +# Add directly to MFS path

128 +ipfs add --to-files=/backups/ myfile.txt

129 +

130 +# Or copy an existing CID into MFS

131 +ipfs files cp /ipfs/QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u /docs/hello.txt

132 +```

133 +

134 +Content in MFS is implicitly pinned and stays organized across node restarts.

135 +

136 +## Options

137 +

138 +Run `ipfs add --help` to see all available options for controlling chunking, DAG layout, CID format, pinning behavior, and more.

139 +

140 +## UnixFS Format

141 +

142 +IPFS uses [UnixFS](https://specs.ipfs.tech/unixfs/) to represent files and directories. UnixFS is an abstraction layer that:

143 +

144 +- Gives names to raw data blobs (so you can have `/foo/bar.txt` instead of just hashes)

145 +- Represents directories as lists of named links to other nodes

146 +- Organizes large files as trees of smaller chunks

147 +- Makes these structures cryptographically verifiable - any tampering is detectable because it would change the hashes

148 +

149 +With `--raw-leaves`, leaf nodes store raw data without the UnixFS wrapper. This is more efficient and is the default when using CIDv1.

150 +

151 +## Code Architecture

152 +

153 +The add flow spans several layers:

154 +

155 +```mermaid

156 +flowchart TD

157 + subgraph CLI ["CLI Layer (kubo)"]

158 + A["core/commands/add.go parses flags, shows progress"]

159 + end

160 + subgraph API ["CoreAPI Layer (kubo)"]

161 + B["core/coreapi/unixfs.go UnixfsAPI.Add() entry point"]

162 + end

163 + subgraph Adder ["Adder (kubo)"]

164 + C["core/coreunix/add.go orchestrates chunking, DAG building, MFS, pinning"]

165 + end

166 + subgraph Boxo ["boxo libraries"]

167 + D["chunker/ - splits data into chunks"]

168 + E["ipld/unixfs/ - DAG layout and UnixFS format"]

169 + F["mfs/ - mutable filesystem abstraction"]

170 + G["pinning/ - pin management"]

171 + H["blockstore/ - block storage"]

172 + end

173 + A --> B --> C --> Boxo

174 +```

175 +

176 +### Key Files

177 +

178 +| Component | Location |

179 +|-----------|----------|

180 +| CLI command | `core/commands/add.go` |

181 +| API implementation | `core/coreapi/unixfs.go` |

182 +| Adder logic | `core/coreunix/add.go` |

183 +| Chunking | [boxo/chunker](https://github.com/ipfs/boxo/tree/main/chunker) |

184 +| DAG layouts | [boxo/ipld/unixfs/importer](https://github.com/ipfs/boxo/tree/main/ipld/unixfs/importer) |

185 +| MFS | [boxo/mfs](https://github.com/ipfs/boxo/tree/main/mfs) |

186 +| Pinning | [boxo/pinning/pinner](https://github.com/ipfs/boxo/tree/main/pinning/pinner) |

187 +

188 +### The Adder

189 +

190 +The `Adder` type in `core/coreunix/add.go` is the workhorse. It:

191 +

192 +1. **Creates an MFS root** - temporary in-memory filesystem for building the DAG

193 +2. **Processes files recursively** - chunks each file and builds DAG nodes

194 +3. **Commits to blockstore** - persists all blocks

195 +4. **Pins the result** - keeps content from being removed

196 +5. **Returns the root CID**

197 +

198 +Key methods:

199 +

200 +- `AddAllAndPin()` - main entry point

201 +- `addFileNode()` - handles a single file or directory

202 +- `add()` - chunks data and builds the DAG using boxo's layout builders

203 +

204 +## Further Reading

205 +

206 +- [UnixFS specification](https://specs.ipfs.tech/unixfs/)

207 +- [IPLD and Merkle DAGs](https://docs.ipfs.tech/concepts/merkle-dag/)

208 +- [Pinning](https://docs.ipfs.tech/concepts/persistence/)

209 +- [MFS (Mutable File System)](https://docs.ipfs.tech/concepts/file-systems/#mutable-file-system-mfs)

docs/customizing.md

+1 -1

@@ -45,7 +45,7 @@ This gives a more Go-centric dependency updating flow to building a new binary w

45 ## Bespoke Extension Points

46 Certain Kubo functionality may have their own extension points. For example:

48 -* Kubo supports the [Routing v1](https://github.com/ipfs/specs/blob/main/routing/ROUTING_V1_HTTP.md) API for delegating content routing to external processes

48 +* Kubo supports the [Routing v1](https://specs.ipfs.tech/routing/http-routing-v1/) API for delegating content routing to external processes

49 * Kubo supports the [Pinning Service API](https://github.com/ipfs/pinning-services-api-spec) for delegating pinning to external processes

50 * Kubo supports [DNSLink](https://dnslink.dev/) for delegating name->CID mappings to DNS

docs/generate-authors.sh deleted

-12

@@ -1,12 +0,0 @@

1 -#!/bin/bash

2 -set -e

3 -

4 -# see also ".mailmap" for how email addresses and names are deduplicated

5 -

6 -

7 -cat >AUTHORS <<-'EOF'

8 -# This file lists all individuals having contributed content to the repository.

9 -# For how it is generated, see `docs/generate-authors.sh`.

10 -

11 -EOF

12 -git log --format='%aN <%aE>' | LC_ALL=C.UTF-8 sort -uf >>AUTHORS