@cryptotaxi247 / kubo
Code Commits Issues Pulls Sessions CI/CD
master
master
md 134 lines 5.11 KB
Rendered Raw
1 # IPFS Repository Migrations
2
3 This directory contains the migration system for IPFS repositories, handling both embedded and external migrations.
4
5 ## Migration System Overview
6
7 ### Embedded vs External Migrations
8
9 Starting from **repo version 17**, Kubo uses **embedded migrations** that are built into the binary, eliminating the need to download external migration tools.
10
11 - **Repo versions <17**: Use external binary migrations downloaded from fs-repo-migrations
12 - **Repo version 17+**: Use embedded migrations built into Kubo
13
14 ### Migration Functions
15
16 #### `migrations.RunEmbeddedMigrations()`
17 - **Purpose**: Runs migrations that are embedded directly in the Kubo binary
18 - **Scope**: Handles repo version 17+ migrations
19 - **Performance**: Fast execution, no network downloads required
20 - **Dependencies**: Self-contained, uses only Kubo's internal dependencies
21 - **Usage**: Primary migration method for modern repo versions
22
23 **Parameters**:
24 - `ctx`: Context for cancellation and timeouts
25 - `targetVersion`: Target repository version to migrate to
26 - `repoPath`: Path to the IPFS repository directory
27 - `allowDowngrade`: Whether to allow downgrade migrations
28
29 ```go
30 err = migrations.RunEmbeddedMigrations(ctx, targetVersion, repoPath, allowDowngrade)
31 if err != nil {
32 // Handle migration failure, may fall back to external migrations
33 }
34 ```
35
36 #### `migrations.RunMigration()` with `migrations.ReadMigrationConfig()`
37 - **Purpose**: Runs external binary migrations downloaded from fs-repo-migrations
38 - **Scope**: Handles legacy repo versions <17 and serves as fallback
39 - **Performance**: Slower due to network downloads and external process execution
40 - **Dependencies**: Requires fs-repo-migrations binaries and network access
41 - **Usage**: Fallback method for legacy migrations
42
43 ```go
44 // Read migration configuration for external migrations
45 migrationCfg, err := migrations.ReadMigrationConfig(repoPath, configFile)
46 fetcher, err := migrations.GetMigrationFetcher(migrationCfg.DownloadSources, ...)
47 err = migrations.RunMigration(ctx, fetcher, targetVersion, repoPath, allowDowngrade)
48 ```
49
50 ## Migration Flow in Daemon Startup
51
52 1. **Primary**: Try embedded migrations first (`RunEmbeddedMigrations`)
53 2. **Fallback**: If embedded migration fails, fall back to external migrations (`RunMigration`)
54 3. **Legacy Support**: External migrations ensure compatibility with older repo versions
55
56 ## Directory Structure
57
58 ```
59 repo/fsrepo/migrations/
60 ├── README.md # This file
61 ├── embedded.go # Embedded migration system
62 ├── embedded_test.go # Tests for embedded migrations
63 ├── migrations.go # External migration system
64 ├── fs-repo-16-to-17/ # First embedded migration (16→17)
65 │ ├── migration/
66 │ │ ├── migration.go # Migration logic
67 │ │ └── migration_test.go # Migration tests
68 │ ├── atomicfile/
69 │ │ └── atomicfile.go # Atomic file operations
70 │ ├── main.go # Standalone migration binary
71 │ └── README.md # Migration-specific documentation
72 └── [other migration utilities]
73 ```
74
75 ## Adding New Embedded Migrations
76
77 To add a new embedded migration (e.g., fs-repo-17-to-18):
78
79 1. **Create migration package**: `fs-repo-17-to-18/migration/migration.go`
80 2. **Implement interface**: Ensure your migration implements the `EmbeddedMigration` interface
81 3. **Register migration**: Add to `embeddedMigrations` map in `embedded.go`
82 4. **Add tests**: Create comprehensive tests for your migration logic
83 5. **Update repo version**: Increment `RepoVersion` in `fsrepo.go`
84
85 ```go
86 // In embedded.go
87 var embeddedMigrations = map[string]EmbeddedMigration{
88 "fs-repo-16-to-17": &mg16.Migration{},
89 "fs-repo-17-to-18": &mg17.Migration{}, // Add new migration
90 }
91 ```
92
93 ## Migration Requirements
94
95 Each embedded migration must:
96 - Implement the `EmbeddedMigration` interface
97 - Be reversible with proper backup handling
98 - Use atomic file operations to prevent corruption
99 - Preserve user customizations
100 - Include comprehensive tests
101 - Follow the established naming pattern
102
103 ## External Migration Support
104
105 External migrations are maintained for:
106 - **Backward compatibility** with repo versions <17
107 - **Fallback mechanism** if embedded migrations fail
108 - **Legacy installations** that cannot be upgraded directly
109
110 The external migration system will continue to work but is not the preferred method for new migrations.
111
112 ## Security and Safety
113
114 All migrations (embedded and external) include:
115 - **Atomic operations**: Prevent repository corruption
116 - **Backup creation**: Allow rollback if migration fails
117 - **Version validation**: Ensure migrations run on correct repo versions
118 - **Error handling**: Graceful failure with informative messages
119 - **User preservation**: Maintain custom configurations during migration
120
121 ## Testing
122
123 Test both embedded and external migration systems:
124
125 ```bash
126 # Test embedded migrations
127 go test ./repo/fsrepo/migrations/ -run TestEmbedded
128
129 # Test specific migration
130 go test ./repo/fsrepo/migrations/fs-repo-16-to-17/migration/
131
132 # Test migration registration
133 go test ./repo/fsrepo/migrations/ -run TestHasEmbedded
134 ```