71b785f8e2 · matheusbsbs/matheus-bagliano

@@ -1,482 +0,0 @@

1 -Notes on Subproject Support

2 -===========================

3 -Junio C Hamano

4 -

5 -Scenario

6 ---------

7 -

8 -The examples in the following discussion show how this proposal

9 -plans to help this:

10 -

11 -. A project to build an embedded Linux appliance "gadget" is

12 - maintained with git.

13 -

14 -. The project uses linux-2.6 kernel as its subcomponent. It

15 - starts from a particular version of the mainline kernel, but

16 - adds its own code and build infrastructure to fit the

17 - appliance's needs.

18 -

19 -. The working tree of the project is laid out this way:

20 -+

21 -------------

22 - Makefile - Builds the whole thing.

23 - linux-2.6/ - The kernel, perhaps modified for the project.

24 - appliance/ - Applications that run on the appliance, and

25 - other bits.

26 -------------

27 -

28 -. The project is willing to maintain its own changes out of tree

29 - of the Linux kernel project, but would want to be able to feed

30 - the changes upstream, and incorporate upstream changes to its

31 - own tree, taking advantage of the fact that both itself and

32 - the Linux kernel project are version controlled with git.

33 -

34 -. To make the story a bit more interesting, later in the history

35 - of development, `linux-2.6/` and `appliance/` directories will

36 - be renamed to `kernel/` and `gadget/`.

37 -

38 -The idea here is to:

39 -

40 -. Keep `linux-2.6/` part as an independent project. The work by

41 - the project on the kernel part can be naturally exchanged with

42 - the other kernel developers this way. Specifically, a tree

43 - object contained in commit objects belonging to this sub-project

44 - does *not* have `linux-2.6/` directory at the top.

45 -

46 -. Keep the `appliance/` part as another independent project.

47 - Applications are supposed to be more or less independent from

48 - the kernel version, but some other bits might be tied to a

49 - specific kernel version. Again, a tree object contained in

50 - commit objects belonging to this sub-project does *not* have

51 - `appliance/` directory at the top.

52 -

53 -. Have another project that combines the whole thing together,

54 - so that the project can keep track of which versions of the

55 - parts are built together. The Makefile is illustrated above,

56 - but there might be other files and directories.

57 -

58 -We will call the project that binds things together the

59 -'toplevel project'. Other projects that hold `linux-2.6/` part

60 -and `appliance/` part are called 'subprojects'.

61 -

62 -

63 -Setting up

64 -----------

65 -

66 -Let's say we have been working on the appliance software,

67 -independently version controlled with git. Also the kernel part

68 -has been version controlled separately, like this:

69 -------------

70 -$ ls -dF current/*/.git current/*

71 -current/Makefile current/appliance/.git/ current/linux-2.6/.git/

72 -current/appliance/ current/linux-2.6/

73 -------------

74 -

75 -Now we would want to get a combined project. First we would

76 -clone from these repositories (which is not strictly needed --

77 -we could use `$GIT_ALTERNATE_OBJECT_DIRECTORIES` instead):

78 -

79 -------------

80 -$ mkdir combined && cd combined

81 -$ cp ../current/Makefile .

82 -$ git init-db

83 -$ mkdir -p .git/refs/subs/{kernel,gadget}/{heads,tags}

84 -$ git clone-pack ../current/linux-2.6/ master | read kernel_commit junk

85 -$ git clone-pack ../current/appliance/ master | read gadget_commit junk

86 -------------

87 -

88 -We will introduce a new command to set up a combined project:

89 -

90 -------------

91 -$ git bind-projects \

92 - $kernel_commit linux-2.6/ \

93 - $gadget_commit appliance/

94 -------------

95 -

96 -This would probably do an equivalent of:

97 -

98 -------------

99 -$ rm -f "$GIT_DIR/index"

100 -$ git read-tree --prefix=linux-2.6/ $kernel_commit

101 -$ git read-tree --prefix=appliance/ $gadget_commit

102 -$ git update-index --bind linux-2.6/ $kernel_commit

103 -$ git update-index --bind appliance/ $gadget_commit

104 -------------

105 -[NOTE]

106 -============

107 -Earlier outlines sent to the git mailing list talked

108 -about `$GIT_DIR/bind` to record what subproject are bound to

109 -which subtree in the current working tree and index. This

110 -proposal instead records that information in the index file

111 -with `update-index --bind` command.

112 -

113 -Also note that in this round of proposal, there is no separate

114 -branches that keep track of heads of subprojects.

115 -

116 -`update-index --bind` is not implemented on the core side yet;

117 -it would involve backward incompatible changes to the index

118 -format.

119 -============

120 -

121 -Let's not forget to add the `Makefile`, and check the whole

122 -thing out from the index file.

123 -------------

124 -$ git add Makefile

125 -$ git checkout-index -f -u -q -a

126 -------------

127 -

128 -Now our directory should be identical with the `current`

129 -directory. After making sure of that, we should be able to

130 -commit the whole thing:

131 -

132 -------------

133 -$ diff -x .git -r ../current ../combined

134 -$ git commit -m 'Initial toplevel project commit'

135 -------------

136 -

137 -Which should create a new commit object that records what is in

138 -the index file as its tree, with `bind` lines to record which

139 -subproject commit objects are bound at what subdirectory, and

140 -updates the `$GIT_DIR/refs/heads/master`. Such a commit object

141 -might look like this:

142 -------------

143 -tree 04803b09c300c8325258ccf2744115acc4c57067

144 -bind 5b2bcc7b2d546c636f79490655b3347acc91d17f linux-2.6/

145 -bind 0bdd79af62e8621359af08f0afca0ce977348ac7 appliance/

146 -author Junio C Hamano <junio@kernel.org> 1137965565 -0800

147 -committer Junio C Hamano <junio@kernel.org> 1137965565 -0800

148 -

149 -Initial toplevel project commit

150 -------------

151 -

152 -Notice that `Makefile` at the top is part of the toplevel

153 -project in this example, but it is not necessary. We could

154 -instead have the appliance subproject include this file. In

155 -such a setup, the appliance subproject would have had `Makefile`

156 -and `appliance/` directory at the toplevel. The `bind` line for

157 -that project would have said "the rest is bound at `/`" and

158 -`write-tree \--exclude=linux-2.6/` would have been used to write

159 -the tree for that subproject out of the combined index.

160 -

161 -

162 -Making further commits

163 -----------------------

164 -

165 -The easiest case is when you updated the Makefile without

166 -changing anything in the subprojects. In such a case, we just

167 -need to create a new commmit object that records the new tree

168 -with the current `HEAD` as its parent, and with the same set of

169 -`bind` lines.

170 -

171 -When we have changes to the subproject part, we would make a

172 -separate commit to the subproject part and then record the whole

173 -thing by making a commit to the toplevel project. The user

174 -interaction might go this way:

175 -------------

176 -$ git commit

177 -error: you have changes to the subproject bound at linux-2.6/.

178 -$ git commit --subproject linux-2.6/

179 -$ git commit

180 -------------

181 -

182 -With the new `\--subproject` option, the directory structure

183 -rooted at `linux-2.6/` part is written out as a tree, and a new

184 -commit object that records that tree object with the commit

185 -bound to that portion of the tree (`5b2bcc7b` in the above

186 -example) as its parent is created. Then the final `git commit`

187 -would record the whole tree with updated `bind` line for the

188 -`linux-2.6/` part.

189 -

190 -

191 -Checking out

192 -------------

193 -

194 -After cloning such a toplevel project, `git clone` without `-n`

195 -option would check out the working tree. This is done by

196 -reading the tree object recorded in the commit object (which

197 -records the whole thing), and adding the information from the

198 -"bind" line to the index file.

199 -

200 -------------

201 -$ cd ..

202 -$ git clone -n combined cloned ;# clone the one we created earlier

203 -$ cd cloned

204 -$ git checkout

205 -------------

206 -

207 -This round of proposal does not maintain separate branch heads

208 -for subprojects. The bound commits and their subdirectories

209 -are recorded in the index file from the commit object, so there

210 -is no need to do anything other than updating the index and the

211 -working tree.

212 -

213 -

214 -Switching branches

215 -------------------

216 -

217 -Along with the traditional two-way merge by `read-tree -m -u`,

218 -we would need to look at:

219 -

220 -. `bind` lines in the current `HEAD` commit.

221 -

222 -. `bind` lines in the commit we are switching to.

223 -

224 -. subproject binding information in the index file.

225 -

226 -to make sure we do sensible things.

227 -

228 -Just like until very recently we did not allow switching

229 -branches when two-way merge would lose local changes, we can

230 -start by refusing to switch branches when the subprojects bound

231 -in the index do not match what is recorded in the `HEAD` commit.

232 -

233 -Because in this round of the proposal we do not use the

234 -`$GIT_DIR/bind` file nor separate branches to keep track of

235 -heads of the subprojects, there is nothing else other than the

236 -working tree and the index file that needs to be updated when

237 -switching branches.

238 -

239 -

240 -Merging

241 --------

242 -

243 -Merging two branches of the toplevel projects can use the

244 -traditional merging mechanism mostly unchanged. The merge base

245 -computation can be done using the `parent` ancestry information

246 -taken from the two toplevel project branch heads being merged,

247 -and merging of the whole tree can be done with a three-way merge

248 -of the whole tree using the merge base and two head commits.

249 -For reasons described later, we would not merge the subproject

250 -parts of the trees during this step, though.

251 -

252 -When the two branch heads use different versions of subproject,

253 -things get a bit tricky. First, let's forget for a moment about

254 -the case where they bind the same project at different location.

255 -We would refuse if they do not have the same number of `bind`

256 -lines that bind something at the same subdirectories.

257 -

258 -------------

259 -$ git merge 'Merge in a side branch' HEAD side

260 -error: the merged heads have subprojects bound at different places.

261 - ours:

262 - linux-2.6/

263 - appliance/

264 - theirs:

265 - kernel/

266 - gadget/

267 - manual/

268 -------------

269 -

270 -Such renaming can be handled by first moving the bind points in

271 -our branch, and redoing the merge (this is a rare operation

272 -anyway). It might go like this:

273 -

274 -------------

275 -$ git reset

276 -$ git update-index --unbind linux-2.6/

277 -$ git update-index --unbind appliance/

278 -$ git update-index --bind $kernel_commit kernel/

279 -$ git update-index --bind $gadget_commit gadget/

280 -$ git commit -m 'Prepare for merge with side branch'

281 -$ git merge 'Merge in a side branch' HEAD side

282 -error: the merged heads have subprojects bound at different places.

283 - ours:

284 - kernel/

285 - gadget/

286 - theirs:

287 - kernel/

288 - gadget/

289 - manual/

290 -------------

291 -[NOTE]

292 -============

293 -Again, `update-index --unbind` is not implemented yet

294 -on the core side.

295 -============

296 -

297 -Their branch added another subproject, so this did not work (or

298 -it could be the other way around -- we might have been the one

299 -with `manual/` subproject while they didn't). This suggests

300 -that we may want an option to `git merge` to allow taking a

301 -union of subprojects. Again, this is a rare operation, and

302 -always taking a union would have created a toplevel project that

303 -had both `kernel/` and `linux-2.6/` bound to the same Linux

304 -kernel project from possibly different vintage, so it would be

305 -prudent to require the set of bound subprojects to exactly match

306 -and give the user an option to take a union.

307 -

308 -------------

309 -$ git merge --union-subprojects 'Merge in a side branch HEAD side

310 -error: the subproject at 'kernel/' needs to be merged first.

311 -------------

312 -

313 -Here, the version of the Linux kernel project in the `side`

314 -branch was different from what our branch had on our `bind`

315 -line. On what kind of difference should we give this error?

316 -Initially, I think we could require one is the fast forward of

317 -the other (ours might be ahead of theirs, or the other way

318 -around), and take the descendant.

319 -

320 -Or we could do an independent merge of subprojects heads, using

321 -the `parent` ancestry of the bound subproject heads to find

322 -their merge-base and doing a three-way merge. This would leave

323 -the merge result in the subproject part of the working tree and

324 -the index.

325 -

326 -[NOTE]

327 -This is the reason we did not do the whole-tree three way merge

328 -earlier. The subproject commit bound to the merge base commit

329 -used for the toplevel project may not be the merge base between

330 -the subproject commits bound to the two toplevel project

331 -commits.

332 -

333 -So let's deal with the case to merge only a subproject part into

334 -our tree first.

335 -

336 -

337 -Merging subprojects

338 --------------------

339 -

340 -An operation of more practical importance is to be able to merge

341 -in changes done outside to the projects bound to our toplevel

342 -project.

343 -

344 -------------

345 -$ git pull --subproject=kernel/ git://git.kernel.org/.../linux-2.6/

346 -------------

347 -

348 -might do:

349 -

350 -. fetch the current `HEAD` commit from Linus.

351 -. find the subproject commit bound at kernel/ subtree.

352 -. perform the usual three-way merge of these two commits, in

353 - `kernel/` part of the working tree.

354 -

355 -After that, `git commit \--subproject` option would be needed to

356 -make a commit.

357 -

358 -[NOTE]

359 -This suggests that we would need to have something similar to

360 -`MERGE_HEAD` for merging the subproject part. In the case of

361 -merging two toplevel project commits, we probably can read the

362 -`bind` lines from the `MERGE_HEAD` commit and either our `HEAD`

363 -commit or our index file. Further, we probably would require

364 -that the latter two must match, just as we currently require the

365 -index file matches our `HEAD` commit before `git merge`.

366 -

367 -Just like the current `pull = fetch + merge` semantics, the

368 -subproject aware version `git pull \--subproject=frotz/` would be

369 -a `git fetch \--subproject=frotz/` followed by a `git merge

370 -\--subproject=frotz/`. So the above would be:

371 -

372 -. Fetch the head.

373 -+

374 -------------

375 -$ git fetch --subproject=kernel/ git://git.kernel.org/.../linux-2.6/

376 -------------

377 -+

378 -which would fetch the commit chain from the remote repository, and

379 -write something like this to `FETCH_HEAD`:

380 -+

381 -------------

382 -3ee68c4...\tfor-merge-into kernel/\tbranch 'master' of git://.../linux-2.6

383 -------------

384 -

385 -. Run `git merge`.

386 -+

387 -------------

388 -$ git merge --subproject=kernel/ \

389 - 'Merge git://.../linux-2.6 into kernel/' HEAD 3ee68c4...

390 -------------

391 -

392 -. In case it does not cleanly automerge, `git merge` would write

393 -the necessary information for a later `git commit` to use in

394 -`MERGE_HEAD`. It may look like this:

395 -+

396 -------------

397 -3ee68c4af3fd7228c1be63254b9f884614f9ebb2 kernel/

398 -------------

399 -+

400 -Similarly, `MERGE_MSG` file will hold the merge message.

401 -

402 -With this, a later invocation of `git commit` to record the

403 -result of hand resolving would be able to notice that:

404 -

405 -. We should be first resolving `kernel/` subproject, not the

406 - whole thing.

407 -. The remote `HEAD` is `3ee68c4\...` commit.

408 -. The merge message is `Merge git://\.../linux-2.6 into kernel/`.

409 -

410 -and would make a merge commit, and register that resulting

411 -commit in the index file using `update-index \--bind` instead of

412 -updating *any* branch head.

413 -

414 -

415 -Management of Subprojects

416 --------------------------

417 -

418 -While the above as a mechanism would support version controlling

419 -of subprojects as a part of *one* larger toplevel project, it

420 -probably is worth pointing out that having a separate repository

421 -to manage the subproject independently would be a good idea.

422 -The same subproject can be incorporated into more than one

423 -toplevel projects, and after all, a subproject should be

424 -something that can stand on its own. In our example scenario,

425 -the `kernel/` project is used as a subproject for the "gadget"

426 -product, but at the same time, the organizaton that runs the

427 -"gadget" project may use Linux on their development machines,

428 -and have their own kernel hackers, not necessarily related to

429 -the use of the kernel in the "gadget" product.

430 -

431 -What this suggests is that not just we need to be able to pull

432 -the kernel development history *into* the subproject of the

433 -"gadget" project, but also we need to be able to push the

434 -development history of the kernel part alone *out* *of* the

435 -"gadget" project to another repository that deals only with the

436 -kernel part.

437 -

438 -It might go this way. First the setup:

439 -

440 -------------

441 -$ git clone git://git.kernel.org/.../linux-2.6 Linux

442 -$ ls -dF *

443 -cloned/ combined/ current/ Linux/

444 -------------

445 -

446 -That is, in addition to the `combined/` which we have been using

447 -to develop the "gadget" product in, we now have a repository for

448 -the kernel, cloned from Linus. In the previous section, we have

449 -outlined how we update the kernel subproject part of `combined/`

450 -repository from the `kernel.org` repository. The same procedure

451 -would work for pulling from `Linux/` repository here.

452 -

453 -We are now going the other way; propagate the kernel work done

454 -in the "gadget" project repository `combined/` back to `Linux/`.

455 -We might do this at the lowest level:

456 -

457 -------------

458 -$ cd combined

459 -$ git cat-file commit HEAD |

460 - sed -ne 's|^bind $[0-9a-f]*$ kernel/$|\1|p' >.git/refs/heads/linux26

461 -$ git push ../Linux linux26:master

462 -------------

463 -

464 -Or, more realistically, since the `Linux` project might already

465 -have their own commits on its `master`:

466 -

467 -------------

468 -$ cd Linux

469 -$ git pull ../combined linux26

470 -------------

471 -

472 -Either way we would need an easy way to maintain the `linux26`

473 -branch in the above example, and that will have to be part of

474 -the wrapper scripts like `git commit` (more likely, that would

475 -be a job for `git commit \--subproject`) for the usability's

476 -sake; in other words, the `cat-file commit` piped to `sed` above

477 -is not something the end user would do, but something that is

478 -done by the wrapper scripts.

479 -

480 -Hopefully the people who work in `Linux/` repository would run

481 -`format-patch` and feed their changes back to the kernel

482 -community.

Remove old subproject design notes.