doc: add large-object-promisors.adoc to the docs build
Commit 5040f9f164 ("doc: add technical design doc for large object promisors", 2025-02-18) added the large object promisors document as a technical document (with a '.txt' extension). The merge commit 2c6fd30198 ("Merge branch 'cc/lop-remote'", 2025-03-05) seems to have renamed the file with an '.adoc' extension. Despite the '.adoc' extension, this document was not being formatted by asciidoc(tor) as part of the docs build. In order to do so, add the document to the make and meson build files. Having added the document to the build, asciidoc and asciidoctor find (slightly different) problems with the syntax of the input document. The first set of warnings (only issued by asciidoc) relate to some 'section title out of sequence: expected level 3, got level 4'. This document uses 'setext' style of section headers, using a series of underline characters, where the character used denotes the level of the title. From document title to level 5 (see [1]), these characters are =, -, ~, ^, +. This does not seem to fit the error message, which implies that those characters denote levels 0 -> 4. Replacing the headings underlined with '+' by the '^' character eliminates these warnings. The second set of warnings (only issued by asciidoctor) relate to some headings which seem to use both arabic and roman numerals as part of a single 'list' sequence. This elicited either 'unterminated listing block' or (for example) 'list item index: expected I, got II' warnings. In order not to mix arabic and roman numerals, remove the numeral from the '0) Non goals' heading. Similarly, the remaining roman numeral entries had the ')' removed and turned into regular headings with I, II, III ... at the beginning. [1] https://asciidoctor.org/docs/asciidoc-recommended-practices/ Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ramsay Jones committed
Oct 16, 2025 at 21:03 UTC
1c1fc86d5557fbbf5ac6ccbaddd824e24fb2b91d
3 files changed
+34
-32
Documentation/Makefile
+1
@@ -122,6 +122,7 @@ TECH_DOCS += technical/bundle-uri
122
TECH_DOCS += technical/commit-graph
123
TECH_DOCS += technical/directory-rename-detection
124
TECH_DOCS += technical/hash-function-transition
125
+TECH_DOCS += technical/large-object-promisors
126
TECH_DOCS += technical/long-running-process-protocol
127
TECH_DOCS += technical/multi-pack-index
128
TECH_DOCS += technical/packfile-uri
Documentation/technical/large-object-promisors.adoc
+32
-32
@@ -34,8 +34,8 @@ a new object representation for large blobs as discussed in:
34
35
https://lore.kernel.org/git/xmqqbkdometi.fsf@gitster.g/
36
37
-0) Non goals
38
-------------
37
+Non goals
38
+---------
39
40
- We will not discuss those client side improvements here, as they
41
would require changes in different parts of Git than this effort.
@@ -90,8 +90,8 @@ later in this document:
90
even more to host content with larger blobs or more large blobs
91
than currently.
92
93
-I) Issues with the current situation
94
-------------------------------------
93
+I Issues with the current situation
94
+-----------------------------------
95
96
- Some statistics made on GitLab repos have shown that more than 75%
97
of the disk space is used by blobs that are larger than 1MB and
@@ -138,8 +138,8 @@ I) Issues with the current situation
138
complaining that these tools require significant effort to set up,
139
learn and use correctly.
140
141
-II) Main features of the "Large Object Promisors" solution
142
-----------------------------------------------------------
141
+II Main features of the "Large Object Promisors" solution
142
+---------------------------------------------------------
143
144
The main features below should give a rough overview of how the
145
solution may work. Details about needed elements can be found in
@@ -166,7 +166,7 @@ format. They should be used along with main remotes that contain the
166
other objects.
167
168
Note 1
169
-++++++
169
+^^^^^^
170
171
To clarify, a LOP is a normal promisor remote, except that:
172
@@ -178,7 +178,7 @@ To clarify, a LOP is a normal promisor remote, except that:
178
itself.
179
180
Note 2
181
-++++++
181
+^^^^^^
182
183
Git already makes it possible for a main remote to also be a promisor
184
remote storing both regular objects and large blobs for a client that
@@ -186,13 +186,13 @@ clones from it with a filter on blob size. But here we explicitly want
186
to avoid that.
187
188
Rationale
189
-+++++++++
189
+^^^^^^^^^
190
191
LOPs aim to be good at handling large blobs while main remotes are
192
already good at handling other objects.
193
194
Implementation
195
-++++++++++++++
195
+^^^^^^^^^^^^^^
196
197
Git already has support for multiple promisor remotes, see
198
link:partial-clone.html#using-many-promisor-remotes[the partial clone documentation].
@@ -213,19 +213,19 @@ remote helper (see linkgit:gitremote-helpers[7]) which makes the
213
underlying object storage appear like a remote to Git.
214
215
Note
216
-++++
216
+^^^^
217
218
A LOP can be a promisor remote accessed using a remote helper by
219
both some clients and the main remote.
220
221
Rationale
222
-+++++++++
222
+^^^^^^^^^
223
224
This looks like the simplest way to create LOPs that can cheaply
225
handle many large blobs.
226
227
Implementation
228
-++++++++++++++
228
+^^^^^^^^^^^^^^
229
230
Remote helpers are quite easy to write as shell scripts, but it might
231
be more efficient and maintainable to write them using other languages
@@ -247,7 +247,7 @@ The underlying object storage that a LOP uses could also serve as
247
storage for large files handled by Git LFS.
248
249
Rationale
250
-+++++++++
250
+^^^^^^^^^
251
252
This would simplify the server side if it wants to both use a LOP and
253
act as a Git LFS server.
@@ -259,7 +259,7 @@ On the server side, a main remote should have a way to offload to a
259
LOP all its blobs with a size over a configurable threshold.
260
261
Rationale
262
-+++++++++
262
+^^^^^^^^^
263
264
This makes it easy to set things up and to clean things up. For
265
example, an admin could use this to manually convert a repo not using
@@ -268,7 +268,7 @@ some users would sometimes push large blobs, a cron job could use this
268
to regularly make sure the large blobs are moved to the LOP.
269
270
Implementation
271
-++++++++++++++
271
+^^^^^^^^^^^^^^
272
273
Using something based on `git repack --filter=...` to separate the
274
blobs we want to offload from the other Git objects could be a good
@@ -284,13 +284,13 @@ should have ways to prevent oversize blobs to be fetched, and also
284
perhaps pushed, into it.
285
286
Rationale
287
-+++++++++
287
+^^^^^^^^^
288
289
A main remote containing many oversize blobs would defeat the purpose
290
of LOPs.
291
292
Implementation
293
-++++++++++++++
293
+^^^^^^^^^^^^^^
294
295
The way to offload to a LOP discussed in 4) above can be used to
296
regularly offload oversize blobs. About preventing oversize blobs from
@@ -326,18 +326,18 @@ large blobs directly from the LOP and the server would not need to
326
fetch those blobs from the LOP to be able to serve the client.
327
328
Note
329
-++++
329
+^^^^
330
331
For fetches instead of clones, a protocol negotiation might not always
332
happen, see the "What about fetches?" FAQ entry below for details.
333
334
Rationale
335
-+++++++++
335
+^^^^^^^^^
336
337
Security, configurability and efficiency of setting things up.
338
339
Implementation
340
-++++++++++++++
340
+^^^^^^^^^^^^^^
341
342
A "promisor-remote" protocol v2 capability looks like a good way to
343
implement this. The way the client and server use this capability
@@ -356,7 +356,7 @@ the client should be able to offload some large blobs it has fetched,
356
but might not need anymore, to the LOP.
357
358
Note
359
-++++
359
+^^^^
360
361
It might depend on the context if it should be OK or not for clients
362
to offload large blobs they have created, instead of fetched, directly
@@ -367,13 +367,13 @@ This should be discussed and refined when we get closer to
367
implementing this feature.
368
369
Rationale
370
-+++++++++
370
+^^^^^^^^^
371
372
On the client, the easiest way to deal with unneeded large blobs is to
373
offload them.
374
375
Implementation
376
-++++++++++++++
376
+^^^^^^^^^^^^^^
377
378
This is very similar to what 4) above is about, except on the client
379
side instead of the server side. So a good solution to 4) could likely
@@ -385,8 +385,8 @@ when cloning (see 6) above). Also if the large blobs were fetched from
385
a LOP, it is likely, and can easily be confirmed, that the LOP still
386
has them, so that they can just be removed from the client.
387
388
-III) Benefits of using LOPs
389
----------------------------
388
+III Benefits of using LOPs
389
+--------------------------
390
391
Many benefits are related to the issues discussed in "I) Issues with
392
the current situation" above:
@@ -406,8 +406,8 @@ the current situation" above:
406
407
- Reduced storage needs on the client side.
408
409
-IV) FAQ
410
--------
409
+IV FAQ
410
+------
411
412
What about using multiple LOPs on the server and client side?
413
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -533,7 +533,7 @@ some objects it already knows about but doesn't have because they are
533
on a promisor remote.
534
535
Regular fetch
536
-+++++++++++++
536
+^^^^^^^^^^^^^
537
538
In a regular fetch, the client will contact the main remote and a
539
protocol negotiation will happen between them. It's a good thing that
@@ -551,7 +551,7 @@ new fetch will happen in the same way as the previous clone or fetch,
551
using, or not using, the same LOP(s) as last time.
552
553
"Backfill" or "lazy" fetch
554
-++++++++++++++++++++++++++
554
+^^^^^^^^^^^^^^^^^^^^^^^^^^
555
556
When there is a backfill fetch, the client doesn't necessarily contact
557
the main remote first. It will try to fetch from its promisor remotes
@@ -576,8 +576,8 @@ from the client when it fetches from them. The client could get the
576
token when performing a protocol negotiation with the main remote (see
577
section II.6 above).
578
579
-V) Future improvements
580
-----------------------
579
+V Future improvements
580
+---------------------
581
582
It is expected that at the beginning using LOPs will be mostly worth
583
it either in a corporate context where the Git version that clients
Documentation/technical/meson.build
+1
@@ -13,6 +13,7 @@ articles = [
13
'commit-graph.adoc',
14
'directory-rename-detection.adoc',
15
'hash-function-transition.adoc',
16
+ 'large-object-promisors.adoc',
17
'long-running-process-protocol.adoc',
18
'multi-pack-index.adoc',
19
'packfile-uri.adoc',