| 1 | gitprotocol-v2(5) |
| 2 | ================= |
| 3 | |
| 4 | NAME |
| 5 | ---- |
| 6 | gitprotocol-v2 - Git Wire Protocol, Version 2 |
| 7 | |
| 8 | SYNOPSIS |
| 9 | -------- |
| 10 | [verse] |
| 11 | <over-the-wire-protocol> |
| 12 | |
| 13 | DESCRIPTION |
| 14 | ----------- |
| 15 | |
| 16 | This document presents a specification for a version 2 of Git's wire |
| 17 | protocol. Protocol v2 will improve upon v1 in the following ways: |
| 18 | |
| 19 | * Instead of multiple service names, multiple commands will be |
| 20 | supported by a single service |
| 21 | * Easily extendable as capabilities are moved into their own section |
| 22 | of the protocol, no longer being hidden behind a NUL byte and |
| 23 | limited by the size of a pkt-line |
| 24 | * Separate out other information hidden behind NUL bytes (e.g. agent |
| 25 | string as a capability and symrefs can be requested using 'ls-refs') |
| 26 | * Reference advertisement will be omitted unless explicitly requested |
| 27 | * ls-refs command to explicitly request some refs |
| 28 | * Designed with http and stateless-rpc in mind. With clear flush |
| 29 | semantics the http remote helper can simply act as a proxy |
| 30 | |
| 31 | In protocol v2 communication is command oriented. When first contacting a |
| 32 | server a list of capabilities will be advertised. Some of these capabilities |
| 33 | will be commands which a client can request be executed. Once a command |
| 34 | has completed, a client can reuse the connection and request that other |
| 35 | commands be executed. |
| 36 | |
| 37 | Packet-Line Framing |
| 38 | ------------------- |
| 39 | |
| 40 | All communication is done using packet-line framing, just as in v1. See |
| 41 | linkgit:gitprotocol-pack[5] and linkgit:gitprotocol-common[5] for more information. |
| 42 | |
| 43 | In protocol v2 these special packets will have the following semantics: |
| 44 | |
| 45 | * '0000' Flush Packet (flush-pkt) - indicates the end of a message |
| 46 | * '0001' Delimiter Packet (delim-pkt) - separates sections of a message |
| 47 | * '0002' Response End Packet (response-end-pkt) - indicates the end of a |
| 48 | response for stateless connections |
| 49 | |
| 50 | Initial Client Request |
| 51 | ---------------------- |
| 52 | |
| 53 | In general a client can request to speak protocol v2 by sending |
| 54 | `version=2` through the respective side-channel for the transport being |
| 55 | used which inevitably sets `GIT_PROTOCOL`. More information can be |
| 56 | found in linkgit:gitprotocol-pack[5] and linkgit:gitprotocol-http[5], as well as the |
| 57 | `GIT_PROTOCOL` definition in linkgit:git[1]. In all cases the |
| 58 | response from the server is the capability advertisement. |
| 59 | |
| 60 | Git Transport |
| 61 | ~~~~~~~~~~~~~ |
| 62 | |
| 63 | When using the git:// transport, you can request to use protocol v2 by |
| 64 | sending "version=2" as an extra parameter: |
| 65 | |
| 66 | 003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0 |
| 67 | |
| 68 | SSH and File Transport |
| 69 | ~~~~~~~~~~~~~~~~~~~~~~ |
| 70 | |
| 71 | When using either the ssh:// or file:// transport, the GIT_PROTOCOL |
| 72 | environment variable must be set explicitly to include "version=2". |
| 73 | The server may need to be configured to allow this environment variable |
| 74 | to pass. |
| 75 | |
| 76 | HTTP Transport |
| 77 | ~~~~~~~~~~~~~~ |
| 78 | |
| 79 | When using the http:// or https:// transport a client makes a "smart" |
| 80 | info/refs request as described in linkgit:gitprotocol-http[5] and requests that |
| 81 | v2 be used by supplying "version=2" in the `Git-Protocol` header. |
| 82 | |
| 83 | C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0 |
| 84 | C: Git-Protocol: version=2 |
| 85 | |
| 86 | A v2 server would reply: |
| 87 | |
| 88 | S: 200 OK |
| 89 | S: <Some headers> |
| 90 | S: ... |
| 91 | S: |
| 92 | S: 000eversion 2\n |
| 93 | S: <capability-advertisement> |
| 94 | |
| 95 | Subsequent requests are then made directly to the service |
| 96 | `$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack). |
| 97 | |
| 98 | Uses the `--http-backend-info-refs` option to |
| 99 | linkgit:git-upload-pack[1]. |
| 100 | |
| 101 | The server may need to be configured to pass this header's contents via |
| 102 | the `GIT_PROTOCOL` variable. See the discussion in linkgit:git-http-backend[1]. |
| 103 | |
| 104 | Capability Advertisement |
| 105 | ------------------------ |
| 106 | |
| 107 | A server which decides to communicate (based on a request from a client) |
| 108 | using protocol version 2, notifies the client by sending a version string |
| 109 | in its initial response followed by an advertisement of its capabilities. |
| 110 | Each capability is a key with an optional value. Clients must ignore all |
| 111 | unknown keys. Semantics of unknown values are left to the definition of |
| 112 | each key. Some capabilities will describe commands which can be requested |
| 113 | to be executed by the client. |
| 114 | |
| 115 | capability-advertisement = protocol-version |
| 116 | capability-list |
| 117 | flush-pkt |
| 118 | |
| 119 | protocol-version = PKT-LINE("version 2" LF) |
| 120 | capability-list = *capability |
| 121 | capability = PKT-LINE(key[=value] LF) |
| 122 | |
| 123 | key = 1*(ALPHA | DIGIT | "-_") |
| 124 | value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;") |
| 125 | |
| 126 | Command Request |
| 127 | --------------- |
| 128 | |
| 129 | After receiving the capability advertisement, a client can then issue a |
| 130 | request to select the command it wants with any particular capabilities |
| 131 | or arguments. There is then an optional section where the client can |
| 132 | provide any command specific parameters or queries. Only a single |
| 133 | command can be requested at a time. |
| 134 | |
| 135 | request = empty-request | command-request |
| 136 | empty-request = flush-pkt |
| 137 | command-request = command |
| 138 | capability-list |
| 139 | delim-pkt |
| 140 | command-args |
| 141 | flush-pkt |
| 142 | command = PKT-LINE("command=" key LF) |
| 143 | command-args = *command-specific-arg |
| 144 | |
| 145 | command-specific-args are packet line framed arguments defined by |
| 146 | each individual command. |
| 147 | |
| 148 | The server will then check to ensure that the client's request is |
| 149 | comprised of a valid command as well as valid capabilities which were |
| 150 | advertised. If the request is valid the server will then execute the |
| 151 | command. A server MUST wait till it has received the client's entire |
| 152 | request before issuing a response. The format of the response is |
| 153 | determined by the command being executed, but in all cases a flush-pkt |
| 154 | indicates the end of the response. |
| 155 | |
| 156 | When a command has finished, and the client has received the entire |
| 157 | response from the server, a client can either request that another |
| 158 | command be executed or can terminate the connection. A client may |
| 159 | optionally send an empty request consisting of just a flush-pkt to |
| 160 | indicate that no more requests will be made. |
| 161 | |
| 162 | Capabilities |
| 163 | ------------ |
| 164 | |
| 165 | There are two different types of capabilities: normal capabilities, |
| 166 | which can be used to convey information or alter the behavior of a |
| 167 | request, and commands, which are the core actions that a client wants to |
| 168 | perform (fetch, push, etc). |
| 169 | |
| 170 | Protocol version 2 is stateless by default. This means that all commands |
| 171 | must only last a single round and be stateless from the perspective of the |
| 172 | server side, unless the client has requested a capability indicating that |
| 173 | state should be maintained by the server. Clients MUST NOT require state |
| 174 | management on the server side in order to function correctly. This |
| 175 | permits simple round-robin load-balancing on the server side, without |
| 176 | needing to worry about state management. |
| 177 | |
| 178 | agent |
| 179 | ~~~~~ |
| 180 | |
| 181 | The server can advertise the `agent` capability with a value `X` (in the |
| 182 | form `agent=X`) to notify the client that the server is running version |
| 183 | `X`. The client may optionally send its own agent string by including |
| 184 | the `agent` capability with a value `Y` (in the form `agent=Y`) in its |
| 185 | request to the server (but it MUST NOT do so if the server did not |
| 186 | advertise the agent capability). The `X` and `Y` strings may contain any |
| 187 | printable ASCII characters except space (i.e., the byte range 33 <= x <= |
| 188 | 126), and are typically of the form "package/version-os" (e.g., |
| 189 | "git/1.8.3.1-Linux") where `os` is the operating system name (e.g., |
| 190 | "Linux"). `X` and `Y` can be configured using the GIT_USER_AGENT |
| 191 | environment variable and it takes priority. The `os` is |
| 192 | retrieved using the 'sysname' field of the `uname(2)` system call |
| 193 | or its equivalent. The agent strings are purely informative for statistics |
| 194 | and debugging purposes, and MUST NOT be used to programmatically assume |
| 195 | the presence or absence of particular features. |
| 196 | |
| 197 | ls-refs |
| 198 | ~~~~~~~ |
| 199 | |
| 200 | `ls-refs` is the command used to request a reference advertisement in v2. |
| 201 | Unlike the current reference advertisement, ls-refs takes in arguments |
| 202 | which can be used to limit the refs sent from the server. |
| 203 | |
| 204 | Additional features not supported in the base command will be advertised |
| 205 | as the value of the command in the capability advertisement in the form |
| 206 | of a space separated list of features: "<command>=<feature-1> <feature-2>" |
| 207 | |
| 208 | ls-refs takes in the following arguments: |
| 209 | |
| 210 | symrefs |
| 211 | In addition to the object pointed by it, show the underlying ref |
| 212 | pointed by it when showing a symbolic ref. |
| 213 | peel |
| 214 | Show peeled tags. |
| 215 | ref-prefix <prefix> |
| 216 | When specified, only references having a prefix matching one of |
| 217 | the provided prefixes are displayed. Multiple instances may be |
| 218 | given, in which case references matching any prefix will be |
| 219 | shown. Note that this is purely for optimization; a server MAY |
| 220 | show refs not matching the prefix if it chooses, and clients |
| 221 | should filter the result themselves. |
| 222 | |
| 223 | If the 'unborn' feature is advertised the following argument can be |
| 224 | included in the client's request. |
| 225 | |
| 226 | unborn |
| 227 | The server will send information about HEAD even if it is a symref |
| 228 | pointing to an unborn branch in the form "unborn HEAD |
| 229 | symref-target:<target>". |
| 230 | |
| 231 | The output of ls-refs is as follows: |
| 232 | |
| 233 | output = *ref |
| 234 | flush-pkt |
| 235 | obj-id-or-unborn = (obj-id | "unborn") |
| 236 | ref = PKT-LINE(obj-id-or-unborn SP refname *(SP ref-attribute) LF) |
| 237 | ref-attribute = (symref | peeled) |
| 238 | symref = "symref-target:" symref-target |
| 239 | peeled = "peeled:" obj-id |
| 240 | |
| 241 | fetch |
| 242 | ~~~~~ |
| 243 | |
| 244 | `fetch` is the command used to fetch a packfile in v2. It can be looked |
| 245 | at as a modified version of the v1 fetch where the ref-advertisement is |
| 246 | stripped out (since the `ls-refs` command fills that role) and the |
| 247 | message format is tweaked to eliminate redundancies and permit easy |
| 248 | addition of future extensions. |
| 249 | |
| 250 | Additional features not supported in the base command will be advertised |
| 251 | as the value of the command in the capability advertisement in the form |
| 252 | of a space separated list of features: "<command>=<feature-1> <feature-2>" |
| 253 | |
| 254 | A `fetch` request can take the following arguments: |
| 255 | |
| 256 | want <oid> |
| 257 | Indicates to the server an object which the client wants to |
| 258 | retrieve. Wants can be anything and are not limited to |
| 259 | advertised objects. |
| 260 | |
| 261 | have <oid> |
| 262 | Indicates to the server an object which the client has locally. |
| 263 | This allows the server to make a packfile which only contains |
| 264 | the objects that the client needs. Multiple 'have' lines can be |
| 265 | supplied. |
| 266 | |
| 267 | done |
| 268 | Indicates to the server that negotiation should terminate (or |
| 269 | not even begin if performing a clone) and that the server should |
| 270 | use the information supplied in the request to construct the |
| 271 | packfile. |
| 272 | |
| 273 | thin-pack |
| 274 | Request that a thin pack be sent, which is a pack with deltas |
| 275 | which reference base objects not contained within the pack (but |
| 276 | are known to exist at the receiving end). This can reduce the |
| 277 | network traffic significantly, but it requires the receiving end |
| 278 | to know how to "thicken" these packs by adding the missing bases |
| 279 | to the pack. |
| 280 | |
| 281 | no-progress |
| 282 | Request that progress information that would normally be sent on |
| 283 | side-band channel 2, during the packfile transfer, should not be |
| 284 | sent. However, the side-band channel 3 is still used for error |
| 285 | responses. |
| 286 | |
| 287 | include-tag |
| 288 | Request that annotated tags should be sent if the objects they |
| 289 | point to are being sent. |
| 290 | |
| 291 | ofs-delta |
| 292 | Indicate that the client understands PACKv2 with delta referring |
| 293 | to its base by position in pack rather than by an oid. That is, |
| 294 | they can read OBJ_OFS_DELTA (aka type 6) in a packfile. |
| 295 | |
| 296 | If the 'shallow' feature is advertised the following arguments can be |
| 297 | included in the clients request as well as the potential addition of the |
| 298 | 'shallow-info' section in the server's response as explained below. |
| 299 | |
| 300 | shallow <oid> |
| 301 | A client must notify the server of all commits for which it only |
| 302 | has shallow copies (meaning that it doesn't have the parents of |
| 303 | a commit) by supplying a 'shallow <oid>' line for each such |
| 304 | object so that the server is aware of the limitations of the |
| 305 | client's history. This is so that the server is aware that the |
| 306 | client may not have all objects reachable from such commits. |
| 307 | |
| 308 | deepen <depth> |
| 309 | Requests that the fetch/clone should be shallow having a commit |
| 310 | depth of <depth> relative to the remote side. |
| 311 | |
| 312 | deepen-relative |
| 313 | Requests that the semantics of the "deepen" command be changed |
| 314 | to indicate that the depth requested is relative to the client's |
| 315 | current shallow boundary, instead of relative to the requested |
| 316 | commits. |
| 317 | |
| 318 | deepen-since <timestamp> |
| 319 | Requests that the shallow clone/fetch should be cut at a |
| 320 | specific time, instead of depth. Internally it's equivalent to |
| 321 | doing "git rev-list --max-age=<timestamp>". Cannot be used with |
| 322 | "deepen". |
| 323 | |
| 324 | deepen-not <rev> |
| 325 | Requests that the shallow clone/fetch should be cut at a |
| 326 | specific revision specified by '<rev>', instead of a depth. |
| 327 | Internally it's equivalent of doing "git rev-list --not <rev>". |
| 328 | Cannot be used with "deepen", but can be used with |
| 329 | "deepen-since". |
| 330 | |
| 331 | If the 'filter' feature is advertised, the following argument can be |
| 332 | included in the client's request: |
| 333 | |
| 334 | filter <filter-spec> |
| 335 | Request that various objects from the packfile be omitted |
| 336 | using one of several filtering techniques. These are intended |
| 337 | for use with partial clone and partial fetch operations. See |
| 338 | `rev-list` for possible "filter-spec" values. When communicating |
| 339 | with other processes, senders SHOULD translate scaled integers |
| 340 | (e.g. "1k") into a fully-expanded form (e.g. "1024") to aid |
| 341 | interoperability with older receivers that may not understand |
| 342 | newly-invented scaling suffixes. However, receivers SHOULD |
| 343 | accept the following suffixes: 'k', 'm', and 'g' for 1024, |
| 344 | 1048576, and 1073741824, respectively. |
| 345 | |
| 346 | If the 'ref-in-want' feature is advertised, the following argument can |
| 347 | be included in the client's request as well as the potential addition of |
| 348 | the 'wanted-refs' section in the server's response as explained below. |
| 349 | |
| 350 | want-ref <ref> |
| 351 | Indicates to the server that the client wants to retrieve a |
| 352 | particular ref, where <ref> is the full name of a ref on the |
| 353 | server. It is a protocol error to send want-ref for the |
| 354 | same ref more than once. |
| 355 | |
| 356 | If the 'sideband-all' feature is advertised, the following argument can be |
| 357 | included in the client's request: |
| 358 | |
| 359 | sideband-all |
| 360 | Instruct the server to send the whole response multiplexed, not just |
| 361 | the packfile section. All non-flush and non-delim PKT-LINE in the |
| 362 | response (not only in the packfile section) will then start with a byte |
| 363 | indicating its sideband (1, 2, or 3), and the server may send "0005\2" |
| 364 | (a PKT-LINE of sideband 2 with no payload) as a keepalive packet. |
| 365 | |
| 366 | If the 'packfile-uris' feature is advertised, the following argument |
| 367 | can be included in the client's request as well as the potential |
| 368 | addition of the 'packfile-uris' section in the server's response as |
| 369 | explained below. Note that at most one `packfile-uris` line can be sent |
| 370 | to the server. |
| 371 | |
| 372 | packfile-uris <comma-separated-list-of-protocols> |
| 373 | Indicates to the server that the client is willing to receive |
| 374 | URIs of any of the given protocols in place of objects in the |
| 375 | sent packfile. Before performing the connectivity check, the |
| 376 | client should download from all given URIs. Currently, the |
| 377 | protocols supported are "http" and "https". |
| 378 | |
| 379 | If the 'wait-for-done' feature is advertised, the following argument |
| 380 | can be included in the client's request. |
| 381 | |
| 382 | wait-for-done |
| 383 | Indicates to the server that it should never send "ready", but |
| 384 | should wait for the client to say "done" before sending the |
| 385 | packfile. |
| 386 | |
| 387 | The response of `fetch` is broken into a number of sections separated by |
| 388 | delimiter packets (0001), with each section beginning with its section |
| 389 | header. Most sections are sent only when the packfile is sent. |
| 390 | |
| 391 | output = acknowledgements flush-pkt | |
| 392 | [acknowledgments delim-pkt] [shallow-info delim-pkt] |
| 393 | [wanted-refs delim-pkt] [packfile-uris delim-pkt] |
| 394 | packfile flush-pkt |
| 395 | |
| 396 | acknowledgments = PKT-LINE("acknowledgments" LF) |
| 397 | (nak | *ack) |
| 398 | (ready) |
| 399 | ready = PKT-LINE("ready" LF) |
| 400 | nak = PKT-LINE("NAK" LF) |
| 401 | ack = PKT-LINE("ACK" SP obj-id LF) |
| 402 | |
| 403 | shallow-info = PKT-LINE("shallow-info" LF) |
| 404 | *PKT-LINE((shallow | unshallow) LF) |
| 405 | shallow = "shallow" SP obj-id |
| 406 | unshallow = "unshallow" SP obj-id |
| 407 | |
| 408 | wanted-refs = PKT-LINE("wanted-refs" LF) |
| 409 | *PKT-LINE(wanted-ref LF) |
| 410 | wanted-ref = obj-id SP refname |
| 411 | |
| 412 | packfile-uris = PKT-LINE("packfile-uris" LF) *packfile-uri |
| 413 | packfile-uri = PKT-LINE(40*(HEXDIGIT) SP *%x20-ff LF) |
| 414 | |
| 415 | packfile = PKT-LINE("packfile" LF) |
| 416 | *PKT-LINE(%x01-03 *%x00-ff) |
| 417 | |
| 418 | acknowledgments section |
| 419 | * If the client determines that it is finished with negotiations by |
| 420 | sending a "done" line (thus requiring the server to send a packfile), |
| 421 | the acknowledgments sections MUST be omitted from the server's |
| 422 | response. |
| 423 | |
| 424 | * Always begins with the section header "acknowledgments" |
| 425 | |
| 426 | * The server will respond with "NAK" if none of the object ids sent |
| 427 | as have lines were common. |
| 428 | |
| 429 | * The server will respond with "ACK obj-id" for all of the |
| 430 | object ids sent as have lines which are common. |
| 431 | |
| 432 | * A response cannot have both "ACK" lines as well as a "NAK" |
| 433 | line. |
| 434 | |
| 435 | * The server will respond with a "ready" line indicating that |
| 436 | the server has found an acceptable common base and is ready to |
| 437 | make and send a packfile (which will be found in the packfile |
| 438 | section of the same response) |
| 439 | |
| 440 | * If the server has found a suitable cut point and has decided |
| 441 | to send a "ready" line, then the server can decide to (as an |
| 442 | optimization) omit any "ACK" lines it would have sent during |
| 443 | its response. This is because the server will have already |
| 444 | determined the objects it plans to send to the client and no |
| 445 | further negotiation is needed. |
| 446 | |
| 447 | shallow-info section |
| 448 | * If the client has requested a shallow fetch/clone, a shallow |
| 449 | client requests a fetch or the server is shallow then the |
| 450 | server's response may include a shallow-info section. The |
| 451 | shallow-info section will be included if (due to one of the |
| 452 | above conditions) the server needs to inform the client of any |
| 453 | shallow boundaries or adjustments to the clients already |
| 454 | existing shallow boundaries. |
| 455 | |
| 456 | * Always begins with the section header "shallow-info" |
| 457 | |
| 458 | * If a positive depth is requested, the server will compute the |
| 459 | set of commits which are no deeper than the desired depth. |
| 460 | |
| 461 | * The server sends a "shallow obj-id" line for each commit whose |
| 462 | parents will not be sent in the following packfile. |
| 463 | |
| 464 | * The server sends an "unshallow obj-id" line for each commit |
| 465 | which the client has indicated is shallow, but is no longer |
| 466 | shallow as a result of the fetch (due to its parents being |
| 467 | sent in the following packfile). |
| 468 | |
| 469 | * The server MUST NOT send any "unshallow" lines for anything |
| 470 | which the client has not indicated was shallow as a part of |
| 471 | its request. |
| 472 | |
| 473 | wanted-refs section |
| 474 | * This section is only included if the client has requested a |
| 475 | ref using a 'want-ref' line and if a packfile section is also |
| 476 | included in the response. |
| 477 | |
| 478 | * Always begins with the section header "wanted-refs". |
| 479 | |
| 480 | * The server will send a ref listing ("<oid> <refname>") for |
| 481 | each reference requested using 'want-ref' lines. |
| 482 | |
| 483 | * The server MUST NOT send any refs which were not requested |
| 484 | using 'want-ref' lines. |
| 485 | |
| 486 | packfile-uris section |
| 487 | * This section is only included if the client sent |
| 488 | 'packfile-uris' and the server has at least one such URI to |
| 489 | send. |
| 490 | |
| 491 | * Always begins with the section header "packfile-uris". |
| 492 | |
| 493 | * For each URI the server sends, it sends a hash of the pack's |
| 494 | contents (as output by git index-pack) followed by the URI. |
| 495 | |
| 496 | * The hashes are 40 hex characters long. When Git upgrades to a new |
| 497 | hash algorithm, this might need to be updated. (It should match |
| 498 | whatever index-pack outputs after "pack\t" or "keep\t". |
| 499 | |
| 500 | packfile section |
| 501 | * This section is only included if the client has sent 'want' |
| 502 | lines in its request and either requested that no more |
| 503 | negotiation be done by sending 'done' or if the server has |
| 504 | decided it has found a sufficient cut point to produce a |
| 505 | packfile. |
| 506 | |
| 507 | * Always begins with the section header "packfile" |
| 508 | |
| 509 | * The transmission of the packfile begins immediately after the |
| 510 | section header |
| 511 | |
| 512 | * The data transfer of the packfile is always multiplexed, using |
| 513 | the same semantics of the 'side-band-64k' capability from |
| 514 | protocol version 1. This means that each packet, during the |
| 515 | packfile data stream, is made up of a leading 4-byte pkt-line |
| 516 | length (typical of the pkt-line format), followed by a 1-byte |
| 517 | stream code, followed by the actual data. |
| 518 | |
| 519 | The stream code can be one of: |
| 520 | 1 - pack data |
| 521 | 2 - progress messages |
| 522 | 3 - fatal error message just before stream aborts |
| 523 | |
| 524 | server-option |
| 525 | ~~~~~~~~~~~~~ |
| 526 | |
| 527 | If advertised, indicates that any number of server specific options can be |
| 528 | included in a request. This is done by sending each option as a |
| 529 | "server-option=<option>" capability line in the capability-list section of |
| 530 | a request. |
| 531 | |
| 532 | The provided options must not contain a NUL or LF character. |
| 533 | |
| 534 | object-format |
| 535 | ~~~~~~~~~~~~~ |
| 536 | |
| 537 | The server can advertise the `object-format` capability with a value `X` (in the |
| 538 | form `object-format=X`) to notify the client that the server is able to deal |
| 539 | with objects using hash algorithm X. If not specified, the server is assumed to |
| 540 | only handle SHA-1. If the client would like to use a hash algorithm other than |
| 541 | SHA-1, it should specify its object-format string. |
| 542 | |
| 543 | session-id=<session-id> |
| 544 | ~~~~~~~~~~~~~~~~~~~~~~~ |
| 545 | |
| 546 | The server may advertise a session ID that can be used to identify this process |
| 547 | across multiple requests. The client may advertise its own session ID back to |
| 548 | the server as well. |
| 549 | |
| 550 | Session IDs should be unique to a given process. They must fit within a |
| 551 | packet-line, and must not contain non-printable or whitespace characters. The |
| 552 | current implementation uses trace2 session IDs (see |
| 553 | link:technical/api-trace2.html[api-trace2] for details), but this may change |
| 554 | and users of the session ID should not rely on this fact. |
| 555 | |
| 556 | object-info |
| 557 | ~~~~~~~~~~~ |
| 558 | |
| 559 | `object-info` is the command to retrieve information about one or more objects. |
| 560 | Its main purpose is to allow a client to make decisions based on this |
| 561 | information without having to fully fetch objects. Object size is the only |
| 562 | information that is currently supported. |
| 563 | |
| 564 | An `object-info` request takes the following arguments: |
| 565 | |
| 566 | size |
| 567 | Requests size information to be returned for each listed object id. |
| 568 | |
| 569 | oid <oid> |
| 570 | Indicates to the server an object which the client wants to obtain |
| 571 | information for. |
| 572 | |
| 573 | The response of `object-info` is a list of the requested object ids |
| 574 | and associated requested information, each separated by a single space. |
| 575 | |
| 576 | output = info flush-pkt |
| 577 | |
| 578 | info = PKT-LINE(attrs) LF) |
| 579 | *PKT-LINE(obj-info LF) |
| 580 | |
| 581 | attrs = attr | attrs SP attrs |
| 582 | |
| 583 | attr = "size" |
| 584 | |
| 585 | obj-info = obj-id SP obj-size |
| 586 | |
| 587 | bundle-uri |
| 588 | ~~~~~~~~~~ |
| 589 | |
| 590 | If the 'bundle-uri' capability is advertised, the server supports the |
| 591 | `bundle-uri' command. |
| 592 | |
| 593 | The capability is currently advertised with no value (i.e. not |
| 594 | "bundle-uri=somevalue"), a value may be added in the future for |
| 595 | supporting command-wide extensions. Clients MUST ignore any unknown |
| 596 | capability values and proceed with the 'bundle-uri` dialog they |
| 597 | support. |
| 598 | |
| 599 | The 'bundle-uri' command is intended to be issued before `fetch` to |
| 600 | get URIs to bundle files (see linkgit:git-bundle[1]) to "seed" and |
| 601 | inform the subsequent `fetch` command. |
| 602 | |
| 603 | The client CAN issue `bundle-uri` before or after any other valid |
| 604 | command. To be useful to clients it's expected that it'll be issued |
| 605 | after an `ls-refs` and before `fetch`, but CAN be issued at any time |
| 606 | in the dialog. |
| 607 | |
| 608 | DISCUSSION of bundle-uri |
| 609 | ^^^^^^^^^^^^^^^^^^^^^^^^ |
| 610 | |
| 611 | The intent of the feature is optimize for server resource consumption |
| 612 | in the common case by changing the common case of fetching a very |
| 613 | large PACK during linkgit:git-clone[1] into a smaller incremental |
| 614 | fetch. |
| 615 | |
| 616 | It also allows servers to achieve better caching in combination with |
| 617 | an `uploadpack.packObjectsHook` (see linkgit:git-config[1]). |
| 618 | |
| 619 | By having new clones or fetches be a more predictable and common |
| 620 | negotiation against the tips of recently produces *.bundle file(s). |
| 621 | Servers might even pre-generate the results of such negotiations for |
| 622 | the `uploadpack.packObjectsHook` as new pushes come in. |
| 623 | |
| 624 | One way that servers could take advantage of these bundles is that the |
| 625 | server would anticipate that fresh clones will download a known bundle, |
| 626 | followed by catching up to the current state of the repository using ref |
| 627 | tips found in that bundle (or bundles). |
| 628 | |
| 629 | PROTOCOL for bundle-uri |
| 630 | ^^^^^^^^^^^^^^^^^^^^^^^ |
| 631 | |
| 632 | A `bundle-uri` request takes no arguments, and as noted above does not |
| 633 | currently advertise a capability value. Both may be added in the |
| 634 | future. |
| 635 | |
| 636 | When the client issues a `command=bundle-uri` request, the response is a |
| 637 | list of key-value pairs provided as packet lines with value |
| 638 | `<key>=<value>`. Each `<key>` should be interpreted as a config key from |
| 639 | the `bundle.*` namespace to construct a list of bundles. These keys are |
| 640 | grouped by a `bundle.<id>.` subsection, where each key corresponding to a |
| 641 | given `<id>` contributes attributes to the bundle defined by that `<id>`. |
| 642 | See linkgit:git-config[1] for the specific details of these keys and how |
| 643 | the Git client will interpret their values. |
| 644 | |
| 645 | Clients MUST parse the line according to the above format, lines that do |
| 646 | not conform to the format SHOULD be discarded. The user MAY be warned in |
| 647 | such a case. |
| 648 | |
| 649 | bundle-uri CLIENT AND SERVER EXPECTATIONS |
| 650 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 651 | |
| 652 | URI CONTENTS:: |
| 653 | The content at the advertised URIs MUST be one of two types. |
| 654 | + |
| 655 | The advertised URI may contain a bundle file that `git bundle verify` |
| 656 | would accept. I.e. they MUST contain one or more reference tips for |
| 657 | use by the client, MUST indicate prerequisites (in any) with standard |
| 658 | "-" prefixes, and MUST indicate their "object-format", if |
| 659 | applicable. |
| 660 | + |
| 661 | The advertised URI may alternatively contain a plaintext file that `git |
| 662 | config list` would accept (with the `--file` option). The key-value |
| 663 | pairs in this list are in the `bundle.*` namespace (see |
| 664 | linkgit:git-config[1]). |
| 665 | |
| 666 | bundle-uri CLIENT ERROR RECOVERY:: |
| 667 | A client MUST above all gracefully degrade on errors, whether that |
| 668 | error is because of bad missing/data in the bundle URI(s), because |
| 669 | that client is too dumb to e.g. understand and fully parse out bundle |
| 670 | headers and their prerequisite relationships, or something else. |
| 671 | + |
| 672 | Server operators should feel confident in turning on "bundle-uri" and |
| 673 | not worry if e.g. their CDN goes down that clones or fetches will run |
| 674 | into hard failures. Even if the server bundle(s) are |
| 675 | incomplete, or bad in some way the client should still end up with a |
| 676 | functioning repository, just as if it had chosen not to use this |
| 677 | protocol extension. |
| 678 | + |
| 679 | All subsequent discussion on client and server interaction MUST keep |
| 680 | this in mind. |
| 681 | |
| 682 | bundle-uri SERVER TO CLIENT:: |
| 683 | The ordering of the returned bundle uris is not significant. Clients |
| 684 | MUST parse their headers to discover their contained OIDS and |
| 685 | prerequisites. A client MUST consider the content of the bundle(s) |
| 686 | themselves and their header as the ultimate source of truth. |
| 687 | + |
| 688 | A server MAY even return bundle(s) that don't have any direct |
| 689 | relationship to the repository being cloned (either through accident, |
| 690 | or intentional "clever" configuration), and expect a client to sort |
| 691 | out what data they'd like from the bundle(s), if any. |
| 692 | |
| 693 | bundle-uri CLIENT TO SERVER:: |
| 694 | The client SHOULD provide reference tips found in the bundle header(s) |
| 695 | as 'have' lines in any subsequent `fetch` request. A client MAY also |
| 696 | ignore the bundle(s) entirely if doing so is deemed worse for some |
| 697 | reason, e.g. if the bundles can't be downloaded, it doesn't like the |
| 698 | tips it finds etc. |
| 699 | |
| 700 | WHEN ADVERTISED BUNDLE(S) REQUIRE NO FURTHER NEGOTIATION:: |
| 701 | If after issuing `bundle-uri` and `ls-refs`, and getting the header(s) |
| 702 | of the bundle(s) the client finds that the ref tips it wants can be |
| 703 | retrieved entirely from advertised bundle(s), the client MAY disconnect |
| 704 | from the Git server. The results of such a 'clone' or 'fetch' should be |
| 705 | indistinguishable from the state attained without using bundle-uri. |
| 706 | |
| 707 | EARLY CLIENT DISCONNECTIONS AND ERROR RECOVERY:: |
| 708 | A client MAY perform an early disconnect while still downloading the |
| 709 | bundle(s) (having streamed and parsed their headers). In such a case |
| 710 | the client MUST gracefully recover from any errors related to |
| 711 | finishing the download and validation of the bundle(s). |
| 712 | + |
| 713 | I.e. a client might need to re-connect and issue a 'fetch' command, |
| 714 | and possibly fall back to not making use of 'bundle-uri' at all. |
| 715 | + |
| 716 | This "MAY" behavior is specified as such (and not a "SHOULD") on the |
| 717 | assumption that a server advertising bundle uris is more likely than |
| 718 | not to be serving up a relatively large repository, and to be pointing |
| 719 | to URIs that have a good chance of being in working order. A client |
| 720 | MAY e.g. look at the payload size of the bundles as a heuristic to see |
| 721 | if an early disconnect is worth it, should falling back on a full |
| 722 | "fetch" dialog be necessary. |
| 723 | |
| 724 | WHEN ADVERTISED BUNDLE(S) REQUIRE FURTHER NEGOTIATION:: |
| 725 | A client SHOULD commence a negotiation of a PACK from the server via |
| 726 | the "fetch" command using the OID tips found in advertised bundles, |
| 727 | even if's still in the process of downloading those bundle(s). |
| 728 | + |
| 729 | This allows for aggressive early disconnects from any interactive |
| 730 | server dialog. The client blindly trusts that the advertised OID tips |
| 731 | are relevant, and issues them as 'have' lines, it then requests any |
| 732 | tips it would like (usually from the "ls-refs" advertisement) via |
| 733 | 'want' lines. The server will then compute a (hopefully small) PACK |
| 734 | with the expected difference between the tips from the bundle(s) and |
| 735 | the data requested. |
| 736 | + |
| 737 | The only connection the client then needs to keep active is to the |
| 738 | concurrently downloading static bundle(s), when those and the |
| 739 | incremental PACK are retrieved they should be inflated and |
| 740 | validated. Any errors at this point should be gracefully recovered |
| 741 | from, see above. |
| 742 | |
| 743 | bundle-uri PROTOCOL FEATURES |
| 744 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 745 | |
| 746 | The client constructs a bundle list from the `<key>=<value>` pairs |
| 747 | provided by the server. These pairs are part of the `bundle.*` namespace |
| 748 | as documented in linkgit:git-config[1]. In this section, we discuss some |
| 749 | of these keys and describe the actions the client will do in response to |
| 750 | this information. |
| 751 | |
| 752 | In particular, the `bundle.version` key specifies an integer value. The |
| 753 | only accepted value at the moment is `1`, but if the client sees an |
| 754 | unexpected value here then the client MUST ignore the bundle list. |
| 755 | |
| 756 | As long as `bundle.version` is understood, all other unknown keys MAY be |
| 757 | ignored by the client. The server will guarantee compatibility with older |
| 758 | clients, though newer clients may be better able to use the extra keys to |
| 759 | minimize downloads. |
| 760 | |
| 761 | Any backwards-incompatible addition of pre-URI key-value will be |
| 762 | guarded by a new `bundle.version` value or values in 'bundle-uri' |
| 763 | capability advertisement itself, and/or by new future `bundle-uri` |
| 764 | request arguments. |
| 765 | |
| 766 | Some example key-value pairs that are not currently implemented but could |
| 767 | be implemented in the future include: |
| 768 | |
| 769 | * Add a "hash=<val>" or "size=<bytes>" advertise the expected hash or |
| 770 | size of the bundle file. |
| 771 | |
| 772 | * Advertise that one or more bundle files are the same (to e.g. have |
| 773 | clients round-robin or otherwise choose one of N possible files). |
| 774 | |
| 775 | * A "oid=<OID>" shortcut and "prerequisite=<OID>" shortcut. For |
| 776 | expressing the common case of a bundle with one tip and no |
| 777 | prerequisites, or one tip and one prerequisite. |
| 778 | + |
| 779 | This would allow for optimizing the common case of servers who'd like |
| 780 | to provide one "big bundle" containing only their "main" branch, |
| 781 | and/or incremental updates thereof. |
| 782 | + |
| 783 | A client receiving such a response MAY assume that they can skip |
| 784 | retrieving the header from a bundle at the indicated URI, and thus |
| 785 | save themselves and the server(s) the request(s) needed to inspect the |
| 786 | headers of that bundle or bundles. |
| 787 | |
| 788 | promisor-remote=<pr-info> |
| 789 | ~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 790 | |
| 791 | The server may advertise some promisor remotes it is using or knows |
| 792 | about to a client which may want to use them as its promisor remotes, |
| 793 | instead of this repository. In this case <pr-info> should be of the |
| 794 | form: |
| 795 | |
| 796 | pr-info = pr-fields | pr-info ";" pr-fields |
| 797 | |
| 798 | pr-fields = pr-field | pr-fields "," pr-field |
| 799 | |
| 800 | pr-field = field-name "=" field-value |
| 801 | |
| 802 | where all the `field-name` and `field-value` in a given `pr-fields` |
| 803 | are field names and values related to a single promisor remote. A |
| 804 | given `field-name` MUST NOT appear more than once in given |
| 805 | `pr-fields`. |
| 806 | |
| 807 | The server MUST advertise at least the "name" and "url" field names |
| 808 | along with the associated field values, which are the name of a valid |
| 809 | remote and its URL, in each `pr-fields`. The "name" and "url" fields |
| 810 | MUST appear first in each pr-fields, in that order. |
| 811 | |
| 812 | After these mandatory fields, the server MAY advertise the following |
| 813 | optional fields in any order: |
| 814 | |
| 815 | `partialCloneFilter`:: The filter specification for the remote. It |
| 816 | corresponds to the "remote.<name>.partialCloneFilter" config setting. |
| 817 | Clients can use this to determine if the remote's filtering strategy |
| 818 | is compatible with their needs (e.g., checking if both use |
| 819 | "blob:none"). Additionally they can use this through the |
| 820 | `--filter=auto` option in linkgit:git-clone[1]. With that option, the |
| 821 | filter specification of the clone will be automatically computed by |
| 822 | combining the filter specifications of the promisor remotes the client |
| 823 | accepts. |
| 824 | |
| 825 | `token`:: An authentication token that clients can use when |
| 826 | connecting to the remote. It corresponds to the "remote.<name>.token" |
| 827 | config setting. |
| 828 | |
| 829 | No other fields are defined by the protocol at this time. Field names |
| 830 | are case-sensitive and MUST be transmitted exactly as specified |
| 831 | above. Clients MUST ignore fields they don't recognize to allow for |
| 832 | future protocol extensions. |
| 833 | |
| 834 | The client can use information transmitted through these fields to |
| 835 | decide if it accepts the advertised promisor remote. Also, the client |
| 836 | can be configured to store the values of these fields or use them |
| 837 | to automatically configure the repository (see "promisor.storeFields" |
| 838 | in linkgit:git-config[1] and `--filter=auto` in linkgit:git-clone[1]). |
| 839 | |
| 840 | Field values MUST be urlencoded. |
| 841 | |
| 842 | If the client decides to use one or more promisor remotes the server |
| 843 | advertised, it can reply with "promisor-remote=<pr-names>" where |
| 844 | <pr-names> should be of the form: |
| 845 | |
| 846 | pr-names = pr-name | pr-names ";" pr-name |
| 847 | |
| 848 | where `pr-name` is the urlencoded name of a promisor remote the server |
| 849 | advertised and the client accepts. |
| 850 | |
| 851 | The promisor remotes that the client accepted will be tried before the |
| 852 | other configured promisor remotes when the client attempts to fetch |
| 853 | missing objects. |
| 854 | |
| 855 | Note that, everywhere in this document, the ';' and ',' characters |
| 856 | MUST be encoded if they appear in `pr-name` or `field-value`. |
| 857 | |
| 858 | If the server doesn't know any promisor remote that could be good for |
| 859 | a client to use, or prefers a client not to use any promisor remote it |
| 860 | uses or knows about, it shouldn't advertise the "promisor-remote" |
| 861 | capability at all. |
| 862 | |
| 863 | In this case, or if the client doesn't want to use any promisor remote |
| 864 | the server advertised, the client shouldn't advertise the |
| 865 | "promisor-remote" capability at all in its reply. |
| 866 | |
| 867 | On the server side, the "promisor.advertise" and "promisor.sendFields" |
| 868 | configuration options can be used to control what it advertises. On |
| 869 | the client side, the "promisor.acceptFromServer" configuration option |
| 870 | can be used to control what it accepts, and the "promisor.storeFields" |
| 871 | option, to control what it stores. See the documentation of these |
| 872 | configuration options in linkgit:git-config[1] for more information. |
| 873 | |
| 874 | Note that in the future it would be nice if the "promisor-remote" |
| 875 | protocol capability could be used by the server, when responding to |
| 876 | `git fetch` or `git clone`, to advertise better-connected remotes that |
| 877 | the client can use as promisor remotes, instead of this repository, so |
| 878 | that the client can lazily fetch objects from these other |
| 879 | better-connected remotes. This would require the server to omit in its |
| 880 | response the objects available on the better-connected remotes that |
| 881 | the client has accepted. This hasn't been implemented yet though. So |
| 882 | for now this "promisor-remote" capability is useful only when the |
| 883 | server advertises some promisor remotes it already uses to borrow |
| 884 | objects from. |
| 885 | |
| 886 | GIT |
| 887 | --- |
| 888 | Part of the linkgit:git[1] suite |