simple-ipc: design documentation for new IPC mechanism
Brief design documentation for new IPC mechanism allowing foreground Git client to talk with an existing daemon process at a known location using a named pipe or unix domain socket. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff Hostetler committed
Mar 15, 2021 at 21:08 UTC
066d5234d04737408f5f537f30f19873a2c7637f
1 file changed
+105
Documentation/technical/api-simple-ipc.txt
new
+105
@@ -0,0 +1,105 @@
1
+Simple-IPC API
2
+==============
3
+
4
+The Simple-IPC API is a collection of `ipc_` prefixed library routines
5
+and a basic communication protocol that allow an IPC-client process to
6
+send an application-specific IPC-request message to an IPC-server
7
+process and receive an application-specific IPC-response message.
8
+
9
+Communication occurs over a named pipe on Windows and a Unix domain
10
+socket on other platforms. IPC-clients and IPC-servers rendezvous at
11
+a previously agreed-to application-specific pathname (which is outside
12
+the scope of this design) that is local to the computer system.
13
+
14
+The IPC-server routines within the server application process create a
15
+thread pool to listen for connections and receive request messages
16
+from multiple concurrent IPC-clients. When received, these messages
17
+are dispatched up to the server application callbacks for handling.
18
+IPC-server routines then incrementally relay responses back to the
19
+IPC-client.
20
+
21
+The IPC-client routines within a client application process connect
22
+to the IPC-server and send a request message and wait for a response.
23
+When received, the response is returned back the caller.
24
+
25
+For example, the `fsmonitor--daemon` feature will be built as a server
26
+application on top of the IPC-server library routines. It will have
27
+threads watching for file system events and a thread pool waiting for
28
+client connections. Clients, such as `git status` will request a list
29
+of file system events since a point in time and the server will
30
+respond with a list of changed files and directories. The formats of
31
+the request and response are application-specific; the IPC-client and
32
+IPC-server routines treat them as opaque byte streams.
33
+
34
+
35
+Comparison with sub-process model
36
+---------------------------------
37
+
38
+The Simple-IPC mechanism differs from the existing `sub-process.c`
39
+model (Documentation/technical/long-running-process-protocol.txt) and
40
+used by applications like Git-LFS. In the LFS-style sub-process model
41
+the helper is started by the foreground process, communication happens
42
+via a pair of file descriptors bound to the stdin/stdout of the
43
+sub-process, the sub-process only serves the current foreground
44
+process, and the sub-process exits when the foreground process
45
+terminates.
46
+
47
+In the Simple-IPC model the server is a very long-running service. It
48
+can service many clients at the same time and has a private socket or
49
+named pipe connection to each active client. It might be started
50
+(on-demand) by the current client process or it might have been
51
+started by a previous client or by the OS at boot time. The server
52
+process is not associated with a terminal and it persists after
53
+clients terminate. Clients do not have access to the stdin/stdout of
54
+the server process and therefore must communicate over sockets or
55
+named pipes.
56
+
57
+
58
+Server startup and shutdown
59
+---------------------------
60
+
61
+How an application server based upon IPC-server is started is also
62
+outside the scope of the Simple-IPC design and is a property of the
63
+application using it. For example, the server might be started or
64
+restarted during routine maintenance operations, or it might be
65
+started as a system service during the system boot-up sequence, or it
66
+might be started on-demand by a foreground Git command when needed.
67
+
68
+Similarly, server shutdown is a property of the application using
69
+the simple-ipc routines. For example, the server might decide to
70
+shutdown when idle or only upon explicit request.
71
+
72
+
73
+Simple-IPC protocol
74
+-------------------
75
+
76
+The Simple-IPC protocol consists of a single request message from the
77
+client and an optional response message from the server. Both the
78
+client and server messages are unlimited in length and are terminated
79
+with a flush packet.
80
+
81
+The pkt-line routines (Documentation/technical/protocol-common.txt)
82
+are used to simplify buffer management during message generation,
83
+transmission, and reception. A flush packet is used to mark the end
84
+of the message. This allows the sender to incrementally generate and
85
+transmit the message. It allows the receiver to incrementally receive
86
+the message in chunks and to know when they have received the entire
87
+message.
88
+
89
+The actual byte format of the client request and server response
90
+messages are application specific. The IPC layer transmits and
91
+receives them as opaque byte buffers without any concern for the
92
+content within. It is the job of the calling application layer to
93
+understand the contents of the request and response messages.
94
+
95
+
96
+Summary
97
+-------
98
+
99
+Conceptually, the Simple-IPC protocol is similar to an HTTP REST
100
+request. Clients connect, make an application-specific and
101
+stateless request, receive an application-specific
102
+response, and disconnect. It is a one round trip facility for
103
+querying the server. The Simple-IPC routines hide the socket,
104
+named pipe, and thread pool details and allow the application
105
+layer to focus on the application at hand.