add `config_set` API for caching config-like files
Currently `git_config()` uses a callback mechanism and file rereads for config values. Due to this approach, it is not uncommon for the config files to be parsed several times during the run of a git program, with different callbacks picking out different variables useful to themselves. Add a `config_set`, that can be used to construct an in-memory cache for config-like files that the caller specifies (i.e., files like `.gitmodules`, `~/.gitconfig` etc.). Add two external functions `git_configset_get_value` and `git_configset_get_value_multi` for querying from the config sets. `git_configset_get_value` follows `last one wins` semantic (i.e. if there are multiple matches for the queried key in the files of the configset the value returned will be the last entry in `value_list`). `git_configset_get_value_multi` returns a list of values sorted in order of increasing priority (i.e. last match will be at the end of the list). Add type specific query functions like `git_configset_get_bool` and similar. Add a default `config_set`, `the_config_set` to cache all key-value pairs read from usual config files (repo specific .git/config, user wide ~/.gitconfig, XDG config and the global /etc/gitconfig). `the_config_set` is populated using `git_config()`. Add two external functions `git_config_get_value` and `git_config_get_value_multi` for querying in a non-callback manner from `the_config_set`. Also, add type specific query functions that are implemented as a thin wrapper around the `config_set` API. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Tanay Abhra <tanayabh@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Tanay Abhra committed
Jul 28, 2014 at 03:10 UTC
3c8687a73eeffc21f5a0b3916d38e46ff985933d
4 files changed
+457
Documentation/technical/api-config.txt
+142
@@ -77,6 +77,86 @@ To read a specific file in git-config format, use
77
`git_config_from_file`. This takes the same callback and data parameters
78
as `git_config`.
79
80
+Querying For Specific Variables
81
+-------------------------------
82
+
83
+For programs wanting to query for specific variables in a non-callback
84
+manner, the config API provides two functions `git_config_get_value`
85
+and `git_config_get_value_multi`. They both read values from an internal
86
+cache generated previously from reading the config files.
87
+
88
+`int git_config_get_value(const char *key, const char **value)`::
89
+
90
+ Finds the highest-priority value for the configuration variable `key`,
91
+ stores the pointer to it in `value` and returns 0. When the
92
+ configuration variable `key` is not found, returns 1 without touching
93
+ `value`. The caller should not free or modify `value`, as it is owned
94
+ by the cache.
95
+
96
+`const struct string_list *git_config_get_value_multi(const char *key)`::
97
+
98
+ Finds and returns the value list, sorted in order of increasing priority
99
+ for the configuration variable `key`. When the configuration variable
100
+ `key` is not found, returns NULL. The caller should not free or modify
101
+ the returned pointer, as it is owned by the cache.
102
+
103
+`void git_config_clear(void)`::
104
+
105
+ Resets and invalidates the config cache.
106
+
107
+The config API also provides type specific API functions which do conversion
108
+as well as retrieval for the queried variable, including:
109
+
110
+`int git_config_get_int(const char *key, int *dest)`::
111
+
112
+ Finds and parses the value to an integer for the configuration variable
113
+ `key`. Dies on error; otherwise, stores the value of the parsed integer in
114
+ `dest` and returns 0. When the configuration variable `key` is not found,
115
+ returns 1 without touching `dest`.
116
+
117
+`int git_config_get_ulong(const char *key, unsigned long *dest)`::
118
+
119
+ Similar to `git_config_get_int` but for unsigned longs.
120
+
121
+`int git_config_get_bool(const char *key, int *dest)`::
122
+
123
+ Finds and parses the value into a boolean value, for the configuration
124
+ variable `key` respecting keywords like "true" and "false". Integer
125
+ values are converted into true/false values (when they are non-zero or
126
+ zero, respectively). Other values cause a die(). If parsing is successful,
127
+ stores the value of the parsed result in `dest` and returns 0. When the
128
+ configuration variable `key` is not found, returns 1 without touching
129
+ `dest`.
130
+
131
+`int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest)`::
132
+
133
+ Similar to `git_config_get_bool`, except that integers are copied as-is,
134
+ and `is_bool` flag is unset.
135
+
136
+`int git_config_get_maybe_bool(const char *key, int *dest)`::
137
+
138
+ Similar to `git_config_get_bool`, except that it returns -1 on error
139
+ rather than dying.
140
+
141
+`int git_config_get_string_const(const char *key, const char **dest)`::
142
+
143
+ Allocates and copies the retrieved string into the `dest` parameter for
144
+ the configuration variable `key`; if NULL string is given, prints an
145
+ error message and returns -1. When the configuration variable `key` is
146
+ not found, returns 1 without touching `dest`.
147
+
148
+`int git_config_get_string(const char *key, char **dest)`::
149
+
150
+ Similar to `git_config_get_string_const`, except that retrieved value
151
+ copied into the `dest` parameter is a mutable string.
152
+
153
+`int git_config_get_pathname(const char *key, const char **dest)`::
154
+
155
+ Similar to `git_config_get_string`, but expands `~` or `~user` into
156
+ the user's home directory when found at the beginning of the path.
157
+
158
+See test-config.c for usage examples.
159
+
160
Value Parsing Helpers
161
---------------------
162
@@ -134,6 +214,68 @@ int read_file_with_include(const char *file, config_fn_t fn, void *data)
214
`git_config` respects includes automatically. The lower-level
215
`git_config_from_file` does not.
216
217
+Custom Configsets
218
+-----------------
219
+
220
+A `config_set` can be used to construct an in-memory cache for
221
+config-like files that the caller specifies (i.e., files like `.gitmodules`,
222
+`~/.gitconfig` etc.). For example,
223
+
224
+---------------------------------------
225
+struct config_set gm_config;
226
+git_configset_init(&gm_config);
227
+int b;
228
+/* we add config files to the config_set */
229
+git_configset_add_file(&gm_config, ".gitmodules");
230
+git_configset_add_file(&gm_config, ".gitmodules_alt");
231
+
232
+if (!git_configset_get_bool(gm_config, "submodule.frotz.ignore", &b)) {
233
+ /* hack hack hack */
234
+}
235
+
236
+/* when we are done with the configset */
237
+git_configset_clear(&gm_config);
238
+----------------------------------------
239
+
240
+Configset API provides functions for the above mentioned work flow, including:
241
+
242
+`void git_configset_init(struct config_set *cs)`::
243
+
244
+ Initializes the config_set `cs`.
245
+
246
+`int git_configset_add_file(struct config_set *cs, const char *filename)`::
247
+
248
+ Parses the file and adds the variable-value pairs to the `config_set`,
249
+ dies if there is an error in parsing the file. Returns 0 on success, or
250
+ -1 if the file does not exist or is inaccessible. The user has to decide
251
+ if he wants to free the incomplete configset or continue using it when
252
+ the function returns -1.
253
+
254
+`int git_configset_get_value(struct config_set *cs, const char *key, const char **value)`::
255
+
256
+ Finds the highest-priority value for the configuration variable `key`
257
+ and config set `cs`, stores the pointer to it in `value` and returns 0.
258
+ When the configuration variable `key` is not found, returns 1 without
259
+ touching `value`. The caller should not free or modify `value`, as it
260
+ is owned by the cache.
261
+
262
+`const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key)`::
263
+
264
+ Finds and returns the value list, sorted in order of increasing priority
265
+ for the configuration variable `key` and config set `cs`. When the
266
+ configuration variable `key` is not found, returns NULL. The caller
267
+ should not free or modify the returned pointer, as it is owned by the cache.
268
+
269
+`void git_configset_clear(struct config_set *cs)`::
270
+
271
+ Clears `config_set` structure, removes all saved variable-value pairs.
272
+
273
+In addition to above functions, the `config_set` API provides type specific
274
+functions in the vein of `git_config_get_int` and family but with an extra
275
+parameter, pointer to struct `config_set`.
276
+They all behave similarly to the `git_config_get*()` family described in
277
+"Querying For Specific Variables" above.
278
+
279
Writing Config Files
280
--------------------
281
cache.h
+32
@@ -1351,6 +1351,38 @@ extern int parse_config_key(const char *var,
1351
const char **subsection, int *subsection_len,
1352
const char **key);
1353
1354
+struct config_set {
1355
+ struct hashmap config_hash;
1356
+ int hash_initialized;
1357
+};
1358
+
1359
+extern void git_configset_init(struct config_set *cs);
1360
+extern int git_configset_add_file(struct config_set *cs, const char *filename);
1361
+extern int git_configset_get_value(struct config_set *cs, const char *key, const char **value);
1362
+extern const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key);
1363
+extern void git_configset_clear(struct config_set *cs);
1364
+extern int git_configset_get_string_const(struct config_set *cs, const char *key, const char **dest);
1365
+extern int git_configset_get_string(struct config_set *cs, const char *key, char **dest);
1366
+extern int git_configset_get_int(struct config_set *cs, const char *key, int *dest);
1367
+extern int git_configset_get_ulong(struct config_set *cs, const char *key, unsigned long *dest);
1368
+extern int git_configset_get_bool(struct config_set *cs, const char *key, int *dest);
1369
+extern int git_configset_get_bool_or_int(struct config_set *cs, const char *key, int *is_bool, int *dest);
1370
+extern int git_configset_get_maybe_bool(struct config_set *cs, const char *key, int *dest);
1371
+extern int git_configset_get_pathname(struct config_set *cs, const char *key, const char **dest);
1372
+
1373
+extern int git_config_get_value(const char *key, const char **value);
1374
+extern const struct string_list *git_config_get_value_multi(const char *key);
1375
+extern void git_config_clear(void);
1376
+extern void git_config_iter(config_fn_t fn, void *data);
1377
+extern int git_config_get_string_const(const char *key, const char **dest);
1378
+extern int git_config_get_string(const char *key, char **dest);
1379
+extern int git_config_get_int(const char *key, int *dest);
1380
+extern int git_config_get_ulong(const char *key, unsigned long *dest);
1381
+extern int git_config_get_bool(const char *key, int *dest);
1382
+extern int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest);
1383
+extern int git_config_get_maybe_bool(const char *key, int *dest);
1384
+extern int git_config_get_pathname(const char *key, const char **dest);
1385
+
1386
extern int committer_ident_sufficiently_given(void);
1387
extern int author_ident_sufficiently_given(void);
1388
config.c
+274
@@ -9,6 +9,8 @@
9
#include "exec_cmd.h"
10
#include "strbuf.h"
11
#include "quote.h"
12
+#include "hashmap.h"
13
+#include "string-list.h"
14
15
struct config_source {
16
struct config_source *prev;
@@ -33,10 +35,23 @@ struct config_source {
35
long (*do_ftell)(struct config_source *c);
36
};
37
38
+struct config_set_element {
39
+ struct hashmap_entry ent;
40
+ char *key;
41
+ struct string_list value_list;
42
+};
43
+
44
static struct config_source *cf;
45
46
static int zlib_compression_seen;
47
48
+/*
49
+ * Default config_set that contains key-value pairs from the usual set of config
50
+ * config files (i.e repo specific .git/config, user wide ~/.gitconfig, XDG
51
+ * config file and the global /etc/gitconfig)
52
+ */
53
+static struct config_set the_config_set;
54
+
55
static int config_file_fgetc(struct config_source *conf)
56
{
57
return fgetc(conf->u.file);
@@ -1212,6 +1227,262 @@ int git_config(config_fn_t fn, void *data)
1227
return git_config_with_options(fn, data, NULL, 1);
1228
}
1229
1230
+static struct config_set_element *configset_find_element(struct config_set *cs, const char *key)
1231
+{
1232
+ struct config_set_element k;
1233
+ struct config_set_element *found_entry;
1234
+ char *normalized_key;
1235
+ int ret;
1236
+ /*
1237
+ * `key` may come from the user, so normalize it before using it
1238
+ * for querying entries from the hashmap.
1239
+ */
1240
+ ret = git_config_parse_key(key, &normalized_key, NULL);
1241
+
1242
+ if (ret)
1243
+ return NULL;
1244
+
1245
+ hashmap_entry_init(&k, strhash(normalized_key));
1246
+ k.key = normalized_key;
1247
+ found_entry = hashmap_get(&cs->config_hash, &k, NULL);
1248
+ free(normalized_key);
1249
+ return found_entry;
1250
+}
1251
+
1252
+static int configset_add_value(struct config_set *cs, const char *key, const char *value)
1253
+{
1254
+ struct config_set_element *e;
1255
+ e = configset_find_element(cs, key);
1256
+ /*
1257
+ * Since the keys are being fed by git_config*() callback mechanism, they
1258
+ * are already normalized. So simply add them without any further munging.
1259
+ */
1260
+ if (!e) {
1261
+ e = xmalloc(sizeof(*e));
1262
+ hashmap_entry_init(e, strhash(key));
1263
+ e->key = xstrdup(key);
1264
+ string_list_init(&e->value_list, 1);
1265
+ hashmap_add(&cs->config_hash, e);
1266
+ }
1267
+ string_list_append_nodup(&e->value_list, value ? xstrdup(value) : NULL);
1268
+
1269
+ return 0;
1270
+}
1271
+
1272
+static int config_set_element_cmp(const struct config_set_element *e1,
1273
+ const struct config_set_element *e2, const void *unused)
1274
+{
1275
+ return strcmp(e1->key, e2->key);
1276
+}
1277
+
1278
+void git_configset_init(struct config_set *cs)
1279
+{
1280
+ hashmap_init(&cs->config_hash, (hashmap_cmp_fn)config_set_element_cmp, 0);
1281
+ cs->hash_initialized = 1;
1282
+}
1283
+
1284
+void git_configset_clear(struct config_set *cs)
1285
+{
1286
+ struct config_set_element *entry;
1287
+ struct hashmap_iter iter;
1288
+ if (!cs->hash_initialized)
1289
+ return;
1290
+
1291
+ hashmap_iter_init(&cs->config_hash, &iter);
1292
+ while ((entry = hashmap_iter_next(&iter))) {
1293
+ free(entry->key);
1294
+ string_list_clear(&entry->value_list, 0);
1295
+ }
1296
+ hashmap_free(&cs->config_hash, 1);
1297
+ cs->hash_initialized = 0;
1298
+}
1299
+
1300
+static int config_set_callback(const char *key, const char *value, void *cb)
1301
+{
1302
+ struct config_set *cs = cb;
1303
+ configset_add_value(cs, key, value);
1304
+ return 0;
1305
+}
1306
+
1307
+int git_configset_add_file(struct config_set *cs, const char *filename)
1308
+{
1309
+ return git_config_from_file(config_set_callback, filename, cs);
1310
+}
1311
+
1312
+int git_configset_get_value(struct config_set *cs, const char *key, const char **value)
1313
+{
1314
+ const struct string_list *values = NULL;
1315
+ /*
1316
+ * Follows "last one wins" semantic, i.e., if there are multiple matches for the
1317
+ * queried key in the files of the configset, the value returned will be the last
1318
+ * value in the value list for that key.
1319
+ */
1320
+ values = git_configset_get_value_multi(cs, key);
1321
+
1322
+ if (!values)
1323
+ return 1;
1324
+ assert(values->nr > 0);
1325
+ *value = values->items[values->nr - 1].string;
1326
+ return 0;
1327
+}
1328
+
1329
+const struct string_list *git_configset_get_value_multi(struct config_set *cs, const char *key)
1330
+{
1331
+ struct config_set_element *e = configset_find_element(cs, key);
1332
+ return e ? &e->value_list : NULL;
1333
+}
1334
+
1335
+int git_configset_get_string_const(struct config_set *cs, const char *key, const char **dest)
1336
+{
1337
+ const char *value;
1338
+ if (!git_configset_get_value(cs, key, &value))
1339
+ return git_config_string(dest, key, value);
1340
+ else
1341
+ return 1;
1342
+}
1343
+
1344
+int git_configset_get_string(struct config_set *cs, const char *key, char **dest)
1345
+{
1346
+ return git_configset_get_string_const(cs, key, (const char **)dest);
1347
+}
1348
+
1349
+int git_configset_get_int(struct config_set *cs, const char *key, int *dest)
1350
+{
1351
+ const char *value;
1352
+ if (!git_configset_get_value(cs, key, &value)) {
1353
+ *dest = git_config_int(key, value);
1354
+ return 0;
1355
+ } else
1356
+ return 1;
1357
+}
1358
+
1359
+int git_configset_get_ulong(struct config_set *cs, const char *key, unsigned long *dest)
1360
+{
1361
+ const char *value;
1362
+ if (!git_configset_get_value(cs, key, &value)) {
1363
+ *dest = git_config_ulong(key, value);
1364
+ return 0;
1365
+ } else
1366
+ return 1;
1367
+}
1368
+
1369
+int git_configset_get_bool(struct config_set *cs, const char *key, int *dest)
1370
+{
1371
+ const char *value;
1372
+ if (!git_configset_get_value(cs, key, &value)) {
1373
+ *dest = git_config_bool(key, value);
1374
+ return 0;
1375
+ } else
1376
+ return 1;
1377
+}
1378
+
1379
+int git_configset_get_bool_or_int(struct config_set *cs, const char *key,
1380
+ int *is_bool, int *dest)
1381
+{
1382
+ const char *value;
1383
+ if (!git_configset_get_value(cs, key, &value)) {
1384
+ *dest = git_config_bool_or_int(key, value, is_bool);
1385
+ return 0;
1386
+ } else
1387
+ return 1;
1388
+}
1389
+
1390
+int git_configset_get_maybe_bool(struct config_set *cs, const char *key, int *dest)
1391
+{
1392
+ const char *value;
1393
+ if (!git_configset_get_value(cs, key, &value)) {
1394
+ *dest = git_config_maybe_bool(key, value);
1395
+ if (*dest == -1)
1396
+ return -1;
1397
+ return 0;
1398
+ } else
1399
+ return 1;
1400
+}
1401
+
1402
+int git_configset_get_pathname(struct config_set *cs, const char *key, const char **dest)
1403
+{
1404
+ const char *value;
1405
+ if (!git_configset_get_value(cs, key, &value))
1406
+ return git_config_pathname(dest, key, value);
1407
+ else
1408
+ return 1;
1409
+}
1410
+
1411
+static void git_config_check_init(void)
1412
+{
1413
+ if (the_config_set.hash_initialized)
1414
+ return;
1415
+ git_configset_init(&the_config_set);
1416
+ git_config(config_set_callback, &the_config_set);
1417
+}
1418
+
1419
+void git_config_clear(void)
1420
+{
1421
+ if (!the_config_set.hash_initialized)
1422
+ return;
1423
+ git_configset_clear(&the_config_set);
1424
+}
1425
+
1426
+int git_config_get_value(const char *key, const char **value)
1427
+{
1428
+ git_config_check_init();
1429
+ return git_configset_get_value(&the_config_set, key, value);
1430
+}
1431
+
1432
+const struct string_list *git_config_get_value_multi(const char *key)
1433
+{
1434
+ git_config_check_init();
1435
+ return git_configset_get_value_multi(&the_config_set, key);
1436
+}
1437
+
1438
+int git_config_get_string_const(const char *key, const char **dest)
1439
+{
1440
+ git_config_check_init();
1441
+ return git_configset_get_string_const(&the_config_set, key, dest);
1442
+}
1443
+
1444
+int git_config_get_string(const char *key, char **dest)
1445
+{
1446
+ git_config_check_init();
1447
+ return git_config_get_string_const(key, (const char **)dest);
1448
+}
1449
+
1450
+int git_config_get_int(const char *key, int *dest)
1451
+{
1452
+ git_config_check_init();
1453
+ return git_configset_get_int(&the_config_set, key, dest);
1454
+}
1455
+
1456
+int git_config_get_ulong(const char *key, unsigned long *dest)
1457
+{
1458
+ git_config_check_init();
1459
+ return git_configset_get_ulong(&the_config_set, key, dest);
1460
+}
1461
+
1462
+int git_config_get_bool(const char *key, int *dest)
1463
+{
1464
+ git_config_check_init();
1465
+ return git_configset_get_bool(&the_config_set, key, dest);
1466
+}
1467
+
1468
+int git_config_get_bool_or_int(const char *key, int *is_bool, int *dest)
1469
+{
1470
+ git_config_check_init();
1471
+ return git_configset_get_bool_or_int(&the_config_set, key, is_bool, dest);
1472
+}
1473
+
1474
+int git_config_get_maybe_bool(const char *key, int *dest)
1475
+{
1476
+ git_config_check_init();
1477
+ return git_configset_get_maybe_bool(&the_config_set, key, dest);
1478
+}
1479
+
1480
+int git_config_get_pathname(const char *key, const char **dest)
1481
+{
1482
+ git_config_check_init();
1483
+ return git_configset_get_pathname(&the_config_set, key, dest);
1484
+}
1485
+
1486
/*
1487
* Find all the stuff for git_config_set() below.
1488
*/
@@ -1707,6 +1978,9 @@ int git_config_set_multivar_in_file(const char *config_filename,
1978
lock = NULL;
1979
ret = 0;
1980
1981
+ /* Invalidate the config cache */
1982
+ git_config_clear();
1983
+
1984
out_free:
1985
if (lock)
1986
rollback_lock_file(lock);
setup.c
+9
@@ -624,6 +624,15 @@ static const char *setup_git_directory_gently_1(int *nongit_ok)
624
dev_t current_device = 0;
625
int one_filesystem = 1;
626
627
+ /*
628
+ * We may have read an incomplete configuration before
629
+ * setting-up the git directory. If so, clear the cache so
630
+ * that the next queries to the configuration reload complete
631
+ * configuration (including the per-repo config file that we
632
+ * ignored previously).
633
+ */
634
+ git_config_clear();
635
+
636
/*
637
* Let's assume that we are in a git repository.
638
* If it turns out later that we are somewhere else, the value will be