@leroysheep / LeeRoySheep
Code Commits Pulls CI/CD 82,027
next
bisect jch maint master next seen test todo
md 332 lines 10.3 KB
Rendered Raw
1 Come out and Clar
2 =================
3
4 In Catalan, "clar" means clear, easy to perceive. Using clar will make it
5 easy to test and make clear the quality of your code.
6
7 > _Historical note_
8 >
9 > Originally the clar project was named "clay" because the word "test" has its
10 > roots in the latin word *"testum"*, meaning "earthen pot", and *"testa"*,
11 > meaning "piece of burned clay"?
12 >
13 > This is because historically, testing implied melting metal in a pot to
14 > check its quality. Clay is what tests are made of.
15
16 ## Quick Usage Overview
17
18 Clar is a minimal C unit testing framework. It's been written to replace the
19 old framework in [libgit2][libgit2], but it's both very versatile and
20 straightforward to use.
21
22 Can you count to funk?
23
24 - **Zero: Initialize test directory**
25
26 ~~~~ sh
27 $ mkdir tests
28 $ cp -r $CLAR_ROOT/clar* tests
29 $ cp $CLAR_ROOT/example/*.c tests
30 ~~~~
31
32 - **One: Write some tests**
33
34 File: tests/adding.c:
35
36 ~~~~ c
37 /* adding.c for the "Adding" suite */
38 #include "clar.h"
39
40 static int *answer;
41
42 void test_adding__initialize(void)
43 {
44 answer = malloc(sizeof(int));
45 cl_assert_(answer != NULL, "No memory left?");
46 *answer = 42;
47 }
48
49 void test_adding__cleanup(void)
50 {
51 free(answer);
52 }
53
54 void test_adding__make_sure_math_still_works(void)
55 {
56 cl_assert_(5 > 3, "Five should probably be greater than three");
57 cl_assert_(-5 < 2, "Negative numbers are small, I think");
58 cl_assert_(*answer == 42, "The universe is doing OK. And the initializer too.");
59 }
60 ~~~~~
61
62 - **Two: Build the test executable**
63
64 ~~~~ sh
65 $ cd tests
66 $ $CLAR_PATH/generate.py .
67 Written `clar.suite` (1 suites)
68 $ gcc -I. clar.c main.c adding.c -o testit
69 ~~~~
70
71 - **Funk: Funk it.**
72
73 ~~~~ sh
74 $ ./testit
75 ~~~~
76
77 ## The Clar Test Suite
78
79 Writing a test suite is pretty straightforward. Each test suite is a `*.c`
80 file with a descriptive name: this encourages modularity.
81
82 Each test suite has optional initialize and cleanup methods. These methods
83 will be called before and after running **each** test in the suite, even if
84 such test fails. As a rule of thumb, if a test needs a different initializer
85 or cleanup method than another test in the same module, that means it
86 doesn't belong in that module. Keep that in mind when grouping tests
87 together.
88
89 The `initialize` and `cleanup` methods have the following syntax, with
90 `suitename` being the current suite name, e.g. `adding` for the `adding.c`
91 suite.
92
93 ~~~~ c
94 void test_suitename__initialize(void)
95 {
96 /* init */
97 }
98
99 void test_suitename__cleanup(void)
100 {
101 /* cleanup */
102 }
103 ~~~~
104
105 These methods are encouraged to use static, global variables to store the state
106 that will be used by all tests inside the suite.
107
108 ~~~~ c
109 static git_repository *_repository;
110
111 void test_status__initialize(void)
112 {
113 create_tmp_repo(STATUS_REPO);
114 git_repository_open(_repository, STATUS_REPO);
115 }
116
117 void test_status__cleanup(void)
118 {
119 git_repository_close(_repository);
120 git_path_rm(STATUS_REPO);
121 }
122
123 void test_status__simple_test(void)
124 {
125 /* do something with _repository */
126 }
127 ~~~~
128
129 Writing the actual tests is just as straightforward. Tests have the
130 `void test_suitename__test_name(void)` signature, and they should **not**
131 be static. Clar will automatically detect and list them.
132
133 Tests are run as they appear on their original suites: they have no return
134 value. A test is considered "passed" if it doesn't raise any errors. Check
135 the "Clar API" section to see the various helper functions to check and
136 raise errors during test execution.
137
138 __Caution:__ If you use assertions inside of `test_suitename__initialize`,
139 make sure that you do not rely on `__initialize` being completely run
140 inside your `test_suitename__cleanup` function. Otherwise you might
141 encounter resource cleanup twice.
142
143 ## How does Clar work?
144
145 To use Clar:
146
147 1. copy the Clar boilerplate to your test directory
148 2. copy (and probably modify) the sample `main.c` (from
149 `$CLAR_PATH/example/main.c`)
150 3. run the Clar mixer (a.k.a. `generate.py`) to scan your test directory and
151 write out the test suite metadata.
152 4. compile your test files and the Clar boilerplate into a single test
153 executable
154 5. run the executable to test!
155
156 The Clar boilerplate gives you a set of useful test assertions and features
157 (like accessing or making sandbox copies of fixture data). It consists of
158 the `clar.c` and `clar.h` files, plus the code in the `clar/` subdirectory.
159 You should not need to edit these files.
160
161 The sample `main.c` (i.e. `$CLAR_PATH/example/main.c`) file invokes
162 `clar_test(argc, argv)` to run the tests. Usually, you will edit this file
163 to perform any framework specific initialization and teardown that you need.
164
165 The Clar mixer (`generate.py`) recursively scans your test directory for
166 any `.c` files, parses them, and writes the `clar.suite` file with all of
167 the metadata about your tests. When you build, the `clar.suite` file is
168 included into `clar.c`.
169
170 The mixer can be run with **Python 2.5, 2.6, 2.7, 3.0, 3.1, 3.2 and PyPy 1.6**.
171
172 Commandline usage of the mixer is as follows:
173
174 $ ./generate.py .
175
176 Where `.` is the folder where all the test suites can be found. The mixer
177 will automatically locate all the relevant source files and build the
178 testing metadata. The metadata will be written to `clar.suite`, in the same
179 folder as all the test suites. This file is included by `clar.c` and so
180 must be accessible via `#include` when building the test executable.
181
182 $ gcc -I. clar.c main.c suite1.c test2.c -o run_tests
183
184 **Note that the Clar mixer only needs to be ran when adding new tests to a
185 suite, in order to regenerate the metadata**. As a result, the `clar.suite`
186 file can be checked into version control if you wish to be able to build
187 your test suite without having to re-run the mixer.
188
189 This is handy when e.g. generating tests in a local computer, and then
190 building and testing them on an embedded device or a platform where Python
191 is not available.
192
193 ### Fixtures
194
195 Clar can create sandboxed fixtures for you to use in your test. You'll need to compile *clar.c* with an additional `CFLAG`, `-DCLAR_FIXTURE_PATH`. This should be an absolute path to your fixtures directory.
196
197 Once that's done, you can use the fixture API as defined below.
198
199 ## The Clar API
200
201 Clar makes the following methods available from all functions in a test
202 suite.
203
204 - `cl_must_pass(call)`, `cl_must_pass_(call, message)`: Verify that the given
205 function call passes, in the POSIX sense (returns a value greater or equal
206 to 0).
207
208 - `cl_must_fail(call)`, `cl_must_fail_(call, message)`: Verify that the given
209 function call fails, in the POSIX sense (returns a value less than 0).
210
211 - `cl_assert(expr)`, `cl_assert_(expr, message)`: Verify that `expr` is true.
212
213 - `cl_check_pass(call)`, `cl_check_pass_(call, message)`: Verify that the
214 given function call passes, in the POSIX sense (returns a value greater or
215 equal to 0). If the function call doesn't succeed, a test failure will be
216 logged but the test's execution will continue.
217
218 - `cl_check_fail(call)`, `cl_check_fail_(call, message)`: Verify that the
219 given function call fails, in the POSIX sense (returns a value less than
220 0). If the function call doesn't fail, a test failure will be logged but
221 the test's execution will continue.
222
223 - `cl_check(expr)`: Verify that `expr` is true. If `expr` is not
224 true, a test failure will be logged but the test's execution will continue.
225
226 - `cl_fail(message)`: Fail the current test with the given message.
227
228 - `cl_warning(message)`: Issue a warning. This warning will be
229 logged as a test failure but the test's execution will continue.
230
231 - `cl_set_cleanup(void (*cleanup)(void *), void *opaque)`: Set the cleanup
232 method for a single test. This method will be called with `opaque` as its
233 argument before the test returns (even if the test has failed).
234 If a global cleanup method is also available, the local cleanup will be
235 called first, and then the global.
236
237 - `cl_assert_equal_i(int,int)`: Verify that two integer values are equal.
238 The advantage of this over a simple `cl_assert` is that it will format
239 a much nicer error report if the values are not equal.
240
241 - `cl_assert_equal_s(const char *,const char *)`: Verify that two strings
242 are equal. The expected value can also be NULL and this will correctly
243 test for that.
244
245 - `cl_fixture_sandbox(const char *)`: Sets up a sandbox for a fixture
246 so that you can mutate the file directly.
247
248 - `cl_fixture_cleanup(const char *)`: Tears down the previous fixture
249 sandbox.
250
251 - `cl_fixture(const char *)`: Gets the full path to a fixture file.
252
253 ### Auxiliary / helper functions
254
255 The clar API is always available while running a test, even when calling
256 "auxiliary" (helper) functions.
257
258 You're encouraged to perform test assertions in those auxiliary
259 methods, instead of returning error values. This is considered good
260 Clar style. _However_, when you do this, you need to call `cl_invoke`
261 to preserve the current state; this ensures that failures are reported
262 as coming from the actual test, instead of the auxiliary method.
263
264 Style Example:
265
266 ~~~~ c
267 /*
268 * Bad style: auxiliary functions return an error code
269 */
270
271 static int check_string(const char *str)
272 {
273 const char *aux = process_string(str);
274
275 if (aux == NULL)
276 return -1;
277
278 return strcmp(my_function(aux), str) == 0 ? 0 : -1;
279 }
280
281 void test_example__a_test_with_auxiliary_methods(void)
282 {
283 cl_must_pass_(
284 check_string("foo"),
285 "String differs after processing"
286 );
287
288 cl_must_pass_(
289 check_string("bar"),
290 "String differs after processing"
291 );
292 }
293 ~~~~
294
295 ~~~~ c
296 /*
297 * Good style: auxiliary functions perform assertions
298 */
299
300 static void check_string(const char *str)
301 {
302 const char *aux = process_string(str);
303
304 cl_assert_(
305 aux != NULL,
306 "String processing failed"
307 );
308
309 cl_assert_(
310 strcmp(my_function(aux), str) == 0,
311 "String differs after processing"
312 );
313 }
314
315 void test_example__a_test_with_auxiliary_methods(void)
316 {
317 cl_invoke(check_string("foo"));
318 cl_invoke(check_string("bar"));
319 }
320 ~~~~
321
322 About Clar
323 ==========
324
325 Clar was originally written by [Vicent Martí](https://github.com/vmg),
326 to replace the old testing framework in [libgit2][libgit2]. It is
327 currently maintained by [Edward Thomson](https://github.com/ethomson),
328 and used by the [libgit2][libgit2] and [git][git] projects, amongst
329 others.
330
331 [libgit2]: https://github.com/libgit2/libgit2
332 [git]: https://github.com/git/git