@cryptotaxi247 / netdata-1 / commits / abf1626db

CMocka tests for Issue 7274 (#7308)

* Start of testing partial requests. Need to stash this to checkout a PR to test. * Disambiguated error messages during header validation. The mocking has blown up in the linker, need to wipe out repo local changes and restart from a known good state. * Test failures. CMocka is really not designed for parametric tests which is making it difficult to test the http validation properly. We have some problems in the web_client.c code that are causing early failures in the testing sequence, and it is causing CMocka to abort the sequence. Need to try a different approach to building the tests... * Pedantic style pass. * Test generation. There must be another value hidden in the system that CMocka uses. This sets up 3278 tests but the results from cmocka_run_group_tests_name show 0 tests were run. * The problem was the "helper"-macro. Calling CMocka directly, moved the setup/teardown into explicit fixtures. Successfully runs the family of tests over the same (empty) state. * Parameterised family of tests runs. The api_next() acts as a counter, the least significant digit is the prefix_len using the web_buffer in the test_family struct as a template to walk throufh. The most significant digit is the number of headers to use in the request. Checked that this walk executes correctly and all the tests run before putting the test payloads back in. We trigger a failure about 3-4 tests in that takes down the process. Currently investigating which parts are not mocked correctly. * Pedantic style pass * Adding a mocking for fatal. That weird thing with the linker has happened again, need to clean repo and rebuild fresh. * Full test sequence executes. The test parameter counter jammed after a failure - we cannot rely on anything in the main test body being executed after we call the functionailty under test. A failure will skip the rest of the execution. Moved the counter stepping to the top of the function (i.e. it is now a ++i instead of a i++). Adjusted the initial state to compensate. This now steps through all of the test-sequence, but it raises an ugly issue - the post-test cleanup will not be executed on a failure. TODO: * Move the test-state into the test_family. * Do the clean-up of the previous test (if necesarry) in the step function. * Fix the assertion on the web_client state. * Pedantic style pass * Test state is now in the test_family. This addresses the issue with leaking on failure and not performing clean-up - we don't really care about memory leaks during unit-testing, but we do care about reseting the system-under-test back to a known state to guarantee independence across the tests. The clean-up is now triggered in api_next(). * Flip the wait flag assertions. Partial requests should leave the web_client waiting to receive more data. * Fixing ACL flags in test-driver. This makes some tests pass - but far too many. Probably need a proper debugging function to show the request / response in a readable format. * Result from the api mocking. Setting a successful return code in the api mocking makes the non-partial tests pass. Zero'ing out the web_client before use has not fixed the initialization errors, there is still some history on the parse_tries that needs to be tracked down. Some of the other errors are spurious - they result from stream multiplexing in the testdriver - be careful with less. * Fix warnings. Switched the build configuration to CFLAGS="-O1 -ggdb -Wall -Wextra -Wformat-signedness -fstack-protector-all -DNETDATA_INTERNAL_CHECKS=1 -D_FORTIFY_SOURCE=2 -DNETDATA_VERIFY_LOCKS=1". The memset introduced last night to zero out the initial web_client state had transposed parameters. Now that the state is initially zero before hitting the http request processing most issues have disappeared. There are 3000+ passing tests and 48 boundary cases to track down. * Pushing log entries from each test into a buffer. This will allow suppression of logs from tests that pass. * Switched to a unique test definition structure per test. This cleans up the code as it means that a list of tests can be constructed during the first walk through the parameter space. There is no need to walk the space twice and keep both walks aligned. Removed the cmocka_unit_test macro and build the CMUnitTest structures directly -> this allows a real name per test instead of the procedure name. The walking/step function api_info_next has been folded back into the test procedure as it is simpler to walk the list in the shared test state. Current TODO: * There is a bug, the check on the wait state in the buffer is not being handled properly, investigate why everything fails. * The results don't match the old code, are we handing the correct web_buffer to each tested piece of code? * Capture the test success state -> dump the log buffer on failures. * State is properly passed through the tests. Spent a long time chasing a horrible bug that seems to be inside CMocka? The state parameter being passed to each unit test is different on each call, i.e. it looks like a unique void** where the void * (*state) has been overwritten with the original value on each iteration through the testing loop. This behaviour does not match the CMocka source code, which does thread the given valud through the unit test calls. It could be a side-effect of the memory check-pointing, but the net-effect is that we cannot change the shared state between tests. It can be set in the setup-fixture and used in each test, but not altered for the subsequent test. This took a long time to diagnose - the fix is simple, we just share the state in a global pointer. This shared state is used to walk through the list of test_def structures so that each unit-test knows where it is in the parameter- space. * With the correct state the bug in triggering the correct assertions is gone. * Dump out the buffered logs on test failure. * The only failing case (relative to these assertions) are the ulta-short partial-requests. * Check the web_client->mode is set properly. * Style pass * Checking values passed to the API despatch point. * Disabled the parametric tests to do some low-level testings. Later on both sets of tests will be active. While the low-level url encoding tests are being developed the dynamically generated set is disabled to make the output easier to read. Working through the W3C URL spec, against RFC3986 and comparing the cases in available url-parsing test-suites to build our test-suite. * Start of the URL test-suite. The percent-decoding in the current implementation is in the wrong place - it happens too early and causes non-delimitor characters in the URL to be treated as delimitors. Current unit-tests seem to cover the range of checks that we need CMocka to make. The handling of output is a little awkward - need something like the dynamic cases that can output the log on a failure or skip it on a pass. * Raw material for low-level testing. * Adding more families in here is getting too messy. About to switch over to multiple testdrivers. * Need to clean repo to work around wrapping failures in CMocka. * CMocka is not compatible with LTO. The weird wrapping issues that come and go are as a result of LTO. My typical netdata-installer command-line that I use to reboot the project state disables LTO, while my normal autoreconf / configure command-line does not causing this problem to reappear seemingly-randomly. To build a single test-driver target this works: autoreconf -ivf && CFLAGS='-O1 -ggdb -Wall -Wextra -Wformat-signedness -fstack-protector-all -DNETDATA_INTERNAL_CHECKS=1 -D_FORTIFY_SOURCE=2 -DNETDATA_VERIFY_LOCKS=1' ./configure --disable-lto && make web/api/tests/web_api_valid_urls The actual change in this commit is just a bug-fix. * Ripping out the parameterized test generator. Each of the URL cases is slightly and subtly different. This can't be done using the parameterization and will need a healthy dose of cut and paste. CMocka does not recognise the mocking for mysendfile, which is necessary to capture the exit route from the URL parsing. * Weird bug in CMocka? For some reason CMocka will not mock out the mysendfile() procedure. We need to mock this to capture the behaviour of the URL parsing as it is one of the exit paths. The wrapping is setup the same way as for the procedures so I cannot see any reason that the library would not overwrite the calls. The only difference that I can find is that mysendfile is in the unit being tested and the other mocked procedures are in different translation units. This should not make a difference, but we have to disable LTO to get CMocka to work and the symbol patchs is some kind of linker hack so there could be an issue if LTO is not running and the patch target is inside the same translation unit. Hiding it for now with a #ifndef UNIT_TESTING, which then compiles find and control flow hits the mock... * Converting the ascii comments into unit_tests. * More nasty cases for unit testing. The commented out case will trigger a buffer overflow in the netdata agent and crash it. * Last of the individual unit tests planned before the demo. * Removing warnings. * Switching on the rest of the parametric set - the other case with CRs. * Fix Travis build failure under docker. * Change the name of a define so it does not collide with existing testing in Travis. * Add CMocka unit tests to CMake * Linting pass * Adding RFC comment to test. * Buffer overflow checks on the captured logs. This fixes the seg-fault seen by @vlvkobal and @thiagoftsm during testing. * Chasing down other valgrind reports. This gets rid of all of the uninitialised variable warnings. We stil have a memory leak, the headers that are set during the unit testing switch on compression. This causes the web_client code to call deflatInit2 and allocate structures for the compressor. We do not have a matching call to deflateEnd anywhere in the code so the memory leaks. * Cleaning up a comment. * Fixing review comments from @vlvkobal. Also noticed that the buffer overflow fix this morning was killing the logfile output, fixed this as well. * Addressing @thiagoftsm's concerns about the changing number of failures. Switched the log dump for failing cases to repr(). Found a bug in the test case generator (not storing the flag for `\r`. Verified that the 58 failing cases are the correct set of failures for the tested code.

Andrew Moss committed Nov 21, 2019 at 09:29 UTC abf1626db13c9bdfc71219a616f6fa2be97e6628

5 files changed +1180 -22

CMakeLists.txt

+59 -1

@@ -897,6 +897,64 @@ if(BUILD_TESTING)

897 target_link_libraries(storage_number_testdriver libnetdata ${NETDATA_COMMON_LIBRARIES} ${CMOCKA_LIBRARIES})

898 add_test(NAME test_storage_number COMMAND storage_number_testdriver)

899

900 - set_target_properties(str2ld_testdriver storage_number_testdriver PROPERTIES RUNTIME_OUTPUT_DIRECTORY tests)

900 + set(WEB_API_TEST_FILES

901 + web/api/tests/web_api.c

902 + web/server/web_client.c

903 + )

904 + add_executable(web_api_testdriver ${WEB_API_TEST_FILES})

905 + target_link_options(

906 + web_api_testdriver

907 + PRIVATE

908 + -Wl,--wrap=rrdhost_find_by_hostname

909 + -Wl,--wrap=finished_web_request_statistics

910 + -Wl,--wrap=config_get

911 + -Wl,--wrap=web_client_api_request_v1

912 + -Wl,--wrap=rrdhost_find_by_guid

913 + -Wl,--wrap=rrdset_find_byname

914 + -Wl,--wrap=rrdset_find

915 + -Wl,--wrap=rrdpush_receiver_thread_spawn

916 + -Wl,--wrap=debug_int

917 + -Wl,--wrap=error_int

918 + -Wl,--wrap=info_int

919 + -Wl,--wrap=fatal_int

920 + )

921 + target_link_libraries(web_api_testdriver libnetdata ${NETDATA_COMMON_LIBRARIES} ${CMOCKA_LIBRARIES})

922 + add_test(NAME test_web_api COMMAND web_api_testdriver)

923 +

924 + set(VALID_URLS_TEST_FILES

925 + web/api/tests/valid_urls.c

926 + web/server/web_client.c

927 + )

928 + add_executable(valid_urls_testdriver ${VALID_URLS_TEST_FILES})

929 + target_link_options(

930 + valid_urls_testdriver

931 + PRIVATE

932 + -Wl,--wrap=rrdhost_find_by_hostname

933 + -Wl,--wrap=finished_web_request_statistics

934 + -Wl,--wrap=config_get

935 + -Wl,--wrap=web_client_api_request_v1

936 + -Wl,--wrap=rrdhost_find_by_guid

937 + -Wl,--wrap=rrdset_find_byname

938 + -Wl,--wrap=rrdset_find

939 + -Wl,--wrap=rrdpush_receiver_thread_spawn

940 + -Wl,--wrap=debug_int

941 + -Wl,--wrap=error_int

942 + -Wl,--wrap=info_int

943 + -Wl,--wrap=fatal_int

944 + -Wl,--wrap=mysendfile

945 + -DREMOVE_MYSENDFILE

946 + )

947 + target_link_libraries(valid_urls_testdriver libnetdata ${NETDATA_COMMON_LIBRARIES} ${CMOCKA_LIBRARIES})

948 + add_test(NAME test_valid_urls COMMAND valid_urls_testdriver)

949 +

950 + set_target_properties(

951 + str2ld_testdriver

952 + storage_number_testdriver

953 + web_api_testdriver

954 + valid_urls_testdriver

955 + PROPERTIES RUNTIME_OUTPUT_DIRECTORY tests

956 + )

957 +

958 +

959 endif()

960 endif()

Makefile.am

+26

@@ -645,10 +645,35 @@ if ENABLE_UNITTESTS

645 libnetdata/tests/str2ld_testdriver \

646 libnetdata/storage_number/tests/storage_number_testdriver \

647 web/api/tests/web_api_testdriver \

648 + web/api/tests/valid_urls_testdriver \

649 $(NULL)

650

651 TESTS = $(check_PROGRAMS)

652

653 + web_api_tests_valid_urls_testdriver_LDFLAGS = \

654 + -Wl,--wrap=rrdhost_find_by_hostname \

655 + -Wl,--wrap=finished_web_request_statistics \

656 + -Wl,--wrap=config_get \

657 + -Wl,--wrap=web_client_api_request_v1 \

658 + -Wl,--wrap=rrdhost_find_by_guid \

659 + -Wl,--wrap=rrdset_find_byname \

660 + -Wl,--wrap=rrdset_find \

661 + -Wl,--wrap=rrdpush_receiver_thread_spawn \

662 + -Wl,--wrap=debug_int \

663 + -Wl,--wrap=error_int \

664 + -Wl,--wrap=info_int \

665 + -Wl,--wrap=fatal_int \

666 + -Wl,--wrap=mysendfile \

667 + -DREMOVE_MYSENDFILE \

668 + $(TEST_LDFLAGS) \

669 + $(NULL)

670 + web_api_tests_valid_urls_testdriver_SOURCES = \

671 + web/api/tests/valid_urls.c \

672 + web/server/web_client.c \

673 + $(LIBNETDATA_FILES) \

674 + $(NULL)

675 + web_api_tests_valid_urls_testdriver_LDADD = $(NETDATA_COMMON_LIBS) $(TEST_LIBS)

676 +

677 web_api_tests_web_api_testdriver_LDFLAGS = \

678 -Wl,--wrap=rrdhost_find_by_hostname \

679 -Wl,--wrap=finished_web_request_statistics \

@@ -661,6 +686,7 @@ if ENABLE_UNITTESTS

686 -Wl,--wrap=debug_int \

687 -Wl,--wrap=error_int \

688 -Wl,--wrap=info_int \

689 + -Wl,--wrap=fatal_int \

690 $(TEST_LDFLAGS) \

691 $(NULL)

692 web_api_tests_web_api_testdriver_SOURCES = \

web/api/tests/valid_urls.c new

+777

@@ -0,0 +1,777 @@

1 +// SPDX-License-Identifier: GPL-3.0-or-later

2 +

3 +#include "../../../libnetdata/libnetdata.h"

4 +#include "../../../libnetdata/required_dummies.h"

5 +#include "../../../database/rrd.h"

6 +#include "../../../web/server/web_client.h"

7 +#include <setjmp.h>

8 +#include <cmocka.h>

9 +#include <stdbool.h>

10 +

11 +void repr(char *result, int result_size, char const *buf, int size)

12 +{

13 + int n;

14 + char *end = result + result_size - 1;

15 + unsigned char const *ubuf = (unsigned char const *)buf;

16 + while (size && result_size > 0) {

17 + if (*ubuf <= 0x20 || *ubuf >= 0x80) {

18 + n = snprintf(result, result_size, "\\%02X", *ubuf);

19 + } else {

20 + *result = *ubuf;

21 + n = 1;

22 + }

23 + result += n;

24 + result_size -= n;

25 + ubuf++;

26 + size--;

27 + }

28 + if (result_size > 0)

29 + *(result++) = 0;

30 + else

31 + *end = 0;

32 +}

33 +

34 +// ---------------------------------- Mocking accesses from web_client ------------------------------------------------

35 +

36 +ssize_t send(int sockfd, const void *buf, size_t len, int flags)

37 +{

38 + info("Mocking send: %zu bytes\n", len);

39 + (void)sockfd;

40 + (void)buf;

41 + (void)flags;

42 + return len;

43 +}

44 +

45 +RRDHOST *__wrap_rrdhost_find_by_hostname(const char *hostname, uint32_t hash)

46 +{

47 + (void)hostname;

48 + (void)hash;

49 + return NULL;

50 +}

51 +

52 +/* Note: we've got some intricate code inside the global statistics module, might be useful to pull it inside the

53 + test set instead of mocking it. */

54 +void __wrap_finished_web_request_statistics(

55 + uint64_t dt, uint64_t bytes_received, uint64_t bytes_sent, uint64_t content_size, uint64_t compressed_content_size)

56 +{

57 + (void)dt;

58 + (void)bytes_received;

59 + (void)bytes_sent;

60 + (void)content_size;

61 + (void)compressed_content_size;

62 +}

63 +

64 +char *__wrap_config_get(struct config *root, const char *section, const char *name, const char *default_value)

65 +{

66 + if (!strcmp(section, CONFIG_SECTION_WEB) && !strcmp(name, "web files owner"))

67 + return "netdata";

68 + (void)root;

69 + (void)default_value;

70 + return "UNKNOWN FIX ME";

71 +}

72 +

73 +int __wrap_web_client_api_request_v1(RRDHOST *host, struct web_client *w, char *url)

74 +{

75 + char url_repr[160];

76 + repr(url_repr, sizeof(url_repr), url, strlen(url));

77 + printf("web_client_api_request_v1(url=\"%s\")\n", url_repr);

78 + check_expected_ptr(host);

79 + check_expected_ptr(w);

80 + check_expected_ptr(url_repr);

81 + return HTTP_RESP_OK;

82 +}

83 +

84 +int __wrap_mysendfile(struct web_client *w, char *filename)

85 +{

86 + (void)w;

87 + printf("mysendfile(filename=\"%s\"\n", filename);

88 + check_expected_ptr(filename);

89 + return HTTP_RESP_OK;

90 +}

91 +

92 +int __wrap_rrdpush_receiver_thread_spawn(RRDHOST *host, struct web_client *w, char *url)

93 +{

94 + (void)host;

95 + (void)w;

96 + (void)url;

97 + return 0;

98 +}

99 +

100 +RRDHOST *__wrap_rrdhost_find_by_guid(const char *guid, uint32_t hash)

101 +{

102 + (void)guid;

103 + (void)hash;

104 + printf("FIXME: rrdset_find_guid\n");

105 + return NULL;

106 +}

107 +

108 +RRDSET *__wrap_rrdset_find_byname(RRDHOST *host, const char *name)

109 +{

110 + (void)host;

111 + (void)name;

112 + printf("FIXME: rrdset_find_byname\n");

113 + return NULL;

114 +}

115 +

116 +RRDSET *__wrap_rrdset_find(RRDHOST *host, const char *id)

117 +{

118 + (void)host;

119 + (void)id;

120 + printf("FIXME: rrdset_find\n");

121 + return NULL;

122 +}

123 +

124 +// -------------------------------- Mocking the log - dump straight through --------------------------------------------

125 +

126 +void __wrap_debug_int(const char *file, const char *function, const unsigned long line, const char *fmt, ...)

127 +{

128 + (void)file;

129 + (void)function;

130 + (void)line;

131 + va_list args;

132 + va_start(args, fmt);

133 + printf(" DEBUG: ");

134 + printf(fmt, args);

135 + printf("\n");

136 + va_end(args);

137 +}

138 +

139 +void __wrap_info_int(const char *file, const char *function, const unsigned long line, const char *fmt, ...)

140 +{

141 + (void)file;

142 + (void)function;

143 + (void)line;

144 + va_list args;

145 + va_start(args, fmt);

146 + printf(" INFO: ");

147 + printf(fmt, args);

148 + printf("\n");

149 + va_end(args);

150 +}

151 +

152 +void __wrap_error_int(

153 + const char *prefix, const char *file, const char *function, const unsigned long line, const char *fmt, ...)

154 +{

155 + (void)prefix;

156 + (void)file;

157 + (void)function;

158 + (void)line;

159 + va_list args;

160 + va_start(args, fmt);

161 + printf(" ERROR: ");

162 + printf(fmt, args);

163 + printf("\n");

164 + va_end(args);

165 +}

166 +

167 +void __wrap_fatal_int(const char *file, const char *function, const unsigned long line, const char *fmt, ...)

168 +{

169 + (void)file;

170 + (void)function;

171 + (void)line;

172 + va_list args;

173 + va_start(args, fmt);

174 + printf("FATAL: ");

175 + printf(fmt, args);

176 + printf("\n");

177 + va_end(args);

178 + fail();

179 +}

180 +

181 +WEB_SERVER_MODE web_server_mode = WEB_SERVER_MODE_STATIC_THREADED;

182 +char *netdata_configured_web_dir = "UNKNOWN FIXME";

183 +RRDHOST *localhost = NULL;

184 +

185 +struct config netdata_config = { .sections = NULL,

186 + .mutex = NETDATA_MUTEX_INITIALIZER,

187 + .index = { .avl_tree = { .root = NULL, .compar = appconfig_section_compare },

188 + .rwlock = AVL_LOCK_INITIALIZER } };

189 +

190 +/* Note: this is not a CMocka group_test_setup/teardown pair. This is performed per-test.

191 +*/

192 +static struct web_client *setup_fresh_web_client()

193 +{

194 + struct web_client *w = (struct web_client *)malloc(sizeof(struct web_client));

195 + memset(w, 0, sizeof(struct web_client));

196 + w->response.data = buffer_create(NETDATA_WEB_RESPONSE_INITIAL_SIZE);

197 + w->response.header = buffer_create(NETDATA_WEB_RESPONSE_HEADER_SIZE);

198 + w->response.header_output = buffer_create(NETDATA_WEB_RESPONSE_HEADER_SIZE);

199 + strcpy(w->origin, "*"); // Simulate web_client_create_on_fd()

200 + w->cookie1[0] = 0; // Simulate web_client_create_on_fd()

201 + w->cookie2[0] = 0; // Simulate web_client_create_on_fd()

202 + w->acl = 0x1f; // Everything on

203 + return w;

204 +}

205 +

206 +static void destroy_web_client(struct web_client *w)

207 +{

208 + buffer_free(w->response.data);

209 + buffer_free(w->response.header);

210 + buffer_free(w->response.header_output);

211 + free(w);

212 +}

213 +

214 +//////////////////////////// Test cases ///////////////////////////////////////////////////////////////////////////////

215 +

216 +static void only_root(void **state)

217 +{

218 + (void)state;

219 +

220 + if (localhost != NULL)

221 + free(localhost);

222 + localhost = malloc(sizeof(RRDHOST));

223 +

224 + struct web_client *w = setup_fresh_web_client();

225 + buffer_strcat(w->response.data, "GET / HTTP/1.1\r\n\r\n");

226 +

227 + char debug[4096];

228 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

229 + printf("-> \"%s\"\n", debug);

230 +

231 + //char expected_url_repr[4096];

232 + //repr(expected_url_repr, sizeof(expected_url_repr), def->url_out_repr, strlen(def->url_out_repr));

233 +

234 + expect_string(__wrap_mysendfile, filename, "/");

235 +

236 + web_client_process_request(w);

237 +

238 + //assert_string_equal(w->decoded_query_string, def->query_out);

239 + destroy_web_client(w);

240 + free(localhost);

241 + localhost = NULL;

242 +}

243 +

244 +static void two_slashes(void **state)

245 +{

246 + (void)state;

247 +

248 + if (localhost != NULL)

249 + free(localhost);

250 + localhost = malloc(sizeof(RRDHOST));

251 +

252 + struct web_client *w = setup_fresh_web_client();

253 + buffer_strcat(w->response.data, "GET // HTTP/1.1\r\n\r\n");

254 +

255 + char debug[4096];

256 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

257 + printf("-> \"%s\"\n", debug);

258 +

259 + //char expected_url_repr[4096];

260 + //repr(expected_url_repr, sizeof(expected_url_repr), def->url_out_repr, strlen(def->url_out_repr));

261 +

262 + expect_string(__wrap_mysendfile, filename, "//");

263 +

264 + web_client_process_request(w);

265 +

266 + //assert_string_equal(w->decoded_query_string, def->query_out);

267 + destroy_web_client(w);

268 + free(localhost);

269 + localhost = NULL;

270 +}

271 +

272 +static void absolute_url(void **state)

273 +{

274 + (void)state;

275 +

276 + if (localhost != NULL)

277 + free(localhost);

278 + localhost = malloc(sizeof(RRDHOST));

279 +

280 + struct web_client *w = setup_fresh_web_client();

281 + buffer_strcat(w->response.data, "GET http://localhost:19999/api/v1/info HTTP/1.1\r\n\r\n");

282 +

283 + char debug[4096];

284 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

285 + printf("-> \"%s\"\n", debug);

286 +

287 + //char expected_url_repr[4096];

288 + //repr(expected_url_repr, sizeof(expected_url_repr), def->url_out_repr, strlen(def->url_out_repr));

289 +

290 + expect_value(__wrap_web_client_api_request_v1, host, localhost);

291 + expect_value(__wrap_web_client_api_request_v1, w, w);

292 + expect_string(__wrap_web_client_api_request_v1, url_repr, "info");

293 +

294 + web_client_process_request(w);

295 +

296 + assert_string_equal(w->decoded_query_string, "?blah");

297 + destroy_web_client(w);

298 + free(localhost);

299 + localhost = NULL;

300 +}

301 +

302 +static void valid_url(void **state)

303 +{

304 + (void)state;

305 +

306 + if (localhost != NULL)

307 + free(localhost);

308 + localhost = malloc(sizeof(RRDHOST));

309 +

310 + struct web_client *w = setup_fresh_web_client();

311 + buffer_strcat(w->response.data, "GET /api/v1/info?blah HTTP/1.1\r\n\r\n");

312 +

313 + char debug[4096];

314 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

315 + printf("-> \"%s\"\n", debug);

316 +

317 + //char expected_url_repr[4096];

318 + //repr(expected_url_repr, sizeof(expected_url_repr), def->url_out_repr, strlen(def->url_out_repr));

319 +

320 + expect_value(__wrap_web_client_api_request_v1, host, localhost);

321 + expect_value(__wrap_web_client_api_request_v1, w, w);

322 + expect_string(__wrap_web_client_api_request_v1, url_repr, "info");

323 +

324 + web_client_process_request(w);

325 +

326 + assert_string_equal(w->decoded_query_string, "?blah");

327 + destroy_web_client(w);

328 + free(localhost);

329 + localhost = NULL;

330 +}

331 +

332 +/* RFC2616, section 4.1:

333 +

334 + In the interest of robustness, servers SHOULD ignore any empty

335 + line(s) received where a Request-Line is expected. In other words, if

336 + the server is reading the protocol stream at the beginning of a

337 + message and receives a CRLF first, it should ignore the CRLF.

338 +*/

339 +static void leading_blanks(void **state)

340 +{

341 + (void)state;

342 +

343 + if (localhost != NULL)

344 + free(localhost);

345 + localhost = malloc(sizeof(RRDHOST));

346 +

347 + struct web_client *w = setup_fresh_web_client();

348 + buffer_strcat(w->response.data, "\r\n\r\nGET /api/v1/info?blah HTTP/1.1\r\n\r\n");

349 +

350 + char debug[4096];

351 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

352 + printf("-> \"%s\"\n", debug);

353 +

354 + //char expected_url_repr[4096];

355 + //repr(expected_url_repr, sizeof(expected_url_repr), def->url_out_repr, strlen(def->url_out_repr));

356 +

357 + expect_value(__wrap_web_client_api_request_v1, host, localhost);

358 + expect_value(__wrap_web_client_api_request_v1, w, w);

359 + expect_string(__wrap_web_client_api_request_v1, url_repr, "info");

360 +

361 + web_client_process_request(w);

362 +

363 + assert_string_equal(w->decoded_query_string, "?blah");

364 + destroy_web_client(w);

365 + free(localhost);

366 + localhost = NULL;

367 +}

368 +

369 +static void empty_url(void **state)

370 +{

371 + (void)state;

372 +

373 + if (localhost != NULL)

374 + free(localhost);

375 + localhost = malloc(sizeof(RRDHOST));

376 +

377 + struct web_client *w = setup_fresh_web_client();

378 + buffer_strcat(w->response.data, "GET HTTP/1.1\r\n\r\n");

379 +

380 + char debug[4096];

381 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

382 + printf("-> \"%s\"\n", debug);

383 +

384 + //char expected_url_repr[4096];

385 + //repr(expected_url_repr, sizeof(expected_url_repr), def->url_out_repr, strlen(def->url_out_repr));

386 +

387 + expect_value(__wrap_web_client_api_request_v1, host, localhost);

388 + expect_value(__wrap_web_client_api_request_v1, w, w);

389 + expect_string(__wrap_web_client_api_request_v1, url_repr, "info");

390 +

391 + web_client_process_request(w);

392 +

393 + assert_string_equal(w->decoded_query_string, "?blah");

394 + destroy_web_client(w);

395 + free(localhost);

396 + localhost = NULL;

397 +}

398 +

399 +/* If the %-escape is being performed at the correct time then the url should not be treated as a query, but instead

400 + as a path "/api/v1/info?blah?" which should despatch into the API with the given values.

401 +*/

402 +static void not_a_query(void **state)

403 +{

404 + (void)state;

405 + localhost = malloc(sizeof(RRDHOST));

406 +

407 + struct web_client *w = setup_fresh_web_client();

408 + buffer_strcat(w->response.data, "GET /api/v1/info%3fblah%3f HTTP/1.1\r\n\r\n");

409 +

410 + char debug[160];

411 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

412 + printf("->%s\n", debug);

413 +

414 + char expected_url_repr[160];

415 + repr(expected_url_repr, sizeof(expected_url_repr), "info?blah?", 10);

416 +

417 + expect_value(__wrap_web_client_api_request_v1, host, localhost);

418 + expect_value(__wrap_web_client_api_request_v1, w, w);

419 + expect_string(__wrap_web_client_api_request_v1, url_repr, expected_url_repr);

420 +

421 + web_client_process_request(w);

422 +

423 + assert_string_equal(w->decoded_query_string, "");

424 + destroy_web_client(w);

425 + free(localhost);

426 +}

427 +

428 +static void cr_in_url(void **state)

429 +{

430 + (void)state;

431 + localhost = malloc(sizeof(RRDHOST));

432 +

433 + struct web_client *w = setup_fresh_web_client();

434 + buffer_strcat(w->response.data, "GET /api/v1/inf\ro\t?blah HTTP/1.1\r\n\r\n");

435 +

436 + char debug[160];

437 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

438 + printf("->%s\n", debug);

439 +

440 + char expected_url_repr[160];

441 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

442 +

443 + web_client_process_request(w);

444 +

445 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

446 +

447 + destroy_web_client(w);

448 + free(localhost);

449 +}

450 +static void newline_in_url(void **state)

451 +{

452 + (void)state;

453 + localhost = malloc(sizeof(RRDHOST));

454 +

455 + struct web_client *w = setup_fresh_web_client();

456 + buffer_strcat(w->response.data, "GET /api/v1/inf\no\t?blah HTTP/1.1\r\n\r\n");

457 +

458 + char debug[160];

459 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

460 + printf("->%s\n", debug);

461 +

462 + char expected_url_repr[160];

463 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

464 +

465 + web_client_process_request(w);

466 +

467 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

468 +

469 + destroy_web_client(w);

470 + free(localhost);

471 +}

472 +

473 +static void bad_version(void **state)

474 +{

475 + (void)state;

476 + localhost = malloc(sizeof(RRDHOST));

477 +

478 + struct web_client *w = setup_fresh_web_client();

479 + buffer_strcat(w->response.data, "GET /api/v1/info?blah HTTP/1.2\r\n\r\n");

480 +

481 + char debug[160];

482 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

483 + printf("->%s\n", debug);

484 +

485 + char expected_url_repr[160];

486 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

487 +

488 + web_client_process_request(w);

489 +

490 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

491 +

492 + destroy_web_client(w);

493 + free(localhost);

494 +}

495 +

496 +static void pathless_query(void **state)

497 +{

498 + (void)state;

499 + localhost = malloc(sizeof(RRDHOST));

500 +

501 + struct web_client *w = setup_fresh_web_client();

502 + buffer_strcat(w->response.data, "GET ?blah HTTP/1.1\r\n\r\n");

503 +

504 + char debug[160];

505 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

506 + printf("->%s\n", debug);

507 +

508 + char expected_url_repr[160];

509 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

510 +

511 + web_client_process_request(w);

512 +

513 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

514 +

515 + destroy_web_client(w);

516 + free(localhost);

517 +}

518 +

519 +static void pathless_fragment(void **state)

520 +{

521 + (void)state;

522 + localhost = malloc(sizeof(RRDHOST));

523 +

524 + struct web_client *w = setup_fresh_web_client();

525 + buffer_strcat(w->response.data, "GET #blah HTTP/1.1\r\n\r\n");

526 +

527 + char debug[160];

528 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

529 + printf("->%s\n", debug);

530 +

531 + char expected_url_repr[160];

532 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

533 +

534 + web_client_process_request(w);

535 +

536 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

537 +

538 + destroy_web_client(w);

539 + free(localhost);

540 +}

541 +

542 +static void short_percent(void **state)

543 +{

544 + (void)state;

545 + localhost = malloc(sizeof(RRDHOST));

546 +

547 + struct web_client *w = setup_fresh_web_client();

548 + buffer_strcat(w->response.data, "GET % HTTP/1.1\r\n\r\n");

549 +

550 + char debug[160];

551 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

552 + printf("->%s\n", debug);

553 +

554 + char expected_url_repr[160];

555 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

556 +

557 + web_client_process_request(w);

558 +

559 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

560 +

561 + destroy_web_client(w);

562 + free(localhost);

563 +}

564 +

565 +static void short_percent2(void **state)

566 +{

567 + (void)state;

568 + localhost = malloc(sizeof(RRDHOST));

569 +

570 + struct web_client *w = setup_fresh_web_client();

571 + buffer_strcat(w->response.data, "GET %0 HTTP/1.1\r\n\r\n");

572 +

573 + char debug[160];

574 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

575 + printf("->%s\n", debug);

576 +

577 + char expected_url_repr[160];

578 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

579 +

580 + web_client_process_request(w);

581 +

582 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

583 +

584 + destroy_web_client(w);

585 + free(localhost);

586 +}

587 +

588 +static void short_percent3(void **state)

589 +{

590 + (void)state;

591 + localhost = malloc(sizeof(RRDHOST));

592 +

593 + struct web_client *w = setup_fresh_web_client();

594 + buffer_strcat(w->response.data, "GET %");

595 +

596 + char debug[160];

597 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

598 + printf("->%s\n", debug);

599 +

600 + char expected_url_repr[160];

601 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

602 +

603 + web_client_process_request(w);

604 +

605 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

606 +

607 + destroy_web_client(w);

608 + free(localhost);

609 +}

610 +

611 +static void percent_nulls(void **state)

612 +{

613 + (void)state;

614 + localhost = malloc(sizeof(RRDHOST));

615 +

616 + struct web_client *w = setup_fresh_web_client();

617 + buffer_strcat(w->response.data, "GET %00%00%00%00%00%00 HTTP/1.1\r\n");

618 +

619 + char debug[160];

620 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

621 + printf("->%s\n", debug);

622 +

623 + char expected_url_repr[160];

624 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

625 +

626 + web_client_process_request(w);

627 +

628 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

629 +

630 + destroy_web_client(w);

631 + free(localhost);

632 +}

633 +

634 +static void percent_invalid(void **state)

635 +{

636 + (void)state;

637 + localhost = malloc(sizeof(RRDHOST));

638 +

639 + struct web_client *w = setup_fresh_web_client();

640 + buffer_strcat(w->response.data, "GET /%x%x%x%x%x%x HTTP/1.1\r\n");

641 +

642 + char debug[160];

643 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

644 + printf("->%s\n", debug);

645 +

646 + char expected_url_repr[160];

647 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

648 +

649 + web_client_process_request(w);

650 +

651 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

652 +

653 + destroy_web_client(w);

654 + free(localhost);

655 +}

656 +

657 +static void space_in_url(void **state)

658 +{

659 + (void)state;

660 + localhost = malloc(sizeof(RRDHOST));

661 +

662 + struct web_client *w = setup_fresh_web_client();

663 + buffer_strcat(w->response.data, "GET / / HTTP/1.1\r\n\r\n");

664 +

665 + char debug[160];

666 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

667 + printf("->%s\n", debug);

668 +

669 + char expected_url_repr[160];

670 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

671 +

672 + web_client_process_request(w);

673 +

674 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

675 +

676 + destroy_web_client(w);

677 + free(localhost);

678 +}

679 +

680 +static void random_sploit1(void **state)

681 +{

682 + (void)state;

683 + localhost = malloc(sizeof(RRDHOST));

684 +

685 + struct web_client *w = setup_fresh_web_client();

686 + // FIXME: Encoding probably needs to go through printf

687 + buffer_need_bytes(w->response.data, 55);

688 + memcpy(

689 + w->response.data->buffer,

690 + "GET \x03\x00\x00/*\xE0\x00\x00\x00\x00\x00Cookie: mstshash=Administr HTTP/1.1\r\n\r\n", 54);

691 + w->response.data->len = 54;

692 +

693 + char debug[160];

694 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

695 + printf("->%s\n", debug);

696 +

697 + char expected_url_repr[160];

698 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

699 +

700 + web_client_process_request(w);

701 +

702 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

703 +

704 + destroy_web_client(w);

705 + free(localhost);

706 +}

707 +

708 +static void null_in_url(void **state)

709 +{

710 + (void)state;

711 + localhost = malloc(sizeof(RRDHOST));

712 +

713 + struct web_client *w = setup_fresh_web_client();

714 + buffer_strcat(w->response.data, "GET / / HTTP/1.1\r\n\r\n");

715 + w->response.data->buffer[5] = 0;

716 +

717 + char debug[160];

718 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

719 + printf("->%s\n", debug);

720 +

721 + char expected_url_repr[160];

722 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

723 +

724 + web_client_process_request(w);

725 +

726 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

727 +

728 + destroy_web_client(w);

729 + free(localhost);

730 +}

731 +static void many_ands(void **state)

732 +{

733 + (void)state;

734 + localhost = malloc(sizeof(RRDHOST));

735 +

736 + struct web_client *w = setup_fresh_web_client();

737 + buffer_strcat(w->response.data, "GET foo?");

738 + for (size_t i = 0; i < 600; i++)

739 + buffer_strcat(w->response.data, "&");

740 + buffer_strcat(w->response.data, " HTTP/1.1\r\n\r\n");

741 +

742 + char debug[2048];

743 + repr(debug, sizeof(debug), w->response.data->buffer, w->response.data->len);

744 + printf("->%s\n", debug);

745 +

746 + char expected_url_repr[160];

747 + repr(expected_url_repr, sizeof(expected_url_repr), "inf\no\t", 6);

748 +

749 + web_client_process_request(w);

750 +

751 + assert_int_equal(w->response.code, HTTP_RESP_BAD_REQUEST);

752 +

753 + destroy_web_client(w);

754 + free(localhost);

755 +}

756 +

757 +int main(void)

758 +{

759 + debug_flags = 0xffffffffffff;

760 + int fails = 0;

761 +

762 + struct CMUnitTest static_tests[] = {

763 + cmocka_unit_test(only_root), cmocka_unit_test(two_slashes), cmocka_unit_test(valid_url),

764 + cmocka_unit_test(leading_blanks), cmocka_unit_test(empty_url), cmocka_unit_test(newline_in_url),

765 + cmocka_unit_test(not_a_query), cmocka_unit_test(cr_in_url), cmocka_unit_test(pathless_query),

766 + cmocka_unit_test(pathless_fragment), cmocka_unit_test(short_percent), cmocka_unit_test(short_percent2),

767 + cmocka_unit_test(short_percent3), cmocka_unit_test(percent_nulls), cmocka_unit_test(percent_invalid),

768 + cmocka_unit_test(space_in_url), cmocka_unit_test(random_sploit1), cmocka_unit_test(null_in_url),

769 + cmocka_unit_test(absolute_url),

770 + // cmocka_unit_test(many_ands), CMocka cannot recover after this crash

771 + cmocka_unit_test(bad_version)

772 + };

773 + (void)many_ands;

774 +

775 + fails += cmocka_run_group_tests_name("static_tests", static_tests, NULL, NULL);

776 + return fails;

777 +}

web/api/tests/web_api.c

+312 -19

@@ -8,8 +8,44 @@

8 #include <cmocka.h>

9 #include <stdbool.h>

11 +void repr(char *result, int result_size, char const *buf, int size)

12 +{

13 + int n;

14 + char *end = result + result_size - 1;

15 + unsigned char const *ubuf = (unsigned char const *)buf;

16 + while (size && result_size > 0) {

17 + if (*ubuf <= 0x20 || *ubuf >= 0x80) {

18 + n = snprintf(result, result_size, "\\%02X", *ubuf);

19 + } else {

20 + *result = *ubuf;

21 + n = 1;

22 + }

23 + result += n;

24 + result_size -= n;

25 + ubuf++;

26 + size--;

27 + }

28 + if (result_size > 0)

29 + *(result++) = 0;

30 + else

31 + *end = 0;

32 +}

33 +

34 +// ---------------------------------- Mocking accesses from web_client ------------------------------------------------

35 +

36 +ssize_t send(int sockfd, const void *buf, size_t len, int flags)

37 +{

38 + info("Mocking send: %zu bytes\n", len);

39 + (void)sockfd;

40 + (void)buf;

41 + (void)flags;

42 + return len;

43 +}

44 +

45 RRDHOST *__wrap_rrdhost_find_by_hostname(const char *hostname, uint32_t hash)

46 {

47 + (void)hostname;

48 + (void)hash;

49 return NULL;

50 }

@@ -18,72 +54,130 @@ RRDHOST *__wrap_rrdhost_find_by_hostname(const char *hostname, uint32_t hash)

54 void __wrap_finished_web_request_statistics(

55 uint64_t dt, uint64_t bytes_received, uint64_t bytes_sent, uint64_t content_size, uint64_t compressed_content_size)

56 {

57 + (void)dt;

58 + (void)bytes_received;

59 + (void)bytes_sent;

60 + (void)content_size;

61 + (void)compressed_content_size;

62 }

64 char *__wrap_config_get(struct config *root, const char *section, const char *name, const char *default_value)

65 {

66 if (!strcmp(section, CONFIG_SECTION_WEB) && !strcmp(name, "web files owner"))

67 return "netdata";

68 + (void)root;

69 + (void)default_value;

70 return "UNKNOWN FIX ME";

71 }

73 int __wrap_web_client_api_request_v1(RRDHOST *host, struct web_client *w, char *url)

74 {

32 - printf("api requests: %s\n", url);

75 + char url_repr[160];

76 + repr(url_repr, sizeof(url_repr), url, strlen(url));

77 + info("web_client_api_request_v1(url=\"%s\")\n", url_repr);

78 + check_expected_ptr(host);

79 + check_expected_ptr(w);

80 + check_expected_ptr(url_repr);

81 + return HTTP_RESP_OK;

82 }

84 int __wrap_rrdpush_receiver_thread_spawn(RRDHOST *host, struct web_client *w, char *url)

85 {

86 + (void)host;

87 + (void)w;

88 + (void)url;

89 return 0;

90 }

92 RRDHOST *__wrap_rrdhost_find_by_guid(const char *guid, uint32_t hash)

93 {

94 + (void)guid;

95 + (void)hash;

96 printf("FIXME: rrdset_find_guid\n");

97 return NULL;

98 }

100 RRDSET *__wrap_rrdset_find_byname(RRDHOST *host, const char *name)

101 {

102 + (void)host;

103 + (void)name;

104 printf("FIXME: rrdset_find_byname\n");

105 return NULL;

106 }

107

108 RRDSET *__wrap_rrdset_find(RRDHOST *host, const char *id)

109 {

110 + (void)host;

111 + (void)id;

112 printf("FIXME: rrdset_find\n");

113 return NULL;

114 }

115

116 +// -------------------------------- Mocking the log - capture per-test ------------------------------------------------

117 +

118 +char log_buffer[10240] = { 0 };

119 void __wrap_debug_int(const char *file, const char *function, const unsigned long line, const char *fmt, ...)

120 {

121 + (void)file;

122 + (void)function;

123 + (void)line;

124 va_list args;

125 va_start(args, fmt);

62 - printf("DEBUG: ");

63 - vprintf(fmt, args);

64 - printf("\n");

126 + size_t cur = strlen(log_buffer);

127 + snprintf(log_buffer + cur, sizeof(log_buffer) - cur, " DEBUG: ");

128 + cur = strlen(log_buffer);

129 + vsnprintf(log_buffer + cur, sizeof(log_buffer) - cur, fmt, args);

130 + cur = strlen(log_buffer);

131 + snprintf(log_buffer + cur, sizeof(log_buffer) - cur, "\n");

132 va_end(args);

133 }

134

135 void __wrap_info_int(const char *file, const char *function, const unsigned long line, const char *fmt, ...)

136 {

137 + (void)file;

138 + (void)function;

139 + (void)line;

140 va_list args;

141 va_start(args, fmt);

72 - printf("INFO: ");

73 - vprintf(fmt, args);

74 - printf("\n");

142 + size_t cur = strlen(log_buffer);

143 + snprintf(log_buffer + cur, sizeof(log_buffer) - cur, " INFO: ");

144 + cur = strlen(log_buffer);

145 + vsnprintf(log_buffer + cur, sizeof(log_buffer) - cur, fmt, args);

146 + cur = strlen(log_buffer);

147 + snprintf(log_buffer + cur, sizeof(log_buffer) - cur, "\n");

148 va_end(args);

149 }

150

151 void __wrap_error_int(

152 const char *prefix, const char *file, const char *function, const unsigned long line, const char *fmt, ...)

153 {

154 + (void)prefix;

155 + (void)file;

156 + (void)function;

157 + (void)line;

158 va_list args;

159 va_start(args, fmt);

83 - printf("ERROR: ");

160 + size_t cur = strlen(log_buffer);

161 + snprintf(log_buffer + cur, sizeof(log_buffer) - cur, " ERROR: ");

162 + cur = strlen(log_buffer);

163 + vsnprintf(log_buffer + cur, sizeof(log_buffer) - cur, fmt, args);

164 + cur = strlen(log_buffer);

165 + snprintf(log_buffer + cur, sizeof(log_buffer) - cur, "\n");

166 + va_end(args);

167 +}

168 +

169 +void __wrap_fatal_int(const char *file, const char *function, const unsigned long line, const char *fmt, ...)

170 +{

171 + (void)file;

172 + (void)function;

173 + (void)line;

174 + va_list args;

175 + va_start(args, fmt);

176 + printf("FATAL: ");

177 vprintf(fmt, args);

178 printf("\n");

179 va_end(args);

180 + fail();

181 }

182

183 WEB_SERVER_MODE web_server_mode = WEB_SERVER_MODE_STATIC_THREADED;

@@ -126,43 +220,242 @@ static void build_request(struct web_buffer *wb, const char *url, bool use_cr, s

220 buffer_strcat(wb, "\n");

221 }

222

129 -static struct web_client *pre_test_setup()

223 +/* Note: this is not a CMocka group_test_setup/teardown pair. This is performed per-test.

224 +*/

225 +static struct web_client *setup_fresh_web_client()

226 {

131 - localhost = malloc(sizeof(RRDHOST));

227 struct web_client *w = (struct web_client *)malloc(sizeof(struct web_client));

228 + memset(w, 0, sizeof(struct web_client));

229 w->response.data = buffer_create(NETDATA_WEB_RESPONSE_INITIAL_SIZE);

230 + w->response.data->date = 0; // Valgrind uninitialised value

231 + w->response.data->expires = 0; // Valgrind uninitialised value

232 + w->response.data->options = 0; // Valgrind uninitialised value

233 w->response.header = buffer_create(NETDATA_WEB_RESPONSE_HEADER_SIZE);

234 w->response.header_output = buffer_create(NETDATA_WEB_RESPONSE_HEADER_SIZE);

235 strcpy(w->origin, "*"); // Simulate web_client_create_on_fd()

236 w->cookie1[0] = 0; // Simulate web_client_create_on_fd()

237 w->cookie2[0] = 0; // Simulate web_client_create_on_fd()

238 + w->acl = 0x1f; // Everything on

239 return w;

240 }

241

142 -static void post_test_cleanup(struct web_client *w)

242 +static void destroy_web_client(struct web_client *w)

243 {

244 buffer_free(w->response.data);

245 buffer_free(w->response.header);

246 buffer_free(w->response.header_output);

247 free(w);

148 - free(localhost);

248 }

249

250 +// ---------------------------------- Parameterized test-families -----------------------------------------------------

251 +// There is no way to pass a parameter block into the setup fixture, we would have to patch CMocka and maintain it

252 +// locally. (The void **current_state in _run_group_tests would be set from a parameter). This is unfortunate as a

253 +// parameteric unit-tester needs to be to pass parameters to the fixtures. We are faking this by calculating the

254 +// space of tests in the launcher, passing an array of identical unit-tests to CMocka and then counting through the

255 +// parameters in the shared state passed between tests. To initialise this counter structure we use this global to

256 +// pass from the launcher (test-builder) to the setup-fixture.

257 +

258 +void *shared_test_state = NULL;

259 +

260 +// -------------------------------- Test family for /api/v1/info ------------------------------------------------------

261 +

262 +struct test_def {

263 + size_t num_headers; // Index coordinate

264 + size_t prefix_len; // Index coordinate

265 + char name[80];

266 + size_t full_len;

267 + struct web_client *instance; // Used within this single test

268 + bool completed, use_cr;

269 + struct test_def *next, *prev;

270 +};

271 +

272 static void api_info(void **state)

273 {

274 + (void)state;

275 + struct test_def *def = (struct test_def *)shared_test_state;

276 + shared_test_state = def->next;

277 +

278 + if (def->prev != NULL && !def->prev->completed && strlen(log_buffer) > 0) {

279 + printf("Log of failing case %s:\n", def->prev->name);

280 + puts(log_buffer);

281 + }

282 + log_buffer[0] = 0;

283 + if (localhost != NULL)

284 + free(localhost);

285 + localhost = calloc(1,sizeof(RRDHOST));

286 +

287 + def->instance = setup_fresh_web_client();

288 + build_request(def->instance->response.data, "/api/v1/info", def->use_cr, def->num_headers);

289 + def->instance->response.data->len = def->prefix_len;

290 +

291 + char buffer_repr[1024];

292 + repr(buffer_repr, sizeof(buffer_repr), def->instance->response.data->buffer,def->prefix_len);

293 + info("Buffer contains: %s [first %zu]", buffer_repr,def->prefix_len);

294 + if (def->prefix_len == def->full_len) {

295 + expect_value(__wrap_web_client_api_request_v1, host, localhost);

296 + expect_value(__wrap_web_client_api_request_v1, w, def->instance);

297 + expect_string(__wrap_web_client_api_request_v1, url_repr, "info");

298 + }

299 +

300 + web_client_process_request(def->instance);

301 +

302 + if (def->prefix_len == def->full_len)

303 + assert_int_equal(def->instance->flags & WEB_CLIENT_FLAG_WAIT_RECEIVE, 0);

304 + else

305 + assert_int_equal(def->instance->flags & WEB_CLIENT_FLAG_WAIT_RECEIVE, WEB_CLIENT_FLAG_WAIT_RECEIVE);

306 + assert_int_equal(def->instance->mode, WEB_CLIENT_MODE_NORMAL);

307 + def->completed = true;

308 + log_buffer[0] = 0;

309 +}

310 +

311 +static int api_info_launcher()

312 +{

313 + size_t num_tests = 0;

314 + struct web_client *template = setup_fresh_web_client();

315 + struct test_def *current, *head = NULL;

316 + struct test_def *prev = NULL;

317 +

318 + for (size_t i = 0; i < MAX_HEADERS; i++) {

319 + build_request(template->response.data, "/api/v1/info", true, i);

320 + for (size_t j = 0; j <= template->response.data->len; j++) {

321 + if (j == 0 && i > 0)

322 + continue; // All zero-length prefixes are identical, skip after first time

323 + current = malloc(sizeof(struct test_def));

324 + if (prev != NULL)

325 + prev->next = current;

326 + else

327 + head = current;

328 + current->prev = prev;

329 + prev = current;

330 +

331 + current->num_headers = i;

332 + current->prefix_len = j;

333 + current->full_len = template->response.data->len;

334 + current->instance = NULL;

335 + current->next = NULL;

336 + current->use_cr = true;

337 + current->completed = false;

338 + sprintf(

339 + current->name, "/api/v1/info@%zu,%zu/%zu+%d", current->num_headers, current->prefix_len,

340 + current->full_len,true);

341 + num_tests++;

342 + }

343 + }

344 for (size_t i = 0; i < MAX_HEADERS; i++) {

154 - struct web_client *w = pre_test_setup();

155 - build_request(w->response.data, "/api/v1/info", true, i);

156 - web_client_process_request(w);

157 - assert_int_equal(w->flags & WEB_CLIENT_FLAG_WAIT_RECEIVE, 0);

158 - post_test_cleanup(w);

345 + build_request(template->response.data, "/api/v1/info", false, i);

346 + for (size_t j = 0; j <= template->response.data->len; j++) {

347 + if (j == 0 && i > 0)

348 + continue; // All zero-length prefixes are identical, skip after first time

349 + current = malloc(sizeof(struct test_def));

350 + if (prev != NULL)

351 + prev->next = current;

352 + else

353 + head = current;

354 + current->prev = prev;

355 + prev = current;

356 +

357 + current->num_headers = i;

358 + current->prefix_len = j;

359 + current->full_len = template->response.data->len;

360 + current->instance = NULL;

361 + current->next = NULL;

362 + current->use_cr = false;

363 + current->completed = false;

364 + sprintf(

365 + current->name, "/api/v1/info@%zu,%zu/%zu+%d", current->num_headers, current->prefix_len,

366 + current->full_len,false);

367 + num_tests++;

368 + }

369 }

370 +

371 + struct CMUnitTest *tests = calloc(num_tests, sizeof(struct CMUnitTest));

372 + current = head;

373 + for (size_t i = 0; i < num_tests; i++) {

374 + tests[i].name = current->name;

375 + tests[i].test_func = api_info;

376 + tests[i].setup_func = NULL;

377 + tests[i].teardown_func = NULL;

378 + tests[i].initial_state = NULL;

379 + current = current->next;

380 + }

381 +

382 + printf("Setup %zu tests in %p\n", num_tests, head);

383 + shared_test_state = head;

384 + int fails = _cmocka_run_group_tests("web_api", tests, num_tests, NULL, NULL);

385 + free(tests);

386 + destroy_web_client(template);

387 + current = head;

388 + while (current != NULL) {

389 + struct test_def *c = current;

390 + current = current->next;

391 + if (c->instance != NULL) // Clean up resources from tests that failed

392 + destroy_web_client(c->instance);

393 + free(c);

394 + }

395 + if (localhost!=NULL)

396 + free(localhost);

397 + return fails;

398 }

399

400 +/* Raw notes for the cases that we did not use in the unit testing suite.

401 + Leaving them here instead of deleting them in-case we expand the suite during the

402 + work on the URL parser.

403 +

404 + Any ' ' in the URI -> invalid response (Description in 5.1 of RFC2616)

405 + Characters that can't be in paths #;?

406 + "GET /apb/../api/v1/info" HTTP/1.1\r\n"

407 +

408 + https://github.com/uriparser/uriparser/blob/uriparser-0.9.3/test/FourSuite.cpp

409 + Not clear why some of these are illegal -> reserved chars?

410 +

411 + ASSERT_TRUE(testBadUri("beepbeep\x07\x07", 8));

412 + ASSERT_TRUE(testBadUri("\n", 0));

413 + ASSERT_TRUE(testBadUri("::", 0)); // not OK, per Roy Fielding on the W3C uri list on 2004-04-01

414 +

415 + // the following test cases are from a Perl script by David A. Wheeler

416 + // at http://www.dwheeler.com/secure-programs/url.pl

417 + ASSERT_TRUE(testBadUri("http://www yahoo.com", 10));

418 + ASSERT_TRUE(testBadUri("http://www.yahoo.com/hello world/", 26));

419 + ASSERT_TRUE(testBadUri("http://www.yahoo.com/yelp.html#\"", 31));

420 +

421 + // the following test cases are from a Haskell program by Graham Klyne

422 + // at http://www.ninebynine.org/Software/HaskellUtils/Network/URITest.hs

423 + ASSERT_TRUE(testBadUri("[2010:836B:4179::836B:4179]", 0));

424 + ASSERT_TRUE(testBadUri(" ", 0));

425 + ASSERT_TRUE(testBadUri("%", 1));

426 + ASSERT_TRUE(testBadUri("A%Z", 2));

427 + ASSERT_TRUE(testBadUri("%ZZ", 1));

428 + ASSERT_TRUE(testBadUri("%AZ", 2));

429 + ASSERT_TRUE(testBadUri("A C", 1));

430 + ASSERT_TRUE(testBadUri("A\\'C", 1)); // r"A\'C"

431 + ASSERT_TRUE(testBadUri("A`C", 1));

432 + ASSERT_TRUE(testBadUri("A<C", 1));

433 + ASSERT_TRUE(testBadUri("A>C", 1));

434 + ASSERT_TRUE(testBadUri("A^C", 1));

435 + ASSERT_TRUE(testBadUri("A\\\\C", 1)); // r'A\\C'

436 + ASSERT_TRUE(testBadUri("A{C", 1));

437 + ASSERT_TRUE(testBadUri("A|C", 1));

438 + ASSERT_TRUE(testBadUri("A}C", 1));

439 + ASSERT_TRUE(testBadUri("A[C", 1));

440 + ASSERT_TRUE(testBadUri("A]C", 1));

441 + ASSERT_TRUE(testBadUri("A[**]C", 1));

442 + ASSERT_TRUE(testBadUri("http://[xyz]/", 8));

443 + ASSERT_TRUE(testBadUri("http://]/", 7));

444 + ASSERT_TRUE(testBadUri("http://example.org/[2010:836B:4179::836B:4179]", 19));

445 + ASSERT_TRUE(testBadUri("http://example.org/abc#[2010:836B:4179::836B:4179]", 23));

446 + ASSERT_TRUE(testBadUri("http://example.org/xxx/[qwerty]#a[b]", 23));

447 +

448 + // from a post to the W3C uri list on 2004-02-17

449 + // breaks at 22 instead of 17 because everything up to that point is a valid userinfo

450 + ASSERT_TRUE(testBadUri("http://w3c.org:80path1/path2", 22));

451 +

452 +*/

453 +

454 int main(void)

455 {

164 - const struct CMUnitTest tests[] = { cmocka_unit_test(api_info) };

456 debug_flags = 0xffffffffffff;

457 + int fails = 0;

458 + fails += api_info_launcher();

459

167 - return cmocka_run_group_tests_name("web_api", tests, NULL, NULL);

460 + return fails;

461 }

web/server/web_client.c

+6 -2

@@ -340,6 +340,8 @@ static inline int access_to_file_is_not_permitted(struct web_client *w, const ch

340 return HTTP_RESP_FORBIDDEN;

341 }

342

343 +// Work around a bug in the CMocka library by removing this function during testing.

344 +#ifndef REMOVE_MYSENDFILE

345 int mysendfile(struct web_client *w, char *filename) {

346 debug(D_WEB_CLIENT, "%llu: Looking for file '%s/%s'", w->id, netdata_configured_web_dir, filename);

347

@@ -448,11 +450,13 @@ int mysendfile(struct web_client *w, char *filename) {

450 w->response.data->date = statbuf.st_mtimespec.tv_sec;

451 #else

452 w->response.data->date = statbuf.st_mtim.tv_sec;

451 -#endif /* __APPLE__ */

453 +#endif

454 buffer_cacheable(w->response.data);

455

456 return HTTP_RESP_OK;

457 }

458 +#endif

459 +

460

461

462 #ifdef NETDATA_WITH_ZLIB

@@ -1266,7 +1270,7 @@ static inline void web_client_send_http_header(struct web_client *w) {

1270 while((bytes = SSL_write(w->ssl.conn, buffer_tostring(w->response.header_output), buffer_strlen(w->response.header_output))) < 0) {

1271 count++;

1272 if(count > 100 || (errno != EAGAIN && errno != EWOULDBLOCK)) {

1269 - error("Cannot send HTTP headers to web client.");

1273 + error("Cannot send HTTPS headers to web client.");

1274 break;

1275 }

1276 }