9cf9013dc · cryptotaxi247/netdata-1

@@ -1,365 +0,0 @@

1 -<!--

2 -title: "The build system"

3 -custom_edit_url: https://github.com/netdata/netdata/edit/master/BUILD.md

4 --->

5 -

6 -# The build system

7 -

8 -We are currently migrating from `autotools` to `CMake` as a build-system. This document

9 -currently describes how we intend to perform this migration, and will be updated after

10 -the migration to explain how the new `CMake` configuration works.

11 -

12 -## Stages during the build

13 -

14 -1. The `netdata-installer.sh`, take in arguments and environment settings to control the

15 - build.

16 -2. The configure step: `autoreconf -ivf ; ./configure` passing arguments into the configure

17 - script. This becomes `generation-time` in CMake. This includes package / system detection

18 - and configuration resulting in the `config.h` in the source root.

19 -3. The build step: recurse through the generated Makefiles and build the executable.

20 -4. The first install step: calls `make install` to handle all the install steps put into

21 - the Makefiles by the configure step (puts binaries / libraries / config into target

22 - tree structure).

23 -5. The second install step: the rest of the installer after the make install handles

24 - system-level configuration (privilege setting, user / groups, fetch/build/install `go.d`

25 - plugins, telemetry, installing service for startup, uninstaller, auto-updates.

26 -

27 -The ideal migration result is to replace all of this with the following steps:

28 -```

29 -mkdir build ; cd build ; cmake .. -D... ; cmake --build . --target install

30 -```

31 -

32 -The `-D...` indicates where the command-line arguments for configuration are passed into

33 -`CMake`.

34 -

35 -## CMake generation time

36 -

37 -At generation time we need to solve the following issues:

38 -

39 -### Feature flags

40 -

41 -Every command-line switch on the installer and the configure script needs to becomes an

42 -argument to the CMake generation, we can do this with variables in the CMake cache:

43 -

44 -CMakeLists.txt:

45 -```

46 -option(ENABLE_DBENGINE "Enable the dbengine storage" ON)

47 -...

48 -if(${ENABLE_DBENGINE})

49 -...

50 -endif()

51 -```

52 -

53 -Command-line interface

54 -```

55 -cmake -DENABLE_DBENGINE

56 -```

57 -

58 -### Dependency detection

59 -

60 -We have a mixture of soft- and hard-dependencies on libraries. For most of these we expect

61 -`pkg-config` information, for some we manually probe for libraries and include files. We

62 -should treat all of the external dependencies consistently:

63 -

64 -1. Default to autodetect using `pkg-config` (e.g. the standard `jemalloc` drops a `.pc`

65 - into the system but we do not check for it.

66 -2. If no `.pc` is found perform a manual search for libraries under known names, and

67 - check for accessible symbols inside them.

68 -3. Check that include paths work.

69 -4. Allow a command-line override (e.g. `-DWITH_JEMALLOC=/...`).

70 -5. If none of the above work then fail the install if the dependency is hard, otherwise

71 - indicate it is not present in the `config.h`.

72 -

73 -Before doing any dependency detection we need to determine which search paths are

74 -really in use for the current compiler, after the `project` declaration we can use:

75 -```

76 -execute_process(COMMAND ${CMAKE_C_COMPILER} "--print-search-dirs"

77 - COMMAND grep "^libraries:"

78 - COMMAND sed "s/^libraries: =//"

79 - COMMAND tr ":" " "

80 - COMMAND tr -d "\n"

81 - OUTPUT_VARIABLE CC_SEARCH_DIRS

82 - RESULTS_VARIABLE CC_SEARCH_RES)

83 -string(REGEX MATCH "^[0-9]+" CC_SEARCH_RES ${CC_SEARCH_RES})

84 -#string(STRIP "${CC_SEARCH_RES}" CC_SEARCH_RES)

85 -if(0 LESS ${CC_SEARCH_RES})

86 - message(STATUS "Warning - cannot determine standard compiler library paths")

87 - # Note: we will probably need a different method for Windows...

88 -endif()

89 -

90 -```

91 -

92 -The output format for this switch works on both `Clang` and `gcc`, it also includes

93 -the include search path, which can be extracted in a similar way. Standard advice here

94 -is to list the `ldconfig` cache or use the `-V` flag to check, but this does not work

95 -consistently across platforms - in particular `gcc` will reconfigure `ld` when it is

96 -called to gcc's internal view of search paths. During experiments each of these

97 -alternative missed / added unused paths. Dumping the compiler's own estimate of the

98 -search paths seems to work consistently across clang/gcc/linux/freebsd configurations.

99 -

100 -The default behaviour in CMake is to search across predefined paths (e.g. `CMAKE_LIBRARY_PATH`)

101 -that are based on heuristics about the current platform. Most projects using CMake seem

102 -to overwrite this with their own estimates.

103 -

104 -We can use the extracted paths as a base, add our own heuristics based on OS and then

105 -`set(CMAKE_LIBRARY_PATH ${OUR_OWN_LIB_SEARCH})` to get the best results. Roughly we do

106 -the following for each external dependency:

107 -```

108 -set(WITH_JSONC "Detect" CACHE STRING "Manually set the path to a json-c installation")

109 -...

110 -if(${WITH_JSONC} STREQUAL "Detect")

111 - pkg_check_modules(JSONC json-c) # Don't set the REQUIRED flag

112 - if(JSONC_FOUND)

113 - message(STATUS "libjsonc found through .pc -> ${JSONC_CFLAGS_OTHER} ${JSONC_LIBRARIES}")

114 - # ... setup using JSONC_CFLAGS_OTHER JSONC_LIBRARIES and JSONC_INCLUDE_DIRS

115 - else()

116 - find_library(LIB_JSONC

117 - NAMES json-c libjson-c

118 - PATHS ${CMAKE_LIBRARY_PATH}) # Includes our additions by this point

119 - if(${LIB_JSONC} STREQUAL "LIB_JSONC-NOTFOUND")

120 - message(STATUS "Library json-c not installed, disabling")

121 - else()

122 - check_library_exists(${LIB_JSONC} json_object_get_type "" HAVE_JSONC)

123 - # ... setup using heuristics for CFLAGS and check include files are available

124 - endif()

125 - endif()

126 -else()

127 - # ... use explicit path as base to check for library and includes ...

128 -endif()

129 -

130 -```

131 -

132 -For checking the include path we have two options, if we overwrite the `CMAKE_`... variables

133 -to change the internal search path we can use:

134 -```

135 -CHECK_INCLUDE_FILE(json/json.h HAVE_JSONC_H)

136 -```

137 -Or we can build a custom search path and then use:

138 -```

139 -find_file(HAVE_JSONC_H json/json.h PATHS ${OUR_INCLUDE_PATHS})

140 -```

141 -

142 -Note: we may have cases where there is no `.pc` but we have access to a `.cmake` (e.g. AWS SDK, mongodb,cmocka) - these need to be checked / pulled inside the repo while building a prototype.

143 -

144 -### Compiler compatibility checks

145 -

146 -In CMakeLists.txt:

147 -

148 -```

149 -CHECK_INCLUDE_FILE(sys/prctl.h HAVE_PRCTL_H)

150 -configure_file(cmake/config.in config.h)

151 -```

152 -

153 -In cmake/config.in:

154 -

155 -```

156 -#cmakedefine HAVE_PRCTL_H 1

157 -```

158 -

159 -If we want to check explicitly if something compiles (e.g. the accept4 check, or the

160 -`strerror_r` typing issue) then we set the `CMAKE_`... paths and then use:

161 -```

162 -check_c_source_compiles(

163 - "

164 - #include <string.h>

165 - int main() { char x = *strerror_r(0, &x, sizeof(x)); return 0; }

166 - "

167 - STRERROR_R_CHAR_P)

168 -

169 -```

170 -This produces a bool that we can use inside CMake or propagate into the `config.h`.

171 -

172 -We can handle the atomic checks with:

173 -```

174 -check_c_source_compiles(

175 - "

176 - int main (int argc, char **argv)

177 - {

178 - volatile unsigned long ul1 = 1, ul2 = 0, ul3 = 2;

179 - __atomic_load_n(&ul1, __ATOMIC_SEQ_CST);

180 - __atomic_compare_exchange(&ul1, &ul2, &ul3, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);

181 - __atomic_fetch_add(&ul1, 1, __ATOMIC_SEQ_CST);

182 - __atomic_fetch_sub(&ul3, 1, __ATOMIC_SEQ_CST);

183 - __atomic_or_fetch(&ul1, ul2, __ATOMIC_SEQ_CST);

184 - __atomic_and_fetch(&ul1, ul2, __ATOMIC_SEQ_CST);

185 - volatile unsigned long long ull1 = 1, ull2 = 0, ull3 = 2;

186 - __atomic_load_n(&ull1, __ATOMIC_SEQ_CST);

187 - __atomic_compare_exchange(&ull1, &ull2, &ull3, 1, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);

188 - __atomic_fetch_add(&ull1, 1, __ATOMIC_SEQ_CST);

189 - __atomic_fetch_sub(&ull3, 1, __ATOMIC_SEQ_CST);

190 - __atomic_or_fetch(&ull1, ull2, __ATOMIC_SEQ_CST);

191 - __atomic_and_fetch(&ull1, ull2, __ATOMIC_SEQ_CST);

192 - return 0;

193 - }

194 - "

195 - HAVE_C__ATOMIC)

196 -```

197 -

198 -For the specific problem of getting the correct type signature in log.c for the `strerror_r`

199 -calls we can replicate what we have now, or we can delete this code completely and use a

200 -better solution that is documented [here](http://www.club.cc.cmu.edu/~cmccabe/blog_strerror.html).

201 -To replicate what we have now:

202 -```

203 -check_c_source_compiles(

204 - "

205 - #include <string.h>

206 - int main() { char x = *strerror_r(0, &x, sizeof(x)); return 0; }

207 - "

208 - STRERROR_R_CHAR_P)

209 -

210 -check_c_source_compiles(

211 - "

212 - #include <string.h>

213 - int main() { int x = strerror_r(0, &x, sizeof(x)); return 0; }

214 - "

215 - STRERROR_R_INT)

216 -

217 -if("${STRERROR_R_CHAR_P}" OR "${STRERROR_R_INT}")

218 - set(HAVE_DECL_STRERROR_R 1)

219 -endif()

220 -message(STATUS "Result was ${HAVE_DECL_STRERROR_R}")

221 -

222 -```

223 -

224 -Note: I did not find an explicit way to select compiler when both `clang` and `gcc` are

225 -present. We might have an implicit way (like redirecting `cc`) but we should put one in.

226 -

227 -

228 -

229 -### Debugging problems in test compilations

230 -

231 -Test compilations attempt to feed a test-input into the targeted compiler and result

232 -in a yes/no decision, this is similar to `AC_LANG_SOURCE(.... if test $ac_...` in .`m4`.

233 -We have two techniques to use in CMake:

234 -```

235 -cmake_minimum_required(VERSION 3.1.0)

236 -include(CheckCCompilerFlag)

237 -project(empty C)

238 -

239 -check_c_source_compiles(

240 - "

241 - #include <string.h>

242 - int main() { char x = *strerror_r(0, &x, sizeof(x)); return 0; }

243 - "

244 - STRERROR_R_CHAR_P)

245 -

246 -try_compile(HAVE_JEMALLOC ${CMAKE_CURRENT_BINARY_DIR}

247 - ${CMAKE_CURRENT_SOURCE_DIR}/quickdemo.c

248 - LINK_LIBRARIES jemalloc)

249 -```

250 -

251 -The `check_c_source_compiles` is light-weight:

252 -

253 -* Inline source for the test, easy to follow.

254 -* Build errors are reported in `CMakeFiles/CMakeErrors.log`

255 -

256 -But we cannot alter the include-paths / library-paths / compiler-flags specifically for

257 -the test without overwriting the current CMake settings. The alternative approach is

258 -slightly more heavy-weight:

259 -

260 -* Can't inline source for `try_compile` - it requires a `.c` file in the tree.

261 -* Build errors are not shown, the recovery process for them is somewhat difficult.

262 -

263 -```

264 -rm -rf * && cmake .. --debug-trycompile

265 -grep jemal CMakeFiles/CMakeTmp/CMakeFiles/*dir/*

266 -cd CMakeFiles/CMakeTmp/CMakeFiles/cmTC_d6f0e.dir # for example

267 -cmake --build ../..

268 -```

269 -

270 -This implies that we can do this to diagnose problems / develop test-programs, but we

271 -have to make them *bullet-proof* as we cannot expose this to end-users. This means that

272 -the results of the compilation must be *crisp* - exactly yes/no if the feature we are

273 -testing is supported.

274 -

275 -### System configuration checks

276 -

277 -For any system configuration checks that fall outside of the above scope (includes, libraries,

278 -packages, test-compilation checks) we have a fall-back that we can use to glue any holes

279 -that we need, e.g. to pull out the packaging strings, inside the `CMakeLists.h`:

280 -```

281 -execute_process(COMMAND cat ${CMAKE_CURRENT_SOURCE_DIR}/packaging/version

282 - COMMAND tr -d '\n'

283 - OUTPUT_VARIABLE VERSION_FROM_FILE)

284 -message(STATUS "Packaging version ${VERSION_FROM_FILE}")

285 -```

286 -and this in the `config.h.in`:

287 -```

288 -#define VERSION_FROM_FILE "@VERSION_FROM_FILE@"

289 -```

290 -

291 -## CMake build time

292 -

293 -We have a working definition of the targets that is in use with CLion and works on modern

294 -CMake (3.15). It breaks on older CMake version (e.g. 3.7) with an error message (issue#7091).

295 -No PoC yet to fix this, but it looks like changing the target properties should do it (in the

296 -worst case we can drop the separate object completely and merge the sources directly into

297 -the final target).

298 -

299 -Steps needed for building a prototype:

300 -

301 -1. Pick a reasonable configuration.

302 -2. Use the PoC techniques above to do a full generation of `CMAKE_` variables in the cache

303 - according to the feature options and dependencies.

304 -3. Push these into the project variables.

305 -4. Work on it until the build succeeds in at least one known configuration.

306 -5. Smoke-test that the output is valid (i.e. the executable loads and runs, and we can

307 - access the dashboard).

308 -6. Do a full comparison of the `config.h` generated by autotools against the CMake version

309 - and document / fix any deviations.

310 -

311 -## CMake install target

312 -

313 -I've only looked at this superficially as we do not have a prototype yet, but each of the

314 -first-stage install steps (in `make install`) and the second-stage (in `netdata-installer.sh`)

315 -look feasible.

316 -

317 -## General issues

318 -

319 -* We need to choose a minimum CMake version that is an available package across all of our

320 - supported environments. There is currently a build issue #7091 that documents a problem

321 - in the compilation phase (we cannot link in libnetdata as an object on old CMake versions

322 - and need to find a different way to express this).

323 -

324 -* The default variable-expansion / comparisons in CMake are awkward, we need this to make it

325 - sane:

326 - ```

327 - cmake_policy(SET CMP0054 "NEW")

328 - ```

329 -* Default paths for libs / includes are not comprehensive on most environments, we still need

330 - some heuristics for common locations, e.g. `/usr/local` on FreeBSD.

331 -

332 -# Recommendations

333 -

334 -We should follow these steps:

335 -

336 -1. Build a prototype.

337 -2. Build a test-environment to check the prototype against environments / configurations that

338 - the team uses.

339 -3. Perform an "internal" release - merge the new CMake into master, but not announce it or

340 - offer to support it.

341 -4. Check it works for the team internally.

342 -5. Do a soft-release: offer it externally as a replacement option for autotools.

343 -6. Gather feedback and usage reports on a wider range of configurations.

344 -7. Do a hard-release: switch over the preferred build-system in the installation instructions.

345 -8. Gather feedback and usage reports on a wider range of configurations (again).

346 -9. Deprecate / remove the autotools build-system completely (so that we can support a single

347 - build-system).

348 -

349 -Some smaller miscellaneous suggestions:

350 -

351 -1. Remove the `_Generic` / `strerror_r` config to make the system simpler (use the technique

352 - on the blog post to make the standard version re-entrant so that it is thread-safe).

353 -2. Pull in jemalloc by source into the repo if it is our preferred malloc implementation.

354 -

355 -# Background

356 -

357 -* [Stack overflow starting point](https://stackoverflow.com/questions/7132862/how-do-i-convert-an-autotools-project-to-a-cmake-project#7680240)

358 -* [CMake wiki including previous autotools conversions](https://gitlab.kitware.com/cmake/community/wikis/Home)

359 -* [Commands section in old CMake docs](https://cmake.org/cmake/help/v2.8.8/cmake.html#section_Commands)

360 -* [try_compile in newer CMake docs](https://cmake.org/cmake/help/v3.7/command/try_compile.html)

361 -* [configure_file in newer CMake docs](https://cmake.org/cmake/help/v3.7/command/configure_file.html?highlight=configure_file)

362 -* [header checks in CMake](https://stackoverflow.com/questions/647892/how-to-check-header-files-and-library-functions-in-cmake-like-it-is-done-in-auto)

363 -* [how to write platform checks](https://gitlab.kitware.com/cmake/community/wikis/doc/tutorials/How-To-Write-Platform-Checks)

364 -

365 -

Delete BUILD.md (#14348)