master
rst 604 lines 23.7 KB
Raw
1 ==================================
2 The QEMU build system architecture
3 ==================================
4
5 This document aims to help developers understand the architecture of the
6 QEMU build system. As with projects using GNU autotools, the QEMU build
7 system has two stages; first the developer runs the "configure" script
8 to determine the local build environment characteristics, then they run
9 "make" to build the project. This is about where the similarities with
10 GNU autotools end, so try to forget what you know about them.
11
12 The two general ways to perform a build are as follows:
13
14 - build artifacts outside of QEMU source tree entirely::
15
16 cd ../
17 mkdir build
18 cd build
19 ../qemu/configure
20 make
21
22 - build artifacts in a subdir of QEMU source tree::
23
24 mkdir build
25 cd build
26 ../configure
27 make
28
29 Most of the actual build process uses Meson under the hood, therefore
30 build artifacts cannot be placed in the source tree itself.
31
32
33 Stage 1: configure
34 ==================
35
36 The configure script has five tasks:
37
38 - detect the host architecture
39
40 - list the targets for which to build emulators; the list of
41 targets also affects which firmware binaries and tests to build
42
43 - find the compilers (native and cross) used to build executables,
44 firmware and tests. The results are written as either Makefile
45 fragments (``config-host.mak``) or a Meson machine file
46 (``config-meson.cross``)
47
48 - create a virtual environment in which all Python code runs during
49 the build, and possibly install packages into it from PyPI
50
51 - invoke Meson in the virtual environment, to perform the actual
52 configuration step for the emulator build
53
54 The configure script automatically recognizes command line options for
55 which a same-named Meson option exists; dashes in the command line are
56 replaced with underscores.
57
58 Almost all QEMU developers that need to modify the build system will
59 only be concerned with Meson, and therefore can skip the rest of this
60 section.
61
62
63 Modifying ``configure``
64 -----------------------
65
66 ``configure`` is a shell script; it uses ``#!/bin/sh`` and therefore
67 should be compatible with any POSIX shell. It is important to avoid
68 using bash-isms to avoid breaking development platforms where bash is
69 not the default shell implementation.
70
71 The configure script provides a variety of functions to help writing
72 portable shell code and providing consistent behavior across architectures
73 and operating systems:
74
75 ``error_exit $MESSAGE $MORE...``
76 Print $MESSAGE to stderr, followed by $MORE... and then exit from the
77 configure script with non-zero status.
78
79 ``has $COMMAND``
80 Determine if $COMMAND exists in the current environment, either as a
81 shell builtin, or executable binary, returning 0 on success. The
82 replacement in Meson is ``find_program()``.
83
84 ``probe_target_compiler $TARGET``
85 Detect a cross compiler and cross tools for the QEMU target $TARGET (e.g.,
86 ``$CPU-softmmu``, ``$CPU-linux-user``, ``$CPU-bsd-user``). If a working
87 compiler is present, return success and set variables ``$target_cc``,
88 ``$target_ar``, etc. to non-empty values.
89
90 ``write_target_makefile``
91 Write a Makefile fragment to stdout, exposing the result of the most recent
92 ``probe_target_compiler`` call as the usual Make variables (``CC``,
93 ``AR``, ``LD``, etc.).
94
95
96 Configure does not generally perform tests for compiler options beyond
97 basic checks to detect the host platform and ensure the compiler is
98 functioning. These are performed using a few more helper functions:
99
100 ``compile_object $CFLAGS``
101 Attempt to compile a test program with the system C compiler using
102 $CFLAGS. The test program must have been previously written to a file
103 called $TMPC.
104
105 ``compile_prog $CFLAGS $LDFLAGS``
106 Attempt to compile a test program with the system C compiler using
107 $CFLAGS and link it with the system linker using $LDFLAGS. The test
108 program must have been previously written to a file called $TMPC.
109
110 ``check_define $NAME``
111 Determine if the macro $NAME is defined by the system C compiler.
112
113 ``do_compiler $CC $ARGS...``
114 Attempt to run the C compiler $CC, passing it $ARGS... This function
115 does not use flags passed via options such as ``--extra-cflags``, and
116 therefore can be used to check for cross compilers. However, most
117 such checks are done at ``make`` time instead (see for example the
118 ``cc-option`` macro in ``pc-bios/option-rom/Makefile``).
119
120 ``write_c_skeleton``
121 Write a minimal C program main() function to the temporary file
122 indicated by $TMPC.
123
124
125 Python virtual environments and the build process
126 -------------------------------------------------
127
128 An important step in ``configure`` is to create a Python virtual
129 environment (venv) during the configuration phase. The Python interpreter
130 comes from the ``--python`` command line option, the ``$PYTHON`` variable
131 from the environment, or the system PATH, in this order. The venv resides
132 in the ``pyvenv`` directory in the build tree, and provides consistency
133 in how the build process runs Python code.
134
135 At this stage, ``configure`` also queries the chosen Python interpreter
136 about QEMU's build dependencies. Note that the build process does *not*
137 look for ``meson`` or ``sphinx-build`` binaries in the PATH;
138 likewise, there are no options such as ``--meson`` or ``--sphinx-build``.
139 This avoids a potential mismatch, where Meson and Sphinx binaries on the
140 PATH might operate in a different Python environment than the one chosen
141 by the user during the build process. On the other hand, it introduces
142 a potential source of confusion where the user installs a dependency but
143 ``configure`` is not able to find it. When this happens, the dependency
144 was installed in the ``site-packages`` directory of another interpreter,
145 or with the wrong ``pip`` program.
146
147 If a package is available for the chosen interpreter, ``configure``
148 prepares a small script that invokes it from the venv itself\ [#distlib]_.
149 If not, ``configure`` can also optionally install dependencies in the
150 virtual environment with ``pip``, either from wheels in ``python/wheels``
151 or by downloading the package with PyPI. Downloading can be disabled with
152 ``--disable-download``; and anyway, it only happens when a ``configure``
153 option (currently, only ``--enable-docs``) is explicitly enabled but
154 the dependencies are not present.
155
156 .. [#distlib] The scripts are created based on the package's metadata,
157 specifically the ``console_script`` entry points. This is the
158 same mechanism that ``pip`` uses when installing a package.
159 Currently, in all cases it would be possible to use ``python -m``
160 instead of an entry point script, which makes this approach a
161 bit overkill. On the other hand, creating the scripts is
162 future proof and it makes the contents of the ``pyvenv/bin``
163 directory more informative. Portability is also not an issue,
164 because the Python Packaging Authority provides a package
165 ``distlib.scripts`` to perform this task.
166
167 The required versions of the packages are stored in a configuration file
168 ``pythondeps.toml``. The format is custom to QEMU, but it is documented
169 at the top of the file itself and it should be easy to understand. The
170 requirements should make it possible to use the version that is packaged
171 by QEMU's supported distros.
172
173 When dependencies are downloaded, instead, ``configure`` uses a "known
174 good" version that is also listed in ``pythondeps.toml``. In this
175 scenario, ``pythondeps.toml`` behaves like the "lock file" used by
176 ``cargo``, ``poetry`` or other dependency management systems.
177
178
179 Bundled Python packages
180 -----------------------
181
182 Python packages that are **mandatory** dependencies to build QEMU,
183 but are not available in all supported distros, are bundled with the
184 QEMU sources. The only one is currently Meson (outdated in Ubuntu
185 22.04 and openSUSE Leap).
186
187 In order to include a new or updated wheel, modify and rerun the
188 ``python/scripts/vendor.py`` script. The script embeds the
189 sha256 hash of package sources and checks it. The pypi.org web site
190 provides an easy way to retrieve the sha256 hash of the sources.
191
192
193 Stage 2: Meson
194 ==============
195
196 The Meson build system describes the build and install process for:
197
198 1) executables, which include:
199
200 - Tools - ``qemu-img``, ``qemu-nbd``, ``qemu-ga`` (guest agent), etc
201
202 - System emulators - ``qemu-system-$ARCH``
203
204 - Userspace emulators - ``qemu-$ARCH``
205
206 - Unit tests
207
208 2) documentation
209
210 3) ROMs, whether provided as binary blobs in the QEMU distributions
211 or cross compiled under the direction of the configure script
212
213 4) other data files, such as icons or desktop files
214
215 All executables are built by default, except for some ``contrib/``
216 binaries that are known to fail to build on some platforms (for example
217 32-bit or big-endian platforms). Tests are also built by default,
218 though that might change in the future.
219
220 The source code is highly modularized, split across many files to
221 facilitate building of all of these components with as little duplicated
222 compilation as possible. Using the Meson "sourceset" functionality,
223 ``meson.build`` files group the source files in rules that are
224 enabled according to the available system libraries and to various
225 configuration symbols. Sourcesets belong to one of four groups:
226
227 Subsystem sourcesets:
228 Various subsystems that are common to both tools and emulators have
229 their own sourceset, for example ``block_ss`` for the block device subsystem,
230 ``chardev_ss`` for the character device subsystem, etc. These sourcesets
231 are then turned into static libraries as follows::
232
233 libchardev = static_library('chardev', chardev_ss.sources(),
234 build_by_default: false)
235
236 chardev = declare_dependency(objects: libchardev.extract_all_objects(recursive: false),
237 dependencies: chardev_ss.dependencies())
238
239 Target-independent emulator sourcesets:
240 Various general purpose helper code is compiled only once and
241 the .o files are linked into all output binaries that need it.
242 This includes error handling infrastructure, standard data structures,
243 platform portability wrapper functions, etc.
244
245 Target-independent code lives in the ``common_ss``, ``system_ss`` and
246 ``user_ss`` sourcesets. ``common_ss`` is linked into all emulators,
247 ``system_ss`` only in system emulators, ``user_ss`` only in user-mode
248 emulators.
249
250 Target-dependent emulator sourcesets:
251 In the target-dependent set lives CPU emulation, some device emulation and
252 much glue code. This sometimes also has to be compiled multiple times,
253 once for each target being built. Target-dependent files are included
254 in the ``specific_ss`` sourceset.
255
256 Each emulator also includes sources for files in the ``hw/`` and ``target/``
257 subdirectories. The subdirectory used for each emulator comes
258 from the target's definition of ``TARGET_BASE_ARCH`` or (if missing)
259 ``TARGET_ARCH``, as found in ``configs/targets/*.mak``.
260
261 Each subdirectory in ``hw/`` adds one sourceset to the ``hw_arch`` dictionary,
262 for example::
263
264 arm_ss = ss.source_set()
265 arm_ss.add(files('boot.c'), fdt)
266 ...
267 hw_arch += {'arm': arm_ss}
268
269 The sourceset is only used for system emulators.
270
271 Each subdirectory in ``target/`` instead should add one sourceset to each
272 of the ``target_arch`` and ``target_system_arch``, which are used respectively
273 for all emulators and for system emulators only. For example::
274
275 arm_ss = ss.source_set()
276 arm_system_ss = ss.source_set()
277 ...
278 target_arch += {'arm': arm_ss}
279 target_system_arch += {'arm': arm_system_ss}
280
281 Module sourcesets:
282 There are two dictionaries for modules: ``modules`` is used for
283 target-independent modules and ``target_modules`` is used for
284 target-dependent modules. When modules are disabled the ``module``
285 source sets are added to ``system_ss`` and the ``target_modules``
286 source sets are added to ``specific_ss``.
287
288 Both dictionaries are nested. One dictionary is created per
289 subdirectory, and these per-subdirectory dictionaries are added to
290 the toplevel dictionaries. For example::
291
292 hw_display_modules = {}
293 qxl_ss = ss.source_set()
294 ...
295 hw_display_modules += { 'qxl': qxl_ss }
296 modules += { 'hw-display': hw_display_modules }
297
298 Utility sourcesets:
299 All binaries link with a static library ``libqemuutil.a``. This library
300 is built from several sourcesets; most of them however host generated
301 code, and the only two of general interest are ``util_ss`` and ``stub_ss``.
302
303 The separation between these two is purely for documentation purposes.
304 ``util_ss`` contains generic utility files. Even though this code is only
305 linked in some binaries, sometimes it requires hooks only in some of
306 these and depend on other functions that are not fully implemented by
307 all QEMU binaries. ``stub_ss`` links dummy stubs that will only be linked
308 into the binary if the real implementation is not present. In a way,
309 the stubs can be thought of as a portable implementation of the weak
310 symbols concept.
311
312
313 The following files concur in the definition of which files are linked
314 into each emulator:
315
316 ``configs/devices/*.mak``
317 The files under ``configs/devices/`` control the boards and devices
318 that are built into each QEMU system emulation targets. They merely contain
319 a list of config variable definitions such as::
320
321 include arm-softmmu.mak
322 CONFIG_XLNX_ZYNQMP_ARM=y
323 CONFIG_XLNX_VERSAL=y
324
325 ``*/Kconfig``
326 These files are processed together with ``configs/devices/*.mak`` and
327 describe the dependencies between various features, subsystems and
328 device models. They are described in :ref:`kconfig`
329
330 ``configs/targets/*.mak``
331 These files mostly define symbols that appear in the ``*-config-target.h``
332 file for each emulator\ [#cfgtarget]_. However, the ``TARGET_ARCH``
333 and ``TARGET_BASE_ARCH`` will also be used to select the ``hw/`` and
334 ``target/`` subdirectories that are compiled into each target.
335
336 .. [#cfgtarget] This header is included by ``qemu/osdep.h`` when
337 compiling files from the target-specific sourcesets.
338
339 These files rarely need changing unless you are adding a completely
340 new target, or enabling new devices or hardware for a particular
341 system/userspace emulation target
342
343
344 Adding checks
345 -------------
346
347 Compiler checks can be as simple as the following::
348
349 config_host_data.set('HAVE_BTRFS_H', cc.has_header('linux/btrfs.h'))
350
351 A more complex task such as adding a new dependency usually
352 comprises the following tasks:
353
354 - Add a Meson build option to meson_options.txt.
355
356 - Add code to perform the actual feature check.
357
358 - Add code to include the feature status in ``config-host.h``
359
360 - Add code to print out the feature status in the configure summary
361 upon completion.
362
363 Taking the probe for SDL2_Image as an example, we have the following
364 in ``meson_options.txt``::
365
366 option('sdl_image', type : 'feature', value : 'auto',
367 description: 'SDL Image support for icons')
368
369 Unless the option was given a non-``auto`` value (on the configure
370 command line), the detection code must be performed only if the
371 dependency will be used::
372
373 sdl_image = not_found
374 if not get_option('sdl_image').auto() or have_system
375 sdl_image = dependency('SDL2_image', required: get_option('sdl_image'),
376 method: 'pkg-config')
377 endif
378
379 This avoids warnings on static builds of user-mode emulators, for example.
380 Most of the libraries used by system-mode emulators are not available for
381 static linking.
382
383 The other supporting code is generally simple::
384
385 # Create config-host.h (if applicable)
386 config_host_data.set('CONFIG_SDL_IMAGE', sdl_image.found())
387
388 # Summary
389 summary_info += {'SDL image support': sdl_image.found()}
390
391 For the configure script to parse the new option, the
392 ``scripts/meson-buildoptions.sh`` file must be up-to-date; ``make
393 update-buildoptions`` (or just ``make``) will take care of updating it.
394
395
396 Support scripts
397 ---------------
398
399 Meson has a special convention for invoking Python scripts: if their
400 first line is ``#! /usr/bin/env python3`` and the file is *not* executable,
401 find_program() arranges to invoke the script under the same Python
402 interpreter that was used to invoke Meson. This is the most common
403 and preferred way to invoke support scripts from Meson build files,
404 because it automatically uses the value of configure's --python= option.
405
406 In case the script is not written in Python, use a ``#! /usr/bin/env ...``
407 line and make the script executable.
408
409 Scripts written in Python, where it is desirable to make the script
410 executable (for example for test scripts that developers may want to
411 invoke from the command line, such as tests/qapi-schema/test-qapi.py),
412 should be invoked through the ``python`` variable in meson.build. For
413 example::
414
415 test('QAPI schema regression tests', python,
416 args: files('test-qapi.py'),
417 env: test_env, suite: ['qapi-schema', 'qapi-frontend'])
418
419 This is needed to obey the --python= option passed to the configure
420 script, which may point to something other than the first python3
421 binary on the path.
422
423 By the time Meson runs, Python dependencies are available in the virtual
424 environment and should be invoked through the scripts that ``configure``
425 places under ``pyvenv``. One way to do so is as follows, using Meson's
426 ``find_program`` function::
427
428 sphinx_build = find_program(
429 fs.parent(python.full_path()) / 'sphinx-build',
430 required: get_option('docs'))
431
432
433 Stage 3: Make
434 =============
435
436 The next step in building QEMU is to invoke make. GNU Make is required
437 to build QEMU, and may be installed as ``gmake`` on some hosts.
438
439 The output of Meson is a ``build.ninja`` file, which is used with the
440 Ninja build tool. However, QEMU's build comprises other components than
441 just the emulators (namely firmware and the tests in ``tests/tcg``) which
442 need different cross compilers. The QEMU Makefile wraps both Ninja and
443 the smaller build systems for firmware and tests; it also takes care of
444 running ``configure`` again when the script changes. Apart from invoking
445 these sub-Makefiles, the resulting build is largely non-recursive.
446
447 Tests, whether defined in ``meson.build`` or not, are also ran by the
448 Makefile with the traditional ``make check`` phony target, while benchmarks
449 are run with ``make bench``. Meson test suites such as ``unit`` can be ran
450 with ``make check-unit``, and ``make check-tcg`` builds and runs "non-Meson"
451 tests for all targets.
452
453 If desired, it is also possible to use ``ninja`` and ``pyvenv/bin/meson test``,
454 respectively to build emulators and run tests defined in meson.build.
455 The main difference is that ``make`` needs the ``-jN`` flag in order to
456 enable parallel builds or tests.
457
458 Useful make targets
459 -------------------
460
461 ``help``
462 Print a help message for the most common build targets.
463
464 ``print-VAR``
465 Print the value of the variable VAR. Useful for debugging the build
466 system.
467
468
469 Important files for the build system
470 ====================================
471
472 Statically defined files
473 ------------------------
474
475 The following key files are statically defined in the source tree, with
476 the rules needed to build QEMU. Their behaviour is influenced by a
477 number of dynamically created files listed later.
478
479 ``Makefile``
480 The main entry point used when invoking make to build all the components
481 of QEMU. The default 'all' target will naturally result in the build of
482 every component.
483
484 ``*/meson.build``
485 The meson.build file in the root directory is the main entry point for the
486 Meson build system, and it coordinates the configuration and build of all
487 executables. Build rules for various subdirectories are included in
488 other meson.build files spread throughout the QEMU source tree.
489
490 ``python/scripts/mkvenv.py``
491 A wrapper for the Python ``venv`` and ``distlib.scripts`` packages.
492 It handles creating the virtual environment, creating scripts in
493 ``pyvenv/bin``, and calling ``pip`` to install dependencies.
494
495 ``tests/Makefile.include``
496 Rules for external test harnesses like the TCG tests.
497
498 ``tests/docker/Makefile.include``
499 Rules for Docker tests. Like ``tests/Makefile.include``, this file is
500 included directly by the top level Makefile, anything defined in this
501 file will influence the entire build system.
502
503 ``tests/vm/Makefile.include``
504 Rules for VM-based tests. Like ``tests/Makefile.include``, this file is
505 included directly by the top level Makefile, anything defined in this
506 file will influence the entire build system.
507
508 Dynamically created files
509 -------------------------
510
511 The following files are generated at run-time in order to control the
512 behaviour of the Makefiles. This avoids the need for QEMU makefiles to
513 go through any pre-processing as seen with autotools, where configure
514 generates ``Makefile`` from ``Makefile.in``.
515
516 Built by configure:
517
518 ``run``
519 Used to run commands / scripts from the git checkout. Sets ``$PATH``
520 to point to locally built binaries, and activates the python venv
521 before running the requested command. Pass the command to run as
522 args, for example::
523
524 $ ./build/run ./script/qmp/qmp-shell-wrap qemu-system-x86_64
525
526 will use the ``python3`` binary and site-packages from the local
527 venv to run ``qmp-shell-wrap`` and spawn the QEMU emulator from
528 the build directory.
529
530 ``config-host.mak``
531 When configure has determined the characteristics of the build host it
532 will write the paths to various tools to this file, for use in ``Makefile``
533 and to a smaller extent ``meson.build``.
534
535 ``config-host.mak`` is also used as a dependency checking mechanism. If make
536 sees that the modification timestamp on configure is newer than that on
537 ``config-host.mak``, then configure will be re-run.
538
539 ``config-meson.cross``
540
541 A Meson "cross file" (or native file) used to communicate the paths to
542 the toolchain and other configuration options.
543
544 ``config.status``
545
546 A small shell script that will invoke configure again with the same
547 environment variables that were set during the first run. It's used to
548 rerun configure after changes to the source code, but it can also be
549 inspected manually to check the contents of the environment.
550
551 ``Makefile.prereqs``
552
553 A set of Makefile dependencies that order the build and execution of
554 firmware and tests after the container images and emulators that they
555 need.
556
557 ``pc-bios/*/config.mak``, ``tests/tcg/config-host.mak``, ``tests/tcg/*/config-target.mak``
558
559 Configuration variables used to build the firmware and TCG tests,
560 including paths to cross compilation toolchains.
561
562 ``pyvenv``
563
564 A Python virtual environment that is used for all Python code running
565 during the build. Using a virtual environment ensures that even code
566 that is run via ``sphinx-build``, ``meson`` etc. uses the same interpreter
567 and packages.
568
569 Built by Meson:
570
571 ``config-host.h``
572 Used by C code to determine the properties of the build environment
573 and the set of enabled features for the entire build.
574
575 ``${TARGET-NAME}-config-devices.mak``
576 TARGET-NAME is the name of a system emulator. The file is
577 generated by Meson using files under ``configs/devices`` as input.
578
579 ``${TARGET-NAME}-config-target.mak``
580 TARGET-NAME is the name of a system or usermode emulator. The file is
581 generated by Meson using files under ``configs/targets`` as input.
582
583 ``$TARGET_NAME-config-target.h``, ``$TARGET_NAME-config-devices.h``
584 Used by C code to determine the properties and enabled
585 features for each target. enabled. They are generated from
586 the contents of the corresponding ``*.mak`` files using Meson's
587 ``configure_file()`` function; each target can include them using
588 the ``CONFIG_TARGET`` and ``CONFIG_DEVICES`` macro respectively.
589
590 ``build.ninja``
591 The build rules.
592
593
594 Built by Makefile:
595
596 ``Makefile.ninja``
597 A Makefile include that bridges to ninja for the actual build. The
598 Makefile is mostly a list of targets that Meson included in build.ninja.
599
600 ``Makefile.mtest``
601 The Makefile definitions that let "make check" run tests defined in
602 meson.build. The rules are produced from Meson's JSON description of
603 tests (obtained with "meson introspect --tests") through the script
604 scripts/mtest2make.py.