| 1 | # Build native (DEB/RPM) packages for testing |
| 2 | |
| 3 | This document provides instructions for developers who need to build native packages locally for testing. |
| 4 | |
| 5 | ## Requirements |
| 6 | |
| 7 | To build native packages locally, you will need the following: |
| 8 | |
| 9 | * A working Docker or Podman host. |
| 10 | * A local copy of the source tree you want to build from. |
| 11 | |
| 12 | ## Building the packages |
| 13 | |
| 14 | In the root of the source tree from which you want to build, clean up any existing files left over from a previous build |
| 15 | and then run: |
| 16 | |
| 17 | ```bash |
| 18 | docker run -it --rm -e VERSION=0.1 -v $PWD:/netdata netdata/package-builders:<tag> |
| 19 | ``` |
| 20 | |
| 21 | or |
| 22 | |
| 23 | ```bash |
| 24 | podman run -it --rm -e VERSION=0.1 -v $PWD:/netdata netdata/package-builders:<tag> |
| 25 | ``` |
| 26 | |
| 27 | The `<tag>` should be the lowercase distribution name with no spaces, followed by the |
| 28 | release of that distribution and then a `-v1` or `-v2` depending on the distro (DEB based distros use `-v2` currently, RPM based distros use `-v1` currently). For example, `centos7-v1` to build on CentOS 7, or `ubuntu20.04-v2` |
| 29 | to build on Ubuntu 20.04. Note that we use Rocky Linux for builds on CentOS/RHEL 8 or newer. See |
| 30 | [netdata/package-builders](https://hub.docker.com/r/netdata/package-builders/tags) for all available tags. |
| 31 | |
| 32 | The value passed in the `VERSION` environment variable can be any version number accepted by the type of package |
| 33 | being built. As a general rule, it needs to start with a digit, and must include a `.` somewhere. |
| 34 | |
| 35 | Once it finishes, the built packages can be found under `artifacts/` in the source tree. |
| 36 | |
| 37 | If an error is encountered and the build is being run interactively, it will drop to a shell to allow you to |
| 38 | inspect the state of the container and look at build logs. |
| 39 | |
| 40 | ### Detailed explanation |
| 41 | |
| 42 | The environments used for building our packages are fully self-contained Docker images built from [Dockerfiles](https://github.com/netdata/helper-images/tree/master/package-builders) |
| 43 | These are published on Docker |
| 44 | Hub with the image name `netdata/package-builders`, and tagged using the name and version of the distribution |
| 45 | (with the tag corresponding to the suffix on the associated Dockerfile). |
| 46 | |
| 47 | The build code expects the following requirements to be met: |
| 48 | |
| 49 | * It expects the source tree it should build from to be located at `/netdata`, and expects that said source tree |
| 50 | is clean (no artifacts left over from previous builds). |
| 51 | * It expects an environment variable named `VERSION` to be defined, and uses this to control what version number |
| 52 | will be shown in the package metadata and filenames. |
| 53 | |
| 54 | Internally, the source tree gets copied to a temporary location for the build process so that the source tree can |
| 55 | be mounted directly from the host without worrying about leaving a dirty tree behind, any templating or file |
| 56 | movements required for the build to work are done, the package build command is invoked with the correct arguments, |
| 57 | and then the resultant packages are copied to the `artifacts/` directory in the original source tree so they are |
| 58 | accessible after the container exits. |
| 59 | |
| 60 | ## Finding build logs after a failed build |
| 61 | |
| 62 | Build logs and artifacts can be found in the build directory, whose location varies by distribution. |
| 63 | |
| 64 | On DEB systems (Ubuntu and Debian), the build directory inside the container is located at `/usr/src/netdata` |
| 65 | |
| 66 | On RPM systems except openSUSE, the build directory inside the container is located under `/root/rpmbuild/BUILD/` |
| 67 | and varies based on the package version number. |
| 68 | |
| 69 | On openSUSE, the build directory inside the container is located under `/usr/src/packages/BUILD`and varies based |
| 70 | on the package version number. |
| 71 | |
| 72 | ## Building for other architectures |
| 73 | |
| 74 | If you need to test a build for an architecture that does not match your host system, you can do so by setting up |
| 75 | QEMU user-mode emulation. This requires a Linux kernel with binfmt\_misc support (all modern distributions provide |
| 76 | this out of the box, but I’m not sure about WSL or Docker Desktop). |
| 77 | |
| 78 | The quick and easy way to do this is to run the following: |
| 79 | |
| 80 | ```bash |
| 81 | docker run --rm --privileged multiarch/qemu-user-static --reset -p yes |
| 82 | ``` |
| 83 | |
| 84 | or |
| 85 | |
| 86 | ```bash |
| 87 | podman run --rm --privileged multiarch/qemu-user-static --reset -p yes |
| 88 | ``` |
| 89 | |
| 90 | This will set up the required QEMU user-mode emulation until you reboot. Note that if using Podman, you will need |
| 91 | to run this as root and not as a rootless container (the package builds work fine in a rootless container though, |
| 92 | even if doing cross-architecture builds). |
| 93 | |
| 94 | Once you have that set up, the command to build the packages is the same as above, you just need to add a correct |
| 95 | `--platform` option to the `docker run` or `podman run` command. The current list of architectures we build for, |
| 96 | and the correct value for the `--platform` option is: |
| 97 | |
| 98 | * 32-bit ARMv7: `linux/arm/v7` |
| 99 | * 64-bit ARMv8: `linux/arm64/v8` |
| 100 | * 32-bit x86: `linux/i386` |
| 101 | * 64-bit x86: `linux/amd64` |