Add a document outlining how to build native packages locally. (#12431)
* Add a document outlining how to build native packages locally. For developers and contributors so they don’t have to bug me every time they need to do it. * Apply suggestions from code review * Further updates to package build documentation. * Further updates. * Apply suggestions from code review Co-authored-by: DShreve2 <david@netdata.cloud> * Even more updates. * Apply suggestions from code review Co-authored-by: DShreve2 <david@netdata.cloud> Co-authored-by: Tina Luedtke <kickoke@users.noreply.github.com> Co-authored-by: DShreve2 <david@netdata.cloud>
Austin S. Hemmelgarn committed
Sep 21, 2022 at 07:25 UTC
4b4873487df0d65f4791e933fc6be424e966f103
1 file changed
+106
packaging/building-native-packages-locally.md
new
+106
@@ -0,0 +1,106 @@
1
+<!--
2
+title: How to build native (DEB/RPM) packages locally for testing
3
+description: Instructions for developers who need to build native packages locally for testing.
4
+custom_edit_url: https://github.com/netdata/netdata/edit/master/packaging/building-native-packages-locally.md
5
+keywords: [Netdata native package, Netdata RPM, Netdata DEB, Testing native packages Netdata]
6
+-->
7
+
8
+# How to build native (DEB/RPM) packages locally for testing
9
+
10
+## Requirements
11
+
12
+To build native packages locally, you will need the following:
13
+
14
+* A working Docker or Podman host.
15
+* A local copy of the source tree you want to build from.
16
+
17
+## Building the packages
18
+
19
+In the root of the source tree from which you want to build, clean up any existing files left over from a previous build
20
+and then run:
21
+
22
+```bash
23
+docker run -it --rm -e VERSION=0.1 -v $PWD:/netdata netdata/package-builders:<tag>
24
+```
25
+
26
+or
27
+
28
+```bash
29
+podman run -it --rm -e VERSION=0.1 -v $PWD:/netdata netdata/package-builders:<tag>
30
+```
31
+
32
+The `<tag>` should be the lowercase distribution name with no spaces, followed by the
33
+release of that distribution. For example, `centos7` to build on CentOS 7, or `ubuntu20.04`
34
+to build on Ubuntu 20.04. Note that we use Alma Linux for builds on CentOS/RHEL 8 or newer. See
35
+[netdata/package-builders](https://hub.docker.com/r/netdata/package-builders/tags) for all available tags.
36
+
37
+The value passed in the `VERSION` environment variable can be any version number accepted by the type of package
38
+being built. As a general rule, it needs to start with a digit, and must include a `.` somewhere.
39
+
40
+Once it finishes, the built packages can be found under `artifacts/` in the source tree.
41
+
42
+If an error is encountered and the build is being run interactively, it will drop to a shell to allow you to
43
+inspect the state of the container and look at build logs.
44
+
45
+### Detailed explanation
46
+
47
+The environments used for building our packages are fully self-contianed Docker images built from [Dockerfiles](https://github.com/netdata/helper-images/tree/master/package-builders)
48
+These are published on Docker
49
+Hub with the image name `netdata/package-builders`, and tagged using the name and version of the distribution
50
+(with the tag corresponding to the suffix on the associated Dockerfile).
51
+
52
+The build code expects the following requirements to be met:
53
+
54
+- It expects the source tree it should build from to be located at `/netdata`, and expects that said source tree
55
+ is clean (no artifacts left over from previous builds).
56
+- It expects an environment variable named `VERSION` to be defined, and uses this to control what version number
57
+ will be shown in the package metadata and filenames.
58
+
59
+Internally, the source tree gets copied to a temporary location for the build process so that the source tree can
60
+be mounted directly from the host without worrying about leaving a dirty tree behind, any templating or file
61
+movements required for the build to work are done, the package build command is invoked with the correct arguments,
62
+and then the resultant packages are copied to the `artifacts/` directory in the original source tree so they are
63
+accessible after the container exits.
64
+
65
+## Finding build logs after a failed build
66
+
67
+Build logs and artifacts can be found in the build directory, whose location varies by distribution.
68
+
69
+On DEB systems (Ubuntu and Debian), the build directory inside the container is located at `/usr/src/netdata`
70
+
71
+On RPM systems except openSUSE, the build directory inside the container is located under `/root/rpmbuild/BUILD/`
72
+and varies based on the package version number.
73
+
74
+On openSUSE, the build directory inside the container is located under `/usr/src/packages/BUILD`and varies based
75
+on the package version number.
76
+
77
+## Building for other architectures
78
+
79
+If you need to test a build for an architecture that does not match your host system, you can do so by setting up
80
+QEMU user-mode emulation. This requires a Linux kernel with binfmt\_misc support (all modern distributions provide
81
+this out of the box, but I’m not sure about WSL or Docker Desktop).
82
+
83
+The quick and easy way to do this is to run the following:
84
+
85
+```bash
86
+docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
87
+```
88
+
89
+or
90
+
91
+```bash
92
+podman run --rm --privileged multiarch/qemu-user-static --reset -p yes
93
+```
94
+
95
+This will set up the required QEMU user-mode emulation until you reboot. Note that if using Podman, you will need
96
+to run this as root and not as a rootless container (the package builds work fine in a rootless container though,
97
+even if doing cross-architecture builds).
98
+
99
+Once you have that set up, the command to build the packages is the same as above, you just need to add a correct
100
+`--platform` option to the `docker run` or `podman run` command. The current list of architectures we build for,
101
+and the correct value for the `--platform` option is:
102
+
103
+- 32-bit ARMv7: `linux/arm/v7`
104
+- 64-bit ARMv8: `linux/arm64/v8`
105
+- 32-bit x86: `linux/i386`
106
+- 64-bit x86: `linux/amd64`