@matheusbsbs / matheus-bagliano / commits / 0e3b67e2aa

ci: add support for GitLab CI

We already support Azure Pipelines and GitHub Workflows in the Git project, but until now we do not have support for GitLab CI. While it is arguably not in the interest of the Git project to maintain a ton of different CI platforms, GitLab has recently ramped up its efforts and tries to contribute to the Git project more regularly. Part of a problem we hit at GitLab rather frequently is that our own, custom CI setup we have is so different to the setup that the Git project has. More esoteric jobs like "linux-TEST-vars" that also set a couple of environment variables do not exist in GitLab's custom CI setup, and maintaining them to keep up with what Git does feels like wasted time. The result is that we regularly send patch series upstream that fail to compile or pass tests in GitHub Workflows. We would thus like to integrate the GitLab CI configuration into the Git project to help us send better patch series upstream and thus reduce overhead for the maintainer. Results of these pipeline runs will be made available (at least) in GitLab's mirror of the Git project at [1]. This commit introduces the integration into our regular CI scripts so that most of the setup continues to be shared across all of the CI solutions. Note that as the builds on GitLab CI run as unprivileged user, we need to pull in both sudo and shadow packages to our Alpine based job to set this up. [1]: https://gitlab.com/gitlab-org/git Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>

Patrick Steinhardt committed Nov 9, 2023 at 09:05 UTC 0e3b67e2aa25edb7e1a5c999c87b52a7b3a7649a
4 files changed +116 -1
.gitlab-ci.yml new
+53
@@ -0,0 +1,53 @@
1 +default:
2 + timeout: 2h
3 +
4 +workflow:
5 + rules:
6 + - if: $CI_PIPELINE_SOURCE == "merge_request_event"
7 + - if: $CI_COMMIT_TAG
8 + - if: $CI_COMMIT_REF_PROTECTED == "true"
9 +
10 +test:
11 + image: $image
12 + before_script:
13 + - ./ci/install-docker-dependencies.sh
14 + script:
15 + - useradd builder --create-home
16 + - chown -R builder "${CI_PROJECT_DIR}"
17 + - sudo --preserve-env --set-home --user=builder ./ci/run-build-and-tests.sh
18 + after_script:
19 + - |
20 + if test "$CI_JOB_STATUS" != 'success'
21 + then
22 + sudo --preserve-env --set-home --user=builder ./ci/print-test-failures.sh
23 + fi
24 + parallel:
25 + matrix:
26 + - jobname: linux-sha256
27 + image: ubuntu:latest
28 + CC: clang
29 + - jobname: linux-gcc
30 + image: ubuntu:20.04
31 + CC: gcc
32 + CC_PACKAGE: gcc-8
33 + - jobname: linux-TEST-vars
34 + image: ubuntu:20.04
35 + CC: gcc
36 + CC_PACKAGE: gcc-8
37 + - jobname: linux-gcc-default
38 + image: ubuntu:latest
39 + CC: gcc
40 + - jobname: linux-leaks
41 + image: ubuntu:latest
42 + CC: gcc
43 + - jobname: linux-asan-ubsan
44 + image: ubuntu:latest
45 + CC: clang
46 + - jobname: pedantic
47 + image: fedora:latest
48 + - jobname: linux-musl
49 + image: alpine:latest
50 + artifacts:
51 + paths:
52 + - t/failed-test-artifacts
53 + when: on_failure
ci/install-docker-dependencies.sh
+12 -1
@@ -16,11 +16,22 @@ linux32)
16 '
17 ;;
18 linux-musl)
19 - apk add --update build-base curl-dev openssl-dev expat-dev gettext \
19 + apk add --update shadow sudo build-base curl-dev openssl-dev expat-dev gettext \
20 pcre2-dev python3 musl-libintl perl-utils ncurses \
21 apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \
22 bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
23 ;;
24 +linux-*)
25 + # Required so that apt doesn't wait for user input on certain packages.
26 + export DEBIAN_FRONTEND=noninteractive
27 +
28 + apt update -q &&
29 + apt install -q -y sudo git make language-pack-is libsvn-perl apache2 libssl-dev \
30 + libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev \
31 + perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl \
32 + libdbd-sqlite3-perl libio-socket-ssl-perl libnet-smtp-ssl-perl ${CC_PACKAGE:-${CC:-gcc}} \
33 + apache2 cvs cvsps gnupg libcgi-pm-perl subversion
34 + ;;
35 pedantic)
36 dnf -yq update >/dev/null &&
37 dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
ci/lib.sh
+45
@@ -14,6 +14,22 @@ then
14 need_to_end_group=
15 echo '::endgroup::' >&2
16 }
17 +elif test true = "$GITLAB_CI"
18 +then
19 + begin_group () {
20 + need_to_end_group=t
21 + printf "\e[0Ksection_start:$(date +%s):$(echo "$1" | tr ' ' _)\r\e[0K$1\n"
22 + trap "end_group '$1'" EXIT
23 + set -x
24 + }
25 +
26 + end_group () {
27 + test -n "$need_to_end_group" || return 0
28 + set +x
29 + need_to_end_group=
30 + printf "\e[0Ksection_end:$(date +%s):$(echo "$1" | tr ' ' _)\r\e[0K\n"
31 + trap - EXIT
32 + }
33 else
34 begin_group () { :; }
35 end_group () { :; }
@@ -229,6 +245,35 @@ then
245
246 GIT_TEST_OPTS="--github-workflow-markup"
247 JOBS=10
248 +elif test true = "$GITLAB_CI"
249 +then
250 + CI_TYPE=gitlab-ci
251 + CI_BRANCH="$CI_COMMIT_REF_NAME"
252 + CI_COMMIT="$CI_COMMIT_SHA"
253 + case "$CI_JOB_IMAGE" in
254 + macos-*)
255 + CI_OS_NAME=osx;;
256 + alpine:*|fedora:*|ubuntu:*)
257 + CI_OS_NAME=linux;;
258 + *)
259 + echo "Could not identify OS image" >&2
260 + env >&2
261 + exit 1
262 + ;;
263 + esac
264 + CI_REPO_SLUG="$CI_PROJECT_PATH"
265 + CI_JOB_ID="$CI_JOB_ID"
266 + CC="${CC_PACKAGE:-${CC:-gcc}}"
267 + DONT_SKIP_TAGS=t
268 + handle_failed_tests () {
269 + create_failed_test_artifacts
270 + return 1
271 + }
272 +
273 + cache_dir="$HOME/none"
274 +
275 + runs_on_pool=$(echo "$CI_JOB_IMAGE" | tr : -)
276 + JOBS=$(nproc)
277 else
278 echo "Could not identify CI type" >&2
279 env >&2
ci/print-test-failures.sh
+6
@@ -51,6 +51,12 @@ do
51 tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
52 continue
53 ;;
54 + gitlab-ci)
55 + mkdir -p failed-test-artifacts
56 + cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/
57 + tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
58 + continue
59 + ;;
60 *)
61 echo "Unhandled CI type: $CI_TYPE" >&2
62 exit 1