@cryptotaxi247 / netdata-1 / commits / 69050e602

netdata/build: A small house keeping around coverity scan (no functional or process change) (#5656)

* netdata/build: a small house keeping around coverity scan (no functional or process change) 1) Split coverity install on seperate scriptlet, call it through travis when needed 2) Remove shellcheck exclussions, they dont seem to be raised any more (i might be using wrong version of shellcheck, will see) 3) Fix authors,copyrights etc 4) Make sure all echo goes to stderr 5) When installing, use mktemp for the temporary directory and put everything there. Remove afterwards * netdata/build: missed - remove temp folder once done * Rebumping with empty commit to trigger build * Temp commit to force execution of modified code for coverity * [ci skip] Revert "Temp commit to force execution of modified code for coverity" This reverts commit 70e770cbbcb2b9cb4c8040d6249688c810d823f3. * Temp commit to force execution of modified code for coverity * Temp commit to force execution of modified code for coverity * packaging/netdata/coverity-scan: Missed PATH export during scan phase * netdata/packaging/coverity-scan: the source folder not so temporary after all Use a more permanent path for the workdir and we will deep dive on the details of this if necessary at a later stage * netdata/packaging: This is a more general change, push REPOSITORY variable up to travis layer to help testing against personal accounts * netdata/packaging/coverity-scan: Missed folder creation (no longer using mktemp, duh), also make sure you do a couple of validations so that you know install is failed or not, otherwise we exit with success but nothing has happened * netdata/coverity-scan: a few more changes towards abstraction 1) move for submit email to travis environment variables 2) Use REPOSITORY in project name 3) Add some new lines on the last log message, to separate it from the curl result * netdata/coverity: missed parameter from echo * Reinstate the flag that controls coverity scan execution (was removed for testing purposes, to run coverity in a controlled environment) * netdata/coverity-scan: Control coverity scan results submission and report back the potential error without blocking the pipeline by failing the step completely

Paul Katsoulakis committed Mar 19, 2019 at 18:22 UTC 69050e602882523815fd3466ce766b440d0e85b2

6 files changed +65 -30

.travis.yml

+1 -2

@@ -27,8 +27,7 @@ jobs:

27 script: ./tests/installer/checksums.sh

28 - name: coverity

29 install: sudo apt-get install -y zlib1g-dev uuid-dev libipmimonitoring-dev libmnl-dev libnetfilter-acct-dev

30 - script: ./coverity-scan.sh || echo "Coverity failed :("

31 - env: INSTALL_COVERITY="true"

30 + script: ./coverity-install.sh && ./coverity-scan.sh || echo "Coverity failed :("

31 if: type = cron

33 - stage: build

.travis/nightlies.sh

-1

@@ -36,7 +36,6 @@ else

36 fi

38 echo "--- BUILD & PUBLISH DOCKER IMAGES ---"

39 -export REPOSITORY="netdata/netdata"

39 packaging/docker/build.sh || BAD_THING_HAPPENED=1

41 echo "--- BUILD ARTIFACTS ---"

.travis/releaser.sh

-1

@@ -64,7 +64,6 @@ if [[ $(git describe) =~ -rc* ]]; then

64 fi

66 echo "---- CREATING TAGGED DOCKER CONTAINERS ----"

67 -export REPOSITORY="netdata/netdata"

67 ./packaging/docker/build.sh

69 echo "---- CREATING RELEASE ARTIFACTS -----"

Makefile.am

@@ -71,6 +71,7 @@ dist_noinst_DATA= \

71 # should be proper init.d/openrc/systemd usable

72 dist_noinst_SCRIPTS= \

73 coverity-scan.sh \

74 + coverity-install.sh \

75 packaging/installer/netdata-updater.sh \

76 packaging/installer/kickstart.sh \

77 packaging/installer/kickstart-static64.sh \

coverity-install.sh new

+42

@@ -0,0 +1,42 @@

1 +#!/usr/bin/env bash

2 +# Coverity installation script

3 +#

4 +# Copyright: SPDX-License-Identifier: GPL-3.0-or-later

5 +#

6 +# Author: Pavlos Emm. Katsoulakis (paul@netdata.cloud)

7 +

8 +token="${COVERITY_SCAN_TOKEN}"

9 +([ -z "${token}" ] && [ -f .coverity-token ]) && token="$(<.coverity-token)"

10 +if [ -z "${token}" ]; then

11 + echo >&2 "Save the coverity token to .coverity-token or export it as COVERITY_SCAN_TOKEN."

12 + exit 1

13 +fi

14 +

15 +covbuild="$(which cov-build 2>/dev/null || command -v cov-build 2>/dev/null)"

16 +([ -z "${covbuild}" ] && [ -f .coverity-build ]) && covbuild="$(<.coverity-build)"

17 +if [ ! -z "${covbuild}" ]; then

18 + echo >&2 "Coverity already installed, nothing to do!"

19 + exit 0

20 +fi

21 +

22 +echo >&2 "Installing coverity..."

23 +WORKDIR="/opt/coverity-source"

24 +mkdir -p "${WORKDIR}"

25 +

26 +curl -SL --data "token=${token}&project=${REPOSITORY}" https://scan.coverity.com/download/linux64 > "${WORKDIR}/coverity_tool.tar.gz"

27 +if [ -f "${WORKDIR}/coverity_tool.tar.gz" ]; then

28 + tar -x -C "${WORKDIR}" -f "${WORKDIR}/coverity_tool.tar.gz"

29 + sudo mv "${WORKDIR}/cov-analysis-linux64-2017.07" /opt/coverity

30 + export PATH=${PATH}:/opt/coverity/bin/

31 +else

32 + echo "Failed to download coverity tool tarball!"

33 +fi

34 +

35 +# Validate the installation

36 +covbuild="$(which cov-build 2>/dev/null || command -v cov-build 2>/dev/null)"

37 +if [ -z "$covbuild" ]; then

38 + echo "Failed to install coverity!"

39 + exit 1

40 +else

41 + echo >&2 "Coverity scan installed!"

42 +fi

coverity-scan.sh

+21 -26

@@ -1,10 +1,15 @@

1 #!/usr/bin/env bash

2 -# shellcheck disable=SC2235

3 -

2 +# Coverity scan script

3 +#

4 # To run this script you need to provide API token. This can be done either by:

5 # - Putting token in ".coverity-token" file

6 # - Assigning token value to COVERITY_SCAN_TOKEN environment variable

7 -# Additionally script can install coverity tool on your computer. To do this just set environment variable INSTALL_COVERITY to "true"

7 +#

8 +# Copyright: SPDX-License-Identifier: GPL-3.0-or-later

9 +#

10 +# Author : Costa Tsaousis (costa@netdata.cloud)

11 +# Author : Pawel Krupa (paulfantom)

12 +# Author : Pavlos Emm. Katsoulakis (paul@netdata.cloud)

14 cpus=$(grep -c ^processor </proc/cpuinfo)

15 [ -z "${cpus}" ] && cpus=1

@@ -16,28 +21,14 @@ if [ -z "${token}" ]; then

21 exit 1

22 fi

19 -# shellcheck disable=SC2230

24 +export PATH=${PATH}:/opt/coverity/bin/

25 covbuild="$(which cov-build 2>/dev/null || command -v cov-build 2>/dev/null)"

26 ([ -z "${covbuild}" ] && [ -f .coverity-build ]) && covbuild="$(<.coverity-build)"

27 if [ -z "${covbuild}" ]; then

23 - echo "Cannot find 'cov-build' binary in \$PATH."

24 - if [ "${INSTALL_COVERITY}" != "" ]; then

25 - echo "Installing coverity..."

26 - mkdir /tmp/coverity

27 - curl -SL --data "token=${token}&project=netdata%2Fnetdata" https://scan.coverity.com/download/linux64 > /tmp/coverity_tool.tar.gz

28 - tar -x -C /tmp/coverity/ -f /tmp/coverity_tool.tar.gz

29 - sudo mv /tmp/coverity/cov-analysis-linux64-2017.07 /opt/coverity

30 - export PATH=${PATH}:/opt/coverity/bin/

31 - # shellcheck disable=SC2230

32 - covbuild="$(which cov-build 2>/dev/null || command -v cov-build 2>/dev/null)"

33 - else

34 - echo "Save command the full filename of cov-build in .coverity-build"

35 - exit 1

36 - fi

37 -fi

38 -

39 -if [ ! -x "${covbuild}" ]; then

40 - echo "The command ${covbuild} is not executable. Save command the full filename of cov-build in .coverity-build"

28 + echo >&2 "Cannot find 'cov-build' binary in \$PATH."

29 + exit 1

30 +elif [ ! -x "${covbuild}" ]; then

31 + echo >&2 "The command ${covbuild} is not executable. Save command the full filename of cov-build in .coverity-build"

32 exit 1

33 fi

@@ -45,7 +36,7 @@ version="$(grep "^#define PACKAGE_VERSION" config.h | cut -d '"' -f 2)"

36 echo >&2 "Working on netdata version: ${version}"

38 echo >&2 "Cleaning up old builds..."

48 -make clean || echo "Nothing to clean"

39 +make clean || echo >&2 "Nothing to clean"

41 [ -d "cov-int" ] && rm -rf "cov-int"

@@ -59,9 +50,13 @@ echo >&2 "Compressing data..."

50 tar czvf netdata-coverity-analysis.tgz cov-int || exit 1

52 echo >&2 "Sending analysis for version ${version} ..."

62 -curl --progress-bar --form token="${token}" \

63 - --form email=costa@tsaousis.gr \

53 +COVERITY_SUBMIT_RESULT=$(curl --progress-bar --form token="${token}" \

54 + --form email=${COVERITY_SCAN_SUBMIT_MAIL} \

55 --form file=@netdata-coverity-analysis.tgz \

56 --form version="${version}" \

57 --form description="netdata, real-time performance monitoring, done right." \

67 - https://scan.coverity.com/builds?project=netdata%2Fnetdata

58 + https://scan.coverity.com/builds?project=${REPOSITORY})

59 +

60 +echo ${COVERITY_SUBMIT_RESULT} | grep -q -e 'Build successfully submitted' || echo >&2 "scan results were not pushed to coverity. Message was: ${COVERITY_SUBMIT_RESULT}"

61 +

62 +echo >&2 "Coverity scan mechanism completed"