| 1 | # How-to: reproduce PR #22423's 864 markdownlint findings locally |
| 2 | |
| 3 | ## When to use |
| 4 | |
| 5 | You want to confirm `analyze-local.sh` matches what Codacy CI reported on a known fixture. PR #22423 is the canonical fixture for this skill: its first CI run reported **864 markdownlint findings**; commit `3a54c9afbc` cleared them by adding `.agents/**` and `docs/netdata-ai/skills/**` to `.codacy.yml`. |
| 6 | |
| 7 | This how-to walks through reproducing those 864 findings on the pre-exclusion state, then confirming the post-exclusion state shows zero on the affected files. |
| 8 | |
| 9 | ## Prerequisite |
| 10 | |
| 11 | - `docker` available (or `codacy-analysis-cli` installed locally). |
| 12 | - `<repo>/.env` need NOT contain `CODACY_TOKEN` -- `analyze-local.sh` runs the CLI anonymously. |
| 13 | |
| 14 | ## Step 1 -- check out the pre-exclusion state |
| 15 | |
| 16 | PR #22423 introduced the exclusion in commit `3a54c9afbc`. The parent commit `d7791e6838` is the "before" state. |
| 17 | |
| 18 | ```bash |
| 19 | git checkout d7791e6838 -- .codacy.yml # restore the pre-exclusion .codacy.yml |
| 20 | # (do NOT switch branches; just stage the older .codacy.yml) |
| 21 | ``` |
| 22 | |
| 23 | ## Step 2 -- run analyze-local on markdownlint only |
| 24 | |
| 25 | ```bash |
| 26 | .agents/skills/codacy-audit/scripts/analyze-local.sh --tool markdownlint |
| 27 | ``` |
| 28 | |
| 29 | Expected: a JSON dump under `<repo>/.local/audits/codacy/local-markdownlint-<ts>.json`. The CLI returns non-zero when findings exist (this is normal; the script tolerates it). |
| 30 | |
| 31 | ## Step 3 -- count findings |
| 32 | |
| 33 | ```bash |
| 34 | DUMP="$(ls -1t .local/audits/codacy/local-markdownlint-*.json | head -1)" |
| 35 | jq ' |
| 36 | if type=="array" then length |
| 37 | elif type=="object" and has("issues") then (.issues | length) |
| 38 | elif type=="object" and has("results") then (.results | length) |
| 39 | else 0 end |
| 40 | ' "$DUMP" |
| 41 | ``` |
| 42 | |
| 43 | Expected: a count close to 864 (within ~10% tolerance for tool-version drift between the CLI bundle and Codacy Cloud). |
| 44 | |
| 45 | ## Step 4 -- restore the exclusion |
| 46 | |
| 47 | ```bash |
| 48 | git checkout HEAD -- .codacy.yml |
| 49 | ``` |
| 50 | |
| 51 | ## Step 5 -- re-run and confirm zero on excluded paths |
| 52 | |
| 53 | ```bash |
| 54 | .agents/skills/codacy-audit/scripts/analyze-local.sh --tool markdownlint |
| 55 | DUMP="$(ls -1t .local/audits/codacy/local-markdownlint-*.json | head -1)" |
| 56 | jq '[ ... | select(.filePath | startswith(".agents/") or startswith("docs/netdata-ai/skills/")) ] | length' "$DUMP" |
| 57 | ``` |
| 58 | |
| 59 | (The exact jq filter depends on the dump shape -- consult the dump structure first via `jq 'keys' "$DUMP"`.) |
| 60 | |
| 61 | Expected: zero rows in the excluded trees. |
| 62 | |
| 63 | ## What this validates |
| 64 | |
| 65 | - `analyze-local.sh` runs end-to-end against the configured runner (docker or local binary). |
| 66 | - The CLI honours `.codacy.yml` exclude_paths (or, if it doesn't, we have empirical evidence to handle that gap in a GitHub issue or branch-local SOW). |
| 67 | - The bundled markdownlint version produces a count consistent with what Codacy CI reports. |
| 68 | |
| 69 | ## Troubleshooting |
| 70 | |
| 71 | - **Docker pull is slow on first run**: `codacy/codacy-analysis-cli:latest` is a few hundred MB. Subsequent runs use the warm cache. |
| 72 | - **Different count than 864**: tool-version drift between the bundled `codacy-analysis-cli` and Codacy Cloud is normal. ~10% tolerance is fine. Anything wider warrants checking the CLI version vs Cloud's reported version. |
| 73 | - **CLI exits with non-zero**: that's expected when findings are present. The script suppresses this; the JSON dump is still valid. |