Skip to content

CI: measure coverage on develop, not only on pull requests - #1162

Merged
Gui-FernandesBR merged 1 commit into
RocketPy-Team:developfrom
thc1006:ci/measure-coverage-on-develop
Aug 15, 2026
Merged

CI: measure coverage on develop, not only on pull requests#1162
Gui-FernandesBR merged 1 commit into
RocketPy-Team:developfrom
thc1006:ci/measure-coverage-on-develop

Conversation

@thc1006

@thc1006 thc1006 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Pull request type

  • ReadMe, Docs and GitHub updates

Checklist

  • Tests for the changes have been added (if needed)
  • Docs have been reviewed and added / updated
  • Lint has passed locally
  • All tests have passed locally

Current behavior

test_pytest.yaml is the only workflow that uploads to Codecov, and it runs on pull_request alone. No commit on develop ever gets a coverage report.

Codecov falls back to the nearest older report it holds and says how far behind that is. The banner on my open pull requests read:

  • 2026-08-14: Report is 53 commits behind head on develop.
  • 2026-08-15: Report is 61 commits behind head on develop.

It grows with every merge, and the comparison is against whatever develop looked like whenever that stale report was uploaded rather than against the base the pull request is actually built on.

.codecov.yml already names master and develop under branches: for both the project and the patch status, so the configuration expects both to be measured. It is only the trigger that never produces the reports.

New behavior

The same matrix also runs on a push to master or develop, under the same paths: filter the pull request trigger uses. Each merge leaves a report at the commit the next pull request will be compared against.

docs.yml pairs pull_request with push in the same shape, so this is the existing idiom in this repository rather than a new one.

Breaking change

  • No

Additional information

Nothing in either job reads pull_request context: the only mention of the event in the file is the trigger itself, so the matrix and the CodecovUpload guard behave the same on a push. Forks still get an empty CODECOV_TOKEN and fail_ci_if_error still stays off for them; a push to develop runs in this repository, so it has the token.

Two things worth weighing:

  • This roughly doubles the matrix runs, once for the pull request and once for the merge. The alternative, a single cheap leg on push, would report lower coverage than the six-leg pull request reports and read as a drop.
  • A merge that touches no .py file still uploads nothing, so the banner can still show a small number. That one is accurate: coverage did not change, and Codecov walking back one commit is the right answer. What this fixes is never getting a report at all.

@thc1006
thc1006 requested a review from a team as a code owner August 14, 2026 17:44
@codecov

codecov Bot commented Aug 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.53%. Comparing base (e0ff281) to head (18d4fb1).
⚠️ Report is 66 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1162      +/-   ##
===========================================
+ Coverage    82.18%   83.53%   +1.35%     
===========================================
  Files          122      130       +8     
  Lines        16355    17086     +731     
===========================================
+ Hits         13441    14273     +832     
+ Misses        2914     2813     -101     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The tests workflow is the only one that uploads to Codecov and it runs on
pull_request alone, so no commit on develop ever gets a report. Codecov then
compares each pull request against the nearest older report it holds and prints
how far behind that has fallen; the banner read 53 commits one day and 61 the
next, and it grows with every merge.

.codecov.yml already names master and develop under branches for both the
project and the patch status, so the configuration expects these to be
measured. docs.yml pairs pull_request with push the same way.

Nothing in the jobs reads pull_request context, so the same matrix runs
unchanged on a push.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
@Gui-FernandesBR
Gui-FernandesBR force-pushed the ci/measure-coverage-on-develop branch from 3dcb0c1 to 18d4fb1 Compare August 15, 2026 01:49
@Gui-FernandesBR
Gui-FernandesBR merged commit a1ae329 into RocketPy-Team:develop Aug 15, 2026
7 checks passed
@thc1006
thc1006 deleted the ci/measure-coverage-on-develop branch August 15, 2026 07:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants