Skip to content

Commit ecb39a3

Browse files
committed
add some comments and documentation for explanation
1 parent 893a36c commit ecb39a3

3 files changed

Lines changed: 24 additions & 7 deletions

File tree

.github/workflows/process.yml

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,8 @@ jobs:
6161
regression-test:
6262
runs-on: ubuntu-latest
6363
needs: [tracking]
64-
# depend on make and tracking job but dont skip if tracking was skipped
64+
# depend on make and tracking job but don't skip if tracking was skipped
65+
# run PROCESS on the regression test input files and compare results to the tracked files
6566
if: always()
6667
continue-on-error: true
6768
strategy:
@@ -80,6 +81,8 @@ jobs:
8081

8182
run-tracking-inputs:
8283
runs-on: ubuntu-latest
84+
# run PROCESS on the regression test input files and archive output MFILEs
85+
# only runs upon merge to main
8386
if: github.ref == 'refs/heads/main'
8487
steps:
8588
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
@@ -114,10 +117,12 @@ jobs:
114117
run: hatch run lint:fmt
115118

116119
tracking:
120+
# update tracker to include the MFILEs created by the run-tracked-files job
117121
concurrency:
118122
group: tracking-jobs
119123
cancel-in-progress: false
120124
runs-on: ubuntu-latest
125+
# depends on run-tracking-inputs; only runs upon merge to main
121126
needs: run-tracking-inputs
122127
if: github.ref == 'refs/heads/main'
123128
env:
@@ -168,6 +173,7 @@ jobs:
168173
group: docs-jobs
169174
cancel-in-progress: false
170175
runs-on: ubuntu-latest
176+
# depends on tracking; only runs upon merge to main
171177
needs: tracking
172178
if: github.ref == 'refs/heads/main'
173179
steps:

documentation/source/development/ci-guide.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,4 +11,9 @@ Our GitHub actions Continuous Integration (CI) pipeline serves to ensure each br
1111
| run-tracking-inputs | Installs PROCESS and runs the regression test input files, archiving the output MFILEs. Only runs on the **main** branch. |
1212
| tracking | Collects MFILEs for input files of interest and creates a dashboard of changes in key values over time (one datapoint for each commit on main). Only runs on the **main** branch. |
1313
| pre-commit-quality-check | ensures the pushed code meets our standards as defined in `.pre-commit-config.yaml`. |
14-
| docs | Builds and deploys the documentation onto GitHub pages. |
14+
| docs | Builds and deploys the documentation onto GitHub pages. |
15+
16+
!!! Info Regression files
17+
When creating a PR, the tracked MFILEs from the point at which you branched off **main** will be used during the regression test job. This job may fail on your PR. If so, the changes will need to be reviewed. If the changes are accepted (i.e. they come from a necessary change to a model), upon merge to main the
18+
`run-tracking-inputs` job will run the regression input files to generate the associated MFILEs to be tracked by the
19+
`tracking job`, and these will now reflect the changes your PR has introduced, so the regression job will pass upon merge to **main**.

documentation/source/development/testing.md

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -24,14 +24,20 @@ filename, then call a method to create a plot file, then check that the plot was
2424

2525
### Regression tests
2626

27-
Regression tests detect changes in the entire program's behaviour by checking that for a given
27+
Regression tests detect changes in the entire program's behaviour by checking that for a given
2828
input, it produces exactly the same output as before those changes. It detects changes in the
29-
program's output. Therefore if your code changes the program's output, it will fail the regression
30-
test. In this case that output difference will need to be reviewed and if accepted, the expected (or
31-
reference) will be updated.
29+
program's output. Therefore, if your code changes the program's output, it will fail the regression
30+
test on the CI for your PR. In this case, that output difference will need to be reviewed and, if accepted,
31+
the expected (or reference) output will be updated to reflect these changes when your code is merged into `main`.
3232

3333
Regression tests compare the output of PROCESS locally when running an input file to a reference output stored in a
34-
[repository](https://github.com/timothy-nunn/process-tracking-data). The test suite will download the reference output for the commit where the your current branch branched off of main. This means **each branch is accountable for only its changes since it branched off of main**.
34+
[repository](https://github.com/timothy-nunn/process-tracking-data). The test suite will download the reference output for the commit where the current branch branched off main. This means **each branch is accountable for only its changes since it branched off main**.
35+
36+
37+
!!! Info CI System
38+
Our tests are run automatically using the Continous Integration (CI) system on GitHub.
39+
More information about the jobs run on our CI can be found [here](https://ukaea.github.io/PROCESS/development/ci-guide/)
40+
3541

3642
## pytest
3743

0 commit comments

Comments
 (0)