You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: documentation/source/development/ci-guide.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,13 +7,13 @@ Our GitHub actions Continuous Integration (CI) pipeline serves to ensure each br
7
7
| ---- | ------------- |
8
8
| unit-test | Installs PROCESS and runs the unit tests. The job will fail if any of the unit tests fail. |
9
9
| integration-test | Installs PROCESS and runs the integration tests. The job will fail if any of the integration tests fail. |
10
-
| regression-test | Installs PROCESS and runs the regression tests with a 0.2% and 5% tolerance, respectively. The job will fail if any of the regression tests fail. The job uses tracked outputs for regression test comparisons. |
10
+
| regression-test | Installs PROCESS and runs the regression tests with a 0.2% and 5% tolerance, respectively. The job will fail if any of the regression tests fail. The job uses tracked MFILEs for regression test comparisons. |
11
11
| run-tracking-inputs | Installs PROCESS and runs the regression test input files, archiving the output MFILEs. Only runs on the **main** branch. |
12
12
| tracking | Collects MFILEs for input files of interest and creates a dashboard of changes in key values over time (one datapoint for each commit on main). Only runs on the **main** branch. |
13
13
| pre-commit-quality-check | ensures the pushed code meets our standards as defined in `.pre-commit-config.yaml`. |
14
14
| docs | Builds and deploys the documentation onto GitHub pages. |
15
15
16
-
!!! Info Regression files
17
-
When creating a PR, the tracked MFILEs from the point at which you branched off **main** will be used during the regression test job. This job may fail on your PR. If so, the changes will need to be reviewed. If the changes are accepted (i.e. they come from a necessary change to a model), upon merge to main the
16
+
!!! Info "Regression Job"
17
+
When creating a PR, the tracked MFILEs from the point at which you branched off main will be used during the regression test job. This job may fail on your PR. If so, these changes to the output will need to be reviewed. If the changes are accepted (i.e. they come from a necessary change to a model), upon merge to main the
18
18
`run-tracking-inputs` job will run the regression input files to generate the associated MFILEs to be tracked by the
19
-
`tracking job`, and these will now reflect the changes your PR has introduced. Therefore when the regression job runs it will use these updated tracked files, and will pass upon merge to main.
19
+
`tracking job`, and these will now reflect the changes your PR has introduced. Therefore when the regression job runs upon merge of your code into main, it will use these updated tracked MFILEs, and the regression tests will pass.
Copy file name to clipboardExpand all lines: documentation/source/development/testing.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -27,16 +27,16 @@ filename, then call a method to create a plot file, then check that the plot was
27
27
Regression tests detect changes in the entire program's behaviour by checking that for a given
28
28
input, it produces exactly the same output as before those changes. It detects changes in the
29
29
program's output. Therefore, if your code changes the program's output, it will fail the regression
30
-
test on the CI for your PR. In this case, that output difference will need to be reviewed and, if accepted,
31
-
the expected (or reference) output will be updated to reflect these changes when your code is merged into `main`.
30
+
test. In this case, that output difference will need to be reviewed and, if accepted,
31
+
the expected (or reference) output will be updated to reflect these changes when your code is merged into main.
32
32
33
33
Regression tests compare the output of PROCESS locally when running an input file to a reference output stored in a
34
34
[repository](https://github.com/timothy-nunn/process-tracking-data). The test suite will download the reference output for the commit where the current branch branched off main. This means **each branch is accountable for only its changes since it branched off main**.
35
35
36
36
37
-
!!! Info CI System
37
+
!!! Info "CI System"
38
38
Our tests are run automatically using the Continous Integration (CI) system on GitHub.
39
-
More information about the jobs run on our CI can be found [here](https://ukaea.github.io/PROCESS/development/ci-guide/)
39
+
More information about how the jobs run on our CI and update the tracked files can be found [here](https://ukaea.github.io/PROCESS/development/ci-guide/).
0 commit comments