Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Leeyabot reboot #747

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from
Draft

Leeyabot reboot #747

wants to merge 7 commits into from

Conversation

ptrscll
Copy link
Collaborator

@ptrscll ptrscll commented Jul 10, 2024

This PR adds code to make leeyabot also pull data from the hector-run-archive repository and make comparisons between that data and the new version of Hector. Additionally, the leeyabot graphs are now included within the resulting html file, and there is a new box plot summarizing the distribution of differences between the new and old data. This box plot (and the other graphs) are created for both the comparison between the new Hector data and the hector-run-archive data and the comparison between the new Hector data and the data generated for the old-new R unit test in hector_comp.csv.

The leeyabot yml file is currently unchanged, so the yml file still only checks whether the output for the new version of Hector matches the data in hector_comp.csv

kdorheim and others added 7 commits February 23, 2024 12:37
Beta = 0.36 seems incorrect, isn't it 0.53.

beta <- fetchvars(core, NA, BETA())
beta
##              scenario year variable value      units
## 1 Unnamed Hector core   NA     beta  0.53 (unitless)

(...)
The current value is set to 0.36 (note that it is a unitless quantity, hence the (unitless) unit).
Fix a minor numeric inconsistency in the hector vignette
* Github Issue 718. Add papers to Hector manuscript lists, including  V3.2.0 manuscript 

* github issue 727. rename FCH4 to RF_CH4 to match all other RF output

* Add a new release checklist

* github issue 734, add output streams

* quite build message

* github issue 740, add matilda refernces

* github issue 735, add ref temperature period to plot
Copy link

Differences in Hector outputs

Hello, this is leeyabot!

Comparison with Previous Dev Version

The current pull request's outputs do not differ from the old-new version of Hector (3.1.1 (d931a00)).

R squared NRMSE
CO2_concentration 1.000 0.000
global_tas 1.000 0.000
RF_CO2 1.000 0.000
RF_tot 1.000 0.000

This table indicates the R^2 and normalized root mean squared error values for each variable. An R^2 of 1 and NRMSE of 0 indicate that the new values are identical to the old values.

Below are several plots comparing the performance of Hector between old and new versions:

Comparison with Previous Release

The current pull request's outputs do not differ from the release version of Hector (3.2.0 (1ef0b4d)).

R squared NRMSE
CO2_concentration 1.000 0.000
global_tas 1.000 0.000
RF_tot 1.000 0.000

This table indicates the R^2 and normalized root mean squared error values for each variable. An R^2 of 1 and NRMSE of 0 indicate that the new values are identical to the release values.

Below are several plots comparing the performance of Hector between the new version and the release version:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants