-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docs for data container and HPC #179
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall great addition to the docs!
One major suggestion:
- I don't like that a lot of code is duplicated between
GModel_xx.jl
. I feel like this obscures the actual differences between the parallelism options. Instead, I think the components that are identical should be stored in some common file and loaded into each of_distfor
,_multithread
, etc.
We could for example call this common fileGModel.jl
and rename the serial case e.g. toGModel_serial.jl
.
I'm happy to review again once my comments and suggestions are addressed! :)
docs/src/data_wrappers.md
Outdated
|
||
To provide a consistent form for data (such as observations, parameter ensembles, model evaluations) across the package, we store the data in simple wrappers internally. | ||
|
||
Data is always stored as columns of `AbstractMatrix`. That is, we obey the format |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Data is always stored as columns of `AbstractMatrix`. That is, we obey the format | |
Data is always stored column-wise in an `AbstractMatrix` subtype. That is, we obey the format |
looking at the constructor code, it appears we only enforce that data is a concrete subtype of AbstractMatrix
, so data will be whatever concrete type was supplied upon construction
bors try |
tryBuild succeeded: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With the final comments, and squashing, LGTM!
I have not looked in detail on the structure of the GModel_common.jl
or distributed_Lorenz_example.jl
codes, which we can revisit in a future PR.
Run the forward model G for an array of parameters by iteratively | ||
calling run_G for each of the N_ensemble parameter values. | ||
Return g_ens, an array of size N_data x N_ensemble, where | ||
g_ens[:,j] = G(params[:,j]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This isn't exactly correct in the current implementation, but I'm happy to leave this for a future PR that cleans up the runscript
parallel & hpc page added lorenz examples to supplement docs examples/Lorenz/distributed_Lorenz_example.jl format docs changes from review typo removed comment remove serial case, add else case format refactor distributed Lorenz example format adjusted docs to suit new code remove ... review comments consistent structure for GModel fixed broken link links to GModels typo
bors r+ |
Build succeeded: |
Purpose and Content
add two docs pages requested in Issues #153 , closes #148
adds an example to demonstrate all of the different parallelism types (serial, multithread, pmap and distributed for)
distributed_Lorenz_example.jl
PR Checklist