Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve loading data from old storage #1208

Closed
ManInFez opened this issue Nov 25, 2020 · 3 comments
Closed

Improve loading data from old storage #1208

ManInFez opened this issue Nov 25, 2020 · 3 comments

Comments

@ManInFez
Copy link
Contributor

The way ert_data.MeasuredData loads data is highly inefficient when there are many obs keys to fetch. This is especially true for SUMMARY_OBS, as for every key, it loads all summary data and all summary observations before it filters one a single key. This needs to be improved.

The MeasuredData links the observations to the responses by iterating over all of them and the fetching the data. A suggestion is would be to fetch all data and all observations separately before linking them together in one big swoop.

Feel free to discuss.

@oyvindeide
Copy link
Collaborator

Depending on the time frame for getting this data directly from new storage, I think this would be a good improvement to make. It should not be very complicated, and if the time frame for getting this data from new storage is on the order of months, it would probably be worth it.

@oysteoh
Copy link
Contributor

oysteoh commented Mar 15, 2021

@oyvindeide should something be done here og just close the issue?

@oyvindeide
Copy link
Collaborator

Should at least be improved now, so safe to close this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants