-
Notifications
You must be signed in to change notification settings - Fork 646
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GSoC] Parallelisation of AnalysisBase with multiprocessing and dask #4162
Conversation
Linter Bot Results:Hi @marinegor! Thanks for making this PR. We linted your code and found the following: Some issues were found with the formatting of your code.
Please have a look at the Please note: The |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #4162 +/- ##
=========================================
Coverage 93.61% 93.62%
=========================================
Files 171 173 +2
Lines 21254 21419 +165
Branches 3937 3978 +41
=========================================
+ Hits 19897 20053 +156
- Misses 898 903 +5
- Partials 459 463 +4 ☔ View full report in Codecov by Sentry. |
Great to see things are moving @marinegor! I'll have a proper look at this PR when back from holidays, but I just wanted to point out that GitHub is not very smart, so "Does not fix #4162" will actually automatically close #4162 when this is merged. "Related to #4162" or something similar (with no "fix" before the PR number) would avoid this issue. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good start. I left a few comments on the code.
@yuxuanzhuang any initial comments so that @marinegor can move forward? |
Co-authored-by: Irfan Alibay <[email protected]> Co-authored-by: Rocco Meli <[email protected]>
…lization.rst Co-authored-by: Irfan Alibay <[email protected]>
Co-authored-by: Rocco Meli <[email protected]>
/azp run |
Azure Pipelines successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Azure pipelines fails seem completely unrelated - I wouldn't let them be blockers
@orbeckst I'm letting you do the final call on merging. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some minor fixes (which I will add) and a few where I need @marinegor to have a quick look (and update with the suggestions if ok).
package/doc/sphinx/source/documentation_pages/analysis/parallelization.rst
Outdated
Show resolved
Hide resolved
@marinegor please check my two suggested changes. If they are ok, please add accept the suggestions or make appropriate changes yourself. Then ping me and assuming that CI still passes (at least non Azure) I will then finally squash-merge (while keeping the co-authors line with the reviewers). |
Co-authored-by: Oliver Beckstein <[email protected]>
Co-authored-by: Oliver Beckstein <[email protected]>
@orbeckst I agree with both of the changes, thanks. I removed the
ping!
🤞🤞🤞 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work @marinegor! An excellent contribution to MDAnalysis
HOORAY!!!! merged |
Wooooooah! 🥳 Congratulations and than you for the massive effort @marinegor! 🚀 |
Fixes #4158
Also fixes # 4259 as a check inno it doesn't, I'll do another PR that does that.AnalysisBase.run()
Related to #4158, does not help f-i-x-i-n-g (cudos to github bot) the issue per se but paves the way towards that.Changes made in this Pull Request:
analysis.base.AnalysisBase
, implementing backend configuration, splitting of the frames for analysis, computation, and results aggregationanalysis.backends
introducesBackendBase
class, as well as built-in backendsBackendMultiprocessing
,BackendSerial
andBackendDask
, implementing theapply
method for computations using various backendsanalysis.results
introducesResultsGroup
class that allows for merging of multiple uniformResults
objects from the same module, given appropriate aggregation functionsPR Checklist
📚 Documentation preview 📚: https://mdanalysis--4162.org.readthedocs.build/en/4162/