Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement low level accelerators for the GGCE backend #57

Open
matthewcarbone opened this issue Oct 15, 2022 · 5 comments
Open

Implement low level accelerators for the GGCE backend #57

matthewcarbone opened this issue Oct 15, 2022 · 5 comments
Assignees
Labels
enhancement New feature or request major A large and significant addition or improvement
Milestone

Comments

@matthewcarbone
Copy link
Owner

This is a major endeavor that will require a lot of work/thought. Essentially we want to do the following: Replace everything in ggce.models and ggce.engine.system with low level language accelerators. For example, by using pybind11 as a submodule we should be able to create Python-importable binaries.

As funny as it is, I think that the current solvers we use in NumPy, SciPy and (obviously) PETSc are faster than most of what we could try to get working in a lower level language! So for now, we can stick with those. But the steps of generating the basis are incredibly time-consuming and could be dramatically sped up.

This idea is still very much in the abstract but I want to document that we want to do it. I'm also closing #6 in favor of this, since this feature will do more than just generate the basis, it will also hook into the modules in e.g. ggce.engine.terms.

@Chiffafox
Copy link
Collaborator

This is really important. Having optimized basis construction -- dare I say, even a parallelizable one?? -- would make our code fully scalable and enable us to do crazy things.

@matthewcarbone
Copy link
Owner Author

Yeah it's not going to be straightforward though. I'm not super comfortable/familiar with building projects that are not pure Python. The good news is that whatever C++ code we use should be pure C++ plus pybind11 for the Python bindings, so we shouldn't have to worry about including a bunch of other header files or something.

That said, I am not entirely sure yet how to properly build the wheels for everything we'd need. If you have any ideas I'm all ears.

@Chiffafox
Copy link
Collaborator

Same here, I am not too experienced with this at the moment. That said, I'd definitely be keen to learn more about Python-C++ integration, so I'd love to work on this together if you think that would be helpful!

@matthewcarbone
Copy link
Owner Author

@Chiffafox your help on this would be amazing. Check out the new master branch. I've pushed the relevant build scripts (to scripts) and I use flit to build the package.

When we include C++ code for example, we will need to build that (make ...), then link it via pybind11, and somehow package all that into a wheel. The system dependencies gets a bit crazy at that point.

I am happy to write most of the C++ code (in fact we more or less already have a lot of it in GMA2 or whatever that old repo was), if you can help me figure out how to package it!

@matthewcarbone
Copy link
Owner Author

This looks extremely promising as a way to do exactly what we want: https://github.com/pybind/python_example

@matthewcarbone matthewcarbone added major A large and significant addition or improvement enhancement New feature or request labels Nov 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request major A large and significant addition or improvement
Projects
None yet
Development

No branches or pull requests

2 participants