Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiprocess Stage POC #1170

Open
wants to merge 12 commits into
base: branch-23.11
Choose a base branch
from

Conversation

cwharris
Copy link
Contributor

@cwharris cwharris commented Sep 1, 2023

Example demonstrates the use of Python Multiprocessing with a custom Stage, which enables offloading of MultiMessage processing to a separate process while continuing the workflow within the same process.

The example results in a segfault originating from Watchable::watcher_epilogue when calling sub.on_next from python in the parent process. It appears that one of m_watchers has been released and we're still trying to access it. Unfortunately I don't know enough about MRC to say what the cause is.

inline void Watchable::watcher_epilogue(WatchableEvent op, bool rc, const void* addr)
{
    for (const auto& obs : m_watchers)
    {
        obs->on_exit(op, rc, addr); // <- segfault occurs here, likely when de-referencing `obs`
    }
}

I was able to work around the issue by simply commenting out the obs->on_exit(...) call and rebuilding MRC and Morpheus together. Everything seemed to work fine, so maybe this is a bug in MRC?

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@cwharris cwharris added non-breaking Non-breaking change feature request New feature or request labels Sep 4, 2023
@jarmak-nv
Copy link
Contributor

Hey @cwharris should we promote this out of draft since you're requested review?

@cwharris cwharris marked this pull request as ready for review September 6, 2023 14:07
@cwharris cwharris requested a review from a team as a code owner September 6, 2023 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request non-breaking Non-breaking change
Projects
Status: Review - Ready for Review
Development

Successfully merging this pull request may close these issues.

None yet

2 participants