Other modules cannot from sys import ___ if dill recurse is enabled

[joseortiz3]

System Description:

• OS : Windows 11 Pro
• Python version: 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)]
• multiprocess version: 0.70.14
• dill version: 0.3.6

When using multiprocess with dill['recurse'] set to True, functions from other modules cannot be imported if that module has the import statement from sys import ____.

Some/most of the exceptions say "pool objects cannot be passed between processes or pickled.", so I'm posting this issue here. Let me know if you think I should post to dill instead or additionally.

Here is a minimal example to reproduce, requires these two python files in the same directory:

mp_error.py

import another_file # another_file.py is in the same directory as this file.

import dill # Error only occurs if using dill with recurse
dill.settings['recurse'] = True 

from multiprocess import Pool

def func_fail(i):
    another_file.empty_func()
    return i

if __name__ == '__main__':

    with Pool() as pool:
        results = pool.map(func_fail, range(2))
    print('func_fail passed.')

another_file.py

#Choose 1 of these lines to uncomment to cause the error # description of error

from sys import modules # NotImplementedError: pool objects cannot be passed between processes or pickled.
#from sys import stdout # TypeError: no default __reduce__ due to non-trivial __cinit__
#from sys import stderr # TypeError: no default __reduce__ due to non-trivial __cinit__
#from sys import stdin # NotImplementedError: pool objects cannot be passed between processes or pickled
#from sys import argv # NotImplementedError: pool objects cannot be passed between processes or pickled
#from sys import exit # NotImplementedError: pool objects cannot be passed between processes or pickled
#from sys import path # NotImplementedError: pool objects cannot be passed between processes or pickled
#from sys import audit # NotImplementedError: pool objects cannot be passed between processes or pickled
#from sys import executable # NotImplementedError: pool objects cannot be passed between processes or pickled
#from sys import * # TypeError: no default __reduce__ due to non-trivial __cinit__
#import sys # Works

def empty_func():
    pass

I hope this is helpful. Also, I am hoping someone could give me some intuition about how dill.recurse works and why other modules might not be able to use certain import statements with recurse enabled.

[mmckerns]

I'm the multiprocess author, and also the dill author. I think this is a dill issue... and also a multiprocess issue... so probably is fine to stay posted here.