Process name not propagated to logging library

[noxdafox]

Greetings,

When investigating the following bug:
noxdafox/pebble#113

It turned out the multiprocess library seems to not propagate the process name to the logging library correctly.

import logging
import multiprocess
import multiprocessing


logging.basicConfig(format='%(processName)s-%(process)d - %(levelname)s:%(message)s', level=logging.DEBUG)


def standardlib():
    logger.error("Using multiprocessing")
    logger.error(multiprocessing.current_process().name)


def externallib():
    logger.error("Using multiprocess")
    logger.error(multiprocess.current_process().name)


if __name__ == "__main__":
    logger = logging.getLogger("main")

    p = multiprocessing.Process(target=standardlib, name="multiprocessing")
    p.start()
    p.join()

    p = multiprocess.Process(target=externallib, name="multiprocess")
    p.start()
    p.join()

Output:

multiprocessing-7814 - ERROR:Using multiprocessing
multiprocessing-7814 - ERROR:multiprocessing
MainProcess-7815 - ERROR:Using multiprocess
MainProcess-7815 - ERROR:multiprocess

The expected output would be:

multiprocessing-7814 - ERROR:Using multiprocessing
multiprocessing-7814 - ERROR:multiprocessing
multiprocess-7815 - ERROR:Using multiprocess
multiprocess-7815 - ERROR:multiprocess

$ python --version
Python 3.9.2

[mmckerns]

The code you have for multiprocessing doesn't work for me. It throws a NameError that logger is not defined -- which is expected because multiprocessing uses pickle and it doesn't have an encapsulated definition of logger. multiprocess should (and does) "work" because it's looking up logger in the global dictionary (from 'main'). However, if I modify your code to pass logger, it gives results like you are printing.

import logging
import multiprocess
import multiprocessing


logging.basicConfig(format='%(processName)s-%(process)d - %(levelname)s:%(message)s', level=logging.DEBUG)


def standardlib(logger):
    if not logger: logger = logging.getLogger('multiprocessing')
    logger.error("Using multiprocessing")
    logger.error(multiprocessing.current_process().name)


def externallib(logger):
    if not logger: logger = logging.getLogger('multiprocess')
    logger.error("Using multiprocess")
    logger.error(multiprocess.current_process().name)


if __name__ == "__main__":
    logger = None #logging.getLogger("main")

    p = multiprocessing.Process(target=standardlib, name="multiprocessing", args=(logger,))
    p.start()
    p.join()

    p = multiprocess.Process(target=externallib, name="multiprocess", args=(logger,))
    p.start()
    p.join()

Interestingly, regardless of whether logger = None in __main__ or the main logger is passed in, the name of the process in externalib is still MainProcess (for python 3.8 - 3.12).

[noxdafox]

I just tried again to make sure and it works on my machine (Linux Debian 11). Are you on Windows/MAC? In such case it might be due to the spawn behaviour.

[mmckerns]

Yes. I was testing on a mac. Spawn has an impact on the picking.

[SvenSchiffner]

I'm facing the same error in python devcontainer with loguru.