pytest-xdist scheduler that runs some tests on dedicated workers.
Can significantly improve runtime by running long tests on separate workers.
Also in conftest.py
you can find handy reporting capabilities to see how tests were scheduled.
pip install xdist-scheduling-exclusive pytest-xdist
To integrate with your pytest setup, update conftest.py as follows:
from xdist_scheduling_exclusive import ExclusiveLoadScopeScheduling
def pytest_xdist_make_scheduler(config, log):
"""xdist-pytest hook to set scheduler."""
return ExclusiveLoadScopeScheduling(config, log)
And place exclusive tests file exclusive_tests.txt
in tests/resources/
.
You can get the list with pytest's
--durations
option that also sort tests by execution time.
Do not forget to clear execution time from the file - there should be only test node IDs
.
See example in this project tests/resources/exclusive_tests.txt
.
Placing the slowest tests in exclusive_tests.txt
will give you the most benefit.
ExclusiveLoadScheduling
Schedule tests fromexclusive_tests.txt
first and on dedicated nodes.ExclusiveLoadFileScheduling
: Place tests fromexclusive_tests.txt
to uniquescopes
. Other tests are grouped as in--dist loadfile
: tests from the same file run on the same node.ExclusiveLoadScopeScheduling
: Schedule tests fromexclusive_tests.txt
first and on dedicated nodes. Other tests are grouped as in--dist loadfile
: tests from the same file run on the same node.
Do not forget to run . ./activate.sh
.
To see how tests were scheduled use something like
python -m pytest -n 4 --xdist-report -s
make help