Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce slowest-tests cli command #30

Merged
merged 3 commits into from
Aug 6, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,11 @@ However, when there are major changes in the suite compared to what's stored in

The splitting algorithm can be controlled with the `--splitting-algorithm` CLI option and defaults to `duration_based_chunks`. For more information about the different algorithms and their tradeoffs, please see the section below.

### CLI commands
#### slowest-tests
Lists the slowest tests based on the information stored in the test durations file. See `slowest-tests help` for more
information.

## Interactions with other pytest plugins
* [`pytest-random-order`](https://github.com/jbasko/pytest-random-order): ⚠️ The **default settings** of that plugin (setting only `--random-order` to activate it) are **incompatible** with `pytest-split`. Test selection in the groups happens after randomization, potentially causing some tests to be selected in several groups and others not at all. Instead, a global random seed needs to be computed before running the tests (for example using `$RANDOM` from the shell) and that single seed then needs to be used for all groups by setting the `--random-order-seed` option.

Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ pre-commit = "^2.13.0"
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry.scripts]
slowest-tests = "pytest_split.cli:list_slowest_tests"
jerry-git marked this conversation as resolved.
Show resolved Hide resolved

[tool.poetry.plugins.pytest11]
pytest-split = "pytest_split.plugin"

Expand Down
34 changes: 34 additions & 0 deletions src/pytest_split/cli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import argparse
import json
from typing import TYPE_CHECKING

if TYPE_CHECKING:
from typing import Dict


def list_slowest_tests():
parser = argparse.ArgumentParser()
parser.add_argument(
"--durations-path",
help=(
"Path to the file in which durations are stored, "
"default is .test_durations in the current working directory"
),
default=".test_durations",
type=argparse.FileType(),
)
parser.add_argument(
"-c",
"--count",
help="How many slowest to list",
default=10,
type=int,
)
args = parser.parse_args()
return _list_slowest_tests(json.load(args.durations_path), args.count)


def _list_slowest_tests(durations: "Dict[str, float]", count: int) -> None:
slowest_tests = tuple(sorted(durations.items(), key=lambda item: item[1], reverse=True))[:count]
for test, duration in slowest_tests:
print(f"{duration:.2f} {test}") # noqa: T001
30 changes: 30 additions & 0 deletions tests/test_cli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import argparse
import json
import sys
from io import StringIO
from unittest.mock import patch

import pytest

from pytest_split import cli


@pytest.fixture
def durations_file(tmpdir):
durations_path = str(tmpdir.join(".durations"))
durations = {f"test_{i}": float(i) for i in range(1, 11)}
with open(durations_path, "w") as f:
json.dump(durations, f)
with open(durations_path, "r") as f:
yield f


def test_slowest_tests(durations_file):
with patch("pytest_split.cli.argparse.ArgumentParser", autospec=True) as arg_parser, patch(
"sys.stdout", new_callable=StringIO
):
arg_parser().parse_args.return_value = argparse.Namespace(durations_path=durations_file, count=3)
cli.list_slowest_tests()

output = sys.stdout.getvalue()
assert output == "10.00 test_10\n" "9.00 test_9\n" "8.00 test_8\n"