Skip to content

Commit

Permalink
use python instead of python3 everywhere
Browse files Browse the repository at this point in the history
Everywhere: on CI, in tests, in docs. Also, we replace bare pip/pip3 and pytest calls with python -m pip and python -m pytest respectively, to make sure we always use the same python executable everywhere.

This fixes #326 (failing Windows tests on CI), because the `python3` executable causes issues (it is not properly copied into the venv directory by venv, which is a CPython bug, see python/cpython#87915), while the python executable does work; see e.g. here https://stackoverflow.com/questions/61669873/python-venv-env-fails-winerror-2-the-system-cannot-find-the-file-specified. `python` should be the same executable under the hood as `python3`, if the setup-python action works as expected, at least.
It also allows us to remove the IS_WINDOWS_CI special case.

Failed attempts at fixing #326 included:
- Setting the shells (assuming something went wrong with environment
  variables)
- Using full paths for both the executable and the venv directory in the
  template test suite.
  • Loading branch information
egpbos committed Jun 9, 2023
1 parent 6cc7753 commit d5e299d
Show file tree
Hide file tree
Showing 7 changed files with 47 additions and 59 deletions.
10 changes: 5 additions & 5 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,12 @@ jobs:
- name: Python info
shell: bash -l {0}
run: |
which python3
python3 --version
which python
python --version
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip setuptools
python3 -m pip install .[dev]
python -m pip install --upgrade pip setuptools
python -m pip install .[dev]
- name: Run pytest
run: |
pytest -v
python -m pytest -v
8 changes: 4 additions & 4 deletions README.dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ We recommend installing `cookiecutter` in user space as per `cookiecutter`'s ins
install `cookiecutter` for every new project.

```shell
python3 -m pip install --user --upgrade cookiecutter
python -m pip install --user --upgrade cookiecutter
```

### Get your own copy of the repository
Expand All @@ -30,17 +30,17 @@ run the tests later.

```shell
# Create a virtual environment, e.g. with
python3 -m venv env
python -m venv env

# activate virtual environment
source env/bin/activate

# make sure to have a recent version of pip and setuptools
python3 -m pip install --upgrade pip setuptools
python -m pip install --upgrade pip setuptools

# (from the project root directory)
# install development dependencies
python3 -m pip install --no-cache-dir .[dev]
python -m pip install --no-cache-dir .[dev]
```

## Running the tests
Expand Down
28 changes: 8 additions & 20 deletions tests/test_project.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,11 @@
import os
import subprocess
from pathlib import Path
from shutil import which
from sys import platform
from typing import Sequence

import pytest

IS_WINDOWS = platform.startswith('win')
IS_WINDOWS_CI = IS_WINDOWS and os.environ.get('CI', False)


def test_project_folder(cookies):
Expand All @@ -33,17 +30,11 @@ def run(args: Sequence[str], dirpath: os.PathLike) -> subprocess.CompletedProces

@pytest.fixture
def project_env_bin_dir(tmp_path):
if IS_WINDOWS_CI:
# Creating virtualenv does not work on Windows CI,
# falling back to using current pip3 dir
pip = Path(which('pip3'))
bin_dir = pip.parent
else:
env_output = run(['python3', '-m', 'venv', 'env'], tmp_path)
assert env_output.returncode == 0
bin_dir = str(tmp_path / 'env' / 'bin')
if IS_WINDOWS:
bin_dir = str(tmp_path / 'env' / 'Scripts')
env_output = run(['python', '-m', 'venv', 'env'], tmp_path)
assert env_output.returncode == 0
bin_dir = str(tmp_path / 'env' / 'bin')
if IS_WINDOWS:
bin_dir = str(tmp_path / 'env' / 'Scripts')
return str(bin_dir) + os.sep


Expand All @@ -52,17 +43,17 @@ def baked_with_development_dependencies(cookies, project_env_bin_dir):
result = cookies.bake()
assert result.exit_code == 0
bin_dir = project_env_bin_dir
latest_pip_output = run([f'{bin_dir}pip3', 'install', '--upgrade', 'pip', 'setuptools'], result.project)
latest_pip_output = run([f'{bin_dir}python', '-m', 'pip', 'install', '--upgrade', 'pip', 'setuptools'], result.project)
assert latest_pip_output.returncode == 0
pip_output = run([f'{bin_dir}pip3', 'install', '--editable', '.[dev]'], result.project)
pip_output = run([f'{bin_dir}python', '-m', 'pip', 'install', '--editable', '.[dev]'], result.project)
assert pip_output.returncode == 0
return result.project


def test_pytest(baked_with_development_dependencies, project_env_bin_dir):
project_dir = baked_with_development_dependencies
bin_dir = project_env_bin_dir
result = run([f'{bin_dir}pytest'], project_dir)
result = run([f'{bin_dir}python', '-m', 'pytest'], project_dir)
assert result.returncode == 0
assert '== 3 passed in' in result.stdout

Expand Down Expand Up @@ -98,9 +89,6 @@ def test_subpackage(baked_with_development_dependencies, project_env_bin_dir):
subsubpackage.mkdir()
(subsubpackage / '__init__.py').write_text('FOO = "bar"', encoding="utf-8")

if IS_WINDOWS_CI:
# On Windows CI python and pip executable are in different paths
bin_dir = ''
# sdist and bdist_wheel both call build command to create build/ dir
# So instead of looking in distribution archives we can look in build/ dir
result = run([f'{bin_dir}python', 'setup.py', 'build'], project_dir)
Expand Down
20 changes: 10 additions & 10 deletions {{cookiecutter.directory_name}}/.github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,16 @@ jobs:
- name: Python info
shell: bash -l {0}
run: |
which python3
python3 --version
which python
python --version
- name: Upgrade pip and install dependencies
run: |
python3 -m pip install --upgrade pip setuptools
python3 -m pip install .[dev,publishing]
python -m pip install --upgrade pip setuptools
python -m pip install .[dev,publishing]
- name: Run unit tests
run: pytest -v
run: python -m pytest -v
- name: Verify that we can build the package
run: python3 setup.py sdist bdist_wheel
run: python setup.py sdist bdist_wheel

lint:
name: Linting build
Expand All @@ -52,12 +52,12 @@ jobs:
- name: Python info
shell: bash -l {0}
run: |
which python3
python3 --version
which python
python --version
- name: Upgrade pip and install dependencies
run: |
python3 -m pip install --upgrade pip setuptools
python3 -m pip install .[dev,publishing]
python -m pip install --upgrade pip setuptools
python -m pip install .[dev,publishing]
- name: Check style against standards using prospector
run: prospector
- name: Check import order
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,12 @@ jobs:
- name: Python info
shell: bash -l {0}
run: |
which python3
python3 --version
which python
python --version
- name: Upgrade pip and install dependencies
run: |
python3 -m pip install --upgrade pip setuptools
python3 -m pip install .[dev,publishing]
python -m pip install --upgrade pip setuptools
python -m pip install .[dev,publishing]
- name: Install pandoc using apt
run: sudo apt install pandoc
- name: Build documentation
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@ jobs:
- name: Python info
shell: bash -l {0}
run: |
which python3
python3 --version
which python
python --version
- name: Install dependencies
run: python3 -m pip install .[dev]
run: python -m pip install .[dev]
- name: Check style against standards using prospector
run: prospector --zero-exit --output-format grouped --output-format pylint:pylint-report.txt
- name: Run unit tests with coverage
run: pytest --cov --cov-report term --cov-report xml --junitxml=xunit-result.xml tests/
run: python -m pytest --cov --cov-report term --cov-report xml --junitxml=xunit-result.xml tests/
- name: Correct coverage paths
run: sed -i "s+$PWD/++g" coverage.xml
- name: SonarCloud Scan
Expand Down
24 changes: 12 additions & 12 deletions {{cookiecutter.directory_name}}/README.dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,19 @@ If you're looking for user documentation, go [here](README.md).

```shell
# Create a virtual environment, e.g. with
python3 -m venv env
python -m venv env

# activate virtual environment
source env/bin/activate

# make sure to have a recent version of pip and setuptools
python3 -m pip install --upgrade pip setuptools
python -m pip install --upgrade pip setuptools

# (from the project root directory)
# install {{ cookiecutter.package_name }} as an editable package
python3 -m pip install --no-cache-dir --editable .
python -m pip install --no-cache-dir --editable .
# install development dependencies
python3 -m pip install --no-cache-dir --editable .[dev]
python -m pip install --no-cache-dir --editable .[dev]
```

Afterwards check that the install directory is present in the `PATH` environment variable.
Expand Down Expand Up @@ -156,22 +156,22 @@ cd $(mktemp -d {{ cookiecutter.package_name }}.XXXXXX)
git clone {{ cookiecutter.repository }} .

# prepare a clean virtual environment and activate it
python3 -m venv env
python -m venv env
source env/bin/activate

# make sure to have a recent version of pip and setuptools
python3 -m pip install --upgrade pip setuptools
python -m pip install --upgrade pip setuptools

# install runtime dependencies and publishing dependencies
python3 -m pip install --no-cache-dir .
python3 -m pip install --no-cache-dir .[publishing]
python -m pip install --no-cache-dir .
python -m pip install --no-cache-dir .[publishing]

# clean up any previously generated artefacts
rm -rf {{ cookiecutter.package_name }}.egg-info
rm -rf dist

# create the source distribution and the wheel
python3 setup.py sdist bdist_wheel
python setup.py sdist bdist_wheel

# upload to test pypi instance (requires credentials)
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
Expand All @@ -187,14 +187,14 @@ In a new terminal, without an activated virtual environment or an env directory:
cd $(mktemp -d {{ cookiecutter.package_name }}-test.XXXXXX)

# prepare a clean virtual environment and activate it
python3 -m venv env
python -m venv env
source env/bin/activate

# make sure to have a recent version of pip and setuptools
pip install --upgrade pip setuptools
python -m pip install --upgrade pip setuptools

# install from test pypi instance:
python3 -m pip -v install --no-cache-dir \
python -m pip -v install --no-cache-dir \
--index-url https://test.pypi.org/simple/ \
--extra-index-url https://pypi.org/simple {{ cookiecutter.package_name }}
```
Expand Down

0 comments on commit d5e299d

Please sign in to comment.