Skip to content

Commit

Permalink
add_docs_GH_actions (#16)
Browse files Browse the repository at this point in the history
* 📝 add gh actions pre-commit  documentation

* ✏️ Fix emojis in readme files
  • Loading branch information
JoseRZapata committed Feb 3, 2024
1 parent 227ffbf commit 26fe63b
Show file tree
Hide file tree
Showing 4 changed files with 72 additions and 38 deletions.
71 changes: 36 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,27 +18,27 @@ Source Code: <https://github.com/JoseRZapata/data-science-project-template>
Table of Contents

- [Data science project template](#data-science-project-template)
- [:tada: Creating a New Project](#tada-creating-a-new-project)
- [🗃️ Creating a New Project](#-creating-a-new-project)
- [Recommendations](#recommendations)
- [:link: Linking an Existing Project](#link-linking-an-existing-project)
- [:card\_file\_box: Project structure](#card_file_box-project-structure)
- [:sparkles: Features and Tools](#sparkles-features-and-tools)
- [:rocket: Project Standardization and Automation](#rocket-project-standardization-and-automation)
- [:hammer: Developer Workflow Automation](#hammer-developer-workflow-automation)
- [:seedling: Conditionally Rendered Python Package or Project Boilerplate](#seedling-conditionally-rendered-python-package-or-project-boilerplate)
- [:wrench: Maintainability](#wrench-maintainability)
- [:label: Type Checking and Data Validation](#label-type-checking-and-data-validation)
- [:white\_check\_mark: Testing/Coverage](#white_check_mark-testingcoverage)
- [:rotating\_light: Linting](#rotating_light-linting)
- [:construction\_worker: CI/CD](#construction_worker-cicd)
- [:lock: Security](#lock-security)
- [:lock\_with\_ink\_pen: Static Application Security Testing (SAST)](#lock_with_ink_pen-static-application-security-testing-sast)
- [:clipboard: Accessibility](#clipboard-accessibility)
- [:memo: Project Documentation](#memo-project-documentation)
- [:card\_file\_box: Templates](#card_file_box-templates)
- [🔗 Linking an Existing Project](#--linking-an-existing-project)
- [🗃️ Project structure](#-project-structure)
- [ Features and Tools](#-features-and-tools)
- [🚀 Project Standardization and Automation](#-project-standardization-and-automation)
- [🔨 Developer Workflow Automation](#-developer-workflow-automation)
- [🌱 Conditionally Rendered Python Package or Project Boilerplate](#-conditionally-rendered-python-package-or-project-boilerplate)
- [🔧 Maintainability](#-maintainability)
- [🏷️ Type Checking and Data Validation](#️--type-checking-and-data-validation)
- [Testing/Coverage](#--testingcoverage)
- [🚨 Linting](#-linting)
- [👷 CI/CD](#-cicd)
- [🔒 Security](#-security)
- [🔏 Static Application Security Testing (SAST)](#-static-application-security-testing-sast)
- [📋 Accessibility](#-accessibility)
- [📝 Project Documentation](#-project-documentation)
- [🗃️ Templates](#-templates)
- [References](#references)

## :tada: Creating a New Project
## 🗃️ Creating a New Project

### Recommendations

Expand Down Expand Up @@ -69,7 +69,7 @@ cookiecutter gh:JoseRZapata/data-science-project-template

Note: **_Cookiecutter_** uses `gh:` as short-hand for `https://github.com/`

## :link: Linking an Existing Project
## 🔗 Linking an Existing Project

If the project was originally installed via [Cookiecutter], you must first use [Cruft] to link the project with the original template:

Expand All @@ -83,7 +83,7 @@ Then/else:
cruft update
```

## :card_file_box: Project structure
## 🗃️ Project structure

Folder structure for data science projects [why?](https://towardsdatascience.com/the-importance-of-layered-thinking-in-data-engineering-a09f685edc71)

Expand All @@ -110,9 +110,10 @@ Folder structure for data science projects [why?](https://towardsdatascience.co
│   ├── actions
│   │   └── python-poetry-env
│   │   └── action.yml # github action to setup python environment
│   ├── dependabot.md # github action to update dependencies
│   ├── pull_request_template.md # template for pull requests
│   └── workflows
│   ├── dependencies.yml # github action to update dependencies
│   ├── docs.yml # github action to build documentation (mkdocs)
│   ├── pre-commit_autoupdate.yml # github action update pre-commit hooks
│   └── test.yml
├── .gitignore # files to ignore in git
Expand All @@ -138,11 +139,11 @@ Folder structure for data science projects [why?](https://towardsdatascience.co
└── settings.json # vscode settings
```

## :sparkles: Features and Tools
## Features and Tools

### :rocket: Project Standardization and Automation
### 🚀 Project Standardization and Automation

#### :hammer: Developer Workflow Automation
#### 🔨 Developer Workflow Automation

- Python packaging, dependency management and environment management
with [Poetry] - [`why?`](https://mathdatasimplified.com/poetry-a-better-way-to-manage-python-dependencies/)
Expand All @@ -156,17 +157,17 @@ Folder structure for data science projects [why?](https://towardsdatascience.co
- Continuous integration and deployment with [GitHub Actions]
- Project configuration files with [Hydra] - [`why?`](https://mathdatasimplified.com/stop-hard-coding-in-a-data-science-project-use-configuration-files-instead/)

#### :seedling: Conditionally Rendered Python Package or Project Boilerplate
#### 🌱 Conditionally Rendered Python Package or Project Boilerplate

- _Optional:_ [Jupyter] support

### :wrench: Maintainability
### 🔧 Maintainability

#### :label: Type Checking and Data Validation
#### 🏷️ Type Checking and Data Validation

- Static type-checking with [Mypy]

#### :white_check_mark: Testing/Coverage
#### Testing/Coverage

- Testing
with [Pytest]
Expand All @@ -177,7 +178,7 @@ Folder structure for data science projects [why?](https://towardsdatascience.co
- Coverage reporting
with [Codecov]

#### :rotating_light: Linting
#### 🚨 Linting

- Code quality:
- [Ruff]
Expand All @@ -202,31 +203,31 @@ Folder structure for data science projects [why?](https://towardsdatascience.co
- Invalid Python files
with [`check-ast`](https://github.com/pre-commit/pre-commit-hooks#check-ast)

#### :construction_worker: CI/CD
#### 👷 CI/CD

- Dependency updates
with [Dependabot]
- Automated [Dependabot] PR merging
with the [Dependabot Auto Merge GitHub Action](https://github.com/ahmadnassri/action-dependabot-auto-merge)

## :lock: Security
## 🔒 Security

### :lock_with_ink_pen: Static Application Security Testing (SAST)
### 🔏 Static Application Security Testing (SAST)

- Code vulnerabilities
with [Bandit]

## :clipboard: Accessibility
## 📋 Accessibility

### :memo: Project Documentation
### 📝 Project Documentation

- Documentation building
with [MkDocs] - [Tutorial](https://realpython.com/python-project-documentation-with-mkdocs/)
- Powered by [mkdocs-material](https://github.com/squidfunk/mkdocs-material)
- Rich automatic documentation from type annotations and docstrings (NumPy, Google, etc.)
with [mkdocstrings]

### :card_file_box: Templates
### 🗃️ Templates

- [Pull Request template]
- [Notebook template]
Expand Down
27 changes: 27 additions & 0 deletions docs/github_actions/gh_action_pre-commit-autoupdate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# GitHub Action:Pre-commit auto-update

This GitHub Action file configures a workflow named "Pre-commit auto-update". This workflow is responsible for automatically updating the pre-commit hooks in your repository.

[pre-commit_autoupdate.yml](https://github.com/JoseRZapata/data-science-project-template/blob/main/{{cookiecutter.repo_name}}/.github/workflows/pre-commit_autoupdate.yml)

## Workflow Details

- `on`: This workflow is triggered in two situations:

- `schedule`: It runs automatically every Monday at 7:00 UTC (according to the cron schedule `"0 7 * * 1"`).
- `workflow_dispatch`: This allows the workflow to be manually run from the GitHub user interface.
- `jobs`: Defines a job named "pre-commit-update".

## Job "pre-commit-update" Details

`runs-on`: This job runs on the latest version of Ubuntu.

`steps`: Defines the steps to be followed in this job.

`Checkout`: This step uses the actions/checkout@v3 action to get a copy of the repository.

`Update pre-commit hooks`: This step uses the `brokenpip3/action-pre-commit-update` action to update the pre-commit hooks. This action requires a GitHub token to function, which is passed through `github-token: ${{ secrets.PAT }}`. [PAT] is a secret stored in the repository settings that contains a personal access token with repository scope.

In summary, this workflow takes care of keeping the pre-commit hooks in the repository up-to-date, running automatically every Monday and also allowing manual execution whenever necessary.

[PAT]: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-fine-grained-personal-access-token
5 changes: 5 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,18 @@ theme:
name: Switch to light mode
features:
- content.code.copy
- navigation.expand
- navigation.path

nav:
- Introduction: 'index.md'
- Pre-commit configuration: 'pre-commit.md'
- Github Actions:
- Pre-commit_autoupdate: 'github_actions/gh_action_pre-commit-autoupdate.md'
- VSCode configuration: 'vscode.md'



markdown_extensions:
- pymdownx.snippets:
check_paths: true
Expand Down
7 changes: 4 additions & 3 deletions {{cookiecutter.repo_name}}/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v1.json)](https://github.com/charliermarsh/ruff)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)

## :sparkles: Features and Tools
## Features and Tools

Features | Package | Why?
--- | --- | ---
Expand Down Expand Up @@ -49,7 +49,7 @@ To install a new package, run:
poetry add <package-name>
```

## :card_file_box: Project structure
## 🗃️ Project structure

```bash
.
Expand All @@ -74,9 +74,10 @@ poetry add <package-name>
│   ├── actions
│   │   └── python-poetry-env
│   │   └── action.yml # github action to setup python environment
│   ├── dependabot.md # github action to update dependencies
│   ├── pull_request_template.md # template for pull requests
│   └── workflows
│   ├── dependencies.yml # github action to update dependencies
│   ├── docs.yml # github action to build documentation (mkdocs)
│   ├── pre-commit_autoupdate.yml # github action update pre-commit hooks
│   └── test.yml
├── .gitignore # files to ignore in git
Expand Down

0 comments on commit 26fe63b

Please sign in to comment.