Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Update workers to respect default command from configuration class #216

Merged
merged 3 commits into from
Oct 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ jobs:
strategy:
matrix:
python-version:
- "3.7"
- "3.8"
- "3.9"
- "3.10"
- "3.11"
fail-fast: false
steps:
- uses: actions/checkout@v4
Expand All @@ -33,8 +33,7 @@ jobs:
env:
PREFECT_SERVER_DATABASE_CONNECTION_URL: "sqlite+aiosqlite:///./collection-tests.db"
run: |
coverage run --branch -m pytest tests -vv
coverage report
pytest --cov=prefect_gcp --no-cov-on-fail --cov-report=term-missing --cov-branch tests -n auto -vv

- name: Run mkdocs build
run: |
Expand Down
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Security

## 0.5.1

Released October 12th, 2023.

### Changed

- Updated workers to respect default command from base worker configuration - [#216](https://github.com/PrefectHQ/prefect-gcp/pull/216)

## 0.5.0

Released October 5th, 2023.

### Fixed

- Empty logging in `list_folders` and `list_blobs` - [#214](https://github.com/PrefectHQ/prefect-gcp/pull/214)

### Added

- Conditional imports to support operating with `pydantic>2` installed - [#215](https://github.com/PrefectHQ/prefect-gcp/pull/215)

## 0.4.7

Released September 22nd, 2023.
Expand Down
6 changes: 2 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
# Coordinate and use GCP in your dataflow with `prefect-gcp`
# `prefect-gcp`

<p align="center">
<img src="https://user-images.githubusercontent.com/15331990/214915616-6369697e-bc84-400a-a584-845d795a68f2.png">
<br>
<a href="https://pypi.python.org/pypi/prefect-gcp/" alt="PyPI version">
<img alt="PyPI" src="https://img.shields.io/pypi/v/prefect-gcp?color=0052FF&labelColor=090422"></a>
<a href="https://github.com/PrefectHQ/prefect-gcp/" alt="Stars">
Expand All @@ -18,7 +16,7 @@
<img src="https://img.shields.io/badge/discourse-browse_forum-red.svg?color=0052FF&labelColor=090422&logo=discourse" /></a>
</p>

The `prefect-gcp` collection makes it easy to leverage the capabilities of Google Cloud Platform (GCP) in your flows, featuring support for Vertex AI, Cloud Run, BigQuery, Cloud Storage, and Secret Manager.
`prefect-gcp` makes it easy to leverage the capabilities of Google Cloud Platform (GCP) in your flows, featuring support for Vertex AI, Cloud Run, BigQuery, Cloud Storage, and Secret Manager.

Visit the full docs [here](https://PrefectHQ.github.io/prefect-gcp).

Expand Down
Binary file modified docs/img/favicon.ico
Binary file not shown.
Binary file removed docs/img/prefect-logo-mark-solid-white-500.png
Binary file not shown.
Binary file added docs/img/prefect-logo-mark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 8 additions & 10 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,22 @@
# Coordinate and use GCP in your dataflow with `prefect-gcp`
# `prefect-gcp`

<p align="center">
<img src="https://user-images.githubusercontent.com/15331990/214915616-6369697e-bc84-400a-a584-845d795a68f2.png">
<br>
<a href="https://pypi.python.org/pypi/prefect-gcp/" alt="PyPI version">
<img alt="PyPI" src="https://img.shields.io/pypi/v/prefect-gcp?color=0052FF&labelColor=090422"></a>
<img alt="PyPI" src="https://img.shields.io/pypi/v/prefect-gcp?color=26272B&labelColor=090422"></a>
<a href="https://github.com/PrefectHQ/prefect-gcp/" alt="Stars">
<img src="https://img.shields.io/github/stars/PrefectHQ/prefect-gcp?color=0052FF&labelColor=090422" /></a>
<img src="https://img.shields.io/github/stars/PrefectHQ/prefect-gcp?color=26272B&labelColor=090422" /></a>
<a href="https://pypistats.org/packages/prefect-gcp/" alt="Downloads">
<img src="https://img.shields.io/pypi/dm/prefect-gcp?color=0052FF&labelColor=090422" /></a>
<img src="https://img.shields.io/pypi/dm/prefect-gcp?color=26272B&labelColor=090422" /></a>
<a href="https://github.com/PrefectHQ/prefect-gcp/pulse" alt="Activity">
<img src="https://img.shields.io/github/commit-activity/m/PrefectHQ/prefect-gcp?color=0052FF&labelColor=090422" /></a>
<img src="https://img.shields.io/github/commit-activity/m/PrefectHQ/prefect-gcp?color=26272B&labelColor=090422" /></a>
<br>
<a href="https://prefect-community.slack.com" alt="Slack">
<img src="https://img.shields.io/badge/slack-join_community-red.svg?color=0052FF&labelColor=090422&logo=slack" /></a>
<img src="https://img.shields.io/badge/slack-join_community-red.svg?color=26272B&labelColor=090422&logo=slack" /></a>
<a href="https://discourse.prefect.io/" alt="Discourse">
<img src="https://img.shields.io/badge/discourse-browse_forum-red.svg?color=0052FF&labelColor=090422&logo=discourse" /></a>
<img src="https://img.shields.io/badge/discourse-browse_forum-red.svg?color=26272B&labelColor=090422&logo=discourse" /></a>
</p>

The `prefect-gcp` collection makes it easy to leverage the capabilities of Google Cloud Platform (GCP) in your flows, featuring support for Vertex AI, Cloud Run, BigQuery, Cloud Storage, and Secret Manager.
`prefect-gcp` makes it easy to leverage the capabilities of Google Cloud Platform (GCP) in your flows, featuring support for Vertex AI, Cloud Run, BigQuery, Cloud Storage, and Secret Manager.

## Getting Started

Expand Down
14 changes: 7 additions & 7 deletions docs/stylesheets/extra.css
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
/* theme */
:root > * {
/* theme */
--md-primary-fg-color: #115AF4;
--md-primary-fg-color--light: #115AF4;
--md-primary-fg-color--dark: #115AF4;
--md-primary-fg-color: #26272B;
--md-primary-fg-color--light: #26272B;
--md-primary-fg-color--dark: #26272B;
}

/* Table formatting */
Expand Down Expand Up @@ -72,7 +72,7 @@ to force column width */
/* dark mode slate theme */
/* dark mode code overrides */
[data-md-color-scheme="slate"] {
--md-code-bg-color: #252a33;
--md-code-bg-color: #1c1d20;
--md-code-fg-color: #eee;
--md-code-hl-color: #3b3d54;
--md-code-hl-name-color: #eee;
Expand Down Expand Up @@ -100,15 +100,15 @@ to force column width */

/* dark mode collection catalog overrides */
[data-md-color-scheme="slate"] .collection-item {
background-color: #3b3d54;
background-color: #26272B;
}

/* dark mode recipe collection overrides */
[data-md-color-scheme="slate"] .recipe-item {
background-color: #3b3d54;
background-color: #26272B;
}

/* dark mode API doc overrides */
[data-md-color-scheme="slate"] .prefect-table th {
background-color: #3b3d54;
background-color: #26272B;
}
2 changes: 1 addition & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ theme:
icon:
repo: fontawesome/brands/github
logo:
img/prefect-logo-mark-solid-white-500.png
img/prefect-logo-mark.png
font:
text: Inter
code: Source Code Pro
Expand Down
6 changes: 1 addition & 5 deletions prefect_gcp/workers/cloud_run.py
Original file line number Diff line number Diff line change
Expand Up @@ -364,11 +364,7 @@ def _populate_or_format_command(self):
if command is None:
self.job_body["spec"]["template"]["spec"]["template"]["spec"][
"containers"
][0]["command"] = [
"python",
"-m",
"prefect.engine",
]
][0]["command"] = shlex.split(self._base_flow_run_command())
elif isinstance(command, str):
self.job_body["spec"]["template"]["spec"]["template"]["spec"][
"containers"
Expand Down
8 changes: 3 additions & 5 deletions prefect_gcp/workers/vertex.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,11 +311,9 @@ def _inject_formatted_command(self):

existing_command = worker_pool_specs[0]["container_spec"].get("command")
if existing_command is None:
worker_pool_specs[0]["container_spec"]["command"] = [
"python",
"-m",
"prefect.engine",
]
worker_pool_specs[0]["container_spec"]["command"] = shlex.split(
self._base_flow_run_command()
)
elif isinstance(existing_command, str):
worker_pool_specs[0]["container_spec"]["command"] = shlex.split(
existing_command
Expand Down
22 changes: 12 additions & 10 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -1,17 +1,19 @@
pandas
pyarrow
pytest
black
coverage
flake8
mypy
interrogate
isort
mkdocs
mkdocs-gen-files
mkdocs-material
mkdocstrings[python]
isort
pre-commit
pytest-asyncio
mock; python_version < '3.8'
mkdocs-gen-files
interrogate
coverage
mypy
pandas
pillow
pre-commit
pyarrow
pytest
pytest-asyncio
pytest-cov
pytest-xdist