Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Commit

Permalink
Merge branch 'main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
milk531 committed Oct 4, 2023
2 parents 633417f + 78fbac0 commit 2357908
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 4 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- Bug that `put_directory` method in `cloud_storage.py`(GcsBucket) do not upload files that in directory

- Fix `list_folders` and `list_blobs` now logging bucket name and bucket path - [#184](https://github.com/PrefectHQ/prefect-gcp/pull/214)

### Security

## 0.4.7
Expand Down
20 changes: 16 additions & 4 deletions prefect_gcp/cloud_storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -870,7 +870,12 @@ async def list_blobs(self, folder: str = "") -> List["Blob"]:
client = self.gcp_credentials.get_cloud_storage_client()

bucket_path = self._join_bucket_folder(folder)
self.logger.info(f"Listing blobs in bucket {bucket_path}.")
if bucket_path is None:
self.logger.info(f"Listing blobs in bucket {self.bucket!r}.")
else:
self.logger.info(
f"Listing blobs in folder {bucket_path!r} in bucket {self.bucket!r}."
)
blobs = await run_sync_in_worker_thread(
client.list_blobs, self.bucket, prefix=bucket_path
)
Expand Down Expand Up @@ -903,12 +908,19 @@ async def list_folders(self, folder: str = "") -> List[str]:
from prefect_gcp.cloud_storage import GcsBucket
gcs_bucket = GcsBucket.load("my-bucket")
gcs_bucket.list_folders('years)
gcs_bucket.list_folders("years")
```
"""

bucket_path = self._join_bucket_folder()
self.logger.info(f"Listing folders in bucket {bucket_path}.")
# Beware of calling _join_bucket_folder twice, see note in method.
# However, we just want to use it to check if we are listing the root folder
bucket_path = self._join_bucket_folder(folder)
if bucket_path is None:
self.logger.info(f"Listing folders in bucket {self.bucket!r}.")
else:
self.logger.info(
f"Listing folders in {bucket_path!r} in bucket {self.bucket!r}."
)

blobs = await self.list_blobs(folder)
# gets all folders with full path
Expand Down

0 comments on commit 2357908

Please sign in to comment.