Skip to content

reweeden/randomdownloadchecker

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 

Repository files navigation

randomdownloadchecker

Lambda for attempting random downloads. This would be EXCEPTIONALLY easy to wrap up in a cloudformation object, but I don't have the time to do that right now. Feel free to push a CF template as a MR.

Setup

Upload lambda_function.py

However you want to upload...

Create SNS

The script uses an SNS to relay success/falure. Create an SNS topic for the script to use

Add Lambda Environment Variables

SNS Topic ARN:

sns_arn = arn:aws:sns:us-east-1:777766665555:RandomDownloadChecker

Which CMR to query:

cmr_api = https://cmr.earthdata.nasa.gov

URS Username and Password:

urs_user = valid_urs_user
usr_pass = SomeCr@zyPassword

Collections to avoid trying to download from (comma seperated list):

skip_collections = C1234567-DAAC,C765432-DAAC,C55555555-DAAC

More complex CMR search filtering parameters (get params):

collection_filter = provider=DAAC

Lambda Config

Lambda IAM: It is imporant to use an IAM role capable of calling SNS. It would probably be easiest to start with a BasicExecution Lambda policy and add sns:Publish.

Lambda Timeout: Downloads take time, the default timeout may not be long enough to run all your downloads!

Create a Cloudwatch Event

Set up a scheduled event to run the lambda at whatever periodicity you desire

Sit back and bask in the glory that is piece of mind knowing your downloads are successful!

About

Lambda for attempting random downloads

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%