Skip to content

Commit

Permalink
fix(custom-resource-handlers): don't recursively process s3 bucket ob…
Browse files Browse the repository at this point in the history
…jects (#30209)

### Issue # (if applicable)
Fixes #30573.

### Reason for this change

I recently had the mispleasure of trying to empty a bucket with ~600000 objects using CDK's `autoDeleteObjects` feature. What I observed was that each lambda invocation would get through a few tens of thousands of objects in relatively good time (a few minutes), then the lambda would grind to a halt doing very little until it reached its 15 minute timeout. This process then repeats with subsequent invocations of the lambda.  I had to empty the bucket in the web console to make real progress toward deleting the bucket.

I have proven that the low memory allocated to the lambda (the default 128mb) plus this recursion is to blame. There is no need to recurse, and doing so will put pressure on the stack, the heap, and (because this is an async function) the event loop.

### Description of changes

Switch the recursion to iteration.

### Description of how you validated changes

#30209 (comment)

### Checklist
- [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
isker committed Jun 17, 2024
1 parent 95938b2 commit 25835e4
Show file tree
Hide file tree
Showing 194 changed files with 36,650 additions and 7,078 deletions.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
},
"S3Key": "96ea260348625427bb7f28ab3b379f62da6f428bcd155cb13f9261aa6a404b0d.zip"
"S3Key": "44e9c4d7a5d3fd2d677e1a7e416b2b56f6b0104bd5eff9cac5557b4c65a9dc61.zip"
},
"Timeout": 900,
"MemorySize": 128,
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

This file was deleted.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
},
"S3Key": "96ea260348625427bb7f28ab3b379f62da6f428bcd155cb13f9261aa6a404b0d.zip"
"S3Key": "44e9c4d7a5d3fd2d677e1a7e416b2b56f6b0104bd5eff9cac5557b4c65a9dc61.zip"
},
"Timeout": 900,
"MemorySize": 128,
Expand Down Expand Up @@ -449,7 +449,7 @@
"Value": "AWSServiceRoleForElasticBeanstalkManagedUpdates"
}
],
"SolutionStackName": "64bit Amazon Linux 2023 v6.1.2 running Node.js 20"
"SolutionStackName": "64bit Amazon Linux 2023 v6.1.6 running Node.js 20"
},
"DependsOn": [
"beastalkapp",
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ const beanstalkEnv = new elasticbeanstalk.CfnEnvironment(stack, 'beanstlk-env',
applicationName: beanstalkApp.applicationName!,
environmentName: 'codepipeline-test-env',
// see https://docs.aws.amazon.com/elasticbeanstalk/latest/platforms/platforms-supported.html#platforms-supported.nodejs
solutionStackName: '64bit Amazon Linux 2023 v6.1.2 running Node.js 20',
solutionStackName: '64bit Amazon Linux 2023 v6.1.6 running Node.js 20',
optionSettings: [
{
namespace: 'aws:autoscaling:launchconfiguration',
Expand Down
Loading

0 comments on commit 25835e4

Please sign in to comment.