-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
@aws-cdk/aws-s3: Unable to delete the existing S3 event notifications #28915
@aws-cdk/aws-s3: Unable to delete the existing S3 event notifications #28915
Comments
Are you able to provide a minimal reproducible code snippets that we can simply copy/paste to reproduce this issue in our environment? That would be very helpful if we share the same code snippets. Thank you. |
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled. |
Hi, we are getting this error everytime we update the lambda target.
Every time
Why does it try to create a new notification config instead of updating the existing one? We tried deleting the existing notification config and allowing cf to create a new one but as soon as we update the lambda it fails again. I think this is related to this change 37be7b9 |
Here is the correct code path : We found out one more issue when we add a S3 event notification to the existing ones causes creation failure. The issue is likely related to the change: 37be7b9 The offending line is Line 39 - we are updating the ID of a JSON object, but the ID is inside of that object when it comes from outside. (https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/custom-resource-handlers/lib/aws-s3/notifications-resource-handler/index.py#L39) We can easily reproduce this issue when we try to add a new S3 event notifications on top of existing ones.
|
I was not able to deploy this with cdk 2.126.0 export class ExampleStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
const handler = this.getDummyHandler();
const importedHandler = lambda.Function.fromFunctionArn(this, "Handler", handler.functionArn);
const bucket = s3.Bucket.fromBucketName(this, "Bucket", my_existing_bucket_name);
bucket.addEventNotification(s3.EventType.OBJECT_CREATED_PUT, new s3n.LambdaDestination(importedHandler));
new CfnOutput(this, 'importedHandlerArn', { value: importedHandler.functionArn });
}
private getDummyHandler(): lambda.Function {
return new lambda.Function(this, 'Func', {
code: lambda.Code.fromInline(`def handler(_, _): return {"foo": "bar"} `),
functionName: 'Func',
handler: 'index.handler',
runtime: lambda.Runtime.PYTHON_3_10,
})
}
} And the log reads:
Upgrading this issue to p1. |
I am facing the same issue when adding a new SNS event notification to my already-created S3 bucket. The bucket is created using CDK only. Error - let bucket = new s3.Bucket(this, "Bucket", {
bucketName: bucketName
}
// new code
const snsTopic = sns.Topic.fromTopicArn(
this,
`snsTopic`,
"arn"
);
bucket.addEventNotification(
s3.EventType.OBJECT_CREATED
new s3n.SnsDestination(snsTopic)
); Is there a workaround? |
@Exter-dg check that there's not an existing rule that's referencing a construct that does not exist anymore. If this is the case, recreate the construct (with the same arn) and then delete the notification setting before removing the construct. |
@manmartgarc There isn't. This is the first notification that is being added to the bucket |
When you check the bucket notification configuration, can you verify that there's nothing there? |
@manmartgarc yes, it is empty. There are no notifications configured |
@kshefchek We have a PR #29022 WIP and will hopefully resolve this issue. |
@pahud thank you! |
What can we do, if PR #29022 is closed? |
Hi @gkaskonas, What you're seeing is actually not related to the change you mentioned. After a bit investigation, I notice that you're using imported lambda function. For imported function, we will set the property Since this property is set to Without necessary permission, creating a s3 event notification will then fail due to |
@Exter-dg I can confirm that I can reproduce the issue. However, one thing I noticed is that the default access policy for SNS topic has the following condition:
This seems to be the issue here and causes configuration error. According to the official documentation, one of the step is to replace the default access policy with the one they provided which uses
I tried it out and this time it deploys successfully. |
To summarize this issue, there are three issues I noticed in the description and comments.
|
Hi, we also have sometimes issues with the notifications. Please check the following python example: from typing import Any
from aws_cdk import App
from aws_cdk import Stack
from aws_cdk import aws_s3 as s3
from aws_cdk import aws_s3_notifications as s3n
from aws_cdk import aws_sns as sns
from constructs import Construct
class S3BucketStack(Stack):
def __init__(
self,
scope: Construct | None = None,
id: str | None = None,
**kwargs: Any,
) -> None:
super().__init__(scope, id, **kwargs)
bucket = s3.Bucket(self, "Bucket")
self.bucket_arn = bucket.bucket_arn
class SnsSubscriptionStack(Stack):
def __init__(
self,
scope: Construct | None = None,
id: str | None = None,
*,
bucket_arn: str,
enable_notification: bool = False,
**kwargs: Any,
) -> None:
super().__init__(scope, id, **kwargs)
bucket = s3.Bucket.from_bucket_arn(self, "Bucket", bucket_arn=bucket_arn)
topic = sns.Topic(self, "Topic")
if enable_notification:
bucket.add_event_notification(
s3.EventType.OBJECT_CREATED,
s3n.SnsDestination(topic),
s3.NotificationKeyFilter(prefix="foo")
)
bucket.add_event_notification(
s3.EventType.OBJECT_REMOVED,
s3n.SnsDestination(topic),
)
app = App()
bucket_stack = S3BucketStack(app, "BucketStack")
SnsSubscriptionStack(app, "SnsSubscriptionStack", bucket_arn=bucket_stack.bucket_arn)
app.synth() I toggle the Error message from CDK:
CloudWatch log:
|
It works fine when creating the notification config the first time (permissions are fine) so not sure what changes when updating it. |
Sure.. Please find the example stack attached.
|
|
1 similar comment
|
Re-opening as the original fix has a bug in the Feature flag implementation. |
|
1 similar comment
|
Describe the bug
I'm facing an issue while trying to delete the existing S3 event notifications.
The problem arises with S3 bucket event notification handler Lambda(AWS CloudFormation handler for "Custom::S3BucketNotifications" resources (@aws-cdk/aws-s3)), where it fails and reports an error: when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.
Custom event notification handler reference : https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/custom-resource-handlers/lib/aws-s3/notifications-resource-handler/index.py
The issue is likely related to the change: 37be7b9
The offending line is Line 39 - we are updating the ID of a JSON object, but the ID is inside of that object when it comes from outside. (https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/custom-resource-handlers/lib/aws-s3/notifications-resource-handler/index.py#L39)
Expected Behavior
Should delete the existing event notifications
Current Behavior
Unable to delete the existing filters:
Error: when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.
Reproduction Steps
Please find the example stack attached to reproduce the issue.
s3-event-notifications.txt
Possible Solution
No response
Additional Information/Context
No response
CDK CLI Version
2.124.0
Framework Version
No response
Node.js Version
v20.11.0
OS
Linux
Language
TypeScript
Language Version
No response
Other information
No response
The text was updated successfully, but these errors were encountered: