Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixed SNS MessageAttributes parsing #94

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

AlexanderKudryavsky
Copy link

@AlexanderKudryavsky AlexanderKudryavsky commented Feb 1, 2021

Summary

This PR fixes/implements the following bugs/features

Explain the motivation for making this change. What existing problem does the pull request solve?

Test plan (required)

@codeclimate
Copy link

codeclimate bot commented Feb 1, 2021

Code Climate has analyzed commit d53339d and detected 0 issues on this pull request.

View more on Code Climate.

@regevbr
Copy link
Contributor

regevbr commented Feb 1, 2021

@AlexanderKudryavsky I am not sure what goes wrong here... If the queue messages are generated by SNS, then they will never by gzipped or stored in S3, you should probably not send messages yourself in the same queue an SNS can send messages in. If you do think this is a valid case, I don't see in AWS docs where the property MessageAttributes exists on the SNS message body. Can you please refer me to it? Same goes for the new way you suggest to parse attributes.

if (opts.unwrapSns) {
const unwrapped: SNSBody = JSON.parse(this.body || EMPTY_BODY);
this.body = unwrapped.Message;
this.subject = unwrapped.Subject;
this.topicArn = unwrapped.TopicArn;
this.attributes = parseMessageAttributes(unwrapped.MessageAttributes);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parseMessageAttributes expects a map and not a string

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also be aware that this will cause an overide of the real message attributes.... so if you send a regular message (not via SNS) this will remove all the attributes. Might be better to merge the attributes (_.extend() for example)

@@ -21,6 +21,7 @@ export interface IMessageOpts {
}

interface SNSBody {
MessageAttributes: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a proper typing for that and then use that interface in the parseMessageAttributes and related functions (please use type guards to parse properly depending on the type we receive)

@@ -27,8 +27,8 @@ export const parseMessageAttributes = (messageAttributes: SQS.MessageBodyAttribu
};

const parseAttributeValue = (unparsedAttribute: SQS.MessageAttributeValue): IMessageAttribute => {
const type = unparsedAttribute.DataType;
const stringValue = unparsedAttribute.StringValue;
const type = unparsedAttribute.DataType || unparsedAttribute.Type;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please parse according to the actual type using type guards

Copy link
Author

@AlexanderKudryavsky AlexanderKudryavsky Feb 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@regevbr
If you include raw in sns, then the structure of the message changes, the attributes are no longer in the body, but it still does not work correctly

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you give exampels please. I'm guessing that if you are using the raw option, you don't need the unwrapSns flag turned on

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@regevbr
unwrapSns: false works. In this case, unwrapSns works somehow differently than I expected, the messages structure I posted to the Issue cannot be handled correctly

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand, so lets try to handle it in this PR

@regevbr
Copy link
Contributor

regevbr commented Feb 1, 2021

@AlexanderKudryavsky please also add tests and make sure the code compiles (npm run build) and passes all the tests. If you need any help, I'm here

@sonarcloud
Copy link

sonarcloud bot commented Mar 19, 2021

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants