Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Improve SQS interoperability #36525

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 23, 2020
Merged

Improve SQS interoperability #36525

merged 1 commit into from
Apr 23, 2020

Conversation

jderusse
Copy link
Member

Q A
Branch? master
Bug fix? no
New feature? yes
Deprecations? no
Tickets NA
License MIT
Doc PR NA

The Symfony Messenger component provides a SerializerInterface to encode/decode the Envelope, this can be used to improve the Interoperability (see article from jolicode (french))

Sadly, the current implementation of SQS adapter json_encode the elements of the Envelope (string body + string[] headers) and store everything in the SQS message Body. That partially defect the interoperability: 3rd party have to also wrap (unwrap) message form json_encoded Body.

This PR leverage the AWS SQS Body and MessageAttribute properties to store message information:

# before
SQS Message:
  Body: {"body": "hello world", "headers": {"foo": "bar"}}
  MessageAttributes: {}

# after
SQS Message:
  Body: hello world
  MessageAttributes:
    foor: bar

@jderusse jderusse requested a review from sroze as a code owner April 22, 2020 12:29
@chalasr chalasr added this to the next milestone Apr 22, 2020
chalasr
chalasr approved these changes Apr 22, 2020
View reviewed changes
Copy link
Member

@chalasr chalasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

chalasr
chalasr requested changes Apr 22, 2020
View reviewed changes
Copy link
Member

@chalasr chalasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests are broken :)

@jderusse
Copy link
Member Author

tests are broken because the FakeSqs service does not support MessageAttributes.... iain/fake_sqs#59

I'm looking for an alternative testing image

chalasr
chalasr approved these changes Apr 22, 2020
View reviewed changes
Copy link
Member

@chalasr chalasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remaining failures are unrelated

@chalasr
Copy link
Member

chalasr commented Apr 23, 2020

Thank you @jderusse.

@chalasr chalasr closed this in 83b37e8 Apr 23, 2020
@chalasr chalasr merged commit 83b37e8 into symfony:master Apr 23, 2020
@jderusse jderusse deleted the sqs-interop branch April 23, 2020 09:11
@nicolas-grekas nicolas-grekas modified the milestones: next, 5.1 May 4, 2020
@fabpot fabpot mentioned this pull request May 5, 2020
Merged
@Swahjak
Copy link

Swahjak commented Jun 1, 2020

@jderusse @chalasr is it possible this introduced the following issue:

Error: "HTTP 400 returned for "https://sqs.eu-west-1.amazonaws.com/".\n\nCode:    InvalidParameterValue\nMessage: Message (user) attribute name contains invalid non-alphanumeric character '#x5C'. You can use only alphanumeric characters, hyphens (-), underscores (_), or periods (.).\nType:    Sender\nDetail:  \n

When trying to send a message that is serialized (using the Symfony serializer) to:

array:2 [
  "body" => "{"id":9948}"
  "headers" => array:3 [
    "type" => "Topper\Bundle\Magento2Bundle\Messenger\Message\ProductUpdatedMessage"
    "X-Message-Stamp-Symfony\Component\Messenger\Stamp\BusNameStamp" => "[{"busName":"event.bus"}]"
    "Content-Type" => "application/json"
  ]

Trying to get a proper description going before opening an issue.

@jderusse jderusse mentioned this pull request Jun 1, 2020
Merged
fabpot added a commit that referenced this pull request Jun 3, 2020
@fabpot
bug #37038 Fix invalid char in SQS Headers (jderusse)
773b4ef 
This PR was squashed before being merged into the 5.1 branch.

Discussion
----------

Fix invalid char in SQS Headers

| Q             | A
| ------------- | ---
| Branch?       | 5.1
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | yes
| Tickets       | #36525 (comment)
| License       | MIT
| Doc PR        | /

From [Amazon documnetation](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-attributes.html) header's name have constraints:
- only `a-zA-Z0-9_\.-` + not start/end with a `.`
- 256 char

This PR serialize ALL headers in a single SQS Attribute.

Commits
-------

76a18b0 Fix invalid char in SQS Headers
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fabpot fabpot fabpot approved these changes

@chalasr chalasr chalasr approved these changes

@sroze sroze Awaiting requested review from sroze

Assignees
No one assigned
Labels
Feature Messenger Status: Reviewed
Projects
None yet
Milestone
5.1
Development

Successfully merging this pull request may close these issues.
6 participants
@jderusse @chalasr @Swahjak @fabpot @nicolas-grekas @carsonbot
Morty Proxy This is a proxified and sanitized view of the page, visit original site.