Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Discussion options

Hi there 👋

I am bit confused about queues & the messenger component as far as the doctrine transport is concerned.

I do use multiple transports with different queue names (as stated in the documentation), and the same DSN as you can see below.

        transports:
            # https://symfony.com/doc/current/messenger.html#transport-configuration
            async:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
                options:
                    queue_name: default
                failure_transport: ~
            webhook_async:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
                options:
                    queue_name: webhook_async
                failure_transport: ~
                retry_strategy:
                  max_retries: 5
                  # milliseconds delay
                  delay: 1000
                  # causes the delay to be higher before each retry
                  # e.g. 1 sec, 4 sec, 16, 64, 256
                  multiplier: 4
                  max_delay: 0
            email_async:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
                options:
                    queue_name: email_async
                failure_transport: 'failed_email_async'
                retry_strategy:
                    max_retries: 3
                    # milliseconds delay
                    delay: 1000
                    # causes the delay to be higher before each retry
                    # e.g. 1 second delay, 2 seconds, 4 seconds
                    multiplier: 10
                    max_delay: 0

            failed_email_async:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
                options:
                    queue_name: failed_email_async

The first issue I had is that, whatever transport I was using (via messenger:consume command) they were all picking into each others queues (eg: the async transport was picking into the email_async transport) so I figured that I had to be more "precise" and tell the consumer to consume a specific queue using the --queues option but then a second issue came out. It seems that I can't because the doctrine transport/receiver do not implement the QueueReceiverInterface.

Any idea where I did wrong ? Should I probably use separate DSNs ? Is this a hole in the racket ?

You must be logged in to vote

Replies: 2 comments

Comment options

I would split the transports by operational concern, not just by message class.

If the queues have different throughput, retry policy, failure handling, or database growth profile, separate Doctrine transports are easier to operate. You get clearer worker processes, clearer failure queues, and cleaner dashboards.

For example:

framework:
  messenger:
    transports:
      emails: '%env(MESSENGER_EMAIL_DSN)%'
      imports: '%env(MESSENGER_IMPORT_DSN)%'
      billing: '%env(MESSENGER_BILLING_DSN)%'

    routing:
      App\Message\SendEmail: emails
      App\Message\ImportJob: imports
      App\Message\BillingJob: billing

The main thing I would avoid is one giant Doctrine transport where slow jobs, urgent jobs, and noisy retries all compete in the same table.

Also give each transport its own failure transport if the recovery path differs. A failed email retry and a failed billing job usually should not be triaged the same way.

You must be logged in to vote
0 replies
Comment options

It looks like a bug since from the description of QueueReceiverInterface, DoctrineTransport should implement it 🤔

I also found #54821 which seems related.

I guess a PR would be the fastest way to make things move.

You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
🙏
Q&A
Labels
None yet
3 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.