Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Discussion options

Hello friends!
I am looking for a way to uncover the real problem here and this is all I get which gives me nothing 👀
How should one get decent error logging while consuming AMQP (RabbitMQ) messages that fail like this one?
I have a working monolog setup that reports everything >=WARNING but consumers have their own rules 🍻
Thanks!

Error thrown while handling message Acme\Bundle\MessageBundle\BookMessage. 
Sending for retry #1 using 2000 ms delay. 
Error: "Redelivered message from AMQP detected that will be rejected and trigger the retry logic."

{
    "class": "Symfony\\Component\\Messenger\\Exception\\RejectRedeliveredMessageException",
    "message": "Redelivered message from AMQP detected that will be rejected and trigger the retry logic.",
    "code": 0,
    "file": "/var/www/html/vendor/symfony/messenger/Middleware/RejectRedeliveredMessageMiddleware.php:38",
    "trace": [
        "/var/www/html/vendor/symfony/messenger/Middleware/AddBusNameStampMiddleware.php:37",
        "/var/www/html/vendor/symfony/messenger/MessageBus.php:70",
        "/var/www/html/vendor/symfony/messenger/RoutableMessageBus.php:54",
        "/var/www/html/vendor/symfony/messenger/Worker.php:161",
        "/var/www/html/vendor/symfony/messenger/Worker.php:108",
        "/var/www/html/vendor/symfony/messenger/Command/ConsumeMessagesCommand.php:238",
        "/var/www/html/vendor/symfony/console/Command/Command.php:326",
        "/var/www/html/vendor/symfony/console/Application.php:1096",
        "/var/www/html/vendor/symfony/framework-bundle/Console/Application.php:126",
        "/var/www/html/vendor/symfony/console/Application.php:324",
        "/var/www/html/vendor/symfony/framework-bundle/Console/Application.php:80",
        "/var/www/html/vendor/symfony/console/Application.php:175",
        "/var/www/html/bin/console:43"
    ]
}
You must be logged in to vote

@andrew-demb I forgot to text earlier but even now for the record and anyone else wasting time on this.

Root Cause: AMQP Heartbeat Loss During Long-Running Message Handlers

I spent a good amount of time debugging this exact issue and finally traced the root cause. Sharing here because the error message is genuinely misleading and there's very little documentation connecting these dots.

Why There's No "Real Exception"

There isn't one. RejectRedeliveredMessageException is not hiding an underlying error — it is the error, but the cause is invisible because it happens at the AMQP protocol level, not in your PHP code.

Here's what actually happens:

  1. Your consumer picks up a message from RabbitMQ

Replies: 2 comments

Comment options

See the description of the middleware

 * AMQP redelivers messages when they do not get acknowledged or rejected. This can happen when the connection times out
 * or an exception is thrown before acknowledging or rejecting. When such errors happen again while handling the
 * redelivered message, the message would get redelivered again and again. The purpose of this middleware is to prevent
 * infinite redelivery loops and to unblock the queue by republishing the redelivered messages as retries with a retry
 * limit and potential delay.

Redelivery will be occurred when consumer didn't ack nor reject the queue message - there is no way to know the root cause

You must be logged in to vote
0 replies
Comment options

@andrew-demb I forgot to text earlier but even now for the record and anyone else wasting time on this.

Root Cause: AMQP Heartbeat Loss During Long-Running Message Handlers

I spent a good amount of time debugging this exact issue and finally traced the root cause. Sharing here because the error message is genuinely misleading and there's very little documentation connecting these dots.

Why There's No "Real Exception"

There isn't one. RejectRedeliveredMessageException is not hiding an underlying error — it is the error, but the cause is invisible because it happens at the AMQP protocol level, not in your PHP code.

Here's what actually happens:

  1. Your consumer picks up a message from RabbitMQ
  2. Your handler starts processing — maybe it makes an HTTP request, calls an external API, runs a slow database query, or does any blocking I/O
  3. While your PHP code is blocked waiting for that operation to complete, the AMQP connection goes idle
  4. RabbitMQ sends heartbeat frames to check if the consumer is still alive
  5. The PECL amqp extension is single-threaded — it can only respond to heartbeat frames during AMQP API calls, not while your PHP code is busy doing something else
  6. After missing 2 heartbeat cycles, RabbitMQ declares your connection dead and closes it
  7. Messages that were prefetched by your consumer get redelivered to another consumer (or the same one after reconnecting) with the AMQP redelivered flag set to true
  8. RejectRedeliveredMessageMiddleware sees this flag and throws RejectRedeliveredMessageException before your handler is ever invoked

That's why you see no application-level exception — your handler code never runs on the redelivered message. The middleware intercepts it first.

The Timing Math

The critical relationship is:

handler_max_execution_time < heartbeat_timeout

RabbitMQ considers a connection dead after heartbeat * 2 seconds of silence (heartbeat frames are sent every heartbeat / 2 seconds, and 2 missed frames = dead).

Example with default/common settings:

Setting Value Effect
heartbeat=15 Frames every ~7.5s, dead after ~30s
Handler makes HTTP call Takes 50s Blocks PHP thread for 50s
Result 50s > 30s Connection killed, messages redelivered

How to Fix It

1. Reduce your handler's maximum blocking time

This is the most important fix. If your handler makes HTTP calls, set explicit timeouts:

// Symfony HttpClient — per request
$response = $this->httpClient->request('POST', $url, [
    'timeout' => 20,        // max idle time between data chunks
    'max_duration' => 25,   // absolute max time for the entire request
    'body' => $payload,
]);

// Or configure on the scoped client (recommended)
// config/packages/framework.yaml
framework:
    http_client:
        scoped_clients:
            my_callback.client:
                timeout: 20
                max_duration: 25

timeout controls idle time (no data received), max_duration caps the total request time regardless of activity. Use both.

2. Tune your AMQP transport DSN

MESSENGER_TRANSPORT_DSN=amqp://user:pass@rabbitmq:5672/%2f/messages?heartbeat=20&connect_timeout=5&read_timeout=10&confirm_timeout=5

Parameter guide:

Parameter Recommended Why
heartbeat 20-30 Dead connection detection at 2× this value. Must be higher than your handler's max execution time. RabbitMQ docs recommend 5-20s range, but for handlers with external I/O you may need higher.
read_timeout heartbeat How long the AMQP socket read blocks waiting for data. If set lower than heartbeat interval, you'll get spurious timeouts during idle periods (heartbeat frames arrive every heartbeat/2 seconds).
connect_timeout 5 TCP connection timeout to RabbitMQ. Keep low to fail fast.
confirm_timeout 5 How long to wait for publish confirms. With clustered RabbitMQ / quorum queues, the leader needs Raft consensus before confirming — 2s can be tight under load.

The golden rule:

handler_max_blocking_time  <  heartbeat * 2  (dead connection threshold)
read_timeout               >= heartbeat      (avoid idle timeout between heartbeat frames)

With the example above: handler max = 25s, dead connection = 40s, read_timeout = 10s > heartbeat/2 = 10s. Everything fits.

3. Consider prefetch_count

If you run multiple workers consuming from the same queue, set prefetch_count=1 on transports with potentially slow handlers. This limits the blast radius — if a consumer's connection dies, only 1 message is stuck in limbo instead of many.

What About --keepalive?

Symfony 7.2+ added a --keepalive option to messenger:consume, but as of today it only supports Beanstalkd, Amazon SQS, Redis, and Doctrine transports. AMQP is not supported. The keepalive mechanism works by resetting delivery timestamps on the transport side, but for AMQP the problem is at the TCP/heartbeat level which this doesn't address.

There is a newer pure-PHP transport (jwage/phpamqplib-messenger) that uses php-amqplib instead of the PECL extension and handles heartbeats in a separate I/O loop — meaning heartbeats would continue during message processing. However, it's still pre-1.0 (0.9.0 as of writing) and has some limitations (e.g., cannot consume multiple transports in one process). Worth watching for the future — the roadmap is to integrate it into Symfony core for 7.4 LTS.

Common Scenarios That Trigger This

Any blocking operation in your handler that exceeds the heartbeat window:

  • HTTP callbacks/webhooks to external services — slow merchants, timeouts, proxy failures
  • External API calls — payment gateways, notification services, third-party integrations
  • Heavy database operations — large imports, complex queries, lock waits
  • File processing — large uploads, PDF generation, image processing
  • DNS resolution — slow or failing DNS can add unexpected seconds

Quick Diagnostic Checklist

If you're seeing RejectRedeliveredMessageException:

  1. Check your heartbeat value in the transport DSN (if not set, it uses the RabbitMQ server default, often 60s)
  2. Look for slow operations in your handler — check HTTP client logs for response times
  3. Compare: is the slowest handler execution time > heartbeat * 2?
  4. If yes → reduce handler blocking time and/or increase heartbeat
  5. If no → check for other causes: network partitions, RabbitMQ node failures, consumer OOM kills

Hope this saves someone the hours I spent staring at a stack trace that told me nothing useful.

You must be logged in to vote
0 replies
Answer selected by 99hops
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
🙏
Q&A
Labels
None yet
2 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.