Description
Symfony version(s) affected: 4.3.5
Description
When using retry_strategy with Symfony messenger, if message handler keeps failing with an exception, the envelope gets filled with RedeliveryStamps. After enough re-tries, the PhpSerializer will start throwing "Allowed memory size of X bytes exhausted".
We are using messenger to ensure successful HTTP callback delivery. Notifications are queued and message handler sends the HTTP notification. If HTTP status code is not 200, Guzzle client will throw an exception, message is re-queued and tried again later. We encountered this issue when a network problem was blocking some notifications for a longer period of time.
How to reproduce
Example project is available here: https://github.com/Jontsa/messenger-retrystrategy-bug-example
You setup a symfony project with messenger and transport like doctrine. Create a message and message handler which will always throw exception. You can now queue one message and using messenger:consume -vvv see how the message size increases at each retry.
You can try and set a lower memory_limit:
$ /usr/bin/php -d memory_limit=10M bin/console messenger:consume -v --limit=100
[OK] Consuming messages from transports "async".
// The worker will automatically exit once it has processed 100 messages or received a stop signal via the
// messenger:stop-workers command.
// Quit the worker with CONTROL-C.
10:24:12 ERROR [messenger] Error thrown while handling message App\NotificationMessage. Dispatching for retry #1 using 1000 ms delay. Error: "I failed, please requeue..." ["message" => App\NotificationMessage^ { …},"class" => "App\NotificationMessage","retryCount" => 1,"delay" => 1000,"error" => "I failed, please requeue...","exception" => Symfony\Component\Messenger\Exception\HandlerFailedException^ { …}]
10:24:13 ERROR [messenger] Error thrown while handling message App\NotificationMessage. Dispatching for retry #2 using 1000 ms delay. Error: "I failed, please requeue..." ["message" => App\NotificationMessage^ { …},"class" => "App\NotificationMessage","retryCount" => 2,"delay" => 1000,"error" => "I failed, please requeue...","exception" => Symfony\Component\Messenger\Exception\HandlerFailedException^ { …}]
10:24:14 ERROR [messenger] Error thrown while handling message App\NotificationMessage. Dispatching for retry #3 using 1000 ms delay. Error: "I failed, please requeue..." ["message" => App\NotificationMessage^ { …},"class" => "App\NotificationMessage","retryCount" => 3,"delay" => 1000,"error" => "I failed, please requeue...","exception" => Symfony\Component\Messenger\Exception\HandlerFailedException^ { …}]
PHP Fatal error: Allowed memory size of 10485760 bytes exhausted (tried to allocate 20480 bytes) in /home/jontsa/Code/testing/messenger-retrystrategy-bug-example/vendor/symfony/messenger/Transport/Serialization/PhpSerializer.php on line 67
PHP Fatal error: Allowed memory size of 10485760 bytes exhausted (tried to allocate 20480 bytes) in /home/jontsa/Code/testing/messenger-retrystrategy-bug-example/vendor/symfony/debug/ErrorHandler.php on line 620
Possible Solution
Additional context