inanzzz | Creating Dead Letter Queue within Symfony applications using PHP AWS SDK

In this example we are going to create a dead-letter queue, a standard queue and associate both of them together with Amazon Simple Queue Service (AWS SQS). The reason why we are doing it is because if a message in standard queue is not processed after certain amount of tries then it is moved to dead-letter queue where it sits and waits.

Files

parameters.yaml

Make sure the credentials below belongs to an AWS IAM user who has "AmazonSSQSFullAccess" permissions.

parameters:

    application_name: 'APP'

    aws_sdk.config.default:
        version: 'latest'
        region: 'eu-west-1'

    aws_sdk.credentials.default:
        credentials:
            key: 'AWS_KEY'
            secret: 'AWS_SECRET'

services.yaml

services:

    Aws\Sdk: ~

    App\Util\AwsSqsUtil:
        arguments:
            $applicationName: '%application_name%'
            $env: '%kernel.environment%'
        calls:
            - [createClient, ['%aws_sdk.config.default%', '%aws_sdk.credentials.default%']]

AwsSqsUtil

declare(strict_types=1);

namespace App\Util;

use Aws\Result;
use Aws\Sdk;
use Aws\Sqs\SqsClient;

class AwsSqsUtil implements AwsSqsUtilInterface
{
    private $sdk;
    /** @var SqsClient */
    private $client;
    private $applicationName;
    private $env;

    public function __construct(Sdk $sdk, string $applicationName, string $env)
    {
        $this->sdk = $sdk;
        $this->applicationName = $applicationName;
        $this->env = $env;
    }

    public function createClient(iterable $config, iterable $credentials): void
    {
        $this->client = $this->sdk->createSqs($config+$credentials);
    }

    /**
     * @link https://docs.aws.amazon.com/aws-sdk-php/v3/api/api-sqs-2012-11-05.html#createqueue
     */
    public function createQueue(string $name, bool $isDeadLetter, string $deadLetterArn = null): ?string
    {
        $attributes = ['VisibilityTimeout' => 60];
        if ($deadLetterArn) {
            $attributes['RedrivePolicy'] = sprintf(
                '{"deadLetterTargetArn":"%s","maxReceiveCount":"5"}',
                $deadLetterArn
            );
        }

        /** @var Result $result */
        $result = $this->client->createQueue([
            'QueueName' => $this->createQueueName($name, $isDeadLetter),
            'Attributes' => $attributes,
        ]);

        return $result->get('QueueUrl');
    }

    /**
     * @link https://docs.aws.amazon.com/aws-sdk-php/v3/api/api-sqs-2012-11-05.html#getqueueattributes
     */
    public function getQueueArn(string $url): string
    {
        /** @var Result $result */
        $result = $this->client->getQueueAttributes([
            'QueueUrl' => $url,
            'AttributeNames' => ['QueueArn']
        ]);

        return $result->get('Attributes')['QueueArn'];
    }

    private function createQueueName(string $name, bool $isDeadLetter): string
    {
        return sprintf(
            '%s_%s_%s%s',
            strtoupper($this->applicationName),
            strtoupper($this->env),
            $name,
            $isDeadLetter ? '_DL' : null
        );
    }
}

VisibilityTimeout: A message is considered to be "in flight" mode after it is received from a queue by a consumer, but not yet deleted from the queue. After given visibility timeout period the message becomes visible to consumers and received again. E.g. VisibilityTimeout => 60 keeps the message (received but not deleted for whatever reason) "in flight" mode for 60 seconds and makes it "available" again. Caution: If message processing takes longer than 60 seconds, your message will be put back into queue and will be reprocessed again which is something we don't want to happen.

RedrivePolicy::maxReceiveCount: The number of times a message can be received by consumers before being moved to the dead-letter queue. E.g. maxReceiveCount: 5 allows a consumer to receive the message maximum of 5 times. If it is still not processed successfully and not deleted then it is moved to associated dead-letter queue.

Usage

This is what we are going to end up with.

Create a dead-letter queue

AwsSqsUtil::createQueue('image_resize', true);

# Response (queue url)
https://sqs.eu-west-1.amazonaws.com/000000000000/APP_DEV_image_resize_DL

Get dead-letter queue ARN

AwsSqsUtil::getQueueArn('https://sqs.eu-west-1.amazonaws.com/000000000000/APP_DEV_image_resize_DL');

# Response (queue arn)
arn:aws:sqs:eu-west-1:000000000000:APP_DEV_image_resize_DL

Create a standard queue

AwsSqsUtil::createQueue('image_resize', false, 'arn:aws:sqs:eu-west-1:000000000000:APP_DEV_image_resize_DL');

# Response (queue url)
https://sqs.eu-west-1.amazonaws.com/000000000000/APP_DEV_image_resize

Dead-letter redrive policy now looks like below.