Moving messages from source queue to DLQ after retries using AWS SNS

19/01/2022 - AWS

In this example, AWS SNS notifications will be forwarded to AWS SQS and processed from there. We are going to move messages from AWS SQS source queue to DLQ after 3 retries. Retries happen if the messages are not processed for some reason. Message will be visible every 30 seconds between each attempts.


Setup


// Create SQS queue
$ aws --profile localstack --endpoint-url http://localhost:4566 sqs create-queue \
    --queue-name test-queue.dlq

// Create SQS DLQ queue
$ aws --profile localstack --endpoint-url http://localhost:4566 sqs create-queue \
    --queue-name test-queue \
    --attributes '{"RedrivePolicy":"{\"deadLetterTargetArn\":\"arn:aws:sqs:eu-west-1:000000000000:test-queue.dlq\",\"maxReceiveCount\":\"3\"}"}'

// Create SNS topic
$ aws --profile localstack --endpoint-url http://localhost:4566 sns create-topic \
    --name test-topic

// Subscribe SQS queue to SNS topic
$ aws --profile localstack --endpoint-url http://localhost:4566 sns subscribe \
    --protocol sqs \
    --topic-arn arn:aws:sns:eu-west-1:000000000000:test-topic \
    --notification-endpoint arn:aws:sqs:eu-west-1:000000000000:test-queue

Test


$ aws --profile localstack --endpoint-url http://localhost:4566 sns publish \
    --topic-arn arn:aws:sns:eu-west-1:000000000000:test-topic \
    --message 'Hello'

Trying to consume messages 3 times.


$ aws --profile localstack --endpoint-url http://localhost:4566 sqs receive-message \
    --queue-url http://localhost:4566/000000000000/test-queue \
    --attribute-names All --message-attribute-names All

$ aws --profile localstack --endpoint-url http://localhost:4566 sqs receive-message \
    --queue-url http://localhost:4566/000000000000/test-queue \
    --attribute-names All --message-attribute-names All

$ aws --profile localstack --endpoint-url http://localhost:4566 sqs receive-message \
    --queue-url http://localhost:4566/000000000000/test-queue \
    --attribute-names All --message-attribute-names All

Message should now be in the DLQ.


$ aws --profile localstack --endpoint-url http://localhost:4566 sqs receive-message \
    --queue-url http://localhost:4566/000000000000/test-queue.dlq \
    --attribute-names All --message-attribute-names All