Building Asynchronous Messaging with Amazon SQS - Designing Standard and FIFO Queues
Learn how to choose between Standard and FIFO queues, use DLQs for failed message handling, and automate processing with Lambda event source mappings.
Overview of SQS
SQS is a fully managed message queuing service. While SNS provides one-to-many fan-out, SQS provides one-to-one point-to-point messaging. It decouples web application request handling from heavy backend processing, improving scalability and fault tolerance.
Queue Design and DLQ
Standard queues provide throughput of tens of thousands of messages per second, with no guarantee of message ordering. FIFO queues guarantee ordering per message group ID and achieve exactly-once processing through deduplication IDs. Throughput is 300 messages per second (3,000 with batching). DLQs automatically move messages that have failed processing maxReceiveCount times, preventing them from blocking the main queue. After investigating DLQ messages and fixing the root cause, you can redrive (resend) them to the main queue.
Lambda Integration and Large Messages
SQS and Lambda event source mappings automatically process queue messages with Lambda functions. By configuring batch size and batch window, you can process multiple messages at once and reduce the number of Lambda invocations. With FIFO queues, Lambda's concurrency is limited to the number of message groups. The Extended Client Library enables sending and receiving messages larger than 256 KB via S3, supporting asynchronous processing of large payloads. Message encryption is configured with SSE-SQS (free) or SSE-KMS. For more on message queuing, you can also explore related books on Amazon.
SQS Pricing
SQS Standard queues cost approximately $0.40 per million requests, and FIFO queues cost approximately $0.50 per million requests. The first 1 million requests per month are free. Each request handles up to 256 KB, with every 64 KB chunk counted as one request. Long polling (WaitTimeSeconds=20) reduces empty responses and optimizes request counts. DLQ messages are billed at the same rate as regular queue messages, so process or delete DLQ messages periodically.
Summary
SQS is a service for building loosely coupled architectures through asynchronous messaging. Standard queues provide throughput of tens of thousands of messages per second, while FIFO queues offer ordering guarantees and deduplication. Lambda event source mappings automate message processing, and DLQs handle failed messages. Long polling optimizes request counts, and the Extended Client Library supports messages larger than 256 KB.