You fire up an EC2 instance to process jobs, but messages pile up in your queue faster than you can drain them. Notifications lag, retries multiply, and suddenly your “simple workflow” looks like a Rube Goldberg machine made of JSON. Welcome to the dark art of AWS SQS/SNS integration.
AWS Simple Queue Service (SQS) gives you reliable, ordered message delivery. Simple Notification Service (SNS) fans those messages out to multiple targets. EC2 is the workhorse that picks them up, runs compute, and stores results. Together, they form a sturdy backbone for distributed systems that need durability and speed without constant babysitting.
A clean integration flow looks like this: SNS publishes an event—maybe a new file in S3 or a webhook from an external service. That event triggers an SQS queue subscribed to the SNS topic. Your EC2 instance polls the queue, processes messages, and reports success or failure. IAM roles define which instance can read from which queue. The magic lies in keeping permissions tight and retries sane.
Keep your instances running with minimal noise by following a few key rules. Use short-lived credentials mapped through IAM role assumption. Set visibility timeouts carefully: too short and you double-process, too long and you block your workers. Monitor dead-letter queues so you can see when things go off script. Always encrypt messages at rest and in transit—SOC 2 auditors love that part.
If something breaks, check the message retention periods before blaming your code. SQS can hold messages for up to fourteen days. Many engineers forget that, leading to ghosts of old jobs haunting new deployments. Clean up queues during releases, or you might find EC2 chewing through stale work at 3 a.m.