AWS SQS AWS Lambda Node.js Symfony Python

CloudTxt

High-performance messaging infrastructure capable of processing millions of messages at scale.

Key Impact & Metrics

Scaled to handle 10M+ messages in 20-minute windows

Achieved 99.9% delivery rate

Built on AWS SQS and Lambda for serverless architecture

Reduced infrastructure costs by 40%

The Challenge

CloudTxt's original messaging infrastructure was built on a monolithic architecture using a single EC2 instance running a Symfony application with a MySQL queue. As the platform grew to serve schools, nonprofits, and community organizations, the system began failing under load. During peak notification windows — typically morning announcements and emergency alerts — the platform needed to deliver millions of SMS and push notifications within tight 20-minute windows. The existing architecture could handle roughly 100,000 messages per day before latency spikes caused cascading failures, messages queued for hours, and delivery rates dropped below 90%. Infrastructure costs were climbing linearly with volume, and the lack of priority queuing meant transactional messages like password resets were delayed alongside bulk marketing sends.

The Approach

We redesigned CloudTxt's messaging pipeline from the ground up using an event-driven serverless architecture. Messages are partitioned into three priority tiers: Critical (OTPs and security alerts with sub-5-second delivery), Standard (notifications with sub-60-second delivery), and Bulk (marketing with sub-30-minute delivery). Each tier has dedicated SQS queues and Lambda worker pools with independent auto-scaling. We implemented a fan-out pattern using SNS to distribute messages across multiple delivery channels (SMS via Twilio, push via Firebase, email via SES), allowing each channel to scale independently. Idempotent processing with Redis prevents duplicate sends, connection pooling manages third-party API rate limits, and real-time monitoring via DataDog provides live visibility into message flow, delivery rates, and queue depths.

The Results

The new architecture handles 10 million+ messages within 20-minute windows with a 99.9% delivery rate, representing a 200x improvement in throughput. Peak throughput reached 2,400 messages per second sustained with end-to-end latency averaging 2.3 seconds. Infrastructure costs dropped 40% despite the massive increase in capacity because the serverless model means paying only for actual message processing rather than maintaining idle EC2 capacity. The system has maintained 99.95% uptime over 18 months with zero message loss and zero manual intervention for scaling.

Technology Stack

AWS SQS
AWS Lambda
Node.js
Symfony
Python

This project was built by Dibyank Padhy using a modern technology stack optimized for performance, scalability, and developer experience. Each technology was selected to address specific architectural requirements identified during the planning phase.

Found this case study interesting? Share it with others

DP

Built by Dibyank Padhy

Dibyank Padhy is an Engineering Manager & Full Stack Developer with 7+ years of experience building scalable software solutions. Passionate about cloud architecture, team leadership, and AI integration.

More Projects

Interested in working together?

I'm always open to discussing new projects, creative ideas, or opportunities to be part of your vision.