VMageste — High-Volume Marketing Analytics SaaS
~3 million events a day, multi-tenant
Key metrics
Context
Agencies and performance teams live switching between Meta, Google and TikTok dashboards, reconciling numbers that never quite match. The platform brings those sources into one place.
The challenge
Ingest ~3 million daily events from Meta, Google, and TikTok APIs into a multi-tenant platform while keeping OLTP latency under control and real-time dashboards responsive without bottlenecks.
The architecture
I led the redesign from a Node.js monolith to an event-driven microservices architecture. Kafka decouples ingestion from processing; Golang consumers handle stream processing at scale; ClickHouse absorbs analytics queries so PostgreSQL handles only transactional workloads. Strangler Fig pattern for zero-downtime migration.
Architecture flow
Traffic sources
- Meta
- TikTok
Ingestion
- Apache Kafka
- Decouples ingestion from processing
Processing
- Golang consumers
- Lead enrichment and deduplication
Storage
- ClickHouse — analytical queries
- PostgreSQL — transactional
Product
- Real-time dashboards
- Alerts and reports
Engineering decisions
Migrating without stopping the product
The Strangler Fig pattern allowed replacing the monolith piece by piece, keeping the old system running while each slice migrated — instead of a big-bang rewrite, which is where this kind of project usually dies.
Splitting writes from analytical reads
Asynchronous ingestion over Kafka and analytical querying on a columnar store. The two workloads stopped competing for the same resource, which was the real cause of the slowness.
Clean leads at the door
Enrichment and deduplication happen in the ingestion pipeline, not in a report. The sales team receives an already-cleaned base instead of discovering duplicates while dialling.