In this blog , I will share my experience on building scalable, distributed and fault-tolerant Analytics engine using Scala, Akka, Play, Kafka and ElasticSearch.
I would like to take you through the journey of building an analytics engine which was primarily used for text analysis. The inputs were structured, unstructured and semi-structured data and we were doing a lot of data crunching using it. The Analytics engine was accessible by the rest-client and web-client(Built In with engine) as shown in below diagram.
Here is a quick overview on technology stack :
- Play Framework as Rest Server & Web Application (Play is MVC framework based on lightweight, stateless and web friendly architecture.)
- Akka cluster as processing engine.(Akka is a toolkit and runtime for building highly concurrent,distributed, and resilient message driven applications on the JVM.)
- ClusterClient (It was contributed module) for communication with Akka cluster. It used to run on rest…
View original post 987 more words
