Real-time data analytics in distributed systems

Abstract

Real-time data analytics involves the processing and analysis of data as it arrives, delivering immediate insights that are crucial for time-sensitive applications. This research explores the platforms and techniques necessary for supporting real-time analytics, extending beyond traditional Event Processing Systems (EPS) to include broader big data contexts that integrate both 'data at rest' and 'data in motion' solutions. A detailed case study is presented, showcasing the application of the Event Swarm complex event processing engine in addressing financial analytics challenges. The study identifies key research challenges in the field, emphasizing the need for innovative approaches and algorithms to enhance real-time data filtering, exploration, statistical analysis, and machine learning.