Scalable big earth observation data mining algorithms: a review

Abstract

Enormous amount of earth information, gathered from satellite sensors, simulations, and other resources, are collectively referred to as Big Earth Observation Data (BEOD). The data contains remarkable insights and spatio-temporal stamps of pertinent Earth phenomena for enhancing our knowledge, responding, and addressing demanding situations of earth sciences and observations. However, However, traditional data mining algorithms are generally time-inefficient, making it difficult to process and analyze BEOD. To address this challenge, we explore two ways to enhance scalability: 1) improving the algorithm with specific parameters or data modifications when run on a single machine, and 2) making the algorithm parallel through distributed execution on multiple machines, such as with cluster-based implementations. We also suggest improvements for existing techniques and widely used algorithms for processing BEOD. In this review, we conduct a systematic review of data mining techniques for classification, clustering, prediction, regression, association rules, pattern mining, and anomaly detection to determine their scalability and performance on various Earth observation use cases. We explored advanced mining techniques, including statistical, machine learning, and deep learning approaches, for handling BEOD at scale. We also identified potential challenges, open research issues, and provide future directions for the development of scalable algorithms for mining BEOD. We observed that applying data mining techniques on BEOD introduces serious concerns since the data has both spatial and temporal components. Statistical and machine learning models such as ARIMA, SARIMA, Naive Bayes, Bayesian networks, KNN, and K-means, as well as SVM, are not suitable for working with the volume and heterogeneity present in the data, but it can be improved by employing big data environments. Nowadays, deep learning-based techniques are popularly used for working with large amounts of data, but it requires specialized systems and upgrades as the data volume increases. This issue can be addressed through a big data platform. A unified deep learning architecture can be employed to handle both the spatial and temporal components of BEOD, and performance can be improved by deploying the architecture on a big data environment. Therefore, this study also reveals that, although deep learning architectures are efficient and trending, traditional statistical methods and machine learning can achieve competitive or sometimes improved performance with the involvement of big data technologies and/or internal data representation.