Abstract
The reduced costs of embedded systems and sensor technology coupled with the increased speed in communication enables businesses and consumers to deploy a large number of sensing devices. This conjunction of technologies has come to be known as the Internet of Things (IoT). Data collected from IoT devices are continuously increasing, and many approaches have been proposed to deal with the big data that is now generated. Multiple artificial intelligent techniques have been proposed and used to extract knowledge out of these continuously growing datasets. In this paper, we demonstrate that a better understanding of data can be achieved through dynamic modeling. This dynamic behavior is observed in many practical scenarios and needs to be taken into account to have a higher accuracy in prediction and analysis for policy making and business-related decisions. We propose and test a novel methodology to detect the dynamic nature of data over time. Machine learning models have been known to suffer from changes in streaming data over time which is defined as concept drift and therefore by detecting this phenomena such models can be improved.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akouemo HN, Povinelli RJ (2016) Probabilistic anomaly detection in natural gas time series data. Int J Forecast 32(3):948–956
Baena-Garcıa M, del Campo-Ávila J, Fidalgo R, Bifet A, Gavalda R, Morales-Bueno R (2006) Early drift detection method. In: Fourth international workshop on knowledge discovery from data streams, vol 6, pp 77–86
Bifet A, Gavalda R (2007) Learning from time-changing data with adaptive windowing. In: Proceedings of the 2007 SIAM international conference on data mining, SIAM, pp 443–448
Caster O, Dietrich J, Kürzinger ML, Lerch M, Maskell S, Norén GN, Tcherny-Lessenot S, Vroman B, Wisniewski A, van Stekelenborg J (2018) Assessment of the utility of social media for broad-ranging statistical signal detection in pharmacovigilance: results from the web-radr project. Drug safety 41(12):1355–1369
Cejnek M, Bukovsky I (2018) Concept drift robust adaptive novelty detection for data streams. Neurocomputing 309:46–53
Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM computing surveys (CSUR) 41(3):15
Delph LF, Galloway LF, Stanton ML (1996) Sexual dimorphism in flower size. Am Nat 148(2):299–320
Dries A, Rückert U (2009) Adaptive concept drift detection. Stat Anal Data Min ASA Data Sci J 2(5–6):311–327
Elwell R, Polikar R (2011) Incremental learning of concept drift in nonstationary environments. IEEE Trans Neural Netw 22(10):1517–1531
Frías-Blanco I, del Campo-Ávila J, Ramos-Jimenez G, Morales-Bueno R, Ortiz-Díaz A, Caballero-Mota Y (2014) Online and non-parametric drift detection methods based on Hoeffding’s bounds. IEEE Trans Knowl Data Eng 27(3):810–823
Gama J, Medas P, Castillo G, Rodrigues P (2004) Learning with drift detection. In: Brazilian symposium on artificial intelligence, Springer, pp 286–295
Gonçalves PM Jr, de Carvalho Santos SG, Barros RS, Vieira DC (2014) A comparative study on concept drift detectors. Expert Syst Appl 41(18):8144–8156
Gubbi J, Buyya R, Marusic S, Palaniswami M (2013) Internet of things (iot): a vision, architectural elements, and future directions. Future Gener Comput Syst 29(7):1645–1660
Holmberg M, Artursson T (2002) Drift compensation, standards, and calibration methods. Handbook of machine olfaction: electronic nose technology. Wiley, New York, pp 325–346
Lemaire V, Salperwyck C, Bondu A (2014) A survey on supervised classification on data streams. In: European Business Intelligence Summer School, Springer, pp 88–125
Liu A, Lu J, Liu F, Zhang G (2018) Accumulating regional density dissimilarity for concept drift detection in data streams. Pattern Recognit 76:256–272
Lu N, Zhang G, Lu J (2014) Concept drift detection via competence models. Artif Intell 209:11–28
Malhotra P, Vig L, Shroff G, Agarwal P (2015) Long short term memory networks for anomaly detection in time series. In: Proceedings, Presses universitaires de Louvain, p 89
Padakandla S, Bhatnagar S, et al. (2019) Reinforcement learning in non-stationary environments. arXiv preprint arXiv:190503970
Shimodaira H (2000) Improving predictive inference under covariate shift by weighting the log-likelihood function. J Stat Plann Inference 90(2):227–244
Souza VMA, Silva DF, Gama J, Batista GEAPA (2015) Data stream classification guided by clustering on nonstationary environments and extreme verification latency. In: Proceedings of SIAM international conference on data mining (SDM), pp 873–881
Sugiyama M, Yamada M, du Plessis MC (2013) Learning under nonstationarity: covariate shift and class-balance change. Wiley Interdiscip Rev Comput Stat 5(6):465–477
Vergara A, Vembu S, Ayhan T, Ryan MA, Homer ML, Huerta R (2012) Chemical gas sensor drift compensation using classifier ensembles. Sens Actuators B Chem 166:320–329
Widmer G, Kubat M (1996) Learning in the presence of concept drift and hidden contexts. Mach Learn 23(1):69–101
Zliobaite I, Bifet A, Gaber M, Gabrys B, Gama J, Minku L, Musial K (2012) Next challenges for adaptive learning systems. ACM SIGKDD Explor Newsl 14(1):48–55
Zuppa M, Distante C, Siciliano P, Persaud KC (2004) Drift counteraction with multiple self-organising maps for an electronic nose. Sens Actuators B Chem 98(2–3):305–317
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Asghari, M., Sierra-Sosa, D., Telahun, M. et al. Aggregate density-based concept drift identification for dynamic sensor data models. Neural Comput & Applic 33, 3267–3279 (2021). https://doi.org/10.1007/s00521-020-05190-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-020-05190-1