Abstract
The main aim of this work is to develop and implement an automatic anomaly detection algorithm for meteorological time-series. To achieve this goal we develop an approach to constructing an ensemble of anomaly detectors in combination with adaptive threshold selection based on artificially generated anomalies. We demonstrate the efficiency of the proposed method by integrating the corresponding implementation into “Minimax-94” road weather information system.
The research was partially supported by the Russian Foundation for Basic Research grants 16-29-09649 ofi m.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aggarwal, C.C., Sathe, S.: Outlier Ensembles: An Introduction. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54765-7
Artemov, A., Burnaev, E.: Ensembles of detectors for online detection of transient changes. In: Eighth International Conference on Machine Vision (ICMV 2015), 98751Z, 8 December 2015, Proceedings SPIE, vol. 9875 (2015)
Artemov, A., Burnaev, E.: Detecting performance degradation of software-intensive systems in the presence of trends and long-range dependence. In: 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), pp. 29–36 (2016). https://doi.org/10.1109/ICDMW.2016.0013
Artemov, A., Burnaev, E.: Optimal estimation of a signal perturbed by a fractional brownian noise. Theor. Probab. Appl. 60(1), 126–134 (2016)
Artemov, A., Burnaev, E., Lokot, A.: Nonparametric decomposition of quasi-periodic time series for change-point detection. In: Eighth International Conference on Machine Vision (ICMV 2015), 987520, 8 December 2015, Proceedings SPIE, vol. 9875 (2015)
Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. SIGMOD Rec. 29(2), 93–104 (2000). https://doi.org/10.1145/335191.335388. http://doi.acm.org/10.1145/335191.335388
Buchanan, F., Gwartz, S.: Road weather information systems at the ministry of transportation, Ontario. In: 2005 Annual Conference of the Transportation Association of Canada (2005)
Burnaev, E., Erofeev, P., Papanov, A.: Influence of resampling on accuracy of imbalanced classification. In: Eighth International Conference on Machine Vision (ICMV 2015), 987521, 8 December 2015, Proceedings SPIE, vol. 9875 (2015)
Burnaev, E., Erofeev, P., Smolyakov, D.: Model selection for anomaly detection. In: Proceedings SPIE. vol. 9875, pp. 9875–9876 (2015). https://doi.org/10.1117/12.2228794
Burnaev, E., Nazarov, I.: Conformalized kernel ridge regression. In: 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 45–52. December 2016. https://doi.org/10.1109/ICMLA.2016.0017
Burnaev, E., Smolyakov, D.: One-class SVM with privileged information and its application to malware detection. In: 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), pp. 273–280. December 2016. https://doi.org/10.1109/ICDMW.2016.0046
Burnaev, E., Vovk, V.: Efficiency of conformalized ridge regression. In: Balcan, M.F., Feldman, V., Szepesvári, C. (eds.) Proceedings of The 27th Conference on Learning Theory. Proceedings of Machine Learning Research, vol. 35, pp. 605–622. PMLR, Barcelona, Spain, 13–15 June 2014. http://proceedings.mlr.press/v35/burnaev14.html
Burnaev, E.V., Golubev, G.K.: On one problem in multichannel signal detection. Problems of Information Transmission 53(4), 368–380 (2017). https://doi.org/10.1134/S0032946017040056
Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794. KDD 2016, ACM, New York, USA (2016). https://doi.org/10.1145/2939672.2939785
Da Silva, N.F., Hruschka, E.R., Hruschka Jr., E.R.: Tweet sentiment analysis with classifier ensembles. Decis. Support Syst. 66, 170–179 (2014)
Ishimtsev, V., Bernstein, A., Burnaev, E., Nazarov, I.: Conformal k-NN anomaly detector for univariate data streams. In: Gammerman, A., Vovk, V., Luo, Z., Papadopoulos, H. (eds.) Proceedings of the Sixth Workshop on Conformal and Probabilistic Prediction and Applications. Proceedings of Machine Learning Research, vol. 60, pp. 213–227. PMLR, Stockholm, Sweden, 13–16 June 2017. http://proceedings.mlr.press/v60/ishimtsev17a.html
Korotin, A., V’yugin, V., Burnaev, E.: Aggregating strategies for long-term forecasting. In: Gammerman, A., Vovk, V., Luo, Z., Smirnov, E., Peeters, R. (eds.) Proceedings of the Seventh Workshop on Conformal and Probabilistic Prediction and Applications. Proceedings of Machine Learning Research, vol. 91, pp. 63–82. PMLR, 11–13 June 2018. http://proceedings.mlr.press/v91/korotin18a.html
Kuleshov, A., Bernstein, A., Burnaev, E.: Conformal prediction in manifold learning. In: Gammerman, A., Vovk, V., Luo, Z., Smirnov, E., Peeters, R. (eds.) Proceedings of the Seventh Workshop on Conformal and Probabilistic Prediction and Applications. Proceedings of Machine Learning Research, vol. 91, pp. 234–253. PMLR, 11–13 June 2018. http://proceedings.mlr.press/v91/kuleshov18a.html
Liu, F.T., Ting, K.M., Zhou, Z.H.: Isolation forest. In: Eighth IEEE International Conference on Data Mining (ICDM), 2008, pp. 413–422. IEEE (2008)
Papadimitriou, S., Kitagawa, H., Gibbons, P.B., Faloutsos, C.: LOCI: fast outlier detection using the local correlation integral. In: Proceedings of 19th International Conference on Data Engineering, 2003, pp. 315–326. IEEE (2003)
Pinet, M., Lo, A.: Development of a road weather information system (RWIS) network for Alberta’s national highway system. In: Intelligent Transportation Systems (2003)
Rivera, R., Nazarov, I., Burnaev, E.: Towards forecast techniques for business analysts of large commercial data sets using matrix factorization methods. J. Phys.: Conf. Series 1117(1), 012010 (2018). http://stacks.iop.org/1742-6596/1117/i=1/a=012010
Rousseeuw, P.J., Driessen, K.V.: A fast algorithm for the minimum covariance determinant estimator. Technometrics 41(3), 212–223 (1999)
Safin, A., Burnaev, E.: Conformal kernel expected similarity for anomaly detection in time-series data. Adv. Syst. Sci. Appl. 17(3), 22–33 (2017). https://doi.org/10.25728/assa.2017.17.3.497
Salehi, M., Zhang, X., Bezdek, J.C., Leckie, C.: Smart sampling: a novel unsupervised boosting approach for outlier detection. In: Kang, B.H., Bai, Q. (eds.) AI 2016. LNCS (LNAI), vol. 9992, pp. 469–481. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-50127-7_40
Schölkopf, B., Williamson, R.C., Smola, A.J., Shawe-Taylor, J., Platt, J.C.: Support vector method for novelty detection. In: Advances in Neural Information Processing Systems, pp. 582–588 (2000)
Seni, G., Elder, J.F.: Ensemble methods in data mining: improving accuracy through combining predictions. Synth. Lect. Data Min. Knowl. Disc. 2(1), 1–126 (2010)
Smolyakov, D., Korotin, A., Erofeev, P., Papanov, A., Burnaev, E.: Meta-learning for resampling recommendation systems. In: Eleventh International Conference on Machine Vision (ICMV 2018); 110411S (2019). Proceedings SPIE, vol. 11041 (2019)
Smolyakov, D., Sviridenko, N., Burikov, E., Burnaev, E.: Anomaly pattern recognition with privileged information for sensor fault detection. In: Pancioni, L., Schwenker, F., Trentin, E. (eds.) ANNPR 2018. LNCS, vol. 11081, pp. 320–332. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99978-4_25
Toivonen, K., Kantonen, J.: Road weather information system in finland. Transp. Res. Rec.: J. Transp. Res. Board 1741, 21–25 (2001)
Volkhonskiy, D., Burnaev, E., Nouretdinov, I., Gammerman, A., Vovk, V.: Inductive conformal martingales for change-point detection. In: Gammerman, A., Vovk, V., Luo, Z., Papadopoulos, H. (eds.) Proceedings of the Sixth Workshop on Conformal and Probabilistic Prediction and Applications. Proceedings of Machine Learning Research, vol. 60, pp. 132–153. PMLR, Stockholm, Sweden, 13–16 June 2017. http://proceedings.mlr.press/v60/volkhonskiy17a.html
Zimek, A., Gaudet, M., Campello, R.J., Sander, J.: Subsampling for efficient and effective unsupervised outlier detection ensembles. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 428–436. ACM (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Smolyakov, D., Sviridenko, N., Ishimtsev, V., Burikov, E., Burnaev, E. (2019). Learning Ensembles of Anomaly Detectors on Synthetic Data. In: Lu, H., Tang, H., Wang, Z. (eds) Advances in Neural Networks – ISNN 2019. ISNN 2019. Lecture Notes in Computer Science(), vol 11555. Springer, Cham. https://doi.org/10.1007/978-3-030-22808-8_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-22808-8_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22807-1
Online ISBN: 978-3-030-22808-8
eBook Packages: Computer ScienceComputer Science (R0)