Abstract
Predictive maintenance solutions have been recently applied in industries for various problems, such as handling the machine status and maintaining the transmission lines. Industrial digital transformation promotes the collection of operational and conditional data generated from different parts of equipment (or power plant) for automatically detecting failures and seeking solutions. Predictive maintenance aims at e.g., minimizing downtime and increasing the whole productivity of manufacturing processes. In this context machine learning techniques have emerged as promising approaches, however it is challenging to select proper methods when data contain imbalanced class labels.
In this paper, we propose a pipeline for constructing machine learning models based on Bayesian optimization approach for imbalanced datasets, in order to improve the classification performance of this model in manufacturing and transmission line applications. In this pipeline, the Bayesian optimization solution is used to suggest the best combination of hyperparameters for model variables. We analyze four multi-output models, such as Adaptive Boosting, Gradient Boosting, Random Forest and MultiLayer Perceptron, to design and develop multi-class and binary imbalanced classifiers.
We have trained each model on two different imbalanced datasets, i.e., AI4I 2020 and electrical power system transmission lines, aiming at constructing a versatile pipeline able to deal with two tasks: failure type and machine (or electrical) status. In the AI4I 2020 case, Random Forest model has performed better than other models for both tasks. In the electrical power system transmission lines case, the MultiLayer Perceptron model has performed better than the others for the failure type task.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amruthnath, N., Gupta, T.: A research study on unsupervised machine learning algorithms for early fault detection in predictive maintenance. In: 2018 5th International Conference on Industrial Engineering and Applications (ICIEA), pp. 355–361 (2018)
Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13, 281–305 (2012)
Borisov, V., Leemann, T., Seßler, K., Haug, J., Pawelczyk, M., Kasneci, G.: Deep neural networks and tabular data: a survey. arXiv preprint arXiv:2110.01889 (2021)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Brochu, E., Cora, V.M., De Freitas, N.: A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599 (2010)
Calabrese, M., et al.: SOPHIA: an event-based IoT and machine learning architecture for predictive maintenance in Industry 4.0. Information 11(4), 202 (2020). https://www.mdpi.com/2078-2489/11/4/202
Cao, H., Nguyen, M., Phua, C., Krishnaswamy, S., Li, X.: An integrated framework for human activity recognition. In: 2012 ACM Conference on Ubiquitous Computing, pp. 621–622 (2012)
Carvalho, T.P., Soares, F.A.A.M.N., Vita, R., P. Francisco, R., Basto, J.P., Alcalá, S.G.S.: A systematic literature review of machine learning methods applied to predictive maintenance. Comput. Ind. Eng. 137, 106024 (2019)
Dua, D., Graff, C.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml
Ghate, V.N., Dudul, S.V.: Optimal MLP neural network classifier for fault detection of three phase induction motor. Expert Syst. Appl. 37(4), 3468–3481 (2010)
Hastie, T., Rosset, S., Zhu, J., Zou, H.: Multi-class AdaBoost. Stat. Interface 2(3), 349–360 (2009)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. SSS, Springer, New York (2009). https://doi.org/10.1007/978-0-387-84858-7
He, H., Ma, Y.: Imbalanced Learning: Foundations, Algorithms, and Applications. Wiley-IEEE Press, Hoboken, 216 pages (2013)
Heaton, J.: Introduction to Neural Networks with Java. Heaton Research, Inc. Chesterfield (2008)
Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-25566-3_40
Jamil, M., Sharma, S.K., Singh, R.: Fault detection and classification in electrical power transmission system using artificial neural network. SpringerPlus 4(1), 1–13 (2015). https://doi.org/10.1186/s40064-015-1080-x
Martin-Diaz, I., Morinigo-Sotelo, D., Duque-Perez, O., de J. Romero-Troncoso, R.: Early fault detection in induction motors using AdaBoost with imbalanced small data and optimized sampling. IEEE Trans. Ind. Appl. 53(3), 3066–3075 (2017)
Mosley, L.: A balanced approach to the multi-class imbalance problem. Ph.D. thesis, Iowa State University (2013)
Nogueira, F.: Bayesian Optimization: open source constrained global optimization tool for Python (2014). https://github.com/fmfn/BayesianOptimization
Orrù, P.F., Zoccheddu, A., Sassu, L., Mattia, C., Cozza, R., Arena, S.: Machine learning approach using MLP and SVM algorithms for the fault prediction of a centrifugal pump in the oil and gas industry. Sustainability 12, 4776 (2020)
Ouadah, A., Leila, Z.-G., Salhi, N.: Selecting an appropriate supervised machine learning algorithm for predictive maintenance. Int. J. Adv. Manufact. Tech. 119, 4277–4301 (2022)
Paolanti, M., Romeo, L., Felicetti, A., Mancini, A., Frontoni, E., Loncarski, J.: Machine learning approach for predictive maintenance in Industry 4.0. In: 2018 14th IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications (MESA), pp. 1–6. IEEE (2018)
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Qin, S., Wang, K., Ma, X., Wang, W., Li, M.: Chapter 9: step standard in design and manufacturing ensemble learning-based wind turbine fault prediction method with adaptive feature selection. Communications in Computer and Information Science Data Science, pp. 572–582 (2017)
Ronzoni, N.: Predictive maintenance experiences on imbalance data with Bayesian optimization. https://gitlab.com/system_anomaly_detection/predictivemeintenance
Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. Adv. Neural Inform. Proc. Syst. 25, 2951–2959 (2012)
Srinivas, N., Krause, A., Kakade, S.M., Seeger, M.: Gaussian process optimization in the bandit setting: no regret and experimental design. arXiv preprint arXiv:0912.3995 (2009)
Susto, G., Schirru, A., Pampuri, S., Mcloone, S., Beghi, A.: Machine learning for predictive maintenance: a multiple classifier approach. IEEE Trans. Ind. Inf. 11, 812–820 (2015)
Urbanowicz, R.J., Moore, J.H.: ExSTraCS 2.0: description and evaluation of a scalable learning classifier system. Evol. Intell. 8(2), 89–116 (2015)
Vasilić, P., Vujnović, S., Popović, N., Marjanović, A., Z̆eljko Durović: Adaboost algorithm in the frame of predictive maintenance tasks. In: 23rd International Scientific-Professional Conference on Information Technology (IT), IEEE, pp. 1–4 (2018)
Acknowledgements
We would like to thank BitBang S.r.l that funded this research; in particular Matteo Casadei, Luca Guerra and the colleagues of Data Science team.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ronzoni, N., De Marco, A., Ronchieri, E. (2022). Predictive Maintenance Experiences on Imbalanced Data with Bayesian Optimization Approach. In: Gervasi, O., Murgante, B., Misra, S., Rocha, A.M.A.C., Garau, C. (eds) Computational Science and Its Applications – ICCSA 2022 Workshops. ICCSA 2022. Lecture Notes in Computer Science, vol 13377. Springer, Cham. https://doi.org/10.1007/978-3-031-10536-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-10536-4_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10535-7
Online ISBN: 978-3-031-10536-4
eBook Packages: Computer ScienceComputer Science (R0)