Skip to main content
SpringerLink
Log in

A new hyper-parameter optimization method for machine learning in fault classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Accurate bearing fault classification is essential for the safe and stable operation of rotating machinery. The success of Machine Learning (ML) in fault classification is mainly dependent on efficient features and the optimal pre-defined hyper-parameters. Various hyper-parameter optimization (HPO) methods have been proposed to tune the ML algorithms’ hyper-parameters in low dimensions but ignore the hyper-parameters of Feature Engineering (FE). The hyper-parameter dimension is high because both FE and the ML algorithm contain many hyper-parameters. This paper proposed a new HPO method for high dimensions based on dimension reduction and partial dependencies. Firstly, the whole hyper-parameter space is separated into two subspaces of FE and the ML algorithm to reduce time consumption. Secondly, the sensitive intervals of hyperparameters can be recognized by partial dependencies due to the nonlinearity of the relationship between the hyperparameters. Then HPO is conducted in intervals to acquire more satisfactory accuracy. The proposed method is verified on three OpenML datasets and the CWRU bearing dataset. The results show that it can automatically construct efficient domain features and outperforms traditional HPO methods and famous ML algorithms. The proposed method is also very time efficient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1:
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Toma RN, Prosvirin AE, Kim JM (2020) Bearing fault diagnosis of induction motors using a genetic algorithm and machine learning classifiers. Sensors 20:1884

    Article  Google Scholar 

  2. Wen L, Li X, Gao L (2021) A new reinforcement learning based learning rate scheduler for convolutional neural network in fault classification. IEEE Trans Ind Electron 68(12):12890–12900. https://doi.org/10.1109/TIE.2020.3044808

  3. Xue Y, Zhang L, Wang B, Zhang Z, Li F (2018) Nonlinear feature selection using Gaussian kernel SVM-RFE for fault diagnosis. Appl Intell 48:3306–3331

    Article  Google Scholar 

  4. Guo J, Yang L, Bie R, Yu J, Gao Y, Shen Y, Kos A (2019) An XGBoost-based physical fitness evaluation model using advanced feature selection and Bayesian hyper-parameter optimization for wearable running monitoring. Comput Netw 151:166–180

    Article  Google Scholar 

  5. Klein A, Falkner S, Bartels S (2017) Fast bayesian optimization of machine learning hyper-parameters on large datasets. Artificial Intell Stat PMLR:528–536

  6. Wang Z, Huang H, Wang Y (2020) Fault diagnosis of planetary gearbox using multi-criteria feature selection and heterogeneous ensemble learning classification. Measurement 173:108654

    Article  Google Scholar 

  7. Zhang X, Zhang Q, Chen M, Sun Y, Qin X, Li H (2018) A two-stage feature selection and intelligent fault diagnosis method for rotating machinery using hybrid filter and wrapper method. Neurocomputing 275:2426–2439

    Article  Google Scholar 

  8. Suo M, Zhu B, An R (2019) Data-driven fault diagnosis of satellite power system using fuzzy Bayes risk and SVM. Aerosp Sci Technol 84:1092–1105

    Article  Google Scholar 

  9. Chen YQ, Fink O, Sansavini G (2017) Combined fault location and classification for power transmission lines fault diagnosis with integrated feature extraction. IEEE Trans Ind Electron 65:561–569

    Article  Google Scholar 

  10. Ali MZ, Shabbir MNSK, Zaman SMK (2020) Single-and multi-fault diagnosis using machine learning for variable frequency drive-fed induction motors. IEEE Trans Ind Appl 56:2324–2337

    Article  Google Scholar 

  11. Chen H, Jiang B, Lu N et al (2018) Deep PCA based real-time incipient fault detection and diagnosis methodology for electrical drive in high-speed trains. IEEE Trans Veh Technol 6:4819–4830

    Article  Google Scholar 

  12. Wen L, Gao L, Li X et al (2021) Convolutional neural network with automatic learning rate scheduler for fault diagnosis. IEEE Trans Instrum Meas 70:1–12. https://doi.org/10.1109/TIM.2020.3048792

    Article  Google Scholar 

  13. Du W, Tao J, Li Y et al (2014) Wavelet leaders multifractal features based fault diagnosis of rotating mechanism. Mech Syst Signal Process 43:57–75

    Article  Google Scholar 

  14. Hu K, Jiang M, Zhang H (2021) Design of fault diagnosis algorithm for electric fan based on LSSVM and Kd-tree. Appl Intell 51:804–818. https://doi.org/10.1007/s10489-020-01830-0

    Article  Google Scholar 

  15. Sun M, Wang H, Liu P, Huang S, Fan P (2019) A sparse stacked denoising autoencoder with optimized transfer learning applied to the fault classification of rolling bearings. Measurement 146:305–314

    Article  Google Scholar 

  16. Xu G, Liu M, Jiang Z, Söffker D, Shen W (2019) Bearing fault diagnosis method based on deep convolutional neural network and random forest ensemble learning. Sensors 19:1088

    Article  Google Scholar 

  17. Deng H, Diao Y, Wu W (2020) A high-speed D-CART online fault diagnosis algorithm for rotor systems. Appl Intell 50:29–41. https://doi.org/10.1007/s10489-019-01516-2

    Article  Google Scholar 

  18. Wen L, Li X, Gao L (2020) A new two-level hierarchical diagnosis network based on convolutional neural network. IEEE Trans Instrum Meas 69:330–338

  19. Huang H, Jia R, Shi X, Liang J, Dang J (2021) Feature selection and hyper parameters optimization for short-term wind power forecast. App Intell: 1-19 51:6752–6770. https://doi.org/10.1007/s10489-021-02191-y

    Article  Google Scholar 

  20. Yangyang L et al (2019) Hyper-parameter optimization using mars surrogate for machine-learning algorithms. IEEE Transac Emerging Top Computa Intell 4:287–297

    Google Scholar 

  21. Qin AS, Mao HL, Hu Q (2021) Cross-domain fault diagnosis of rolling bearing using similar features-based transfer approach. Measurement 172:108900

    Article  Google Scholar 

  22. Yang L, Shami A (2020) On hyper-parameter optimization of machine learning algorithms: theory and practice. arXiv preprint arXiv:2007.15745

  23. Saufi SR, bin Ahmad ZA, Leong MS et al (2018) Differential evolution optimization for resilient stacked sparse autoencoder and its applications on bearing fault diagnosis. Meas Sci Technol 29:125002

    Article  Google Scholar 

  24. Hutter F, Kotthoff L, Vanschoren J (2019) Automated machine learning: methods, systems, challenges, Springer Nature

  25. Yao Q, Wang M, Chen Y et al (2018) Taking human out of learning applications: a survey on automated machine learning. arXiv preprint arXiv:1810.13306

  26. Gao L, Ding Y (2020) Disease prediction via Bayesian hyper-parameter optimization and ensemble learning. BMC Research Notes 13:1–6

    Article  Google Scholar 

  27. Balandat M, Karrer B, Jiang D et al (2021) BoTorch: a framework for efficient Monte-Carlo Bayesian optimization. Adv Neural Inf Proces Syst 6:1–34

    Google Scholar 

  28. Nayebi A, Munteanu A, Poloczek M (2019) A framework for Bayesian optimization in embedded subspaces. Intern Conf Mach Learn 5:4752–4761

    Google Scholar 

  29. Falkner S, Klein A, Hutter F (2018) BOHB: robust and efficient hyper-parameter optimization at scale. Intern Conf Mach Learn 12:1437–1446

    Google Scholar 

  30. Zhang W, Wu C, Zhong H, Li Y, Wang L (2021) Prediction of undrained shear strength using extreme gradient boosting and random forest based on Bayesian optimization. Geosci Front 12:469–477

    Article  Google Scholar 

  31. Feurer M, Klein A, Eggensperger K et al (2018) Auto-sklearn: efficient and robust automated machine learning. Automate Mach Learn 36:113–134

    Google Scholar 

  32. Li S, Xing W, Kirby R et al (2020) Multi-fidelity Bayesian optimization via deep neural networks. Adv Neural Inf Proces Syst 33:145–156

    Google Scholar 

  33. Li C, De Oliveira JV, Cerrada M et al (2018) A systematic review of fuzzy formalisms for bearing fault diagnosis. IEEE Trans Fuzzy Syst 27:1362–1382

    Article  Google Scholar 

  34. Zeng C, Yang L, Zhu AX (2017) Construction of membership functions for soil mapping using the partial dependence of soil on environmental covariates calculated by random forest. Soil Sci Soc Am J 81:341–353

    Article  Google Scholar 

  35. Bernard C, Denuit M, Vanduffel S et al (2018) Measuring portfolio risk under partial dependence information. J Risk Insur 85:843–863

    Article  Google Scholar 

  36. Zhou J, Li E, Yang S, Wang M, Shi X, Yao S, Mitri HS (2019) Slope stability prediction for circular mode failure using gradient boosting machine approach based on an updated database of case histories. Saf Sci 118:505–518

    Article  Google Scholar 

  37. Allen T, Murray KA, Zambrana-Torrelio C et al (2017) Global hotspots and correlates of emerging zoonotic diseases. Nat Commun 8:1–10

    Article  Google Scholar 

  38. Vanschoren J, Van Rijn JN, Bischl B et al (2013) OpenML: networked science in machine learning. SIGKDD Explo 15:49–60

    Article  Google Scholar 

  39. Case Western Reserve University Bearing Data Center Website Available at: http://csegroups.case.edu/bearingdatacenter/home

  40. Wu X, Jiang B, Zhong Y, et al (2020) Tolerant Markov boundary discovery for feature selection, proceedings of the 29th ACM international conference on Information & Knowledge Management: 2261-2264

  41. Zhang R, Zhang Z (2020) Feature selection with symmetrical complementary coefficient for quantifying feature interactions. Appl Intell 50:101–118

    Article  Google Scholar 

  42. Zhang L, Lu X (2018) New fast feature selection methods based on multiple support vector data description. Appl Intell 48:1776–1790

    Article  Google Scholar 

  43. Schöne M, Kohlhase M (2020) Least squares approach for multivariate Split selection in regression trees, international conference on intelligent data engineering and automated learning: 41-50

  44. Kumar S, Agrawal K, Mandan N (2020) Red wine quality prediction using machine learning techniques, international conference on computer communication and informatics: 1-6

  45. Chen T, Wang Z, Yang X, Jiang K (2019) A deep capsule neural network with stochastic delta rule for bearing fault diagnosis on raw vibration signals. Measurement 148:106857

    Article  Google Scholar 

  46. Zhang M, Wang D, Lu W, Yang J, Li Z, Liang B (2019) A deep transfer model with wasserstein distance guided multi-adversarial networks for bearing fault diagnosis under different working conditions. IEEE Access 7:65303–65318

    Article  Google Scholar 

  47. Zhu Z, Peng G, Chen Y, Gao H (2019) A convolutional neural network based on a capsule network with strong generalization for bearing fault diagnosis. Neurocomputing 323:62–75

    Article  Google Scholar 

  48. Zhang W, Li C, Peng G, Chen Y, Zhang Z (2018) A deep convolutional neural network with new training algorithms for bearing fault diagnosis under noisy environment and different working load. Mech Syst Signal Process 100:439–453

    Article  Google Scholar 

  49. Liu H, Yao D, Yang J, Li X (2019) Lightweight convolutional neural network and its application in rolling bearing fault diagnosis under variable working conditions. Sensors 19:4827

    Article  Google Scholar 

  50. Wu Y, Zhao R, Jin W, He T, Ma S, Shi M (2021) Intelligent fault diagnosis of rolling bearings using a semi-supervised convolutional neural network. Appl Intell 51:2144–2160

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Natural Science Foundation of China (NSFC) under Grants No.51805192, the National Key R&D Program of China under Grant number 2019YFB1704600 and sponsored by the State Key Laboratory of Digital Manufacturing Equipment and Technology (DMET) of Huazhong University of Science and Technology (HUST) under Grant No. DMETKF2020029.

Author information

Authors and Affiliations

  1. The State Key Laboratory of Digital Manufacturing Equipment & Technology, Huazhong University of Science and Technology, 1037# Luoyu Road, Wuhan, 430074, China

    Xingchen Ye, Liang Gao & Xinyu Li

  2. School of Mechanical Engineering and Electronic Information, China University of Geosciences, No. 388 Lumo Road, Wuhan, 430074, China

    Long Wen

Authors
  1. Xingchen Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Long Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Long Wen.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ye, X., Gao, L., Li, X. et al. A new hyper-parameter optimization method for machine learning in fault classification. Appl Intell 53, 14182–14200 (2023). https://doi.org/10.1007/s10489-022-04238-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04238-0

Keywords

Search

Navigation