Skip to main content
SpringerLink
Log in

A deep reinforcement learning approach for maintenance planning of multi-component systems with complex structure

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In recent years, the trend toward greater integration and complexity of mechanical systems has brought challenges to the formulation of preventive maintenance plans. It is very difficult to realize the traditional condition-based maintenance method that relies on calculating the optimal maintenance threshold to achieve optimal maintenance. However, in solving highly complex and challenging control and decision-making problems, the deep reinforcement learning (DRL) method shows its powerful ability and provides a new idea for the maintenance planning of complex systems. Numerical results show that DRL-based maintenance model can obtain optimization strategies through continuous exploration and realize the trade-off between component maintenance cost and the loss caused by system failure, whether in simple or complex multi-component systems. The policy minimizes the overall cost of the system by choosing actions that minimize the total long-term cost. The comparison with other maintenance strategies shows that the proposed model is superior to various baseline policies and reduces the system lifecycle cost.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

We declare that the data generated and analyzed in this study can be obtained from the article.

References

  1. Gong Y, Zha M, Lv Z (2022) Fractional-order optimal control model for the equipment management optimization problem with preventive maintenance. Neural Comput Appl 34:4693–4714

    Article  Google Scholar 

  2. Goyal D, Pabla BS, Dhami SS et al (2017) Optimization of condition-based maintenance using soft computing. Neural Comput Appl 28(1):829–844

    Article  Google Scholar 

  3. Zheng R, Chen B, Gu L (2020) Condition-based maintenance with dynamic thresholds for a system using the proportional hazards model. Reliab Eng Syst Saf 204:107123

    Article  Google Scholar 

  4. Chen N, Ye ZS, Xiang Y et al (2015) Condition-based maintenance using the inverse Gaussian degradation model. Eur J Oper Res 243(1):190–199

    Article  MathSciNet  MATH  Google Scholar 

  5. Ahmad R, Kamaruddin S (2012) An overview of time-based and condition-based maintenance in industrial application. Comput Ind Eng 63(1):135–149

    Article  Google Scholar 

  6. Nan C, Yong C, Li Z et al (2011) Optimal variability sensitive condition-based maintenance over time-based maintenance with a Cox PH model. Int J Prod Res 49(6–8):2083–2100

    Google Scholar 

  7. Hong PEI, Chang-Hua HU, Xiao-Sheng SI, Zheng-Xin Z, Dang-Bo DU (2018) Remaining life prediction information-based maintenance decision model for equipment under imperfect maintenance. Acta Autom Sin 44(4):719–729

    Google Scholar 

  8. Grall A, Bérenguer C, Dieulle L (2002) A condition-based maintenance policy for stochastically deteriorating systems. Reliab Eng Syst Saf 76(2):167–180

    Article  Google Scholar 

  9. Du DB, Pei H, Zhang JX, et al (2020) A new condition-based maintenance decision model for degraded equipment subjected to random shocks. In: 2020 Chinese Control Decision Conference (CCDC)

  10. Jonge BD, Teunter R, Tinga T (2017) The influence of practical factors on the benefits of condition-based maintenance over time-based maintenance. Reliab Eng Syst Saf 158:21–30

    Article  Google Scholar 

  11. Liu B, Wu S, Xie M et al (2017) A condition-based maintenance policy for degrading systems with age- and state-dependent operating cost. Eur J Oper Res 263(3):879–887

    Article  MathSciNet  MATH  Google Scholar 

  12. Shahraki AF, Yadav OP, Vogiatzis C (2020) Selective maintenance optimization for multi-state systems considering stochastically dependent components and stochastic imperfect maintenance actions. Reliab Eng Syst Saf 196:106738

    Article  Google Scholar 

  13. Tian Z, Liao H (2011) Condition based maintenance optimization for multi-component systems using proportional hazards model. Reliab Eng Syst Saf 96(5):581–589

    Article  Google Scholar 

  14. Xu J, Liang Z, Li YF et al (2021) Generalized condition-based maintenance optimization for multi-component systems considering stochastic dependency and imperfect maintenance. Reliab Eng Syst Saf 211:107592

    Article  Google Scholar 

  15. Hai CV, Phuc D, Anne B (2016) A stationary grouping maintenance strategy using mean residual life and the birnbaum importance measure for complex structures. IEEE Trans Reliab 65(1):217–234

    Article  Google Scholar 

  16. Rasmekomen N, Parlikad AK (2016) Condition-based maintenance of multi-component systems with degradation state-rate interactions. Reliab Eng Syst Saf 148:1–10

    Article  Google Scholar 

  17. Fan D, Zhang A, Feng Q et al (2021) Group maintenance optimization of subsea Xmas trees with stochastic dependency. Reliab Eng Syst Saf 209(7):107450

    Article  Google Scholar 

  18. Mengkai Xu, Jin X, Kamarthi S et al (2018) A failure-dependency modeling and state discretization approach for condition-based maintenance optimization of multi-component systems. J Manuf Syst 47:141–152

    Article  Google Scholar 

  19. Broek MAJUH, Teunter RH, Jonge BD et al (2021) Joint condition-based maintenance and load-sharing optimization for two-unit systems with economic dependency. Eur J Oper Res

  20. Mokhtar EHA, Chateauneuf A, Laggoune R (2018) Condition based opportunistic preventive maintenance policy for utility systems with both economic and structural dependencies. Appl Gas Supply Netw 165:214–223

    Google Scholar 

  21. Guo QC, Bing HZ, Ling L (2017) Joint optimization of lot sizing and condition-based maintenance for multi-component production systems. Comput Ind Eng 110:538–549

    Article  Google Scholar 

  22. Zhang N, Si W (2020) Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks. Reliab Eng Syst Saf 203:107094

    Article  Google Scholar 

  23. Andrzejczak K (2015) Stochastic modelling of the repairable system. J Konbin 35(1):5–14

    Article  MathSciNet  Google Scholar 

  24. Andriotisa CP, Papakonstantinou KG (2021) Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints. Reliab Eng Syst Saf 212:107551

    Article  Google Scholar 

  25. Hsieh MH, Jeng SL (2007) Accelerated discrete degradation models for leakage current of ultra-thin gate oxides. IEEE Trans Reliab 56(3):369–380

    Article  Google Scholar 

  26. Huang J, Chang Q, Arinez J (2020) Deep reinforcement learning based preventive maintenance policy for serial production lines. Expert Syst Appl 160:113701

    Article  Google Scholar 

  27. Han X, Liu H, Sun F et al (2019) Active object detection with multi-step action prediction using deep Q-network. IEEE Trans Industr Inf 15(6):3723–3731

    Article  Google Scholar 

  28. Hasselt HV, Guez A, Silver D (2015) Deep reinforcement learning with double Q-learning. In: Thirtieth AAAI conference on artificial intelligence

Download references

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi Province, China

    Jiahao Chen & Yu Wang

Authors
  1. Jiahao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu Wang.

Ethics declarations

Conflict of interest

We declare that we have no conflicts of interest associated with this publication. There was no funding for conducting this study. This manuscript is submitted only in this journal, and there is no case of multiple submissions of one manuscript. We confirm that we have taken full consideration of the intellectual property protection related to this work. There are no impediments to publication.

Ethical approval

Our research belongs to the improvement and application innovation of intelligent algorithms, so it does not involve ethical issues.

Informed consent

We confirm that the article has been read and approved to submit by all authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, J., Wang, Y. A deep reinforcement learning approach for maintenance planning of multi-component systems with complex structure. Neural Comput & Applic 35, 15549–15562 (2023). https://doi.org/10.1007/s00521-023-08542-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08542-9

Keywords

Search

Navigation