Skip to main content
Log in

An Optimal Disassembly Sequence Planning for Complex Products using Enhanced Deep Reinforcement Learning Framework

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Disassembly Sequence Planning (DSP) is a crucial problem in the field of repair and maintenance. There is a pressing need for an efficient technique to solve the Complete Disassembly Sequence Planning (CDSP) problem for large, highly complex products without compromising time and computational resources. Since exact methods fail to handle complex products, and meta-heuristic approaches often do not produce optimal results, its solution requires a deep reinforcement learning approach. This work proposes a novel Enhanced Deep Reinforcement Learning (EDRL) approach in which the Actor-Critic Networks with an attention mechanism is employed in both networks to assign weightage to important actions, generating optimal sequences based on these actions. To further improve the model and accurately predict the loss value, a hybrid loss function is developed by combining the Categorical Cross-entropy and Log-Cosh loss functions. This work considers DSP attributes like stability, liaison, geometric feasibility, and precedence for experiments on various products. The proposed EDRL-CDSP method outperforms existing techniques in terms of optimality while requiring less time to generate the solution. The optimized disassembly sequence, when followed, consumes less time for the total disassembly of all the parts of the product. These findings suggest that the proposed EDRL-CDSP approach can effectively address the challenges in DSP and provide practical benefits for industrial applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

The data supporting the findings of this study are not publicly available due to proprietary and privacy concerns. However, upon valid request, the data can be made available for verification and further research. Requests for access to the data should be directed to the first author and will be subject to any necessary approvals, conditions, or restrictions.

References

  1. Home-E-Waste Monitor. https://ewastemonitor.info/. Last accessed 2023/11/30.

  2. Sataloff RT, Johns MM, Kost KM, Lambert AJD, Gupta SM. Disassembly modeling for assembly, maintenance, reuse and recycling (resource management). London: CRC Press; 2004.

    Google Scholar 

  3. Sanchez B, Haas C. A novel selective disassembly sequence planning method for adaptive reuse of buildings. J Clean Prod. 2018;183:998–1010. https://doi.org/10.1016/J.JCLEPRO.2018.02.201.

    Article  Google Scholar 

  4. Zhang XF, Zhang SY. Product cooperative disassembly sequence planning based on branch-and-bound algorithm. Int J Adv Manuf Technol. 2010;51:1139–47. https://doi.org/10.1007/s00170-010-2682-7.

    Article  Google Scholar 

  5. Tian G, Zhou M, Chu J. A chance constrained programming approach to determine the optimal disassembly sequence. IEEE Trans Autom Sci Eng. 2013;10:1004–13. https://doi.org/10.1109/TASE.2013.2249663.

    Article  Google Scholar 

  6. Smith S, Smith G, Chen WH. Disassembly sequence structure graphs: an optimal approach for multiple-target selective disassembly sequence planning. Adv Eng Inform. 2012;26:306–16. https://doi.org/10.1016/j.aei.2011.11.003.

    Article  Google Scholar 

  7. Shan H, Li S, Huang J, Gao Z, Li W. Ant colony optimization algorithm-based disassembly sequence planning. In: Proceedings of the 2007 IEEE international conference on mechatronics and automation, ICMA 2007; 2007. p. 867–72. https://doi.org/10.1109/ICMA.2007.4303659.

  8. Hui W, Dong X, Guanghong D. A genetic algorithm for product disassembly sequence planning. Neurocomputing. 2008;71:2720–6. https://doi.org/10.1016/j.neucom.2007.11.042.

    Article  Google Scholar 

  9. Tian Y, Zhang X, Liu Z, Jiang X, Xue J. Product cooperative disassembly sequence and task planning based on genetic algorithm. Int J Adv Manuf Technol. 2019;105:2103–20. https://doi.org/10.1007/s00170-019-04241-9.

    Article  Google Scholar 

  10. Parsa S, Saadat M. Intelligent selective disassembly planning based on disassemblability characteristics of product components. Int J Adv Manuf Technol. 2019;104:1769–83. https://doi.org/10.1007/s00170-019-03857-1.

    Article  Google Scholar 

  11. Alshibli M, El Sayed A, Kongar E, Sobh TM, Gupta SM. Disassembly sequencing using Tabu search. J Intell Robot Syst Theory Appl. 2016;82:69–79. https://doi.org/10.1007/s10846-015-0289-9.

    Article  Google Scholar 

  12. Azab A, Ziout A, ElMaraghy W. Modeling and optimization for disassembly planning. Jordan J Mech Indust Eng. 2011;5:1–8.

    Google Scholar 

  13. Zhou X, Huang K, Xi L, Lee J. Preventive maintenance modeling for multi-component systems with considering stochastic failures and disassembly sequence. Reliab Eng Syst Saf. 2015;142:231–7. https://doi.org/10.1016/j.ress.2015.05.005.

    Article  Google Scholar 

  14. Allagui A, Belhadj I, Plateaux R, Hammadi M, Penas O, Aifaoui N. Reinforcement learning for disassembly sequence planning optimization. Comput Ind. 2023. https://doi.org/10.1016/j.compind.2023.103992.

    Article  Google Scholar 

  15. Chen Z, Li L, Zhao F, Sutherland JW, Yin F. Disassembly sequence planning for target parts of end-of-life smartphones using Q-learning algorithm. In: Procedia CIRP. London: Elsevier; 2023. p. 684–9. https://doi.org/10.1016/j.procir.2023.02.115.

  16. Mao H, Liu Z, Qiu C. Adaptive disassembly sequence planning for VR maintenance training via deep reinforcement learning. Int J Adv Manuf Technol. 2023;124:3039–48. https://doi.org/10.1007/s00170-021-08290-x.

    Article  Google Scholar 

  17. Smith S, Hung PY. A novel selective parallel disassembly planning method for green design. J Eng Des. 2015;26:283–301. https://doi.org/10.1080/09544828.2015.1045841.

    Article  Google Scholar 

  18. Guo J, Zhong J, Li Y, Du B, Guo S. A hybrid artificial fish swam algorithm for disassembly sequence planning considering setup time. Assem Autom. 2019;39:140–53. https://doi.org/10.1108/AA-12-2017-180.

    Article  Google Scholar 

  19. Zhong L, Youchao S, Gabriel OE, Haiqiao W. Disassembly sequence planning for maintenance based on metaheuristic method. Aircr Eng Aerosp Technol. 2011;83:138–45. https://doi.org/10.1108/00022661111131221.

    Article  Google Scholar 

  20. Liu J, Zhou Z, Pham DT, Xu W, Yan J, Liu A, Ji C, Liu Q. An improved multi-objective discrete bees algorithm for robotic disassembly line balancing problem in remanufacturing. Int J Adv Manuf Technol. 2018;97:3937–62. https://doi.org/10.1007/s00170-018-2183-7.

    Article  Google Scholar 

  21. Ghandi S, Masehian E. Review and taxonomies of assembly and disassembly path planning problems and approaches. CAD Comput Aided Des. 2015;67–68:58–86. https://doi.org/10.1016/j.cad.2015.05.001.

    Article  Google Scholar 

  22. Wang H, Peng Q, Zhang J, Gu P. Selective disassembly planning for the end-of-life product. Proc CIRP. 2017;60:512–7. https://doi.org/10.1016/j.procir.2017.02.003.

    Article  Google Scholar 

  23. Zhang C, Fathollahi-Fard AM, Li J, Tian G, Zhang T. Disassembly sequence planning for intelligent manufacturing using social engineering optimizer. Symmetry (Basel). 2021. https://doi.org/10.3390/SYM13040663.

    Article  Google Scholar 

  24. Zhan C, Zhang X, Tian G, Pham DT, Ivanov M, Aleksandrov A, Fu C, Zhang J, Wu Z. Environment-oriented disassembly planning for end-of-life vehicle batteries based on an improved northern goshawk optimisation algorithm. Environ Sci Pollut Res. 2023. https://doi.org/10.1007/s11356-023-25599-0.

    Article  Google Scholar 

  25. Xie J, Li X, Gao L. Disassembly sequence planning based on a modified grey wolf optimizer. Int J Adv Manuf Technol. 2021;116:3731–50. https://doi.org/10.1007/S00170-021-07696-X.

    Article  Google Scholar 

  26. Gunji BM, Pabba SK, Rajaram IRS, Sorakayala PS, Dubey A, Deepak BBVL, Biswal BB, Bahubalendruni MVAR. Optimal disassembly sequence generation and disposal of parts using stability graph cut-set method for End of Life product. Sadhana Acad Proc Eng Sci. 2021;46:1–15. https://doi.org/10.1007/S12046-020-01525-9.

    Article  Google Scholar 

  27. Gulivindala AK, Bahubalendruni MVAR, Madhu Balan P, Eswaran M. Mechanical disassembly sequence planning for end-of-life products to maximize recyclability. Sadhana Acad Proc Eng Sci. 2023;48:1–11. https://doi.org/10.1007/S12046-023-02160-W.

    Article  Google Scholar 

  28. Tseng HE, Chang CC, Lee SC, Huang YM. Hybrid bidirectional ant colony optimization (hybrid BACO): an algorithm for disassembly sequence planning. Eng Appl Artif Intell. 2019;83:45–56. https://doi.org/10.1016/j.engappai.2019.04.015.

    Article  Google Scholar 

  29. Addou EH, Serghini A, Mermri EB. Hybrid approaches based on simulated annealing, Tabu search and ant colony optimization for solving the k-minimum spanning tree problem. Int J Adv Comput Sci Appl. 2021;12:708–12. https://doi.org/10.14569/IJACSA.2021.0120288.

    Article  Google Scholar 

  30. Tseng HE, Huang YM, Chang CC, Lee SC. Disassembly sequence planning using a Flatworm algorithm. J Manuf Syst. 2020;57:416–28. https://doi.org/10.1016/j.jmsy.2020.10.014.

    Article  Google Scholar 

  31. Tao F, Bi L, Zuo Y, Nee AYC. Partial/parallel disassembly sequence planning for complex products. J Manuf Sci Eng Trans ASME. 2018. https://doi.org/10.1115/1.4037608.

    Article  Google Scholar 

  32. Gulivindala AK, Bahubalendruni MVAR, Chandrasekar R, Ahmed E, Abidi MH, Al-Ahmari A. Automated disassembly sequence prediction for industry 4.0 using enhanced genetic algorithm. Comput Mater Continua. 2021;69:2531–48. https://doi.org/10.32604/cmc.2021.018014.

    Article  Google Scholar 

  33. Yu D, Zhang X, Tian G, Jiang Z, Liu Z, Qiang T, Zhan C. Disassembly sequence planning for green remanufacturing using an improved whale optimisation algorithm. Processes. 2022. https://doi.org/10.3390/pr10101998.

    Article  Google Scholar 

  34. Syed Shahul Hameed AS, Rajagopalan N. SPGD: search party gradient descent algorithm, a simple gradient-based parallel algorithm for bound-constrained optimization. Mathematics. 2022. https://doi.org/10.3390/math10050800.

    Article  Google Scholar 

  35. Bansal JC, Sharma H, Jadon SS, Clerc M. Spider monkey optimization algorithm for numerical optimization. Memet Comput. 2014;6:31–47. https://doi.org/10.1007/S12293-013-0128-0.

    Article  Google Scholar 

  36. Sutton RS, Barto AG. Reinforcement learning: an introduction. London: MIT Press; 2018.

    Google Scholar 

  37. Xia K, Gao L, Li W, Wang L, Chao KM. A Q-learning based selective disassembly planning service in the cloud based remanufacturing system for WEEE. In: ASME 2014 international manufacturing science and engineering conference, MSEC 2014 collocated with the JSME 2014 international conference on materials and processing and the 42nd North American manufacturing research conference, vol. 1; 2014. https://doi.org/10.1115/MSEC2014-4008.

  38. Chand M, Ravi C. A novel reinforcement learning framework for disassembly sequence planning using Q-learning technique optimized using an enhanced simulated annealing algorithm. Artif Intell Eng Des Anal Manuf. 2024;38:5. https://doi.org/10.1017/S0890060424000039.

    Article  Google Scholar 

  39. Syed Shahul Hameed AS, Rajagopalan N. MABSearch: the bandit way of learning the learning rate—a harmony between reinforcement learning and gradient descent. Natl Acad Sci Lett. 2023;1:1–6. https://doi.org/10.1007/S40009-023-01292-1.

    Article  Google Scholar 

  40. Hameed ASSS, Rajagopalan N. NPROS: a not so pure random orthogonal search algorithm—a suite of random optimization algorithms driven by reinforcement learning. Optim Lett. 2023. https://doi.org/10.1007/s11590-023-02038-0.

    Article  Google Scholar 

  41. Chand M, Ravi C. A state-of-the-art literature survey on artificial intelligence techniques for disassembly sequence planning. CIRP J Manuf Sci Technol. 2023;41:292–310. https://doi.org/10.1016/j.cirpj.2022.11.017.

    Article  Google Scholar 

  42. Kumar GA, Bahubalendruni MVAR, Vara Prasad VSS, Ashok D, Sankaranarayanasamy K. A novel geometric feasibility method to perform assembly sequence planning through oblique orientations. Eng Sci Technol Int J. 2022. https://doi.org/10.1016/j.jestch.2021.04.013.

    Article  Google Scholar 

  43. Bello I, Pham H, Le QV, Norouzi M, Bengio S. Neural combinatorial optimization with reinforcement learning. In: 5th international conference on learning representations, ICLR 2017—workshop track proceedings; 2016.

  44. Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on international conference on machine learning—volume 37. JMLR.org; 2015. p. 448–56.

  45. Deudon M, Cournut P, Lacoste A, Adulyasak Y, Rousseau LM. Learning heuristics for the tsp by policy gradient. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics). 10848 LNCS. 2018. p. 170–81. https://doi.org/10.1007/978-3-319-93031-2_12.

  46. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I. Attention is all you need. In: Proceedings of the 31st international conference on neural information processing systems. Red Hook, NY, USA: Curran Associates Inc.; 2017. p. 6000–10.

  47. de da Costa PRO, Rhuggenaath J, Zhang Y, Akcay A. Learning 2-opt heuristics for the traveling salesman problem via deep reinforcement learning. In: Asian conference on machine learning, 18–20 November 2020, Bangkok, Thailand. PMLR; 2020. p. 465–80.

  48. Williams RJ. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn. 1992;8:229–56. https://doi.org/10.1007/BF00992696.

    Article  Google Scholar 

  49. Huber PJ. Robust estimation of a location parameter. Breakthro Stat Methodol Distrib. 1964;35:73–101. https://doi.org/10.1214/AOMS/1177703732.

    Article  MathSciNet  Google Scholar 

  50. Good IJ. Maximum entropy for hypothesis formulation. Espec Multidimens Conting Tables. 1963;34:911–34. https://doi.org/10.1214/AOMS/1177704014.

    Article  Google Scholar 

  51. Lin S. Computer solutions of the traveling salesman problem. Bell Syst Tech J. 1965;44:2245–69. https://doi.org/10.1002/J.1538-7305.1965.TB04146.X.

    Article  MathSciNet  Google Scholar 

  52. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. Adv Neural Inf Process Syst. 2014;27:1.

    Google Scholar 

  53. Li B, Ding L, Hu D, Zheng S. Backtracking algorithm-based disassembly sequence planning. Proc CIRP. 2018;69:932–7. https://doi.org/10.1016/j.procir.2017.12.007.

    Article  Google Scholar 

  54. Bahubalendruni MVAR, Varupala VP. Disassembly sequence planning for safe disposal of end-of-life waste electric and electronic equipment. Natl Acade Sci Lett. 2021;44:243–7. https://doi.org/10.1007/S40009-020-00994-0.

    Article  Google Scholar 

  55. Anil Kumar G, Bahubalendruni MVAR, Prasad VSS, Sankaranarayanasamy K. A multi-layered disassembly sequence planning method to support decision making in de-manufacturing. Sadhana Acad Proc Eng Sci. 2021;46:1–16. https://doi.org/10.1007/S12046-021-01622-3.

    Article  MathSciNet  Google Scholar 

  56. Pistolesi F, Lazzerini B. TeMA: a tensorial memetic algorithm for many-objective parallel disassembly sequence planning in product refurbishment. IEEE Trans Industr Inform. 2019;15:3743–53. https://doi.org/10.1109/TII.2019.2904631.

    Article  Google Scholar 

  57. Smith SS, Chen WH. Rule-based recursive selective disassembly sequence planning for green design. Adv Eng Inform. 2011;25:77–87. https://doi.org/10.1016/J.AEI.2010.03.002.

    Article  Google Scholar 

  58. Rickli JL, Camelio JA. Multi-objective partial disassembly optimization based on sequence feasibility. J Manuf Syst. 2013;32:281. https://doi.org/10.1016/j.jmsy.2012.11.005.

    Article  Google Scholar 

Download references

Funding

No funds, grants, or other support was received.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Mirothali Chand. The first draft of the manuscript was written by Mirothali Chand and other author commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Chandrasekar Ravi.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical Approval

Not applicable.

Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Advance in Artificial Intelligence for Machine Vision Applications” guest edited by Koushlendra Kumar Singh, B. Ramchandra Reddy, V. M. Gadre and Akbar Sheikh Akbari.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chand, M., Ravi, C. An Optimal Disassembly Sequence Planning for Complex Products using Enhanced Deep Reinforcement Learning Framework. SN COMPUT. SCI. 5, 581 (2024). https://doi.org/10.1007/s42979-024-02924-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-024-02924-z

Keywords

Navigation