Skip to main content
SpringerLink
Log in

Evolutionary digital twin model with an agent-based discrete-event simulation method

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

A digital twin model provides the ability to adjust candidate behavior based on feedback from its physical part. However, small interactions between different subsystems and using real-time data about physical workshops are the primary problems in digital twin models. The essence of the digital twin model is the combination of the physical simulation method and the data-driven simulation method, and agent-based discrete-event modeling theory is an advanced way to build a digital twin model. Thus, the theoretical framework of the Digital Twin Workshop model is improved from the underlying modeling logic based on this new theory. By combining reinforcement learning with the digital twin workshop model, an evolutionary digital twin workshop model is developed in this study. This model is then applied to a real-world case. A comparison is made between the Digital Twin Workshop model with reinforcement learning policy and a heuristic policy and a random policy. The simulation results verify the validity and performance of the proposed model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Lu Y, Liu C, Kevin I, Wang K, Huang H, Xu X (2020) Digital twin-driven smart manufacturing: Connotation, reference model, applications and research issues. Robot Comput Integr Manuf 61:101837. https://doi.org/10.1016/j.rcim.2019.101837

    Article  Google Scholar 

  2. Grieves M W (2005) Product lifecycle management: the new paradigm for enterprises. International J Product Development 2(1-2):71–84. https://doi.org/10.1504/IJPD.2005.006669

    Article  Google Scholar 

  3. Liu S, Bao J, Lu Y, Li J, Lu S, Sun X (2021) Digital twin modeling method based on biomimicry for machining aerospace components. J Manufacturing Systems 58:180–195. https://doi.org/10.1016/j.jmsy.2020.04.014

    Article  Google Scholar 

  4. Tao F, Zhang M, Cheng J, Qi Q (2017) Digital twin workshop: a new paradigm for future workshop. Comput Integr Manuf Syst 23(1):1–9

    Google Scholar 

  5. Kong T, Hu T, Zhou T, Ye Y (2021) Data construction method for the applications of workshop digital twin system. J Manuf Syst 58:323–328. https://doi.org/10.1016/j.jmsy.2020.02.003

    Article  Google Scholar 

  6. Zhang H, Liu Q, Chen X, Zhang D, Leng J (2017) A digital twin-based approach for designing and multi-objective optimization of hollow glass production line. Ieee Access 5:26901–26911. https://doi.org/10.1109/ACCESS.2017.2766453

    Article  Google Scholar 

  7. Park KT, Lee SH, Do Noh S (2021) Information fusion and systematic logic library-generation methods for self-configuration of autonomous digital twin. J Intell Manuf, pp 1–31

  8. Liu C, Jiang P, Jiang W (2020) Web-based digital twin modeling and remote control of cyber-physical production systems. Robotics and computer-integrated manufacturing 64:101956. https://doi.org/10.1016/j.rcim.2020.101956

    Article  Google Scholar 

  9. Liu C, Vengayil H, Zhong RY, Xu X (2018) A systematic development method for cyber-physical machine tools. J Manufacturing Systems 48:13–24. https://doi.org/10.1016/j.jmsy.2018.02.001

    Article  Google Scholar 

  10. Bao J, Guo D, Li J, Zhang J (2019) The modelling and operations for the digital twin in the context of manufacturing. Enterprise Information Systems 13(4):534–556. https://doi.org/10.1080/17517575.2018.1526324

    Article  Google Scholar 

  11. Chakraborty S, Adhikari S (2021) Machine learning based digital twin for dynamical systems with multiple time-scales. Computers & Structures 243:106410. https://doi.org/10.1016/J.COMPSTRUC.2020.106410

    Article  Google Scholar 

  12. Jiang H, Qin S, Fu J, Zhang J, Ding G (2021) How to model and implement connections between physical and virtual models for digital twin application. J Manuf Syst 58:36–51. https://doi.org/10.1016/j.jmsy.2020.05.012

    Article  Google Scholar 

  13. Kim H, Choi B-K (2016) Colored acd and its application. Simul Model Pract Theory 63:133–148. https://doi.org/10.1016/j.simpat.2016.02.009

    Article  Google Scholar 

  14. Ryan J, Heavey C (2006) Process modeling for simulation. Computers in industry 57(5):437–450. https://doi.org/10.1016/j.compind.2006.02.002

    Article  Google Scholar 

  15. Buyurgan N, Saygin C (2008) Application of the analytical hierarchy process for real-time scheduling and part routing in advanced manufacturing systems. J Manuf Syst 27 (3):101–110. https://doi.org/10.1016/j.jmsy.2008.08.002

    Article  Google Scholar 

  16. Abdoos M, Mozayani N, Bazzan ALC (2011) Traffic light control in non-stationary environments based on multi agent q-learning. In: 2011 14th International IEEE conference on intelligent transportation systems (ITSC), IEEE, pp 1580–1585

  17. Borshchev A, Filippov A (2004) From system dynamics and discrete event to practical agent based modeling: reasons, techniques, tools. In: Proceedings of the 22nd international conference of the system dynamics society, vol 22, Citeseer, pp 25–29

  18. Robinson S (2005) Discrete-event simulation: from the pioneers to the present, what next?. J Oper Res Soc 56(6):619–629

    Article  MATH  Google Scholar 

  19. Savaglio C, Ganzha M, Paprzycki M, Bădică C, Ivanović M, Fortino G (2020) Agent-based internet of things: State-of-the-art and research challenges. Futur Gener Comput Syst 102:1038–1053. https://doi.org/10.1016/j.future.2019.09.016

    Article  Google Scholar 

  20. Tesfatsion L (2007) Agents come to bits: Towards a constructive comprehensive taxonomy of economic entities. J Economic Behavior & Organization 63(2):333–346. https://doi.org/10.1016/j.jebo.2005.12.016

    Article  Google Scholar 

  21. Schieritz N, Grobler A (2003) Emergent structures in supply chains-a study integrating agent-based and system dynamics modeling. In: 36th Annual Hawaii international conference on system sciences, 2003. Proceedings of the 9–pp , IEEE

  22. Majid MA, Fakhreldin M, Zuhairi KZ (2016) Comparing discrete event and agent based simulation in modelling human behaviour at airport check-in counter. In: International conference on human-computer interaction, Springer, pp 510–522

  23. Jaensch F, Csiszar A, Scheifele C, Verl A (2018) Digital twins of manufacturing systems as a base for machine learning. In: 2018 25th International conference on mechatronics and machine vision in practice (M2VIP), pp 1–6

  24. Um J, Weyer S, Quint F (2017) Plug-and-simulate within modular assembly line enabled by digital twins and the use of automationml. IFAC-PapersOnLine 50(1):15904–15909. https://doi.org/10.1016/j.ifacol.2017.08.2360

    Article  Google Scholar 

  25. Lin TY, Jia Z, Yang C, Xiao Y, Lan S, Shi G, Zeng B, Li H (2021) Evolutionary digital twin: A new approach for intelligent industrial product development. Adv Eng Inform 47:101209. https://doi.org/10.1016/J.AEI.2020.101209

    Article  Google Scholar 

  26. Zou X, Yang R, Yin C, Wu Q (2019) Research on node deployment in different terrain of manet based on relational deep reinforcement learning. In: Proceedings of the 2019 2nd international conference on algorithms, computing and artificial intelligence. https://doi.org/10.1145/3377713.3377758, pp 565–569

  27. Cronrath C, Aderiani A R, Lennartson B (2019) Enhancing digital twins through reinforcement learning. In: 2019 IEEE 15th International conference on automation science and engineering (CASE), pp 293–298

  28. Park KT, Son YH, Ko SW, Noh SD (2021) Digital twin and reinforcement learning-based resilient production control for micro smart factory. Appl Sci, 11(7). https://doi.org/10.3390/app11072977, https://www.mdpi.com/2076-3417/11/7/2977

  29. Ho D, Liang E, Chen X, Stoica I, Abbeel P (2019) Population based augmentation: Efficient learning of augmentation policy schedules. In: International conference on machine learning, PMLR, pp 2731–2741

  30. Gao K, Cao Z, Zhang L, Chen Z, Han Y, Pan Q (2019) A review on swarm intelligence and evolutionary algorithms for solving flexible job shop scheduling problems. IEEE/CAA J Automatica Sinica 6(4):904–916. https://doi.org/10.1109/JAS.2019.1911540

    Article  Google Scholar 

  31. Li F, Xu Z, Li H (2021) A multi-agent based cooperative approach to decentralized multi-project scheduling and resource allocation. Computers & Industrial Engineering 151:106961

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China under Grant No. 52005447, No. 71871203, No. L1924063 and the Natural Science Foundation of Zhejiang Province under Grant No. LY18G020018, No. LQ21E050014.

Author information

Authors and Affiliations

  1. College of Mechanical Engineering, Zhejiang University of Technology, No. 288, Liuhe Road, Hangzhou, China

    Hongbin Qiu, Yong Chen, Wenchao Yi & Yingde Li

  2. Shanghai Tobacco Group Co., Ltd,, No. 717, Changyang Road, Shanghai, China

    Huaxiang Zhang

Authors
  1. Hongbin Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huaxiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenchao Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yingde Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenchao Yi.

Ethics declarations

Conflict of Interests

The authors declare no conflicts of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qiu, H., Chen, Y., Zhang, H. et al. Evolutionary digital twin model with an agent-based discrete-event simulation method. Appl Intell 53, 6178–6194 (2023). https://doi.org/10.1007/s10489-022-03507-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03507-2

Keywords

Search

Navigation