Abstract
Demands for more accurate machine learning models have given rise to rethinking current modeling approaches that were deemed unsuitable, primarily due to their computational complexity and the lack of availability and accessibility to representative data. In Industry 4.0, rapid advancements in Digital Twin (DT) technologies and the pervasiveness of cost-effective sensor technologies have pushed the incorporation of artificial intelligence, particularly data-driven machine learning models, for use in smart manufacturing. However, the persistent issue with such models is their high sensitivity to the training data and the lack of interpretability in the outcomes, at times generating unrealistic results. The incorporation of knowledge into the machine learning pipeline has been earmarked as the most promising approach to address such issues. This paper aims to answer this call through a Knowledge-embedded Machine Learning (KML) framework for smart manufacturing, which embeds knowledge from experience and, or physics information into the machine learning pipeline, thus making the outcomes from these models more representative of real applications. The merits of KML were then presented through comparative studies showing its capability to outperform knowledge-based and data-driven models. This promising outcome led to the development of frameworks that can potentially incorporate KML for smart manufacturing applications such as Prognostics and Health Management (PHM) and DT, further supporting the usefulness of the proposed KML framework.

















Similar content being viewed by others
Abbreviations
- AI:
-
Artificial Intelligence
- BPFI:
-
Ball Pass Frequency at Inner
- BPFO:
-
Ball Pass Frequency at Outer
- CPS:
-
Cyber-Physical-Systems
- CRM:
-
Customer Relation Management
- DDML:
-
Data-Driven Machine Learning
- DT:
-
Digital Twin
- ERP:
-
Enterprise Resource Planning
- IIoT:
-
Industrial Internet of Things
- KML:
-
Knowledge-embedded Machine Learning
- MSE:
-
Mean Square Error
- PHM:
-
Prognostics and Health Management
- RUL:
-
Remaining Useful Life
- SM:
-
Smart Manufacturing
- SMAI:
-
Smart Manufacturing AND Artificial Intelligence
- SMAIK:
-
Smart Manufacturing AND Artificial Intelligence AND Knowledge
- WIP:
-
Work-in-process
- WoS:
-
Web of science
References
Aggarwal, C. C. (2021). Machine learning: The inductive view. Artificial Intelligence, 1, 167–210.
Ahmad, W., Khan, S. A., & Kim, J.-M. (2018). A hybrid prognostics technique for rolling element bearings using adaptive predictive models. IEEE Transactions on Industrial Electronics, 65(2), 1577–1584.
Aivaliotis, P., Georgoulias, K., & Chryssolouris, G. (2017). A RUL calculation approach based on physical–based simulation models for predictive maintenance. Proceedings of the 23rd international conference on engineering, technology and innovation (ICE/ITMC), Madeira, Portugal, June 27–29, pp. 1284–1287.
Aivaliotis, P., Georgoulias, K., Arkouli, Z., & Makris, S. (2019). Methodology for enabling digital twin using advanced physics-based modelling in predictive maintenance. Procedia of the 52nd CIRP Conference on Manufacturing Systems, 81, 417–422.
Alam, K. M., & El Saddik, A. (2017). C2PS: A digital twin architecture reference model for the cloud-based cyber-physical systems. IEEE Access, 5, 2050–2062.
Atamuradov, V., Medjaher, K., Dersin, P., Lamoureux, B., & Zerhouni, N. (2017). Prognostics and health management for maintenance practitioners - review, implementation and tools evaluation. International Journal of Prognostics and Health Management, 8(3), 1–31.
Barricelli, B. R., Casiraghi, E., & Fogli, D. (2019). A survey on digital twin: Definitions, characteristics, applications, and design implications. IEEE Access, 7, 167653–167671.
Bassi, L. (2017). Industry 4.0: Hope, hype or revolution? Proceedings of the 2017 IEEE 3rd international forum on research and technologies for society and industry (RTSI’2017), Modena, Italy, September 11–13.
Bechhoefer, E. (2013). Condition based maintenance fault database for testing of diagnostic and prognostics algorithms. https://mfpt.org/fault-data-sets/.
Bechhoefer, E. (2018). A quick introduction to bearing envelope analysis. (Society for Machinery Failure Prevention Technology). https://mfpt.org/wp-content/uploads/2018/03/MFPT-Bearing-Envelope-Analysis.pdf.
Bechhoefer, E., He, D., & Dempsey, P. (September, 2011). Gear health threshold setting based on a probability of false alarm. Proceedings of the annual conference of the Prognostics and Health Management Society, Montreal, Canada, pp. 275–281.
Bektas, O., Marshall, J., & Jones, J. A. (2020). Comparison of computational prognostic methods for complex systems under dynamic regimes: A review of perspectives. Archives of Computational Methods in Engineering, 27, 999–1011.
Bikmukhametov, T., & Jäschke, J. (2020). Combining machine learning and process engineering physics towards enhanced accuracy and explainability of data-driven models. Computers & Chemical Engineering, 138, 106834.
Boschert, S. & Rosen, R. (2016). Digital Twin – The Simulation Aspect. Mechatronic Futures, pp. 59–74.
Camci, F., Medjaher, K., Zerhouni, N., & Nectoux, P. (2013). Feature evaluation for effective bearing prognostics. Quality and Reliability Engineering International, 29(4), 477–486.
Camps-Valls, G., et al. (2018). Physics-aware Gaussian processes in remote sensing. Applied Soft Computing, 68, 69–82.
Chan, S. & Elsheikh, A. (2017). Parametrization and generation of geological models with generative adversarial networks. arXiv:1708.01810
Chen, T. & Guestrin, C. (August 2016). XGBoost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (KDD’16), San Francisco, USA, pp. 785–794.
Chen, C., Vachtsevanos, G., & Orchard, M. E. (2012). Machine remaining useful life prediction: An integrated adaptive neuro-fuzzy and high-order particle filtering approach. Mechanical Systems and Signal Processing, 28, 597–607.
Chen, C., Zhang, B., Vachtsevanos, G., & Orchard, M. E. (2011). Machine condition prediction based on adaptive neuro-fuzzy and high-order particle filtering. IEEE Transactions on Industrial Electronics, 58(9), 4353–4364.
Chinnam, R. B., & Baruah, P. (2004). A neuro-fuzzy approach for estimating mean residual life in condition-based maintenance systems. International Journal of Materials and Product Technology, 20(1), 166–179.
Clarivate Analytics. Web of Science (2021). http://apps.webofknowledge.com.
Daigle, M. J., & Goebel, K. (2013). Model-based prognostics with concurrent damage progression processes. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 43(3), 535–546.
Dai, W.-Z., Xu, Q., Yu, Y., & Zhou, Z.-H. (2019). Bridging machine learning and logical reasoning by abductive learning. Advances in Neural Information Processing Systems, 32, 2019.
Das, A. K., Feng, Y., Habibullah, M. S., Yu, Z. & Farbiz, F. (October 2019). Tool Wear Health Monitoring with Limited Degradation Data. Proceedings of 2019 IEEE region 10 conference (TENCON–2019), Kochi, India, pp. 17–20.
Diamanti, K., & Soutis, C. (2010). Structural health monitoring techniques for aircraft composite structures. Progress in Aerospace Sciences, 46(8), 342–352.
Ding, H., et al. (2020). State of AI-based monitoring in smart manufacturing and introduction to focused section. IEEE/ASME Transactions on Mechatronics, 25(5), 2143–2154.
Downey, A., Lui, Y.-H., Hu, C., Laflamme, S., & Hu, S. (2019). Physics-based prognostics of lithium-ion battery using non-linear least squares with dynamic bounds. Reliability Engineering and System Safety, Elsevier, 182(C), 1–12.
Durkin, J. (1990). Research review: Application of expert systems in the sciences. The Ohio Journal of Science, 90(5), 171–179.
Erge, O. & van Oort, E. (January 2020). Combining Physics–Based and Data–Driven Modeling for Pressure Prediction in Well Construction. Proceedings of the 19th Python in Science Conference (SCIPY–2020), pp. 125–131.
Fang, Y., Kuan, K., Lin, J., Tanand, C., & Chandrasekhar, V. (2017). Object detection meets knowledge graphs. Proceedings of the 26th international joint conference on artificial intelligence (IJCAI-17), Melbourne, Australia, August 19–25, pp. 1661–1667.
Gabor, T., Belzner, L., Kiermeier, M., Beck, M. T., & Neitz, A. (2016). A Simulation–Based Architecture for Smart Cyber–Physical Systems. Proceedings of the 2016 IEEE international conference on autonomic computing (ICAC2016), Wuerzburg, Germany, July 17–22 .
Grieves, M. (2015). Digital twin: Manufacturing excellence through virtual factory replication. Digital Twin White Paper.
Gross, R. (2005). Psychology: The science of mind & behaviour (5th ed.). London: Hodder Education Publishers.
Habibullah, M. S., et al. (2021). A perspective into analysing tool wear condition in hard-turning process - The key lessons learnt. In C. Toro, W. Wang, & H. Akhtar (Eds.), Implementing Industry 4.0. Intelligent Systems Reference Library (Vol. 202, pp. 79–111). Cham: Springer.
Hansen, K., et al. (2013). Assessment and validation of machine learning methods for predicting molecular atomization energies. Journal of Chemical Theory and Computation, 9(8), 3404–3419.
Hosseinkhani, K., & Ng, E. (2015). A combined empirical and numerical approach for tool wear prediction in machining. Procedia CIRP, 31, 304–309.
Jia, X., et al. (2019). Physics guided RNNs for modeling dynamical systems: A case study in simulating lake temperature profiles. Proceedings of the 2019 SIAM International Conference on Data Mining (Society for Industrial and Applied Mathematics), May 2019, pp. 558–566.
Karpatne, A., Watkins, W., Read, J., & Kumar, V. (2017). Physics-guided Neural Networks (PGNN): An application in lake temperature modeling. arXiv:1710.11431
Karpatne, A., et al. (2017). Theory-guided data science: A new paradigm for scientific discovery from data. IEEE Transactions on Knowledge and Data Engineering, 29(10), 2318–2331.
Ketokivi, M., & Choi, T. (2014). Renaissance of case research as a scientific method. Journal of Operations Management, 32(5), 232–240.
Kiritsis, D. (2011). Closed-loop PLM for intelligent products in the era of the Internet of things. Computer Aided Design, 43(5), 479–501.
Kritzinger, W., Karner, M., Traar, G., Henjes, J., & Sihn, W. (2018). Digital Twin in manufacturing: A categorical literature review and classification. IFAC-PapersOnLine, 51(11), 1016–1022.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90.
Kusiak, A. (2017). Smart manufacturing must embrace big data. Nature, 544, 23–25.
Kusiak, A. (2018). Smart manufacturing. International Journal of Production Research, 56(1–2), 508–517.
Ladj, A., et al. (2021). A knowledge-based Digital Shadow for machining industry in a Digital Twin perspective. Journal of Manufacturing Systems, 58(Part B), 168–179.
Lee, J., et al. (2014). Prognostics and health management design for rotary machinery systems-Reviews, methodology and applications. Mechanical Systems and Signal Processing, 42(1–2), 314–334.
Lee, J., Davari, H., Singh, J., & Pandhare, V. (2018). Industrial artificial intelligence for industry 4.0-based manufacturing systems. Manufacturing Letters, 18, 20–23.
Liao, L., & Kottig, F. (2014). Review of hybrid prognostics approaches for remaining useful life prediction of engineered systems, and an application to battery life prediction. IEEE Transactions on Reliability, 63(1), 191–207.
Liu, C., Wang, K., Wang, Y., & Yuan, X. (2021). Learning deep multi-manifold structure feature representation for quality prediction with an industrial application. IEEE Transactions on Industrial Informatics. https://doi.org/10.1109/TII.2021.3130411.
Liu, T., Zhu, K., & Zeng, L. (2018). Diagnosis and prognosis of degradation process via hidden Semi-Markov Model. IEEE/ASME Transactions on Mechatronic, 23(3), 1456–1466.
Lopatin, D. V., & Zhirov, A. I. (2017). Geomorphology in the system of Earth sciences. Geography and Natural Resources, 38, 313–318.
Lu, Y., Liu, C., Wang, K.I.-K., Huang, H., & Xu, X. (2020). Digital Twin-driven smart manufacturing: Connotation, reference model, applications and research issues. Robotics and Computer-Integrated Manufacturing, 1, 101837.
Machamer, P., & Silberstein, M. (2002). The Blackwell guide to the philosophy of science (Vol. 19). Oxford: Blackwell.
Magnani, L. (2009). Abductive cognition: The epistemological and eco-cognitive dimensions of hypothetical reasoning (1st ed.). Berlin: Springer.
Me, M. (2015). A review on applications of the wavelet transform technique in spectral analysis. Journal of Applied & Computational Mathematics, 4(4), 1–6.
Negri, E., Fumagalli, L., & Macchi, M. (2017). A review of the roles of digital twin in CPS-based production systems. Proceedings of the 27th international conference on flexible automation and intelligent manufacturing (FAIM–2017), Modena, Italy, June 27–30, pp. 939–948.
Pecht, M., & Jaai, R. (2010). A prognostics and health management roadmap for information and electronics-rich systems. Microelectronics Reliability, 50(3), 317–323.
Poole, D. L., & Mackworth, A. (2017). Artificial intelligence: Foundations of computational agents (2nd ed.). Cambridge: Cambridge University Press.
Qiao, Q., Wang, J., Ye, L., & Gao, R. X. (2019). Digital Twin for machining tool condition prediction. Procedia CIRP, 81, 1388–1393.
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686–707.
Randall, R. B., & Antoni, J. (2011). Rolling element bearing diagnostics - A tutorial. Mechanical Systems and Signal Processing, 25(2), 485–520.
Read, C. (2016). Logic deductive and inductive (2nd ed.). Scotts Valley: CreateSpace Independent Publishing Platform.
Riad, A. M., Elminir, H. K., & Elattar, H. M. (2010). Evaluation of neural networks in the subject of prognostics as compared to linear regression model. International Journal of Engineering & Technology, 10(6), 52–56.
Rios, J., Morate, F. M., Oliva, M., & Hernandez, J. C. (2016). Framework to support the aircraft digital counterpart concept with an industrial design view. International Journal of Agile Systems and Management, 9(3), 212–231.
Schuh, G., Kelzenberg, C., Wiese, J., & Ochel, T. (2019). Data structure of the digital shadow for systematic knowledge management systems in single and small batch production. Procedia CIRP, 84, 1094–1100.
Shrouf, F., Ordieres, J., & Miragliotta, G. (2014). Smart Factories in Industry 4.0: A Review of the Concept and of Energy Management Approached in Production Based on the Internet of Things Paradigm. Proceedings of the IEEE International Conference on Industrial Engineering and Engineering Management (IEEM’2014), Selangor, Malaysia, December 9–12 697–701 .
Siddhpura, A., & Paurobally, R. (2013). A review of flank wear prediction methods for tool condition monitoring in a turning process. International Journal of Advanced Manufacturing Technology, 65, 371–393.
Stefik, M. (1995). Introduction to knowledge systems (1st ed.). San Francisco: Morgan Kaufmann Publishers Inc.
Steup, M. & Neta, R. (2020). The Stanford Encyclopedia of Philosophy: Epistemology (Stanford Encyclopedia of Philosophy (Online), Stanford, CA, USA, 2020). https://plato.stanford.edu/entries/epistemology.
Stoecklein, D., Lore, K. G., Davies, M., Sarkar, S., & Ganapathysubramanian, B. (2017). Deep learning for flow sculpting: Insights into efficient learning using scientific simulation data. Scientific Reports, 7, 1.
Tao, F., & Qi, Q. (2019). Make more digital twins. Nature, 573, 490–491.
Torabi, A. J., Er, M. J., Li, X., Lim, B. S., & Peen, G. O. (2016). Application of clustering methods for online tool condition monitoring and fault diagnosis in high-speed milling processes. IEEE Systems Journal, 10(2), 721–732.
Tsui, K. L., Chen, N., Zhou, Q., Hai, Y., & Wang, W. (2015). Prognostics and health management: A review on data driven approaches. Mathematical Problems in Engineering, 1(17), 1–17.
Uhlemann, T.H.-J., Lehmann, C., & Steinhilper, R. (2017). The Digital Twin: Realizing the Cyber-Physical Production System for Industry 4.0. Procedia CIRP (24th CIRP Conference on Life Cycle Engineering), 61, 335–340.
Vathoopan, M., Johny, M., Zoitl, A., & Knoll, A. (2018). Modular fault ascription and corrective maintenance using a digital twin. IFAC-PapersOnLine, 51(11), 1041–1046.
von Rueden, L., et al. (2021). Informed machine learning— A taxonomy and survey of integrating prior knowledge into learning systems. IEEE Transactions on Knowledge and Data Engineering.
Wang, T., Yu, J., Siegel, D., & Lee, J. (October, 2008). A similarity-based prognostics approach for remaining useful life estimation of engineered systems. In Proceedings of the international conference on prognostics health management (PHM–2008) pp. 1–6.
Whitmore, A., Agarwal, A., & Xu, L. D. (2015). The internet of things - A survey of topics and trends. Information Systems Frontiers, 17, 261–274.
Willard, J., Jia, X., Steinbach, M., Kumar, V., & Xu, S. (2020). Integrating physics-based modeling with machine learning: A survey. arXiv:2003.04919v4
Wu, Q., Yang, X., & Zhou, Q. (2012). Pattern recognition and its application in fault diagnosis of electromechanical system. Journal of Information and Computational Science, 9(8), 2221–2228.
Xi, Z., Jing, R., Wang, P., & Hu, C. (2013). A Copula-based sampling method for data–driven prognostics and health management. Proceedings of the 2013 IEEE conference on prognostics and health management (PHM–2013), Gaithersburg, USA, June 24–27, pp. 2–82.
Yan, J., Meng, Y., Lu, L., & Guo, C. (2017). Big–Data–Driven Based Intelligent Prognostics Scheme in Industry 4.0 Environment. Proceedings of the 2017 Prognostics and System Health Management Conference (PHM–2017), Harbin, China, July 9–12 1–5 .
Yang, F., et al. (2016). Health index-based prognostics for remaining useful life predictions in electrical machines. IEEE Transactions on Industrial Electronics, 63(4), 2633–2644.
Yang, F., Habibullah, M. S., & Shen, Y. (2021). Remaining useful life prediction of induction motors using nonlinear degradation of health index. Mechanical Systems and Signal Processing, 148, 1–17.
Zagzebski, L. (2017). What is Knowledge? The Blackwell Guide to Epistemology (Book Editors: J. Greco, E. Sosa)(Chapter 3) (pp. 92–116). (ISBN: 9780631202912) .
Zhang, C. et al. (2018). A multi-state diagnosis and prognosis framework with feature learning for tool condition monitoring. arXiv:1805.00367 (pp. 1–12).
Zhang, G., et al. (2019). Effects of uniaxial and triaxial compression tests on the frozen sandstone combining with CT scanning. International Journal of Physical Modelling in Geotechnics, 19(5), 261–274.
Zhang, C., Zhou, G., He, J., Li, Z., & Cheng, W. (2019). A data- and knowledge-driven framework for digital twin manufacturing cell. Procedia CIRP, 83, 345–350.
Zheng, P., et al. (2018). Smart manufacturing systems for Industry 4.0: Conceptual framework, scenarios, and future perspectives. Frontiers of Mechanical Engineering, 13, 137–150.
Zhou, J. (2013). Digitalization and intelligentization of manufacturing industry. Advances in Manufacturing, 1, 1–7.
Acknowledgements
The authors would like to thank Mr. Eric Bechhoefer (Green Power Monitoring Systems (GPMS) International, Inc., USA) for the detailed information on the bearing dataset, data acquisition system, and the related sampling rate.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Farbiz, F., Habibullah, M.S., Hamadicharef, B. et al. Knowledge-embedded machine learning and its applications in smart manufacturing. J Intell Manuf 34, 2889–2906 (2023). https://doi.org/10.1007/s10845-022-01973-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-022-01973-6