skip to main content
research-article

Digital Twin of Intelligent Small Surface Defect Detection with Cyber-manufacturing Systems

Published: 17 November 2023 Publication History

Abstract

With the remarkable technological development in cyber-physical systems, industry 4.0 has evolved by use of a significant concept named digital twin (DT). However, it is still difficult to construct a relationship between twin simulation and a real scenario considering dynamic variations, especially when dealing with small surface defect detection tasks with high performance and computation resource requirements. In this article, we aim to construct cyber-manufacturing systems to achieve a DT solution for small surface defect detection task. Focusing on DT-based solution, the proposed system consists of an Edge–Cloud architecture and a surface defect detection algorithm. Considering dynamic characteristics and real-time response requirement, Edge–Cloud architecture is built to achieve smart manufacturing by efficiently collecting, processing, analyzing, and storing data produced by factory. A deep learning–based algorithm is then constructed to detect surface defeats based on multi-modal data, i.e., imaging and depth data. Experiments show the proposed algorithm could achieve high accuracy and recall in small defeat detection task, thus constructing DT in cyber-manufacturing.

References

[1]
Samah Aloufi and Abdulmotaleb El-Saddik. 2022. MMSUM digital twins: A multi-view multi-modality summarization framework for sporting events. ACM Trans. Multim. Comput. Commun. Appl. 18, 1 (2022), 5:1–5:25.
[2]
Peter Bauer, Bjorn Stevens, and Wilco Hazeleger. 2021. A digital twin of earth for the green transition. Nat. Clim. Change 11, 2 (2021), 80–83.
[3]
Navaneeth Bodla, Bharat Singh, Rama Chellappa, and Larry S. Davis. 2017. Soft-NMS–improving object detection with one line of code. In Proceedings of the IEEE International Conference on Computer Vision. 5561–5569.
[4]
Chen Chen, Jiange Jiang, Yang Zhou, Ning Lv, Xiaoxu Liang, and Shaohua Wan. 2022. An edge intelligence empowered flooding process prediction using Internet of things in smart city. J. Parallel Distrib. Comput. 165 (2022), 66–78.
[5]
Xiaozhi Chen, Kaustav Kundu, Ziyu Zhang, Huimin Ma, Sanja Fidler, and Raquel Urtasun. 2016. Monocular 3d object detection for autonomous driving. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2147–2156.
[6]
Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, and Tian Xia. 2017. Multi-view 3d object detection network for autonomous driving. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1907–1915.
[7]
Yuwei Cheng, Hu Xu, and Yimin Liu. 2021. Robust small object detection on the water surface through fusion of camera and millimeter wave radar. In Proceedings of IEEE/CVF International Conference on Computer Vision. IEEE, 15243–15252.
[8]
Songtao Ding, Hongyu Wang, Hu Lu, Michele Nappi, and Shaohua Wan. 2022. Two path gland segmentation algorithm of colon pathological image based on local semantic guidance. IEEE J. Biomed. Health Inf. (2022).
[9]
Martin Engelcke, Dushyant Rao, Dominic Zeng Wang, Chi Hay Tong, and Ingmar Posner. 2017. Vote3deep: Fast object detection in 3d point clouds using efficient convolutional neural networks. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’17). IEEE, 1355–1361.
[10]
Shahrzad Faghih-Roohi, Siamak Hajizadeh, Alfredo Núñez, Robert Babuska, and Bart De Schutter. 2016. Deep convolutional neural networks for detection of rail surface defects. In Proceedings of the International Joint Conference on Neural Networks. 2584–2589.
[11]
Liuhao Ge, Yujun Cai, Junwu Weng, and Junsong Yuan. 2018. Hand pointnet: 3d hand pose estimation using point sets. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8417–8426.
[12]
Ross Girshick. 2015. Fast r-cnn. In Proceedings of the IEEE International Conference on Computer Vision. 1440–1448.
[13]
Yu He, Kechen Song, Qinggang Meng, and Yunhui Yan. 2019. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Trans. Instrum. Meas. 69, 4 (2019), 1493–1504.
[14]
Yihui He, Chenchen Zhu, Jianren Wang, Marios Savvides, and Xiangyu Zhang. 2019. Bounding box regression with uncertainty for accurate object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2888–2897.
[15]
Sabina Jeschke, Christian Brecher, Tobias Meisen, Denis Özdemir, and Tim Eschert. 2017. Industrial internet of things and cyber manufacturing systems. In Industrial Internet of Things. Springer, 3–19.
[16]
Xiaogang Jia, Xianqiang Yang, Xinghu Yu, and Huijun Gao. 2020. A modified centernet for crack detection of sanitary ceramics. In Proceedings of the Annual Conference of the IEEE Industrial Electronics Society. 5311–5316.
[17]
Eric Guiffo Kaigom and Jürgen Roßmann. 2021. Value-driven robotic digital twins in cyber-physical applications. IEEE Trans. Ind. Inf. 17, 5 (2021), 3609–3619.
[18]
Hei Law and Jia Deng. 2020. CornerNet: Detecting objects as paired keypoints. Int. J. Comput. Vis. 128, 3 (2020), 642–656.
[19]
Jay Lee, Behrad Bagheri, and Chao Jin. 2016. Introduction to cyber manufacturing. Manufact. Lett. 8 (2016), 11–15.
[20]
Xiaoming Li, Hao Liu, Weixi Wang, Ye Zheng, Haibin Lv, and Zhihan Lv. 2022. Big data analysis of the Internet of Things in the digital twins of smart city based on deep learning. Fut. Gener. Comput. Syst. 128 (2022), 167–177.
[21]
Hui Lin, Bin Li, Xinggang Wang, Yufeng Shu, and Shuanglong Niu. 2019. Automated defect inspection of LED chip using deep convolutional neural network. J. Intell. Manuf. 30, 6 (2019), 2525–2534.
[22]
Hsien-I. Lin and Fauzy Satrio Wibowo. 2021. Image data assessment approach for deep learning-based metal surface defect-detection systems. IEEE Access 9 (2021), 47621–47638.
[23]
Su Liu, Jiong Yu, Xiaoheng Deng, and Shaohua Wan. 2021. FedCPF: An efficient-communication federated learning approach for vehicular edge computing in 6G communication networks. IEEE Trans. Intell. Transport. Syst. 23, 2 (2021), 1616–1629.
[24]
László Monostori, Botond Kádár, Thomas Bauernhansl, Shinsuke Kondoh, Soundar Kumara, Gunther Reinhart, Olaf Sauer, Gunther Schuh, Wilfried Sihn, and Kenichi Ueda. 2016. Cyber-physical systems in manufacturing. Cirp Ann. 65, 2 (2016), 621–641.
[25]
Arsalan Mousavian, Dragomir Anguelov, John Flynn, and Jana Kosecka. 2017. 3d bounding box estimation using deep learning and geometry. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7074–7082.
[26]
Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. 2017. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 652–660.
[27]
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Proceedings of Neural Information Processing Systems, 5099–5108.
[28]
Domen Racki, Dejan Tomazevic, and Danijel Skocaj. 2018. A compact convolutional neural network for textured surface anomaly detection. In Proceedings of IEEE Winter Conference on Applications of Computer Vision. 1331–1339.
[29]
Roberto Saracco. 2019. Digital twins: Bridging physical space and cyberspace. Computer 52, 12 (2019), 58–64.
[30]
Michael Schluse, Marc Priggemeyer, Linus Atorf, and Jürgen Rossmann. 2018. Experimentable digital twins - streamlining simulation-based systems engineering for industry 4.0. IEEE Trans. Ind. Inf. 14, 4 (2018), 1722–1731.
[31]
Wen Sun, Ning Xu, Lu Wang, Haibin Zhang, and Yan Zhang. 2022. Dynamic digital twin and federated learning with incentives for air-ground networks. IEEE Trans. Netw. Sci. Eng. 9, 1 (2022), 321–333.
[32]
Domen Tabernik, Samo Sela, Jure Skvarc, and Danijel Skocaj. 2020. Segmentation-based deep-learning approach for surface-defect detection. J. Intell. Manufact. 31, 3 (2020), 759–776.
[33]
Dazhong Wu, Shaopeng Liu, Li Zhang, Janis Terpenny, Robert X. Gao, Thomas Kurfess, and Judith A. Guzzo. 2017. A fog computing-based framework for process monitoring and prognosis in cyber-manufacturing. J. Manufact. Syst. 43 (2017), 25–34.
[34]
Wu Mingtao and Moon Young. 2018. Intrusion detection for cybermanufacturing system. J. Manufacturing Sc. Eng. 141 (2018), 11.
[35]
Yirui Wu, Haifeng Guo, Chinmay Chakraborty, Mohammad Khosravi, Stefano Berretti, and Shaohua Wan. 2022. Edge computing driven low-light image dynamic enhancement for object detection. IEEE Trans. Netw. Sci. Eng. (2022).
[36]
Yan Yan, Yuxing Mao, and Bo Li. 2018. Second: Sparsely embedded convolutional detection. Sensors 18, 10 (2018), 3337.
[37]
Chao Zhang, Guanghui Zhou, Han Li, and Yan Cao. 2020. Manufacturing blockchain of things for the configuration of a data- and knowledge-driven digital twin manufacturing cell. IEEE IoT J. 7, 12 (2020), 11884–11894.
[38]
Liang Zhao, Chengcheng Wang, Kanglian Zhao, Daniele Tarchi, Shaohua Wan, and Neeraj Kumar. 2022. INTERLINK: A digital twin-assisted storage strategy for satellite-terrestrial networks. IEEE Trans. Aerosp. Electron. Syst. (2022).
[39]
Xiaochi Zhou, Andreas Eibeck, Mei Qi Lim, Nenad B. Krdzavac, and Markus Kraft. 2019. An agent composition framework for the j-park simulator-a knowledge graph for the process industry. Comput. Chem. Eng. 130 (2019), 106577.
[40]
Xingyi Zhou, Vladlen Koltun, Philipp Krähenbühl. 2020. Tracking objects as points. In Proceedings of European Conference on Computer Vision, 474–490.
[41]
Xiaokang Zhou, Xuesong Xu, Wei Liang, Zhi Zeng, Shohei Shimizu, Laurence T. Yang, and Qun Jin. 2022. Intelligent small object detection for digital twin in smart manufacturing with industrial cyber-physical systems. IEEE Trans. Ind. Inf. 18, 2 (2022), 1377–1386.

Cited By

View all
  • (2025)Cyber-Physical System Security for Manufacturing Industry 4.0 Using LSTM-CNN Parallel OrchestrationIEEE Access10.1109/ACCESS.2025.352552013(3788-3801)Online publication date: 2025
  • (2025)Digital Twin-driven multi-scale characterization of machining quality: current status, challenges, and future perspectivesRobotics and Computer-Integrated Manufacturing10.1016/j.rcim.2024.10290293(102902)Online publication date: Jun-2025
  • (2025)A novel domain independent scene text localizerPattern Recognition10.1016/j.patcog.2024.111015158(111015)Online publication date: Feb-2025
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Internet Technology
ACM Transactions on Internet Technology  Volume 23, Issue 4
November 2023
249 pages
ISSN:1533-5399
EISSN:1557-6051
DOI:10.1145/3633308
  • Editor:
  • Ling Liu
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 November 2023
Online AM: 17 November 2022
Accepted: 27 October 2022
Revised: 06 October 2022
Received: 10 April 2022
Published in TOIT Volume 23, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Defect detection
  2. cyber manufacturing
  3. digital twin
  4. 3D point cloud

Qualifiers

  • Research-article

Funding Sources

  • National Key R&D Program of China
  • National Natural Science Foundation of China
  • Fundamental Research Funds for the Central Universities
  • Fundamental Research Funds for the Central Universities

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)675
  • Downloads (Last 6 weeks)41
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Cyber-Physical System Security for Manufacturing Industry 4.0 Using LSTM-CNN Parallel OrchestrationIEEE Access10.1109/ACCESS.2025.352552013(3788-3801)Online publication date: 2025
  • (2025)Digital Twin-driven multi-scale characterization of machining quality: current status, challenges, and future perspectivesRobotics and Computer-Integrated Manufacturing10.1016/j.rcim.2024.10290293(102902)Online publication date: Jun-2025
  • (2025)A novel domain independent scene text localizerPattern Recognition10.1016/j.patcog.2024.111015158(111015)Online publication date: Feb-2025
  • (2024)Automatic Positioning System for Industrial CT Image Defects Based on Machine VisionProceedings of the 2nd International Conference on Signal Processing, Computer Networks and Communications10.1145/3654446.3654488(235-239)Online publication date: 3-May-2024
  • (2024)GBT-CHAIN: A System Framework for Solving the General Trilemma in Permissioned BlockchainsDistributed Ledger Technologies: Research and Practice10.1145/36158713:2(1-15)Online publication date: 18-Jun-2024
  • (2024)Feature Fusion Pyramid Network for End-to-End Scene Text DetectionACM Transactions on Asian and Low-Resource Language Information Processing10.1145/358200323:11(1-16)Online publication date: 21-Nov-2024
  • (2024)Edge Computing and Few-Shot Learning Featured Intelligent Framework in Digital Twin Empowered Mobile NetworksIEEE Transactions on Network and Service Management10.1109/TNSM.2024.345099321:6(6505-6514)Online publication date: Dec-2024
  • (2024)Attention-Based Deep Neural Network Combined Local and Global Features for Indoor Scene RecognitionIEEE Transactions on Industrial Informatics10.1109/TII.2024.342419720:11(12684-12693)Online publication date: Nov-2024
  • (2024)Highly Sensitive Real-Time Detection of Surface Texture Changes2024 IEEE 17th International Conference on Advanced Trends in Radioelectronics, Telecommunications and Computer Engineering (TCSET)10.1109/TCSET64720.2024.10755562(1-4)Online publication date: 8-Oct-2024
  • (2024)Detection Method of Steel Surface Defects Based on Semi-Supervised Frame2024 IEEE 4th International Conference on Software Engineering and Artificial Intelligence (SEAI)10.1109/SEAI62072.2024.10674129(128-133)Online publication date: 21-Jun-2024
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media