Kangcheng Liu
ABSTRACT
The defect diagnosis of modern infrastructures is crucial to public safety. In this work, we propose a complete crack inspection system with three main components, including the autonomous system setup, the geographic-information-system-based 3D reconstruction, and the database construction as well as domain adaptive algorithms design. To fulfill the unsupervised domain adaptation (UDA) task of cracks recognition in infrastructural inspections, we propose a robust unsupervised domain adaptive learning strategy termed Crack-DA to increase the generalization capacity of the model in unseen test circumstances. Specifically, firstly, we propose leveraging the self-supervised depth information to help the learning of semantics. Secondly, we propose using the edge information to suppress the non-edge background objects and noises. Thirdly, we propose using the data augmentation-based consistency learning to increase the prediction robustness. Finally, we use the disparity in depth to evaluate the domain gap in semantics and explicitly consider the domain gap in the optimization of the network. Also, we propose a source database consisting of 11,298 crack images with detailed pixel-level labels for network training in domain adaptations. Extensive experiments on UAV-captured highway cracks and real-site UAV inspections of building cracks demonstrate the robustness and effectiveness of the proposed domain adaptive crack recognition approach.
Supplemental Material
Available for Download
- Paul Bao, Lei Zhang, and Xiaolin Wu. 2005. Canny edge detection enhancement by scale multiplication. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 27, 9 (2005), 1485--1490.Google Scholar
Digital Library
- Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, and Jurgen Gall. 2019. SemanticKITTI: A dataset for semantic scene understanding of lidar sequences. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 9297--9307.Google ScholarCross Ref
- Guowei Cai, BenMChen, and Tong Heng Lee. 2011. Unmanned Rotorcraft Systems. Springer Science & Business Media.Google Scholar
- Guowei Cai, Ben M Chen, Kemao Peng, Miaobo Dong, and Tong H Lee. 2008. Modeling and Control of the Yaw Channel of a UAV Helicopter. IEEE Transactions on Industrial Electronics (TIE) 55, 9 (2008), 3426--3434.Google ScholarCross Ref
- Ben M Chen. 2000. Robust and H ∞ Control. Springer.Google ScholarCross Ref
- Fu-Chen Chen and Mohammad R Jahanshahi. 2018. NB-CNN: Deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Transactions on Industrial Electronics (TIE) 65, 5 (2018), 4392--4400.Google ScholarCross Ref
- Liang-Chieh Chen, Jonathan T Barron, George Papandreou, Kevin Murphy, and Alan L Yuille. 2016. Semantic image segmentation with task-specific edge detection using cnns and a discriminatively trained domain transform. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 4545--4554.Google ScholarCross Ref
- Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2018. Deeplab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 40, 4 (2018), 834--848.Google ScholarCross Ref
- Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. 2018. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. In Proceedings of the European Conference on Computer Vision (ECCV). 801--818.Google ScholarDigital Library
- Wooram Choi and Young-Jin Cha. 2020. SDDNet: Real-time crack segmentation. IEEE Transactions on Industrial Electronics (TIE) 67, 9 (2020), 8016--8025.Google ScholarCross Ref
- Lijuan Duan, Huiling Geng, Jun Zeng, Junbiao Pang, and Qingming Huang. 2019. Fast and Accurately Measuring Crack Width via Cascade Principal Component Analysis. In Proceedings of the ACM Multimedia Asia. 1--7.Google ScholarDigital Library
- Fei Gao, LuqiWang, Boyu Zhou, Xin Zhou, Jie Pan, and Shaojie Shen. 2020. Teach-repeat-replan: A complete and robust system for aggressive flight in complex environments. IEEE Transactions on Robotics 36, 5 (2020), 1526--1545.Google ScholarCross Ref
- Andreas Geiger, Philip Lenz, and Raquel Urtasun. 2012. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, 3354--3361.Google ScholarCross Ref
- Dayan Guan, Jiaxing Huang, Aoran Xiao, and Shijian Lu. 2021. Domain adaptive video segmentation via temporal consistency regularization. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8053--8064.Google ScholarCross Ref
- Zhonghua Hong, Fan Yang, Haiyan Pan, Ruyan Zhou, Yun Zhang, Yanling Han, Jing Wang, Shuhu Yang, Peng Chen, Xiaohua Tong, et al. 2022. Highway Crack Segmentation from Unmanned Aerial Vehicle Images Using Deep Learning. IEEE Geoscience and Remote Sensing Letters (GRSL) (2022).Google Scholar
- Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu. 2021. Model adaptation: Historical contrastive learning for unsupervised domain adaptation without source data. Advances in Neural Information Processing Systems 34 (2021), 3635--3649.Google Scholar
- Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu. 2022. Multi-level adversarial network for domain adaptive semantic segmentation. Pattern Recognition 123 (2022), 108384.Google ScholarDigital Library
- Jiaxing Huang, Dayan Guan, Aoran Xiao, Shijian Lu, and Ling Shao. 2022. Category contrast for unsupervised domain adaptation in visual tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1203--1214.Google ScholarCross Ref
- Jiaxing Huang, Shijian Lu, Dayan Guan, and Xiaobing Zhang. 2020. Contextual relation consistent domain adaptation for semantic segmentation. In European conference on computer vision. Springer, 705--722.Google ScholarDigital Library
- Sophie Lambert-Lacroix and Laurent Zwald. 2016. The adaptive BerHu penalty in robust regression. Journal of Nonparametric Statistics 28, 3 (2016), 487--514.Google ScholarCross Ref
- Boyi Li, Felix Wu, Ser-Nam Lim, Serge Belongie, and Kilian Q Weinberger. 2021. On feature normalization and data augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 12383--12392.Google ScholarCross Ref
- Zhaoshuo Li, Xingtong Liu, Nathan Drenkow, Andy Ding, Francis X Creighton, Russell H Taylor, and Mathias Unberath. 2021. Revisiting stereo depth estimation from a sequence-to-sequence perspective with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 6197--6206.Google ScholarCross Ref
- Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2980--2988.Google ScholarCross Ref
- Kangcheng Liu, Zhi Gao, Feng Lin, and BenMChen. 2020. Fg-net: Fast large-scale lidar point clouds understanding network leveraging correlated feature mining and geometric-aware modelling. arXiv preprint arXiv:2012.09439 (2020).Google Scholar
- Kangcheng Liu, Zhi Gao, Feng Lin, and Ben M Chen. 2021. FG-Conv: Large-Scale LiDAR Point Clouds Understanding Leveraging Feature Correlation Mining and Geometric-Aware Modeling. In 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 12896--12902.Google Scholar
- Kangcheng Liu, Zhi Gao, Feng Lin, and Ben M Chen. 2022. FG-Net: A Fast and Accurate Framework for Large-Scale LiDAR Point Cloud Understanding. IEEE Transactions on Cybernetics (2022).Google Scholar
- Kangcheng Liu, Xiaodong Han, and Ben M Chen. 2019. Deep learning based automatic crack detection and segmentation for unmanned aerial vehicle inspections. In 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 381--387.Google ScholarDigital Library
- Kangcheng Liu, Yanbin Qu, Hak-Man Kim, and Huihui Song. 2017. Avoiding frequency second dip in power unreserved control during wind power rotational speed recovery. IEEE transactions on power systems 33, 3 (2017), 3097--3106.Google Scholar
- Kangcheng Liu, Yuzhi Zhao, Qiang Nie, Zhi Gao, and Ben M. Chen. 2022. WS3D: Weakly Supervised 3D Scene Segmentation with Region-Level Boundary Awareness and Instance Discrimination. In European Conference on Computer Vision (ECCV).Google Scholar
- Kangcheng Liu, Xunkuai Zhou, and Ben M Chen. 2022. An Enhanced LiDAR Inertial Localization and Mapping System for Unmanned Ground Vehicles. In 2022 17th IEEE International Conference on Control and Automation (ICCA). IEEE.Google Scholar
- Kangcheng Liu, Xunkuai Zhou, Benyun Zhao, Huosen Ou, and Ben M Chen. 2022. An Integrated Visual System for Unmanned Aerial Vehicles Following Ground Vehicles: Simulations and Experiments. In 2022 17th IEEE International Conference on Control and Automation (ICCA). IEEE.Google Scholar
- Yahui Liu, Jian Yao, Xiaohu Lu, Renping Xie, and Li Li. 2019. DeepCrack: A deep hierarchical feature learning architecture for crack segmentation. Neurocomputing 338 (2019), 139--153.Google ScholarDigital Library
- Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 10012--10022.Google ScholarCross Ref
- Zhicheng Liu, Xilong Liu, Zhiqiang Cao, Xurong Gong, Min Tan, and Junzhi Yu. 2022. High Precision Calibration for 3D Vision-Guided Robot System. IEEE Transactions on Industrial Electronics (TIE) (2022).Google Scholar
- Zhengzhe Liu, Xiaojuan Qi, and Chi-Wing Fu. 2021. One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1726--1736.Google ScholarCross Ref
- Zhipeng Luo, Zhongang Cai, Changqing Zhou, Gongjie Zhang, Haiyu Zhao, Shuai Yi, Shijian Lu, Hongsheng Li, Shanghang Zhang, and Ziwei Liu. 2021. Unsupervised domain adaptive 3d detection with multi-level consistency. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8866--8875.Google ScholarCross Ref
- Lorenz Meier, Petri Tanskanen, Lionel Heng, Gim Hee Lee, Friedrich Fraundorfer, and Marc Pollefeys. 2012. PIXHAWK: A micro aerial vehicle design for autonomous flight using onboard computer vision. Autonomous Robots 33, 1 (2012), 21--39.Google ScholarDigital Library
- Martin Mundt, Sagnik Majumder, Sreenivas Murali, Panagiotis Panetsos, and Visvanathan Ramesh. 2019. Meta-learning convolutional neural architectures for multi-target concrete defect classification with the concrete defect bridge image dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 11196--11205.Google ScholarCross Ref
- Quang-Hieu Pham, Thanh Nguyen, Binh-Son Hua, Gemma Roig, and Sai-Kit Yeung. 2019. JSIS3D: Joint semantic-instance segmentation of 3D point clouds with multi-task point-wise networks and multi-value conditional random fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (ICCV). 8827--8836.Google ScholarCross Ref
- Paolo Sassi, Paolo Tripicchio, and Carlo Alberto Avizzano. 2019. A smart monitoring system for automatic welding defect detection. IEEE Transactions on Industrial Electronics (TIE) 66, 12 (2019), 9641--9650.Google ScholarCross Ref
- Jasper RR Uijlings, Koen EA Van De Sande, Theo Gevers, and Arnold WM Smeulders. 2013. Selective search for object recognition. International Journal of Computer Vision (IJCV) 104, 2 (2013), 154--171.Google ScholarDigital Library
- Qian Wan, Liang Gao, Xinyu Li, and Long Wen. 2022. Industrial Image Anomaly Localization Based on Gaussian Clustering of Pre-trained Feature. IEEE Transactions on Industrial Electronics (TIE) (2022).Google Scholar
- Long Wang and Zijun Zhang. 2017. Automatic detection of wind turbine blade surface cracks based on UAV-taken images. IEEE Transactions on Industrial Electronics (TIE) 64, 9 (2017), 7293--7303.Google ScholarCross Ref
- Yuxing Wang, Yawen Lu, Zhihua Xie, and Guoyu Lu. 2021. Deep Unsupervised 3D SfM Face Reconstruction Based on Massive Landmark Bundle Adjustment. In Proceedings of the 29th ACM International Conference on Multimedia. 1350--1358.Google ScholarDigital Library
- Long Wen, Xinyu Li, and Liang Gao. 2021. A new reinforcement learning based learning rate scheduler for convolutional neural network in fault classification. Transactions on Industrial Electronics (TIE) 68, 12 (2021), 12890--12900.Google ScholarCross Ref
- Qingsong Wen, Liang Sun, Fan Yang, Xiaomin Song, Jingkun Gao, Xue Wang, and Huan Xu. 2020. Time series data augmentation for deep learning: A survey. arXiv preprint arXiv:2002.12478 (2020).Google Scholar
- Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. 2021. SegFormer: Simple and efficient design for semantic segmentation with transformers. Advances in Neural Information Processing Systems (Neurips) 34 (2021).Google Scholar
- Guanglei Yang, Hao Tang, Mingli Ding, Nicu Sebe, and Elisa Ricci. 2021. Transformer-based attention networks for continuous pixel-wise prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 16269--16279.Google ScholarCross Ref
- Ming-Jia Yang, Yu-Xiao Guo, Bin Zhou, and Xin Tong. 2021. Indoor Scene Generation from a Collection of Semantic-Segmented Depth Images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 15203--15212.Google ScholarCross Ref
- Fangneng Zhan, Chuhui Xue, and Shijian Lu. 2019. Ga-dan: Geometry-aware domain adaptation network for scene text detection and recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 9105--9115.Google ScholarCross Ref
- Chong Zhang, Zongxian Li, Jingjing Liu, Peixi Peng, Qixiang Ye, Shijian Lu, Tiejun Huang, and Yonghong Tian. 2021. Self-guided adaptation: Progressive representation alignment for domain adaptive object detection. IEEE Transactions on Multimedia 24 (2021), 2246--2258.Google ScholarDigital Library
- Zhenyu Zhang, Shouwei Gao, and Zheng Huang. 2021. An automatic glioma segmentation system using a multilevel attention pyramid scene parsing network. Current Medical Imaging 17, 6 (2021), 751--761.Google ScholarCross Ref
- Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. 2017. Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2881--2890.Google ScholarCross Ref
- Yuzhi Zhao, Lai-Man Po, Tingyu Lin, XuehuiWang, Kangcheng Liu, Yujia Zhang, Wing-Yin Yu, Pengfei Xian, and Jingjing Xiong. 2021. Legacy photo editing with learned noise prior. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2103--2112.Google ScholarCross Ref
- Mingmin Zhen, Jinglu Wang, Lei Zhou, Shiwei Li, Tianwei Shen, Jiaxiang Shang, Tian Fang, and Long Quan. 2020. Joint semantic segmentation and boundary detection using iterative pyramid contexts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 13666--13675.Google ScholarCross Ref
- Qin Zou, Zheng Zhang, Qingquan Li, Xianbiao Qi, Qian Wang, and Song Wang. 2019. Deepcrack: Learning hierarchical convolutional features for crack detection. IEEE Transactions on Image Processing (TIP) 28, 3 (2019), 1498--1512.Google ScholarDigital Library
Index Terms
- Robust Industrial UAV/UGV-Based Unsupervised Domain Adaptive Crack Recognitions with Depth and Edge Awareness: From System and Database Constructions to Real-Site Inspections
Recommendations
ROAD: Robust Unsupervised Domain Adaptation with Noisy Labels
MM '23: Proceedings of the 31st ACM International Conference on MultimediaIn recent years, Unsupervised Domain Adaptation (UDA) has emerged as a popular technique for transferring knowledge from a labeled source domain to an unlabeled target domain. However, almost all of the existing approaches implicitly assume that the ...
Unsupervised and Semi-Supervised Robust Spherical Space Domain Adaptation
Adversarial domain adaptation has been an effective approach for learning domain-invariant features by adversarial training. In this paper, we propose a novel adversarial domain adaptation approach defined in the spherical feature space, in which we ...
Domain consistency regularization for unsupervised multi-source domain adaptive classification
Highlights- We propose a novel multi-source domain adaptation method for classification.
- We ...
AbstractDeep learning-based multi-source unsupervised domain adaptation (MUDA) has been actively studied in recent years. Compared with single-source unsupervised domain adaptation (SUDA), domain shift in MUDA exists not only between the ...
Comments