Abstract
Anomaly detection is a challenging task in the field of data analysis, especially when it comes to unsupervised pixel-level segmentation of anomalies in images. In this paper, we present a novel multi-stage image resynthesis framework for detecting and segmenting image anomalies. In contrast to existing reconstruction-based approaches, our method is based on repairing suspicious regions of defective images so that the defects can be localized in the residual map between inputs and the repaired outputs. To avoid the reconstruction artifacts caused by defects, we propose to generate each pixel of the image by its context in the first coarse reconstruction stage. Then, while excluding all safe pixels, our method repairs suspicious regions that have large deviations to the original input image in subsequent stages. After several iterations, the defects will be detected in the final residual map. The experimental results show that we achieved better performances than the state-of-the-art benchmarks using the publicly available MVTec dataset as well as a real-world equipment surface dataset. In addition, the method also demonstrates an excellent capability of repairing defects in abnormal samples.
Similar content being viewed by others
References
Ayush, H., Ankit, G., Trisha, P.: CAMLPAD: cybersecurity autonomous machine learning platform for anomaly detection. In: Advances in Information and Communication. Springer, Cham, pp. 705–720 (2020)
Bergmann, P., Fauser, M., Sattlegger, D., Steger, C.: Mvtec ad—a comprehensive real-world dataset for unsupervised anomaly detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9584–9592 (2019). https://doi.org/10.1109/CVPR.2019.00982
Bergmann, P., Löwe, S., Fauser, M., Sattlegger, D., Steger, C.: Improving unsupervised defect segmentation by applying structural similarity to autoencoders. CoRR (2018). arXiv:1807.02011
Böttger, T., Ulrich, M.: Real-time texture error detection on textured surfaces with compressed sensing. Pattern Recognit. Image Anal 26, 88–94 (2016). https://doi.org/10.1134/S1054661816010053
Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Warde-Farley, B.X.D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: NIPS’14. MIT Press, Cambridge, MA, USA, pp. 2672–2680 (2014)
Hu, G., Huang, J., Wang, Q., Li, J., Xu, Z., Huang, X.: Unsupervised fabric defect detection based on a deep convolutional generative adversarial network. Text. Res. J 90(3–4), 247–270 (2020). https://doi.org/10.1177/0040517519862880
Ibrahim, A., Ali, A., Esam, A., Raed, A., Mohamed, Z., Hua, M.: Ad-IoT: Anomaly detection of IoT cyberattacks in smart city using machine learning. In: 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0305–0310 (2019)
Liu, W., Li, R., Zheng, M., Karanam, S., Wu, Z., Bhanu, B., Radke, R.J., Camps, O.: Towards visually explaining variational autoencoders. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8639–8648 (2020). https://doi.org/10.1109/CVPR42600.2020.00867
Mei, S., Yang, H., Yin, Z.: An unsupervised-learning-based approach for automated defect inspection on textured surfaces. IEEE Trans. Instrum. Meas. 67(6), 1266–1277 (2018)
Mujeeb, A., Dai, W., Erdt, M., Sourin, A.: One class based feature learning approach for defect detection using deep autoencoders. Adv. Eng. Inform. 42, 100933 (2019). https://doi.org/10.1016/j.aei.2019.100933
Napoletano, P., Piccoli, F., Schettini, R.: Anomaly detection in nanofibrous materials by cnn-based self-similarity. Sensors 18(1), 209 (2018)
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: feature learning by inpainting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Samet, A., Amir, A.A., Toby, B.: GANomaly: semi-supervised anomaly detection via adversarial training. In: Computer Vision—ACCV 2018. Springer, Cham, pp. 622–637 (2019)
Satoshi, I., Edgar, S., Hiroshi, I.: Globally and locally consistent image completion. ACM Trans. Graph. 36(4), 3073659 (2017). https://doi.org/10.1145/3072959.3073659
Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Information Processing in Medical Imaging. Springer, Cham, pp. 146–157 (2017)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings (2015). arXiv:1409.1556
Steger, C.: Similarity measures for occlusion, clutter, and illumination invariant object recognition. In: Pattern Recognition. Springer Berlin Heidelberg, pp. 148–154 (2001)
Thomas, S., Philipp, S., Waldstein, S.M., Ursula, S.E., Georg, L.: Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Niethammer, M., Styner, M., Aylward, S., Zhu, H., Oguz, I., Yap, P.T., Shen, D. (eds.) Inf. Process. Med. Imaging, pp. 146–157. Springer, Cham (2017)
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)
Yeh, R.A., Chen, C., Lim, T.Y., H.-Johnson, M., Do, M.: Semantic image inpainting with perceptual and contextual losses. (2016) arXiv arXiv:1607.07539
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Generative image inpainting with contextual attention. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Zenati, H., Foo, C.S., Lecouat, B., Manek, G., Chandrasekhar, V.R.: Efficient gan-based anomaly detection. CoRR (2018) arXiv:1802.06222
Zhao, Z., Li, B., Dong, R., Zhao, P.: A surface defect detection method based on positive samples. In: PRICAI 2018: Trends in Artificial Intelligence. Springer, Cham, pp. 473–481 (2018)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Dai, W., Erdt, M. & Sourin, A. Detection and segmentation of image anomalies based on unsupervised defect reparation. Vis Comput 37, 3093–3102 (2021). https://doi.org/10.1007/s00371-021-02257-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-021-02257-5