Skip to main content
Log in

Detection and segmentation of image anomalies based on unsupervised defect reparation

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Anomaly detection is a challenging task in the field of data analysis, especially when it comes to unsupervised pixel-level segmentation of anomalies in images. In this paper, we present a novel multi-stage image resynthesis framework for detecting and segmenting image anomalies. In contrast to existing reconstruction-based approaches, our method is based on repairing suspicious regions of defective images so that the defects can be localized in the residual map between inputs and the repaired outputs. To avoid the reconstruction artifacts caused by defects, we propose to generate each pixel of the image by its context in the first coarse reconstruction stage. Then, while excluding all safe pixels, our method repairs suspicious regions that have large deviations to the original input image in subsequent stages. After several iterations, the defects will be detected in the final residual map. The experimental results show that we achieved better performances than the state-of-the-art benchmarks using the publicly available MVTec dataset as well as a real-world equipment surface dataset. In addition, the method also demonstrates an excellent capability of repairing defects in abnormal samples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ayush, H., Ankit, G., Trisha, P.: CAMLPAD: cybersecurity autonomous machine learning platform for anomaly detection. In: Advances in Information and Communication. Springer, Cham, pp. 705–720 (2020)

  2. Bergmann, P., Fauser, M., Sattlegger, D., Steger, C.: Mvtec ad—a comprehensive real-world dataset for unsupervised anomaly detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9584–9592 (2019). https://doi.org/10.1109/CVPR.2019.00982

  3. Bergmann, P., Löwe, S., Fauser, M., Sattlegger, D., Steger, C.: Improving unsupervised defect segmentation by applying structural similarity to autoencoders. CoRR (2018). arXiv:1807.02011

  4. Böttger, T., Ulrich, M.: Real-time texture error detection on textured surfaces with compressed sensing. Pattern Recognit. Image Anal 26, 88–94 (2016). https://doi.org/10.1134/S1054661816010053

    Article  Google Scholar 

  5. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Warde-Farley, B.X.D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: NIPS’14. MIT Press, Cambridge, MA, USA, pp. 2672–2680 (2014)

  6. Hu, G., Huang, J., Wang, Q., Li, J., Xu, Z., Huang, X.: Unsupervised fabric defect detection based on a deep convolutional generative adversarial network. Text. Res. J 90(3–4), 247–270 (2020). https://doi.org/10.1177/0040517519862880

    Article  Google Scholar 

  7. Ibrahim, A., Ali, A., Esam, A., Raed, A., Mohamed, Z., Hua, M.: Ad-IoT: Anomaly detection of IoT cyberattacks in smart city using machine learning. In: 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0305–0310 (2019)

  8. Liu, W., Li, R., Zheng, M., Karanam, S., Wu, Z., Bhanu, B., Radke, R.J., Camps, O.: Towards visually explaining variational autoencoders. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8639–8648 (2020). https://doi.org/10.1109/CVPR42600.2020.00867

  9. Mei, S., Yang, H., Yin, Z.: An unsupervised-learning-based approach for automated defect inspection on textured surfaces. IEEE Trans. Instrum. Meas. 67(6), 1266–1277 (2018)

    Article  Google Scholar 

  10. Mujeeb, A., Dai, W., Erdt, M., Sourin, A.: One class based feature learning approach for defect detection using deep autoencoders. Adv. Eng. Inform. 42, 100933 (2019). https://doi.org/10.1016/j.aei.2019.100933

    Article  Google Scholar 

  11. Napoletano, P., Piccoli, F., Schettini, R.: Anomaly detection in nanofibrous materials by cnn-based self-similarity. Sensors 18(1), 209 (2018)

    Article  Google Scholar 

  12. Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: feature learning by inpainting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  13. Samet, A., Amir, A.A., Toby, B.: GANomaly: semi-supervised anomaly detection via adversarial training. In: Computer Vision—ACCV 2018. Springer, Cham, pp. 622–637 (2019)

  14. Satoshi, I., Edgar, S., Hiroshi, I.: Globally and locally consistent image completion. ACM Trans. Graph. 36(4), 3073659 (2017). https://doi.org/10.1145/3072959.3073659

    Article  Google Scholar 

  15. Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Information Processing in Medical Imaging. Springer, Cham, pp. 146–157 (2017)

  16. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings (2015). arXiv:1409.1556

  17. Steger, C.: Similarity measures for occlusion, clutter, and illumination invariant object recognition. In: Pattern Recognition. Springer Berlin Heidelberg, pp. 148–154 (2001)

  18. Thomas, S., Philipp, S., Waldstein, S.M., Ursula, S.E., Georg, L.: Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Niethammer, M., Styner, M., Aylward, S., Zhu, H., Oguz, I., Yap, P.T., Shen, D. (eds.) Inf. Process. Med. Imaging, pp. 146–157. Springer, Cham (2017)

  19. Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)

    MathSciNet  MATH  Google Scholar 

  20. Yeh, R.A., Chen, C., Lim, T.Y., H.-Johnson, M., Do, M.: Semantic image inpainting with perceptual and contextual losses. (2016) arXiv arXiv:1607.07539

  21. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Generative image inpainting with contextual attention. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

  22. Zenati, H., Foo, C.S., Lecouat, B., Manek, G., Chandrasekhar, V.R.: Efficient gan-based anomaly detection. CoRR (2018) arXiv:1802.06222

  23. Zhao, Z., Li, B., Dong, R., Zhao, P.: A surface defect detection method based on positive samples. In: PRICAI 2018: Trends in Artificial Intelligence. Springer, Cham, pp. 473–481 (2018)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Wenting Dai.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dai, W., Erdt, M. & Sourin, A. Detection and segmentation of image anomalies based on unsupervised defect reparation. Vis Comput 37, 3093–3102 (2021). https://doi.org/10.1007/s00371-021-02257-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02257-5

Keywords

Navigation