Wenzheng Li
ABSTRACT
In industrial production processes, defect inspection plays an important role in reducing the occurrence of failures and improving production efficiency. Data-driven algorithms represented by deep learning have made great progress in recent years, but need to face the problems of small quantity and poor quality of datasets when applied to industrial defect inspection. This paper proposes a layer mask blending-based generative adversarial network (LMBGAN) and optimizes the training process to generate high-quality surface defect samples. LMBGAN generates defect images and layer masks using the defect image decoder and layer mask decoder with the Pixel Shuffle operation. Inspired by the layer mask in computer painting, LMBGAN adopts the input image as the base layer and blends the defect foreground through the layer mask, giving it the ability to focus more on generating upper-layer defect images and reducing unnecessary background changes. LMBGAN additionally introduces adaptive discriminator augmentation and non-saturating logistic loss to promote model convergence under small datasets, effectively alleviating the problem of GAN training difficulties with limited data. The experiment results show that the proposed method can generate high-quality and diverse defect image samples through easily accessible normal samples, thus reducing the difficulty of obtaining rare defect image samples.
- Teo, T. W., & Abdullah, M. Z. (2018). Solar cell micro-crack detection using localised texture analysis. Journal of Image and Graphics, 6(1), 54-58.Google Scholar
Cross Ref
- Zhou Y. Research on Image-Based Automatic Wafer Surface Defect Detection Algorithm[J]. Journal of Image and Graphics, 2019, 7(1): 26-31.Google ScholarDigital Library
- Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009, June). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition (pp. 248-255). Ieee.Google ScholarCross Ref
- Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., ... & Zitnick, C. L. (2014, September). Microsoft coco: Common objects in context. In European conference on computer vision (pp. 740-755). Springer, Cham.Google ScholarCross Ref
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2020). Generative adversarial networks. Communications of the ACM, 63(11), 139-144.Google ScholarDigital Library
- Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., & Aila, T. (2020). Training generative adversarial networks with limited data. Advances in Neural Information Processing Systems, 33, 12104-12114.Google Scholar
- Gui, J., Sun, Z., Wen, Y., Tao, D., & Ye, J. (2021). A review on generative adversarial networks: Algorithms, theory, and applications. IEEE Transactions on Knowledge and Data Engineering.Google Scholar
- Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., ... & Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1874-1883).Google ScholarCross Ref
- Mescheder, L., Geiger, A., & Nowozin, S. (2018, July). Which training methods for GANs do actually converge?. In International conference on machine learning (pp. 3481-3490). PMLR.Google Scholar
- He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).Google ScholarCross Ref
- Tan, M., & Le, Q. (2019, May). Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning (pp. 6105-6114). PMLR.Google Scholar
- Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 779-788).Google ScholarCross Ref
- Li, Z., Tian, X., Liu, X., Liu, Y., & Shi, X. (2022). A two-stage industrial defect detection framework based on improved-yolov5 and optimized-inception-resnetv2 models. Applied Sciences, 12(2), 834.Google ScholarCross Ref
- Jing, J., Wang, Z., Rätsch, M., & Zhang, H. (2022). Mobile-Unet: An efficient convolutional neural network for fabric defect detection. Textile Research Journal, 92(1-2), 30-42.Google ScholarCross Ref
- Karras, T., Laine, S., & Aila, T. (2019). A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4401-4410).Google ScholarCross Ref
- Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020). Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 8110-8119).Google ScholarCross Ref
- Karras, T., Aittala, M., Laine, S., Härkönen, E., Hellsten, J., Lehtinen, J., & Aila, T. (2021). Alias-free generative adversarial networks. Advances in Neural Information Processing Systems, 34, 852-863.Google Scholar
- Park, T., Liu, M. Y., Wang, T. C., & Zhu, J. Y. (2019). Semantic image synthesis with spatially-adaptive normalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 2337-2346).Google ScholarCross Ref
- Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision (pp. 2223-2232).Google ScholarCross Ref
- Choi, Y., Choi, M., Kim, M., Ha, J. W., Kim, S., & Choo, J. (2018). Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 8789-8797).Google ScholarCross Ref
- Choi, Y., Uh, Y., Yoo, J., & Ha, J. W. (2020). Stargan v2: Diverse image synthesis for multiple domains. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 8188-8197).Google ScholarCross Ref
- Han, J., Shoeiby, M., Petersson, L., & Armin, M. A. (2021). Dual contrastive learning for unsupervised image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 746-755).Google ScholarCross Ref
- Sandfort, V., Yan, K., Pickhardt, P. J., & Summers, R. M. (2019). Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks. Scientific reports, 9(1), 1-9.Google Scholar
- Niu, S., Li, B., Wang, X., & Lin, H. (2020). Defect image sample generation with GAN for improving defect recognition. IEEE Transactions on Automation Science and Engineering, 17(3), 1611-1622.Google Scholar
- Zhang, G., Cui, K., Hung, T. Y., & Lu, S. (2021). Defect-GAN: High-fidelity defect synthesis for automated defect inspection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 2524-2534).Google ScholarCross Ref
- Ikeda, Y., Doman, K., Mekada, Y., & Nawano, S. (2021). Lesion image generation using conditional GAN for metastatic liver cancer detection. Journal of Image and Graphics, 9(1), 27-30.Google ScholarCross Ref
- Bora, A., Price, E., & Dimakis, A. G. (2018, February). AmbientGAN: Generative models from lossy measurements. In International conference on learning representations.Google Scholar
- Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30.Google Scholar
Index Terms
- Image Sample Generation of Stator Surface Defects Based on Layer Mask Blending Generative Adversarial Network
Recommendations
Normal Image Generation-Based Defect Detection by Generative Adversarial Network with Chaotic Random Images
Advances in Visual ComputingAbstractWe propose a defect detection method called ChaosGAN (Generative Adversarial Network with Chaotic Random Images) for image generation that can output a normal image with high reconstruction performance regardless of whether the input image is ...
A Method for Face Image Inpainting Based on Autoencoder and Generative Adversarial Network
Image and Video TechnologyMask-guided network for image captioning
AbstractAttention mechanisms have been widely adopted for image captioning because of their powerful performance. In this paper, we propose a Mask Captioning Network (MaC) consisting of an object layer and a background layer to capture the ...
Highlights- This paper proposes a new image captioning model consisting of the mask and scene layers.
Comments