Skip to main content

Multi-modal Quality Prediction Algorithm Based on Anomalous Energy Tracking Attention

  • Conference paper
  • First Online:
Advanced Intelligent Computing Technology and Applications (ICIC 2024)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14876))

Included in the following conference series:

  • 459 Accesses

Abstract

With the integration of manufacturing and the new generation of information technology, it becomes possible to utilize quality prediction technology instead of inspection technology. This paper focuses on achieving efficient product quality prediction by utilizing various relevant data from the product production process to overcome the limitations of manual quality inspection. Aiming to address the issues related to one-sided information of unimodal data and the close relationship between abnormal fluctuations in the production process and quality problems, a product quality prediction model called MAETAFormer is proposed. This model is based on the attention mechanism of anomalous energy tracking. It enhances the information of anomalous energy through the wavelet transform and a focusing module that targets sparse high-frequency anomalous fluctuations. The model further integrates the anomalous energy using an attention mechanism between the dual streams of time series data and image data. Finally, it deeply fuses the two modalities through the bilinear pooling module. The model is tested on the collected Strand Multimodal Dataset and demonstrates superior performance compared to advanced multimodal networks like LXMERT.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Chien, C., Trappey, A., Wang, C.: ARIMA-AdaBoost hybrid approach for product quality prediction in advanced transformer manufacturing. Adv. Eng. Inform. 57, 102055 (2023)

    Article  Google Scholar 

  2. Rehmer, A., Klute, M., Kroll, A., Heim, H.P.: An internal dynamics approach to predicting batch-end product quality in plastic injection molding using Recurrent Neural Networks. In: IEEE Conference on Control Technology and Applications, pp. 1427–1432. IEEE, Trieste (2022)

    Google Scholar 

  3. Ren, L., Meng, Z., Wang, X., Lu, R., Yang, L.T.: A wide-deep-sequence model-based quality prediction method in industrial process analysis. IEEE Trans. Neural Netw. Learn. Syst. 31(9), 3721–3731 (2020)

    Article  MathSciNet  Google Scholar 

  4. Vaswani, A., et al.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)

    Google Scholar 

  5. Zhang, G., Tang, Z., Fan, Y., Liu, J., Jahanshahi, H., Aly, A.A.: Steel wire rope surface defect detection based on segmentation template and spatiotemporal gray sample set. Sensors 21(16), 5401(2021)

    Google Scholar 

  6. Jiang, J., Jin, Z., Wang, B., Ma, L., Cui, Y.: A Sobel operator combined with patch statistics algorithm for fabric defect detection. KSII Trans. Internet Inf. Syst. 14(2), 687–701 (2020)

    Google Scholar 

  7. Qin, W., Wen, H., Li, F.: Fabric defect detection algorithm based on residual energy distribution and Gabor feature fusion. Vis. Comput. 39(11), 5971–5985 (2023)

    Article  Google Scholar 

  8. Cui, Y., Lu, S., Liu, S.: Real-time detection of wood defects based on SPP-improved YOLO algorithm. Multim. Tools Appl. 82(14), 21031–21044 (2023)

    Article  Google Scholar 

  9. Yang, J., Shi, Y., Qi, Z.: DFR: Deep feature reconstruction for unsupervised anomaly segmentation. Neural Comput. 424, 9–22 (2021)

    Google Scholar 

  10. Cohen, N., Hoshen, Y.: Sub-image anomaly detection with deep pyramid correspondences. Arxiv preprint arXiv:2005.02357 (2020)

    Google Scholar 

  11. Yu, Z., Yu, J., Fan, J., Tao, D.: Multi-modal factorized bilinear pooling with co-attention learning for visual question answering. In: IEEE International Conference on Computer Vision, pp.1821–1830. IEEE, Venice (2017)

    Google Scholar 

  12. Lu, J., Batra, D., Parikh, D., Lee, S.: Vilbert: pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Adv. Neural Inf. Process. Syst. 32 (2019)

    Google Scholar 

  13. Tan, H., Bansal, M.: Lxmert: learning cross-modality encoder representations from transformers. arXiv preprint arXiv:1908.07490 (2019)

  14. Chen, Y.C., et al.: Uniter: universal image-text representation learning. In: European Conference on Computer Vision, pp.104–120. Springer, Glasgow (2020)

    Google Scholar 

  15. Kim, W., Son, B., Kim, I.: Vilt: vision-and-language transformer without convolution or region supervision. In: International Conference on Machine Learning, pp. 5583–5594. PMLR (2021)

    Google Scholar 

Download references

Acknowledgments

The authors have no competing interests to declare that are relevant to the content of this article.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Qifei Zhang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, H., Zhang, Q., Li, W., Liang, X. (2024). Multi-modal Quality Prediction Algorithm Based on Anomalous Energy Tracking Attention. In: Huang, DS., Zhang, C., Pan, Y. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2024. Lecture Notes in Computer Science(), vol 14876. Springer, Singapore. https://doi.org/10.1007/978-981-97-5666-7_13

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-5666-7_13

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-5665-0

  • Online ISBN: 978-981-97-5666-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics