Skip to main content
SpringerLink
Log in

Accelerating a Meta Learning Model for Ultrasonic Non-Destructive Testing Applications Using Model Compression and FPGA Hardware

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

Non-Destructive Evaluation/Testing (NDE/NDT) is comprised of advanced sensor technologies that can evaluate structures, materials and components for defects and analyze their properties. In recent years, researchers have been applying deep learning algorithms on NDT technologies to improve the capability of detecting and classifying complex sensor data. However, deep learning models often require large computational resources including specialized hardware accelerators, dedicated memory blocks and increased power consumption. It is very challenging to implement these deep learning algorithms in real-time testing scenarios in the field due to limited access to aforementioned computational resources. To address this issue, we introduce a model compression algorithm and the corresponding Field Programmable Gate Array (FPGA) accelerators for a novel deep learning model targeting ultrasonic NDT techniques. The ultrasonic deep learning algorithm which is based on Meta Learning is capable of detecting and classifying different flaw types (e.g. cracks, holes) within the specimen. The results have shown that the model compression has significantly reduced the required operations with minimal accuracy loss, and the low-cost FPGA hardware platform is able to accelerate the inference using compressed model with high efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13

Similar content being viewed by others

Data Availability

Data that support the findings of this study are available upon reasonable request from the corresponding author (E.O.).

References

  1. Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86, 2278–2324. https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  2. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 1097–1105.

    Google Scholar 

  3. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. https://arxiv.org/pdf/1706.03762.pdf

  4. Pyle, R. J., Bevan, R. L. T., Hughes, R. R., Rachev, R. K., Ali, A. A. S., & Wilcox, P. D. (2021). Deep learning for ultrasonic crack characterization in NDE. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 68, 1854–1865. https://doi.org/10.1109/TUFFC.2020.3045847

    Article  PubMed  Google Scholar 

  5. Cantero-Chinchilla, S., Wilcox, P. D., & Croxford, A. J. (2022). Deep learning in automated ultrasonic NDE–developments, axioms and opportunities. NDT & E International, 102703.

  6. Meng, M., Chua, Y. J., Wouterson, E., & Ong, C. P. K. (2017). Ultrasonic signal classification and imaging system for composite materials via deep convolutional neural networks. Neurocomputing, 257, 128–135.

    Article  Google Scholar 

  7. Ye, J., Ito, S., & Toyama, N. (2018). Computerized ultrasonic imaging inspection: from shallow to deep learning. Sensors, 18, 3820.

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  8. Amiri, N., Farrahi, G., Kashyzadeh, K. R., & Chizari, M. (2020). Applications of ultrasonic testing and machine learning methods to predict the static & fatigue behavior of spot-welded joints. Journal of Manufacturing Processes, 52, 26–34.

    Article  Google Scholar 

  9. Virupakshappa, K., Marino, M., & Oruklu, E. (2018). A multi-resolution convolutional neural network architecture for ultrasonic flaw detection. In: 2018 IEEE International Ultrasonics Symposium (IUS) (pp. 1–4). https://doi.org/10.1109/ULTSYM.2018.8579888

  10. Farabet, C., Poulet, C., Han, J. Y., & LeCun, Y. (2009). CNP: An FPGA-based processor for convolutional networks. In 2009 International Conference on Field Programmable Logic and Applications (pp. 32–37).

  11. Lisp Universal SHell. (2002). http://lush.sourceforge.net/

  12. Lian, X., Liu, Z., Song, Z., Dai, J., Zhou, W., & Ji, X. (2019). High-performance FPGA-based CNN accelerator with block-floating-point arithmetic. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 27, 1874–1885. https://doi.org/10.1109/TVLSI.2019.2913958

  13. Moss, D. J., Krishnan, S., Nurvitadhi, E., Ratuszniak, P., Johnson, C., Sim, J., Mishra, A., Marr, D., Subhaschandra, S., & Leong, P. H. (2018). A customizable matrix multiplication framework for the Intel HARPv2 Xeon+ FPGA platform: A deep learning case study. In Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (pp. 107–116).

  14. Kala, S., Jose, B. R., Mathew, J., & Nalesh, S. (2019). High-performance CNN Accelerator on FPGA Using Unified Winograd-GEMM Architecture. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 27, 2816–2828.

  15. AlBdairi, A. J. A., Xiao, Z., Alkhayyat, A., Humaidi, A. J., Fadhel, M. A., Taher, B. H., Alzubaidi, L., Santamaría, J., & Al-Shamma, O. (2022). Face recognition based on deep learning and FPGA for ethnicity identification. Applied Sciences, 12, 2605.

  16. He, D., He, J., Liu, J., Yang, J., Yan, Q., & Yang, Y. (2021). An FPGA-based LSTM acceleration engine for deep learning frameworks. Electronics, 10, 681.

  17. Yuan, Y., Virupakshappa, K., & Oruklu, E. (2022). FPGA implementation of an ultrasonic flaw detection algorithm based on convolutional neural networks. Journal of Signal Processing Systems, 94, 1447–1457.

    Article  Google Scholar 

  18. Virupakshappa, K., & Oruklu, E. (2021). Localization of ultrasonic flaws using grid based deep learning. In 2021 IEEE International Ultrasonics Symposium (IUS) (pp. 1–3). https://doi.org/10.1109/IUS52206.2021.9593753

  19. OnScale Solve. (2023). Retrieved March 3, 2022, from https://onscale.com

  20. Treeby, B. E., Budisky, J., Wise, E. S., Jaros, J., & Cox, B. T. (2018). Rapid calculation of acoustic fields from arbitrary continuous-wave sources. Journal of the Acoustical Society of America, 143, 529–537.

    Article  ADS  PubMed  Google Scholar 

  21. Virupakshappa, K., & Oruklu, E. (2019). Multi-class classification of defect types in ultrasonic NDT signals with convolutional neural networks. In 2019 IEEE International Ultrasonics Symposium (IUS) (pp. 1647–1650).

  22. Koch, G., Zemel, R., Salakhutdinov, R., et al. (2015). Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop (Vol. 2, p. 0). Lille.

  23. Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P. H. S., & Hospedales, T. M. (2017). Learning to compare: Relation network for few-shot learning. CoRR abs/1711.06025. http://arxiv.org/abs/1711.06025

  24. Chicco, D. (2021). Siamese neural networks: An overview. Artificial Neural Networks, 73–94.

  25. LeNail, A. (2019). NN-SVG: Publication-ready neural network architecture schematics. Journal of Open Source Software, 4, 747.

    Article  ADS  Google Scholar 

  26. Zhang, M., Li, L., Wang, H., Liu, Y., Qin, H., & Zhao, W. (2019). Optimized compression for implementing convolutional neural networks on FPGA. Electronics, 8. https://doi.org/10.3390/electronics8030295. https://www.mdpi.com/2079-9292/8/3/295

  27. Zhou, Y., Zhang, Y., Wang, Y., & Tian, Q. (2019). Accelerate CNN via recursive bayesian pruning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).

  28. Huang, Q., Zhou, K., You, S., & Neumann, U. (2018). Learning to prune filters in convolutional neural networks. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 709–718). https://doi.org/10.1109/WACV.2018.00083

  29. Han, S., Pool, J., Tran, J., & Dally, W. (2015). Learning both weights and connections for efficient neural network. Advances in Neural Information Processing Systems, 28.

  30. Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015). Deep learning with limited numerical precision. In International Conference On Machine Learning (pp. 1737–1746). PMLR.

  31. Guo, K., Sui, L., Qiu, J., Yu, J., Wang, J., Yao, S., Han, S., Wang, Y., & Yang, H. (2018). Angel-eye: A complete design flow for mapping CNN onto embedded FPGA. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 37, 35–47. https://doi.org/10.1109/TCAD.2017.2705069

    Article  Google Scholar 

  32. Yuan, Y., Virupakshappa, K., & Oruklu, E. (2022). Model compression and FPGA implementation of an ultrasonic flaw detection algorithm based on meta learning. In 2022 IEEE International Ultrasonics Symposium (IUS) (pp. 1–4). https://doi.org/10.1109/IUS54386.2022.9958305

  33. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., & Zheng, X. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. https://www.tensorflow.org/, software available from tensorflow.org.

  34. Huang, S., Pearson, C., Nagi, R., Xiong, J., Chen, D., & Hwu, W.-M. (2019). Accelerating sparse deep neural networks on FPGAs. In 2019 IEEE High Performance Extreme Computing Conference (HPEC) (pp. 1–7). https://doi.org/10.1109/HPEC.2019.8916419

  35. Peng, H., Huang, S., Geng, T., Li, A., Jiang, W., Liu, H., Wang, S., & Ding, C. (2021). Accelerating transformer-based deep learning models on FPGAs using column balanced block pruning. In 2021 22nd International Symposium on Quality Electronic Design (ISQED) (pp. 142–148). https://doi.org/10.1109/ISQED51717.2021.9424344

  36. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.

  37. Alveo U200 Data Center Accelerator Card. (2018). Retrieved March 7, 2022, from https://www.xilinx.com/products/boards-and-kits/alveo/u200.html

  38. Intel®Core™i5-5257U Processor. (2015). Retrieved March 7, 2022, from https://ark.intel.com/content/www/us/en/ark/products/84985/intel-core-i55257u-processor-3m-cache-up-to-3-10-ghz.html

  39. Lu, L., Xie, J., Huang, R., Zhang, J., Lin, W., & Liang, Y. (2019). An efficient hardware accelerator for sparse convolutional neural networks on FPGAs. In 2019 IEEE 27th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) (pp. 17–25). https://doi.org/10.1109/FCCM.2019.00013

  40. Deep learning with int8 optimization on Xilinx devices white paper (wp485). (2017). Retrieved October 11, 2022, from https://www.xilinx.com/support/documentation/whitepapers/wp486-deep-learning-int8.pdf

  41. Zedboard. (2020). Retrieved June 14, 2020, from http://zedboard.org/product/zedboard

  42. Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, M., Wilson, J., Millman, K. J., Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R., Larson, E., … van Mulbregt, P. (2020). SciPy 1.0 Contributors, SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods, 17, 261–272. https://doi.org/10.1038/s41592-019-0686-2

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Li, S., Wen, W., Wang, Y., Han, S., Chen, Y., & Li, H. (2017). An FPGA design framework for CNN sparsification and acceleration. In 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) (pp. 28–28). IEEE.

  44. Guan, Y., Liang, H., Xu, N., Wang, W., Shi, S., Chen, X., Sun, G., Zhang, W., & Cong, J. (2017). FP-DNN: An automated framework for mapping deep neural networks onto FPGAs with RTL-HLS hybrid templates. In 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) (pp. 152–159). IEEE.

  45. Jiang, C., Ojika, D., Patel, B., & Lam, H. (2021). Optimized FPGA-based deep learning accelerator for sparse CNN using high bandwidth memory. In 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) (pp. 157–164). https://doi.org/10.1109/FCCM51124.2021.00026

Download references

Funding

No funding was received for conducting this study.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Illinois Institute of Technology, 3301 South Dearborn Street, Chicago, 60616, IL, USA

    Yu Yuan, Kushal Virupakshappa & Erdal Oruklu

Authors
  1. Yu Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kushal Virupakshappa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erdal Oruklu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yu Yuan is primarly responsible for the hardware realization and model compression methods. Kushal Virupakshappa is responsible for the development of the deep learning algorithm for ultrasonic NDT. Erdal Oruklu has contributed to hardware & software design, algorithm development, analysis and verification. The first draft of the manuscript was written by Yu Yuan and all authors read and approved the final manuscript.

Corresponding author

Correspondence to Erdal Oruklu.

Ethics declarations

Conflict of Interest

This study did not include any research involving human participants and/or animals. Authors have no financial or proprietary interests in any material discussed in this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, Y., Virupakshappa, K. & Oruklu, E. Accelerating a Meta Learning Model for Ultrasonic Non-Destructive Testing Applications Using Model Compression and FPGA Hardware. J Sign Process Syst 96, 15–29 (2024). https://doi.org/10.1007/s11265-023-01901-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-023-01901-8

Keywords

Search

Navigation