Skip to main content
SpringerLink
Log in

An effective automatic object detection algorithm for continuous sonar image sequences

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Object detection of continuous sonar image sequences has become an efficient way for underwater environment exploration. However, the task always suffers from the influence of th e complex underwater environment. In particular, the existing algorithms mainly focus on image-based detection and can not balance the detection speed and accuracy in continuous sonar image sequences. To solve the problem, this paper proposes a novel automatic detection algorithm based on deep learning for continuous sonar image sequences. Firstly, the convLSTM (convolution Long Short-Term Memory) is improved to fuse sonar features obtained from the cross-detection model, which are from three aspects: 1) The original convolution is replaced by depthwise separable convolution; 2) The original network input is divided into G groups and processed by group convolution; 3) A connection layer between the Bottleneck convolution layer and output is added to further capture feature information between frames. Then, to fully extract sonar features, a cross-detection network is established by fusing two different feature extraction networks MobileNetV3-Large and MobileNetV3-Small. Finally, we combine thecross-detection network with the improved convLSTM to establish the whole model, which can fully extract and utilize temporal information in continuous sonar image sequences. The experimental results show that the proposed model has effectively improved the detection speed in sonar image sequences at 150 FPS, simultaneously keeping an 85.8% mAP.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The availability of these data is limited, and they are used according to the license of the current study, so they are not publicly available.

References

  1. Bochkovskiy A, Wang C-Y, Mark Liao H-Y (2020) Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934

  2. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258

  3. Dura E, Zhang Y, Liao X, Dobeck GJ, Carin L (2005) Active learning for detection of mine-like objects in side-scan sonar imagery. IEEE J Ocean Eng 30(2):360–371

    Article  Google Scholar 

  4. Fan Z, Xia W, Liu X, Li H (2021) Detection and segmentation of underwater objects from forward-looking sonar based on a modified mask rcnn. Signal, Image and Video Processing, pp 1–9

    Google Scholar 

  5. Feichtenhofer C, Pinz A, Zisserman A (2017) Detect to track and track to detect. In Proceedings of the IEEE International Conference on Computer Vision, pp 3038–3046

  6. Guo J, Li Y, Lin W, Chen Y, Li J (2018) Network decoupling: From regular to depthwise separable convolutions. arXiv preprintar Xiv:1808.05517

  7. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  8. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

  9. Howard A, Sandler M (2019) Searching for mobilenetv3. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1314–1324

  10. Hurtós N, Palomeras N, Nagappa S, Salvi J (2013) Automatic detection of underwater chain links using a forward-looking sonar. In 2013 MTS/IEEE OCEANS-Bergen, pp 1–7

  11. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  12. Hu K, Wang Z (2019) Graph sequence recurrent neural network for vision-based freezing of gait detection. IEEE Trans Image Process 29:1890–1901

    Article  MathSciNet  Google Scholar 

  13. Jiang Z, Liu Y, Yang C (2020) Learning where to focus for efficient video object detection. In European Conference on Computer Vision, pp 18–34

  14. Kim J, Yu S-C (2016) Convolutional neural network-based real-time rov detection using forward-looking sonar image. In 2016 IEEE/OES Autonomous Underwater Vehicles (AUV), pp 396–400

  15. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  16. Liu M, Zhu M (2018) Mobile video object detection with temporally-aware feature maps. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5686–5695

  17. Maki T, Horimoto H (2020) Tracking a sea turtle by an auv with a multibeam imaging sonar: Toward robotic observation of marine life. Int J Control Autom Syst 18(3):597–604

    Article  Google Scholar 

  18. Ma Q, Jiang L, Yu W (2020) Training with noise adversarial network: A generalization method for object detection on sonar image. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp 729–738

  19. McKay J, Gerg I, Monga V, Raj RG (2017) What’s mine is yours: Pretrained cnns for limited training sonar atr. In OCEANS 2017-Anchorage, pp 1–7

  20. Myers V, Fawcett J (2010) A template matching procedure for automatic target recognition in synthetic aperture sonar imagery. IEEE Signal Process Lett 17(7):683–686

    Article  Google Scholar 

  21. Qin R, Zhao X, Zhu W (2021) Multiple receptive field network (mrf-net) for autonomous underwater vehicle fishing net detection using forward-looking sonar images. Sensors 21(6):1933

    Article  Google Scholar 

  22. Sung M, Kim J, Lee M (2020) Realistic sonar image simulation using deep learning for underwater object detection. Int J Control Autom Syst 18(3):523–534

    Article  Google Scholar 

  23. Tian M, Chen H, Wang Q (2019) Detection and recognition of flower image based on ssd network in video stream. J Physics: Conf Ser 1237:032045

    Google Scholar 

  24. Valdenegro-Toro M (2016) Objectness scoring and detection proposals in forward-looking sonar images with convolutional neural networks. In IAPR workshop on artificial neural networks in pattern recognition, pp 209–219

  25. Wang C-Y, Bochkovskiy A, Liao H-YM (2022) Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprintar Xiv:2207.02696

  26. Wang Y-X, Girshick R, Hebert M, Hariharan B (2018) Low-shot learning from imaginary data. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7278–7286

  27. Wang Z, Zhang S, Huang W, Guo J, Zeng L (2022) Sonar image target detection based on adaptive global feature enhancement network. IEEE Sens J 22(2):1509–1530

    Article  Google Scholar 

  28. Williams DP, Fakiris E (2014) Exploiting environmental information for improved underwater target classification in sonar imagery. IEEE Trans Geosci Remote Sens 52(10):6284–6297

    Article  Google Scholar 

  29. Wu S, Liu Y, Li S, Zhang S (2022) Lsh-detr: object detection algorithm for marine benthic organisms based on improved detr. J Electron Imaging 31(6):063030

    Article  Google Scholar 

  30. Zacchini L, Topini A, Franchi M, Secciani N, Manzari V, Bazzarello L, Stifani M, Ridolfi A (2022) Autonomous underwater environment perceiving and modeling: An experimental campaign with feelhippo auv for forward looking sonar-based automatic target recognition and data association. IEEE Journal of Oceanic Engineering, pp 1–20

  31. Zhou T, Si J, Wang L, Xu C, Yu X (2022) Automatic detection of underwater small targets using forward-looking sonar images. IEEE Trans Geosci Remote Sens 60:1–12

    Google Scholar 

  32. Zhou W, Wang Z (2021) Research on autonomous detection method of underwater unmanned vehicle. In 2021 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), pp 1–5

  33. Zhu X, Xiong Y (2017) Deep feature flow for video recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2349–2358

Download references

Acknowledgements

This work has been supported by the National Key R&D Program of China (2022YFB4703405) and the Fundamental Research Fund for the Central Universities (B220202020).

Author information

Author notes

  1. Huanru Sun, Xinnan Fan and Qi He contributed equally to this work.

Authors and Affiliations

  1. College of Internet of Things Engineering, HoHai University, Jinling North Road, Jiangsu, 213022, Changzhou, China

    Pengfei Shi, Huanru Sun, Xinnan Fan, Qi He, Xuan Zhou & Liang Lu

Authors
  1. Pengfei Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huanru Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinnan Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qi He
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xuan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Liang Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinnan Fan.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, P., Sun, H., Fan, X. et al. An effective automatic object detection algorithm for continuous sonar image sequences. Multimed Tools Appl 83, 10233–10246 (2024). https://doi.org/10.1007/s11042-023-15837-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15837-5

Keywords

Search

Navigation