Skip to main content
SpringerLink
Log in

Occlusion aware underwater object tracking using hybrid adaptive deep SORT -YOLOv3 approach

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Underwater object tracking and recognition are challenging due to the distinctive characteristics of underwater environments. The water medium exhibits diffraction and scattering of light when it travels deep in the water. This results in unclear, occluded videos and images further causing challenges in interpretation. Tracking the object of interest from the consecutive frames in underwater scenarios often lead to occlusion of objects. Addressing these issues, an improved hybrid adaptive deep SORT-YOLOv3 (HADSYv3) detection and tracking scheme for occluded underwater objects is proposed. The neural network-based training model YOLOv3 is applied to extract and categorize the underwater object. An adaptive deep SORT algorithm with the long-short term memory (LSTM) based deep learning approach is used to determine the position of the objects in the underwater sequences. The proposed Hybrid Adaptive DeepSORT-YOLOv3 (HADSYv3) method incorporates both the YOLOv3 algorithm meant for object detection and the adaptive deep SORT algorithm for tracking applications. Though the deep SORT algorithm track objects in real time applications standalone, if it gets combined with suitable detection schemes, the overall efficiency can be improved by confirming all the possible detections. The proposed method is compared with other state of art underwater object recognition schemes and the occluded object detection for various view angles is evaluated quantitatively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Blanc K, Lingrand D, Precioso F (2014) Fish species recognition from video using SVM classifier. In: 3rd ACM International Workshop on Multimedia Analysis for Ecological Data, ACM, 1– 6

  2. Carlevaris-Bianco N, Mohan A, Eustice RM (2010) September. Initial results in underwater single image dehazing. In: Oceans 2010 Mts/IEEE Seattle. IEEE, pp 1–8. https://doi.org/10.1109/OCEANS.2010.5664428

  3. Dai J, Wang R, Zheng H, Ji G, Qiao X (2016) ZooplanktonNet: Deep convolutional network for zooplankton classification. OCEANS 2016 - Shanghai, IEEE. https://doi.org/10.1109/OCEANSAP.2016.7485680

  4. Dhanalakshmi S, Venkatesh C (2013) Classification of ultrasound carotid artery images using texture features. International Review on Computers and Software (IRECOS) 8(4)933–940

  5. Duan K, Keerthi SS (2005) Which is the best multiclass SVM method? Proceedings of the Sixth International Workshop on Multiple Classifier Systems, p 278– 285

  6. Fish for Knowledge data set. Available on http://f4k.dieei.unict.it/datasets/bkgmodeling/. Accessed 1 Oct  2010 - 10 Jul 2013

  7. Hou G-J, Luan X, Song§ D-L, Xue-Yan M (2015) Underwater Man-made Object Recognition on the basis of color and shape features. J Coastal Res Coastal Res Foundation. https://doi.org/10.2112/JCOASTRES-D-14-00249.1

    Article  Google Scholar 

  8. Hsiao Y, Chen C, Lin S, Lin F (2014)Real-world underwater fish recognition and identification using sparse representation. Ecol Inf 23:13–21. https://doi.org/10.1016/j.ecoinf.2013.10.002

    Article  Google Scholar 

  9. Hu W, Li X, Luo W, Zhang X, Maybank S, Zhang Z (2012) Single and multiple object tracking using log-Euclidean Riemannian subspace and block-division appearance model. IEEE Trans Pattern Anal Mach Intell 34(12):2420–2440

    Article  Google Scholar 

  10. Kim B, Yu S (2017) Imaging sonar based real-time underwater object detection utilizing AdaBoost method. IEEE Underwater Technology (UT), Busan, pp 1–5. https://doi.org/10.1109/UT.2017.7890300

  11. Lee D, Kim G, Kim D, Myung H, Choi H-T(2012)Vision-based object detection and tracking for autonomous navigation of underwater robots. Ocean Eng 48:59–68. ACM. https://doi.org/10.1016/j.oceaneng.2012.04.006

  12. Lee H, Park M, Kim J (2016) Plankton classification on imbalanced large scale database via convolutional neural networks with transfer learning. 2016 IEEE International Conference on Image Processing (ICIP). https://doi.org/10.1109/ICIP.2016.7533053

  13. Li X, Shang M, Qin H, Chen L (2015) Fast accurate fish detection and recognition of Un-derwater images with fast R-CNN. OCEANS 2015 -MTS/IEEE Washington. IEEE Press, Washington, DC, pp 1–5

    Google Scholar 

  14. Li Y, Lu H, Li J, Li X, Li Y, Serikawa S (2016) Underwater image de-scattering and classification by deep neural network. Comput Electr Eng 54:68–77, ISSN 0045-7906. https://doi.org/10.1016/j.compeleceng.2016.08.008

  15. Mahmood A, Bennamoun M, An S, Sohel F, Boussaid F, Hovey R, Kendrick G, Fisher RB (2016) Coral Classification with hybrid feature representations. 2016 IEEE International Conference on Image Processing (ICIP). https://doi.org/10.1109/ICIP.2016.7532411

  16. Mathias A, Samiappan D (2019) Underwater image restoration based on diffraction bounded optimization algorithm with dark channel prior. Optik 192:162925

    Article  Google Scholar 

  17. Mathias A, Dhanalakshmi S, Kumar R, Narayanamoorthi R (2021) Underwater object detection based on bi-dimensional empirical mode decomposition and Gaussian Mixture Model approach. Ecol Inform 66:101469

  18. Mathias A, Dhanalakshmi S, Kumar R, Narayanamoorthi R (2022) Deep neural network driven automated underwater object detection. Comput Mater Continua 70(3):5251–5267

    Article  Google Scholar 

  19. Nikolovska (2015) AUV based flushed and buried object detection. OCEANS 2015 - Genova, Genoa, pp 1–5. https://doi.org/10.1109/OCEANS-Genova.2015.7271651

  20. NOAA dataset available on https://videos.fisheries.noaa.gov/. Accessed 2013

  21. Pan Y, He F, Yu H (2019) A novel enhanced collaborative autoencoder with knowledge distillation for top-N recommender systems. Neurocomputing 332:137–148

    Article  Google Scholar 

  22. Pan Y, He F, Yu H (2020) Learning social representations with deep autoencoder for recommender system. World Wide Web 23(4):2259–2279

    Article  Google Scholar 

  23. Quan Q, He F, Li H (2021) A multi-phase blending method with incremental intensity for training detection networks. Vis Comput 37(2):245–259

    Article  Google Scholar 

  24. Ravanbakhsh M, Shortis M, Shafait F, Mian A, Harvey E, Seager J (2015) Automated fish detection in underwater images using shape-based level sets. Photogram Rec 30(149):46–62

    Article  Google Scholar 

  25. Samiappan D, Chakrapani V (2016) Classification of carotid artery abnormalities in ultrasound images using an artificial neural classifier. Int Arab J Inf Technol 13(6A):756–762

    Google Scholar 

  26. Schechner YY, Karpel N (2005) Recovery of underwater visibility and structure by polarization analysis. IEEE J Oceanic Eng 30(3):570–587. https://doi.org/10.1109/JOE.2005.850871

    Article  Google Scholar 

  27. Song B, Jeng TY, Staudt E, Roy-Chowdhury AK (2010) A stochastic graph evolution framework for robust multi-target tracking. In: European Conference on Computer Vision. Springer, Berlin, pp 605–619

  28. Vasamsetti S, Setia S, Mittal N, Sardana HK, Babbar G (2018) Automatic underwater moving object detection using multi-feature integration framework in complex backgrounds. IET Comput Vis 12(6):770–778, The Institution of Engineering and Technology 2018

  29. Wojke N, Bewley A, Paulus D (2017) Simple online and realtime tracking with a deep association metric. In: 2017 IEEE international conference on image processing (ICIP). IEEE, pp 3645–3649

  30. Wright J, Yang AY, Ganesh A (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 23:210–227. https://doi.org/10.1109/TPAMI.2008.79

    Article  Google Scholar 

  31. Yafei Zhu L, Chang J, Dai H, Zheng BZ (2016) Automatic object detection and segmentation from underwater images via saliency-based region merging. OCEANS 2016 - Shanghai, IEEE. https://doi.org/10.1109/OCEANSAP.2016.7485598

  32. Yang H, Ping L, Hu YZ, Fu JN. Research on underwater object recognition based on YOLOv3” Microsystem Technologies. https://doi.org/10.1007/s00542-019-04694-8

  33. Zhang L, van der Maaten L (2013) Structure preserving object tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1838–1845

  34. Zhang S, He F (2020) DRCDN: learning deep residual convolutional dehazing networks. Vis Comput 36(9):1797–1808

    Article  Google Scholar 

  35. Zhou H, Llewellyn L, Wei L, Creighton D, Nahavandi S (2015) Marine object detection using background Modellingblob analysis. 2015 IEEE International Conference on Systems, Man, Cybernetics, IEEE. https://doi.org/10.1109/SMC.2015.86

  36. Zu Y, Ma J, Tian J, Liu H, Yun, Zhang (2014) A gravity gradient differential ratio method for underwater object detection. IEEE Geosci Remote Sens Lett 11. https://doi.org/10.1109/LGRS.2013.2279485

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, College of Engineering and Technology, SRM Institute of Science and Technology SRM Nagar, Kattankulathur, Kanchipuram, Chennai , TN, 603203, India

    Ajisha Mathias, Samiappan Dhanalakshmi & R. Kumar

Authors
  1. Ajisha Mathias
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samiappan Dhanalakshmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Samiappan Dhanalakshmi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mathias, A., Dhanalakshmi, S. & Kumar, R. Occlusion aware underwater object tracking using hybrid adaptive deep SORT -YOLOv3 approach. Multimed Tools Appl 81, 44109–44121 (2022). https://doi.org/10.1007/s11042-022-13281-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13281-5

Keywords

Search

Navigation