Skip to main content

Advertisement

SpringerLink
Log in

Crosspooled FishNet: transfer learning based fish species classification model

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Fish species classifiction is an important task for biologists and marine ecologists to frequently estimate the relative abundance of fish species in their natural habitats and monitor changes in their populations. Traditional methods used for fish species classifiction were laboriuos, time consuming and expensive. So, there is need for an automated system that can not only detect and track but also categorize fish as well as other aquatic species in underwater imagery, minimizing the manual interference. Absorption and scattering of light in deep sea environment leads to low resolution images making fish species recognition and classification a challenging task. Further, performance of traditional computer vision techniques tends to degrade in underwater conditions due to the presence of high background clutter and highly indistinct features of marine species. For such classification problems, Artificial Neural Networks (ANN) or deep neural network are being increasingly employed for improved performance. But the limited dataset of fish images makes it difficult to train such networks as they require huge datasets for training. Thus to reduce the requirement for a huge amount of training data, an algorithm using cross convolutional layer pooling on a pre-trained Convolutional Neural Networks (CNN) is proposed. The present paper focuses on the development of automatic system for classification, which can detect and classify fish from underwater images captured through videos. Thorough analysis on image dataset of 27,370 fish images gives a validation accuracy of 98.03%. The proposed method will be an efficient replacement to strenuous and time consuming method of manual recognition by marine experts and thus be advantageous for monitoring fish biodiversity in their natural habitats.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Allken V, Handegard NO, Rosen S, Schreyeck T, Mahiout T, Malde K (2019) Fish species identification using a convolutional neural network trained on synthetic data. ICES J Mar Sci 76(1):342–349

    Article  Google Scholar 

  2. Anderson Aparecido dos Santos AA, Gonçalves WN (2019) Improving pantanal fish species recognition through taxonomic ranks in convolutional neural networks. Ecological Informatics 100977:53

    Google Scholar 

  3. Boom BJ, Huang PX, He J, Fisher RB (2012) Supporting ground-truth annotation of image datasets using clustering. In: Proceedings of the 21st international conference on pattern recognition (ICPR), pp 1542–1545

  4. Deng J, Dong W, Socher R, Li L, Li K, Li F-F (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE computer society conference on computer vision and pattern recognition workshops (CVPR workshops), pp 248–255

  5. Fan D-P, Cheng M-M, Liu J-J, Gao S-H, Hou Q, Borji A (2018) Salient objects in clutter: bringing salient object detection to the foreground. In: The european conference on computer vision (ECCV), pp 186–202

  6. Fleiss JL (1981) Statistical methods for rates and proportions, 2nd edn. Wiley, New York

    MATH  Google Scholar 

  7. Gandarias JM, García-Cerezo AJ, Gómez-de-Gabriel JM (2019) Cnn-based methods for object recognition with high-resolution tactile sensors. IEEE Sensors J 19(16):6872–6882

    Article  Google Scholar 

  8. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778

  9. Jalal A, Salman A, Mian A, Shortis M, Shafait F (2020) Fish detection and species classification in underwater environments using deep learning with temporal information. Ecological Informatics 101088:57

    Google Scholar 

  10. Jin L, Liang H (2017) Deep learning for underwater image recognition in small sample size situations. In: OCEANS 2017 - Aberdeen, pp 1–4

  11. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  12. Larsen R, Olafsdottir H, Ersbøll BK (2009) Shape and texture based classification of fish species. In: Image analysis. Springer , Berlin, pp 745–749

  13. Li X, Shang M, Qin H, Chen L (2015) Fast accurate fish detection and recognition of underwater images with fast r-cnn. In: OCEANS, pp 1–5

  14. Liu L, Shen C, van den Hengel A (2015) The treasure beneath convolutional layers: cross-convolutional-layer pooling for image classification. In: IEEE conference on computer vision and pattern Recognition (CVPR)

  15. Liu L, Shen C, van den Hengel A (2017) Cross-convolutional-layer pooling for image recognition. IEEE Trans Pattern Anal Mach Intell 39(11):2305–2313

    Article  Google Scholar 

  16. Mathur NGM (2019) Automatic fish species classification in underwater images using transfer learning. In: International conference on signal processing, VLSI and communication engineering

  17. Nagaoka Y, Miyazaki T, Sugaya Y, Omachi S (2019) Automatic mackerel sorting machine using global and local features. IEEE Access 7:63767–63777

    Article  Google Scholar 

  18. Qin JLH, Li X, Zhang C (2016) Deepfish: accurate underwater live fish recognition with a deep architecture. Neurocomputing 187:49–58

    Article  Google Scholar 

  19. Qiu C, Zhang S, Wang C, Yu Z, Zheng H, Zheng B (2018) Improving transfer learning and squeeze- and-excitation networks for small-scale fine-grained fish image classification. IEEE Access 6:78503–78512

    Article  Google Scholar 

  20. Rathi D, Jain S, Indu S (2017) Underwater fish species classification using convolutional neural network and deep learning. In: Ninth international conference on advances in pattern recognition (ICAPR)

  21. Rauf HT, Ikram Ullah Lali M, Zahoor S, Shah SZH, Rehman AU, Bukhari SAC (2019) Visual features based automated identification of fish species using deep convolutional neural networks. Comput Electron Agric 167:105075

    Article  Google Scholar 

  22. Richard Landis J, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159–174

    Article  Google Scholar 

  23. Rodrigues MTA, Freitas MHG, Pádua F, Gomes RM, Carrano EG (2015) Evaluating cluster detection algorithms and feature extraction techniques in automatic classification of fish species. Pattern Anal Applic 18(4):783–797

    Article  MathSciNet  Google Scholar 

  24. Rova A, Mori G, Dill LM (2007) One fish, two fish, butterfish, trumpeter: recognizing fish in underwater video. In: IAPR conference on machine vision applications

  25. Salman A, Jalal A, Shafait F, Mian A, Shortis M, Seager J, Harvey E (2016) Fish species classification in unconstrained underwater environments based on deep learning. Limnology and Oceanography: Methods 14(9):570–585

    Google Scholar 

  26. Schettini R, Corchs S (2010) Underwater image processing: state of the art of restoration and image enhancement methods. EURASIP Journal on Advances in Signal Processing 2010(1):746052

    Article  Google Scholar 

  27. Siddiqui S, Salman A, Malik I, Shafait F, Mian A, Shortis M, Harvey E (2018) Automatic fish species classification in underwater videos: exploiting pretrained deep neural network models to compensate for limited labelled data. ICES J Mar Sci 75:1–16

    Article  Google Scholar 

  28. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556:1--14

  29. Spampinato C, Giordano D, Salvo RD, Chen-Burger Y-HJ, Fisher RB, Nadarajan G (2010) Automatic fish classification for underwater species behavior understanding. In: Proceedings of the first ACM international workshop on analysis and retrieval of tracked events and motion in imagery streams, pp 45–50

  30. Storbeck F, Daan B (2011) Fish species recognition using computer vision and a neural network. Fish Res 51:11–15

    Article  Google Scholar 

  31. Strachan NJC, Kell L (1995) A potential method for the differentiation between haddock fish stocks by computer vision using canonical discriminant analysis. ICES J Mar Sci 52(1):145–149

    Article  Google Scholar 

  32. Tamou B, Ben A (2018) Nasreddine underwater live fish recognition by deep learning. In: Image and signal processing, pp 275–283

  33. Tharwat A, Hemedan AA, Hassanien AE, Gabel T (2018) A biometric-based model for fish species classification. Fish Res 204:324–336

    Article  Google Scholar 

  34. Villon S, Mouillot D, Chaumont M, Darling ES, Subsol G, Claverie T, Villéger S (2018) A deep learning method for accurate and fast identification of coral reef fishes in underwater images. Ecological Informatics 48:238–244

    Article  Google Scholar 

  35. White C, Svellingen DJ, Strachan NJC (2006) Automated measurement of species and length of fish by computer vision. Fish Res 80:203–310

    Article  Google Scholar 

  36. Zhao J, Cao Y, Fan D, Cheng M, Li X, Zhang L (2019) Contrast prior and fluid pyramid integration for rgbd salient object detection. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 3922–3931

  37. Zhao J-X, Liu J-J, Fan D-P, Cao Y, Yang J, Cheng M-M (2019) Egnet: edge guidance network for salient object detection. In: The IEEE international conference on computer vision (ICCV), pp 8779–8788

Download references

Author information

Authors and Affiliations

  1. Department of ECE, IGDTUW, Kashmere Gate, Delhi-06, Delhi, India

    Monika Mathur, Diksha Vasudev, Sonalika Sahoo, Divanshi Jain & Nidhi Goel

Authors
  1. Monika Mathur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Diksha Vasudev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sonalika Sahoo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Divanshi Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nidhi Goel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nidhi Goel.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mathur, M., Vasudev, D., Sahoo, S. et al. Crosspooled FishNet: transfer learning based fish species classification model. Multimed Tools Appl 79, 31625–31643 (2020). https://doi.org/10.1007/s11042-020-09371-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09371-x

Keywords

Search

Navigation