Skip to main content
SpringerLink
Log in

Recognition of Aquatic Invasive Species Larvae Using Autoencoder-Based Feature Averaging

  • Conference paper
  • First Online:
Advances in Visual Computing (ISVC 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13598))

Included in the following conference series:

Abstract

The spread of invasive aquatic species disrupts ecological balance, damages natural resources, and adversely affects agricultural activity. There is a need for automated systems that can detect and classify invasive and non-invasive aquatic species using underwater videos without human supervision. In this paper, we intend to classify the larvae of invasive species like Zebra and Quagga mussels. These organisms are native to eastern Europe, but are invasive in United States waterways. It’s important to identify invasive species at the larval stage when they are mobile in the water and before they have established a presence, to avoid infestations. Video-based underwater species classification has several challenges due to variation of illumination, angle of view and background noise. In the case of invasive larvae, there is added difficulty due to the microscopic size and small differences between aquatic species larvae. Additionally, there are challenges of data imbalance since invasive species are typically less abundant than native species. In video-based surveillance methods, each organism may have multiple video frames offering different views that show different angles, conditions, etc. Since, there are multiple images per organism, we propose using image set based classification which can accurately classify invasive and non-invasive organisms based on sets of images. Image set classification can often have higher accuracy even if single image classification accuracy is lower. Our system classifies image sets with a feature averaging pipeline that begins with an autoencoder to extract features from the images. These features are then averaged for each set corresponding to a single organism. The final prediction is made by a classifier trained on the image set features. Our experiments show that feature averaging provides a significant improvement over other models of image classification, achieving more than \(97\%\) F1 score to predict invasive organisms on our video imaging data for a quagga mussel survey.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arizona Game and Fish: Invasive zebra mussels found in “moss ball” aquarium product. azgfd.com/invasive-zebra-mussels-found-in-moss-ball-aquarium-product-sold-at-aquarium-and-pet-supply-stores/

    Google Scholar 

  2. Benton, G., Finzi, M., Izmailov, P., Wilson, A.G.: Learning invariances in neural networks (2020). https://doi.org/10.48550/ARXIV.2010.11882

  3. Bochinski, E., Bacha, G., Eiselein, V., Walles, T.J., Nejstgaard, J.C., Sikora, T.: Deep active learning for in situ plankton classification. In: International Conference on Pattern Recognition, pp. 5–15 (2018)

    Google Scholar 

  4. Cevikalp, H., Triggs, B.: Face recognition based on image sets. In: CVPR Proceedings, pp. 2567–2573 (2010)

    Google Scholar 

  5. Chai, J., Zeng, H., Li, A., Ngai, E.W.: Deep learning in computer vision: a critical review of emerging techniques and application scenarios. Mach. Learn. Appl. 6, 100134 (2021)

    Google Scholar 

  6. Chuang, M.C., Hwang, J.N., Williams, K.: Supervised and unsupervised feature extraction methods for underwater fish species recognition. In: ICPR Workshop on Computer Vision for Analysis of Underwater Imagery, pp. 33–40 (2014)

    Google Scholar 

  7. Conn, D., Lutz, R., Hu, Y.P., Kennedy, V.: Guide to the Identification of Larval and Postlarval Stages of Zebra Mussels, Dreissena spp. and the Dark False Mussel, Mytilopsis leucophaeata. NEW YORK SEA GRANT, STONY BROOK, NY (USA), January 1993

    Google Scholar 

  8. Connelly, N.A., O’Neill, C.R., Knuth, B.A., Brown, T.L.: Economic impacts of zebra mussels on drinking water treatment and electric power generation facilities. Environ. Manag. 40(1), 105–112 (2007)

    Article  Google Scholar 

  9. Cowart, D.A., Breedveld, K.G., Ellis, M.J., Hull, J.M., Larson, E.R.: Environmental DNA (eDNA) applications for the conservation of imperiled crayfish (decapoda: Astacidea) through monitoring of invasive species barriers and relocated populations. J. Crustac. Biol. 38(3), 257–266 (2018)

    Article  Google Scholar 

  10. Daliri, M.R., Torre, V.: Robust symbolic representation for shape recognition and retrieval. Pattern Recogn. 41(5), 1782–1798 (2008)

    Article  MATH  Google Scholar 

  11. Deng, Y., Loy, C.C., Tang, X.: Image aesthetic assessment: an experimental survey. IEEE Sig. Process. Mag. 34, 80–106 (2017)

    Article  Google Scholar 

  12. Durán, C., Lanao, M., Anadón, A., Touyá, V.: Management strategies for the zebra mussel invasion in the Ebro river basin. Aquat. Invasions 5(3), 309–16 (2010)

    Article  Google Scholar 

  13. ENSR International: Rapid response plan for the zebra mussel (dreissena polymorpha) in Massachusetts (2005). https://www.mass.gov/doc/zebra-mussel-3/download

  14. Feist, S.M., Lance, R.F.: Advanced molecular-based surveillance of quagga and zebra mussels: a review of environmental DNA/RNA (eDNA/eRNA) studies and considerations for future directions. NeoBiota 66, 117 (2021)

    Article  Google Scholar 

  15. Fitzgibbon, A.W., Zisserman, A.: Joint manifold distance: a new approach to appearance based clustering. In: CVPR Proceedings, vol. 1, p. I (2003)

    Google Scholar 

  16. Gingera, T.D., Bajno, R., Docker, M.F., Reist, J.D.: Environmental DNA as a detection tool for zebra mussels dreissena polymorpha (Pallas, 1771) at the forefront of an invasion event in Lake Winnipeg, Manitoba. Canada. Manag. Biol. Invasions 8(3), 287 (2017)

    Article  Google Scholar 

  17. Goroshin, R., LeCun, Y.: Saturating auto-encoders (2013). https://doi.org/10.48550/ARXIV.1301.3577

  18. Gutierrez, P., Cordier, A., Caldeira, T., Sautory, T.: Data augmentation and pre-trained networks for extremely low data regimes unsupervised visual inspection. In: Optical Metrology (2021)

    Google Scholar 

  19. Hadid, A., Pietikainen, M.: From still image to video-based face recognition: an experimental analysis. In: Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 813–818 (2004)

    Google Scholar 

  20. Hinton, G.E., Dayan, P., Revow, M.: Modeling the manifolds of images of handwritten digits. IEEE Trans. Neural Netw. 8(1), 65–74 (1997)

    Article  Google Scholar 

  21. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  22. Johnson, L.E.: Enhanced early detection and enumeration of zebra mussel (dreissena spp.) veligers using cross-polarized light microscopy. Hydrobiologia 312(2), 139–146 (1995)

    Google Scholar 

  23. Khashman, A.: Face recognition using neural networks and pattern averaging. In: ISNN Proceedings (2006)

    Google Scholar 

  24. Khashman, A.: Intelligent face recognition: local versus global pattern averaging. In: Australian Conference on Artificial Intelligence (2006)

    Google Scholar 

  25. Kim, T.K., Kittler, J., Cipolla, R.: Discriminative learning and recognition of image set classes using canonical correlations. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1005–1018 (2007)

    Article  Google Scholar 

  26. Kovenko, V., Bogach, I.: A comprehensive study of autoencoders applications related to images. In: Proceeding of the International Conference on Information Technology and Interactions (IT &I-2020), pp. 43–54 (2020)

    Google Scholar 

  27. Lin, W., Hasenstab, K., Moura Cunha, G., Schwartzman, A.: Comparison of handcrafted features and convolutional neural networks for liver MR image adequacy assessment. Sci. Rep. 10(1), 1–11 (2020)

    Article  Google Scholar 

  28. Liu, J., Jaiswal, A., Yao, K.T., Raghavendra, C.S.: Autoencoder-derived features as inputs to classification algorithms for predicting well failures. In: SPE Western Regional Meeting (2015)

    Google Scholar 

  29. Nwankpa, C., Ijomah, W., Gachagan, A., Marshall, S.: Activation functions: comparison of trends in practice and research for deep learning (2018). https://doi.org/10.48550/ARXIV.1811.03378

  30. O’Neill, Jr., C.R., Dextrase, A.: The introduction and spread of the zebra mussel in North America. In: Proceedings of The Fourth International Zebra Mussel Conference, March 1994

    Google Scholar 

  31. Pacific Gas and Electric: Prevent the spread of quagga and zebra mussels. pge.com/en_US/about-pge/environment/what-we-are-doing/quagga-and-zebra-mussel-prevention-program/quagga-and-zebra-mussel-prevention-program.page

    Google Scholar 

  32. Raitoharju, J., et al.: Data enrichment in fine-grained classification of aquatic macroinvertebrates. In: 2nd Workshop on Computer Vision for Analysis of Underwater Imagery, pp. 43–48 (2016)

    Google Scholar 

  33. Rifai, S., Vincent, P., Muller, X., Glorot, X., Bengio, Y.: Contractive auto-encoders: explicit invariance during feature extraction. In: ICML (2011)

    Google Scholar 

  34. Schoening, T., Kuhn, T., Nattkemper, T.W.: Seabed classification using a bag-of-prototypes feature representation. In: ICPR Workshop on Computer Vision for Analysis of Underwater Imagery, pp. 17–24 (2014)

    Google Scholar 

  35. Shafait, F., et al.: Fish identification from videos captured in uncontrolled underwater environments. ICES J. Mar. Sci. 73(10), 2737–2746 (2016)

    Article  Google Scholar 

  36. Shah, S.A., Nadeem, U., Bennamoun, M., Sohel, F., Togneri, R.: Efficient image set classification using linear regression based image reconstruction. In: CVPR Workshop Proceedings, pp. 99–108 (2017)

    Google Scholar 

  37. Shoeibi, A., et al.: A comprehensive comparison of handcrafted features and convolutional autoencoders for epileptic seizures detection in EEG signals. Expert Syst. Appl. 163, 113788 (2021)

    Article  Google Scholar 

  38. Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6, 1–48 (2019)

    Article  Google Scholar 

  39. Siddiqui, S.A., et al.: Automatic fish species classification in underwater videos: exploiting pre-trained deep neural network models to compensate for limited labelled data. ICES J. Marine Sci. 75(1), 374–389 (07 2017)

    Google Scholar 

  40. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). https://doi.org/10.48550/ARXIV.1409.1556

  41. Sun, H., Zhen, X., Zheng, Y., Yang, G., Yin, Y., Li, S.: Learning deep match kernels for image-set classification. In: CVPR Proceedings, pp. 3307–3316 (2017)

    Google Scholar 

  42. Taylor, L., Nitschke, G.S.: Improving deep learning with generic data augmentation. IEEE Symposium Series on Computational Intelligence, pp. 1542–1547 (2018)

    Google Scholar 

  43. Texas Parks and Wildlife Department: TPWD aquatic invasive species management FY 2020–2021. tpwd.texas.gov/landwater/water/aquatic-invasives/media/Statewide

    Google Scholar 

  44. Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: ICML Proceedings, pp. 1096–1103 (2008)

    Google Scholar 

  45. Wang, R., Shan, S., Chen, X., Gao, W.: Manifold-manifold distance with application to face recognition based on image set. In: CVPR Proceedings, pp. 1–8 (2008)

    Google Scholar 

  46. Yamaguchi, O., Fukui, K., Maeda, K.I.: Face recognition using temporal image sequence. In: Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 318–323 (1998)

    Google Scholar 

  47. Zhao, Z.Q., Xu, S.T., Liu, D., Tian, W.D., Jiang, Z.D.: A review of image set classification. Neurocomputing 335, 251–260 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

Funding for this project is provided by Texas Parks and Wildlife Department (TPWD). We would like to acknowledge the significant help from Ryan McManamay, Micah Bowman, Jordan Jatko, and Mark Lueders from Baylor’s Department of Environmental Science.

Author information

Authors and Affiliations

  1. Baylor University, Waco, TX, 76798, USA

    Shaif Chowdhury & Greg Hamerly

Authors
  1. Shaif Chowdhury
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Greg Hamerly
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shaif Chowdhury or Greg Hamerly .

Editor information

Editors and Affiliations

  1. University of Nevada, Reno, NV, USA

    George Bebis

  2. University of Illinois Urbana-Champaign, Urbana, IL, USA

    Bo Li

  3. National University of Singapore, Singapore, Singapore

    Angela Yao

  4. Microsoft Research Asia, Beijing, China

    Yang Liu

  5. University of Missouri, Columbia, MO, USA

    Ye Duan

  6. City University of Hong Kong, Kowloon, Hong Kong

    Manfred Lau

  7. Idaho National Laboratory, Idaho Falls, ID, USA

    Rajiv Khadka

  8. Salesforce, Seattle, WA, USA

    Ana Crisan

  9. Tufts University, Medford, MA, USA

    Remco Chang

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chowdhury, S., Hamerly, G. (2022). Recognition of Aquatic Invasive Species Larvae Using Autoencoder-Based Feature Averaging. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2022. Lecture Notes in Computer Science, vol 13598. Springer, Cham. https://doi.org/10.1007/978-3-031-20713-6_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20713-6_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20712-9

  • Online ISBN: 978-3-031-20713-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation