Abstract
Coonamessett Farm Foundation (CFF) conducts one of the optical surveys of the sea scallop resource using a HabCam towed vehicle. The CFF HabCam v3 collects 5+ images per second, providing a continuous track of imagery as it is towed over scallop grounds. Annotations from HabCam images are translated into biomass estimates through a multi-step process that relies on adequate image subsampling and accurate scallop counts and measurements. Annotating images is the most time-consuming part of generating these reliable scallop biomass estimates. Reliably automating this process has the potential to increase annotation rates and improve biomass estimation, addressing questions about scallop patchiness and distribution over small-to-large scales on scallop grounds. Furthermore, optical survey images provide a wealth of raw data that is not consistently utilized. An additional high priority goal is to provide data for improving flounder stock assessments because the incidental bycatch of non-target species could have negative impacts on the long-term sustainability of the fishery. Kitware, Inc developed the Video and Image Analytics for Marine Environments (VIAME) system for analysis of underwater imagery with support from the NOAA Automated Image Analysis Strategic Initiative. CFF collaborated with Kitware to develop improved automated detectors for scallop classes (live scallops, swimming scallops, and clappers) and flounder by incorporating depth disparity information obtained from stereo image pairs collected during surveys. The accuracy and precision of these new detectors, with depth information incorporated, was contrasted against that of single-image detectors when utilized upgraded deep learning networks. A few methods were investigated, including early and late fusion of the depth information.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bradski, G., Kaehler, A.: Learning OpenCV: Computer vision with the OpenCV library. O’Reilly Media, Inc. (2008)
Cai, Z., Vasconcelos, N.: Cascade r-cnn: delving into high quality object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6154–6162 (2018)
Chang, J.H., Shank, B.V., Hart, D.R.: A comparison of methods to estimate abundance and biomass from belt transect surveys. Limnol. Oceanogr. Methods 15, 480–494 (2017)
Chen, K., et al.: MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155 (2019)
Dawkins, M., et al.: An open-source platform for underwater image and video analytics. IEEE Winter Conf. Appl. Comput. Vis. 2017, 898–906 (2017)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)
Hennen, D.R., Hart, D.R.: Shell height-to-weight relationships for Atlantic sea scallops (Placopecten magellanicus) in offshore US waters. J. Shellfish Res. 31, 1133–1144 (2012)
Hirschmuller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2007)
Li, Y., Zhang, J., Cheng, Y., Huang, K., Tan, T.: Semantics-guided multi-level RGB-D feature fusion for indoor semantic segmentation. IEEE Int. Conf. Image Process. 2017, 1262–1266 (2017)
Maguire, J.J.: Summary Report of the Review of Sea Scallop Survey Methodologies and Their Integration for Stock Assessment and Fishery Management (2015). https://www.nefsc.noaa.gov/saw/scallop-2015/pdfs/scallop-surveys-review-summary-report-april-9.pdf
National Marine Fisheries Service (NMFS). Fisheries of the United States 2018. Current Fishery Statistics No. 2018. 167 p. (2020)
New England Fishery Management Council (NEFMC). Atlantic Sea Scallop Fishery Management Plan Framework Adjustment 30 (2019). https://s3.amazonaws.com/nefmc.org/190307-FW30-Final-Submission.pdf
NEFMC. Scallop Fishery Management Plan Framework Adjustment 32 (2020). https://s3.amazonaws.com/nefmc.org/Framework-32-Final-Submission_signed-FONSI.pdf
O’Keefe, C., DeCelles, G.: Forming a partnership to avoid bycatch. Fisheries 38, 434–444 (2013)
Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. 32 (2019)
Shumway, S.E., Parsons, G.J. (eds.): Scallops: Biology, Ecology, Aquaculture, and Fisheries, p. 1196. Elsevier Publishing, Amsterdam Netherlands (2016)
Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492–1500 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this paper
Cite this paper
Dawkins, M., Crall, J., Leotta, M., O’Hara, T., Siemann, L. (2023). Towards Depth Fusion into Object Detectors for Improved Benthic Species Classification. In: Rousseau, JJ., Kapralos, B. (eds) Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and Challenges. ICPR 2022. Lecture Notes in Computer Science, vol 13645. Springer, Cham. https://doi.org/10.1007/978-3-031-37731-0_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-37731-0_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-37730-3
Online ISBN: 978-3-031-37731-0
eBook Packages: Computer ScienceComputer Science (R0)
