Abstract
This paper presents the first-ever approach for autonomous 3D semantic mapping of coral reefs. The position of corals in 3D coordinates and the type of the coral are presented in such a3D semantic map. The intended application of this work is coral reef health monitoring, as the current assessment is based entirely on direct or indirect human observation. The proposed system joins a convolutional neural network (CNN) with a direct visual odometry approach and a correlation filter based tracker, Kernelized Correlation Filter (KCF), to identify the different coral species detected. In addition to the coral classification, the 3D position of each coral is identified producing a semantic map of the observed reef. Each coral is identified once and tracked to prevent a recount. The number of different coral species encountered in two separate traversed areas is reported. Furthermore, the shape and size of a coral can be extracted from the 3D reconstruction enabling the extraction of volumetric data for subsequent studies. Experimental results from the coral reefs of Barbados verify the robustness and accuracy of the proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Coral reef monitoring|reef recharge. http://reefrecharge.com/coral-reef-monitoring/. Accessed 18 April 2019
Corals of the world—variation in species. http://coral.aims.gov.au/info/taxonomy-variation.jsp. Accessed 30 November 2017
Beijbom, O., Edmunds, P.J., Kline, D., Mitchell, B.G., Kriegman, D., et al.: Automated annotation of coral reef survey images. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1170–1177 (2012)
Beijbom, O., Edmunds, P.J., Kline, D.I., Mitchell, B.G., Kriegman, D.: Automated annotation of coral reef survey images. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1170–1177 (2012)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR09 (2009)
Engel, J., Koltun, V., Cremers, D.: Direct sparse odometry. 40(3), 611–625 (2018)
Engel, J., Schöps, T., Cremers, D.: LSD-SLAM: Large-Scale Direct Monocular SLAM, vol. 8690, pp. 834–849. Springer Int. Publishing (2014)
Everingham, M., Eslami, S.A., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111(1), 98–136 (2015)
Forster, C., Zhang, Z., Gassner, M., Werlberger, M., Scaramuzza, D.: SVO: semidirect visual odometry for monocular and multicamera systems. 33(2) (2017)
Fu, C.Y., Liu, W., Ranga, A., Tyagi, A., Berg, A.C.: DSSD: Deconvolutional single shot detector. arXiv preprint arXiv:1701.06659 (2017)
Gao, X., Wang, R., Demmel, N., Cremers, D.: LDSO: Direct sparse odometry with loop closure. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2198–2204. IEEE (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2015)
Joshi, B., Rahman, S., Cain, B., Johnson, J., Kalitazkis, M., Xanthidis, M., Karapetyan, N., Hernandez, A., Quattrini Li, A., Vitzilaios, N., Rekleitis, I.: Experimental comparison of open source vision-inertial-based state estimation algorithms (2019) (under review)
Leutenegger, S., Lynen, S., Bosse, M., Siegwart, R., Furgale, P.: Keyframe-based visual-inertial odometry using nonlinear optimization. 34(3), 314–334 (2015)
Lin, T.Y., Dollár, P., Girshick, R.B., He, K., Hariharan, B., Belongie, S.J.: Feature pyramid networks for object detection. In: CVPR, vol. 1, p. 4 (2017)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: SSD: Single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37. Springer
Mahmood, A., et al.: Coral classification with hybrid feature representations. In: IEEE International Conference on Image Processing (ICIP), pp. 519–523 (2016)
Marcos, M.S.A., David, L., Peñaflor, E., Ticzon, V., Soriano, M.: Automated benthic counting of living and non-living components in ngedarrak reef, palau via subsurface underwater video. Environ. Monitor. Assess. 145(1–3), 177–184 (2008)
Mehta, A., Ribeiro, E., Gilner, J., van Woesik, R.: Coral reef texture classification using support vector machines. In: International Conference on Computer Vision Theory and Applications (VISAPP), pp. 302–310 (2007)
Modasshir, M., Li, A.Q., Rekleitis, I.: Mdnet: Multi-patch dense network for coral classification. In: MTS/IEEE Oceans Charleston, Charleston, SC, USA (Oct. 2018) (accepted)
Modasshir, M., Rahman, S., Youngquist, O., Rekleitis, I.: Coral identification and counting with an autonomous underwater vehicle. In: IEEE International Conference on Robotics and Biomimetics (ROBIO), Kuala Lumpur, Malaysia, pp. 524–529 (2018)
Mur-Artal, R., Montiel, J., Tardos, J.: ORB-SLAM: a versatile and accurate monocular SLAM system, vol. 31, pp. 1147–1163 (2015)
Pizarro, O., Rigby, P., Johnson-Roberson, M., Williams, S.B., Colquhoun, J.: Towards image-based marine habitat classification. In: OCEANS 2008, pp. 1–7. IEEE (2008)
Rahman, S., Li, A.Q., Rekleitis, I.: Sonar visual inertial SLAM of underwater structures. In: IEEE International Conference on Robotics and Automation, Brisbane, Australia, pp. 5190–5196 (May 2018)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Rogers, C., Garrison, G., Grober, R., Hillis, Z., F.ie, M.: Coral reef monitoring manual for the Caribbean and western Atlantic, 110 pp. Virgin Islands National Park Ilus (1994)
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: Overfeat: integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)
Stokes, M.D., Deane, G.B.: Automated processing of coral reef benthic images. Limnol. Oceanogr. Methods 7(2), 157–168 (2009)
Sun, C., Wang, D., Lu, H., Yang, M.H.: Correlation tracking via joint discrimination and reliability learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 489–497 (2018)
Acknowledgements
This work was made possible through the generous support of National Science Foundation grants (NSF 1513203).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Modasshir, M., Rahman, S., Rekleitis, I. (2021). Autonomous 3D Semantic Mapping of Coral Reefs. In: Ishigami, G., Yoshida, K. (eds) Field and Service Robotics. Springer Proceedings in Advanced Robotics, vol 16. Springer, Singapore. https://doi.org/10.1007/978-981-15-9460-1_26
Download citation
DOI: https://doi.org/10.1007/978-981-15-9460-1_26
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-9459-5
Online ISBN: 978-981-15-9460-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)