Skip to main content
SpringerLink
Log in

Matching Cross-Domain Data with Cooperative Training of Triplet Networks: A Case Study on Underwater Robotics

  • Regular paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Recently, Deep Convolutional Neural Networks have been successfully applied to various robotics problems, such as robot vision and simultaneous localization and mapping. Among these, siamese and triplet networks have obtained great traction in intra-domain matching. However, it is impossible to directly use these networks in cross-domain problems. Thus, this paper proposes a new method to train a set of triplet networks to perform cross-domain matching and ranking focused in underwater robotics. The method is used to train a pair of networks to perform matching of acoustic and segmented aerial images aiming to support an underwater robot localization algorithm. Results show that the method is able to achieve up to 83% accuracy in matching acoustic and segmented aerial images and up to 85% recall in ranking relevant aerial images given an acoustic image.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bertinetto, L, Valmadre, J, Henriques, JF, Vedaldi, A, Torr, PH: Fully-convolutional siamese networks for object tracking. In: European Conference on Computer Vision. Springer, pp. 850–865 (2016)

  2. Bromley, J., Bentz, J.W., Bottou, L., Guyon, I., LeCun, Y., Moore, C., Säckinger, E, Shah, R.: Signature verification using a “siamese” time delay neural network. Int. J. Pattern Recognit. Artif. Intell. 7(04), 669–688 (1993)

    Article  Google Scholar 

  3. De Giacomo, G.G., dos Santos, M.M., Drews-Jr, P.L., Botelho, S.S.: Cooperative training of triplet networks for cross-domain matching. In: 2020 Latin American Robotics Symposium (LARS), 2020 Brazilian Symposium on Robotics (SBR) and, vol. 2020, pp 1–6. Workshop on Robotics in Education (WRE), IEEE (2020)

  4. Dos Santos, M.M., De Giacomo, G.G., Drews, P., Botelho, S.S.: Satellite and underwater sonar image matching using deep learning. In: 2019 Latin American Robotics Symposium (LARS), 2019 Brazilian Symposium on Robotics (SBR) and 2019 Workshop on Robotics in Education (WRE), pp 109–114. IEEE (2019)

  5. Dos Santos, M.M., De Giacomo, G.G., Drews, P.L., Botelho, SS: Underwater sonar and aerial images data fusion for robot localization. IEEE (2019)

  6. Dos Santos, M.M., De Giacomo, G.G., Drews-Jr, P.L., Botelho, S.S.: Matching color aerial images and underwater sonar images using deep learning for underwater localization. IEEE Robot. Autom. Lett. 5(4), 6365–6370 (2020)

    Article  Google Scholar 

  7. Fox, D.: Kld-sampling: Adaptive particle filters. Adv. Neural Inform. Process. Syst. 14, 713–720 (2001)

    Google Scholar 

  8. Glorot, X, Bengio, Y: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)

  9. Hoffer, E, Ailon, N: Deep metric learning using triplet network. In: International Workshop on Similarity-Based Pattern Recognition. Springer, pp. 84–92 (2015)

  10. Liu, L, Jiang, H, He, P, Chen, W, Liu, X, Gao, J, Han, J: On the variance of the adaptive learning rate and beyond. arXiv:190803265 (2019)

  11. Ribeiro, P.O., dos Santos, M.M., Drews, P.L., Botelho, S.S., Longaray, L.M., Giacomo, G.G., Pias, MR: Underwater place recognition in unknown environments with triplet based acoustic image retrieval. IEEE (2018)

  12. Ronneberger, O, Fischer, P, Brox, T: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp 234–241 (2015)

  13. dos Santos, M.M., De Giacomo, G.G., Drews, P., Botelho, SS, 2019: Semantic segmentation of static and dynamic structures of marina satellite images using deep learning. IEEE

  14. Schroff, F, Kalenichenko, D, Philbin, J: Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)

  15. Shi, Y, Yu, X, Campbell, D, Li, H: Where am i looking at? Joint location and orientation estimation by cross-view matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4064–4072 (2020)

  16. Silveira, L., Guth, F., Drews-Jr, P., Ballester, P., Machado, M., Codevilla, F., Duarte-Filho, N., Botelho, S.: An open-source bio-inspired solution to underwater slam. IFAC-PapersOnLine 48(2), 212–217 (2015)

    Article  Google Scholar 

  17. de Souza Ribeiro, POC, dos Santos, MM, Drews, PLJ, da Costa Botelho, SS: Forward looking sonar scene matching using deep learning. In: 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 574–579. https://doi.org/10.1109/ICMLA.2017.00-99 (2017)

  18. Steffens, C, Messias, L, Drews-Jr, P, Botelho, S: Contrast enhancement and image completion: A cnn based model to restore ill exposed images. In: 2019 IEEE 17th International Conference on Industrial Informatics (INDIN), pp. 226–232. https://doi.org/10.1109/INDIN41052.2019.8972228 (2019)

  19. Steffens, C, Messias, L, Drews-Jr, P, Botelho, S: Cnn based image restoration: Adjusting ill-exposed srgb images in post-processing. Journal of Intelligent & Robotic Systems. https://doi.org/10.1007/s10846-019-01124-9 (2020)

  20. Wang, P., Li, W., Wan, J., Ogunbona, P., Liu, X.: Cooperative training of deep aggregation networks for rgb-d action recognition. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

  21. Yu, F, Koltun, V: Multi-scale context aggregation by dilated convolutions. arXiv:151107122 (2015)

  22. Zhang, M, Lucas, J, Ba, J, Hinton, GE: Lookahead optimizer: k steps forward, 1 step back. In: Advances in Neural Information Processing Systems, pp. 9597–9608 (2019)

  23. Zhu, Z, Wang, Q, Li, B, Wu, W, Yan, J, Hu, W: Distractor-aware siamese networks for visual object tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 101–117 (2018)

Download references

Funding

This study was partly supported by the National Council for Scientific and Technological Development (CNPq) and Coordenacao de Aperfeiçoamento de Pessoal de Nivel Superior - Brasil (CAPES) - Finance Code 001. This paper is also a contribution of the INCT-Mar COI funded by CNPq Grant Number 610012/2011-8.

Author information

Authors and Affiliations

  1. Federal University of Rio Grande (FURG), Center for Computational Science, Av. Itália Km 8, 96203-900, Rio Grande, RS, Brazil

    Giovanni G. De Giacomo, Matheus M. dos Santos, Paulo L. J. Drews-Jr & Silvia S. C. Botelho

Authors
  1. Giovanni G. De Giacomo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matheus M. dos Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paulo L. J. Drews-Jr
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Silvia S. C. Botelho
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Giovanni G. De Giacomo: implementation and execution of the Deep Learning experiments; writing of the manuscript. Matheus M. dos Santos: development of the dataset and associated tools; helped writing the manuscript. Paulo L. J. Drews-Jr: theoretical support on the idea; revising the manuscript. Silvia S. C. Botelho: theoretical support on the idea; revising the manuscript.

Corresponding author

Correspondence to Giovanni G. De Giacomo.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for Publication

Not applicable.

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Availability of data and materials

Python 3/TensorFlow 2.x implementation of the model and datasets are available for download at: https://github.com/giovgiac/neptune.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

De Giacomo, G.G., dos Santos, M.M., Drews-Jr, P.L.J. et al. Matching Cross-Domain Data with Cooperative Training of Triplet Networks: A Case Study on Underwater Robotics. J Intell Robot Syst 104, 55 (2022). https://doi.org/10.1007/s10846-022-01591-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-022-01591-7

Keywords

Mathematics Subject Classification (2010)

Search

Navigation