Skip to main content
SpringerLink
Log in

A survey of maritime vision datasets

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The field of computer vision has been applied in many topics and scenes, especially in the shipping business which occupies a large position in the world trade. With the development of ship intellectualization, the task of detection, tracking, segmentation and classification of interested targets become more and more important. Publicly available dataset is the foundation to promote research in shipping. Based on this intention, we systematically present a review of maritime datasets on maritime perception. In this paper, comparison is made in terms of data type, environment, ground authenticity, and applicable research directions. The aim of writing this paper is to help researchers quickly identify the most suitable dataset for their work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Antonio-Javier G, Antonio P, Pablo G (2018) Automatic ship classification from optical aerial images with convolutional neural networks. Remote Sens 10(4):511

    Article  Google Scholar 

  2. Badrinarayanan V, Kendall A, Cipolla R (2019) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  3. Bloisi D, Iocchi L (2009) Argos - a video surveillance system for boat traffic monitoring in Venice. Int J Pattern Recognit Artif Intell 23(7):1477–1502

    Article  Google Scholar 

  4. Bloisi DD, Iocchi L, Pennisi A, Tombolini L (2015) ARGOS-Venice boat classification. IEEE International Conference on Advanced Video & Signal Based Surveillance

  5. Bovcon B, Kristan M (2018) Obstacle Detection for USVs by Joint Stereo-View Semantic Segmentation. 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE

  6. Bovcon B, Kristan M (2019) Benchmarking semantic segmentation methods for obstacle detection on a marine environment. In: Proc. 24th Comput. Vis. Winter workshop. 1-9

  7. Bovcon B, Mandeljc R, Pers J, Kristan M (2017) Improving vision-based obstacle detection on usv using inertial sensor. International Symposium on Image & Signal Processing & Analysis pp 1–6. https://doi.org/10.1109/ISPA.2017.8073559

  8. Bovcon B, Mandeljc R, Perš J, Kristan M (2018) Stereo obstacle detection for unmanned surface vehicles by imu-assisted semantic segmentation - sciencedirect. Robot Auton Syst 104:1–13

    Article  Google Scholar 

  9. Bovcon B, Muhovic J, Pers J, Kristan M (2019) The MaSTr1325 dataset for training deep USV obstacle detection models. 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS)

  10. Bürkle A, Essendorfer B (2010) Maritime surveillance with integrated systems. In: Proc. of the 2nd International Conference on Waterside Security Conference. https://doi.org/10.1109/WSSC.2010.5730231

  11. Cane T, Ferryman J (2019) Evaluating deep semantic segmentation networks for object detection in maritime surveillance. 2018 15th IEEE international conference on advanced video and signal based surveillance (AVSS)

  12. Carthel C, Coraluppi S, Grignan P (2007) Multisensor tracking and fusion for maritime surveillance. International Conference on Information Fusion

  13. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  14. Chen LC, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European conference on computer vision (ECCV). 801-818

  15. Dao-Duc C, Xiaohui H, Morère O (2015) Maritime vessel images classification using deep convolutional neural networks. The sixth international symposium. ACM

  16. Dong J, Liu K, Wang J (2013) Simulation of the foggy scene under outdoor natural scenes. J Comput-Aided Des Comput Graph 25(3):397–409

    Google Scholar 

  17. Farahnakian F, Haghbayan M, Poikonen J, Laurinen M, Nevalainen P, Heikkonen J (2018) Object detection based on multi-sensor proposal fusion in maritime environment, 2018 17th IEEE international conference on machine learning and applications (ICMLA), Orlando, FL, 2018, 971–976

  18. Farhadi A, Endres I, Hoiem D, Forsyth D (2009) Describing objects by their attributes. In: 2009 IEEE Conference on Computer Vision and Pattern Recogn 1778–1785

  19. Fefilatyev S, Goldgof DB, Lembke C (2009, April) Autonomous buoy platform for low-cost visual maritime surveillance: design and initial deployment. In: Ocean Sensing and Monitoring, vol 7317, p 73170A

    Chapter  Google Scholar 

  20. Fefilatyev S, Goldgof D, Shreve M, Lembke C (2012) Detection and tracking of ships in open sea with rapidly moving buoy-mounted camera system. Ocean Eng 54:1–12

    Article  Google Scholar 

  21. FleetMon (2018) Vessel Photo Database. https://www.fleetmon.com/community/photos. Accessed 21 April 2020

  22. Gallego AJ, Pertusa A, Gil P (2018) Automatic ship classification from optical aerial images with convolutional neural networks. Remote Sens 10(4):511

    Article  Google Scholar 

  23. Haghbayan M et al (2018) An Efficient Multi-sensor Fusion Approach for Object Detection in Maritime Environments. 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI. 2163–2170

  24. Hartemink M (2012) Robust automatic object detection in a maritime environment: polynomial background estimation and the reduction of false detections by means of classification. Master of Science Thesis, Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS), Delft University of Technology, Delft-the Netherlands

  25. IHS Markit (2018) HIS Maritime World Register of Ships. https://ihsmarkit.com/products/maritime-portal-desktop.html. Accessed 21 April 2020

  26. Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 1125–1134

  27. Jampens R, Hernandez F, Vandecasteele F, Verstockt S (2016) Automatic detection, tracking and counting of birds in marine video content. In: 2016 sixth international conference on image processing theory, tools and applications (IPTA). 1–6

  28. Kristan M, Kenk VS, Kovačič S, Perš J (2015) Fast image-based obstacle detection from unmanned surface vehicles. IEEE Trans Cybern 46(3):641–654

    Article  Google Scholar 

  29. Krüger W, Orlov Z (2010) Robust layer-based boat detection and multi-target-tracking in maritime environments. In: 2010 International WaterSide Security Conference 1–7

  30. Kunz GJ, Bekman HHT, Benoist KW, Coen LH, van den Heuvel JC, van Putten FJ (2005) Detection of small targets in a marine environment using laser radar. In: Remote Sensing of the Coastal Oceanic Environment 5885, 58850F

  31. Lampert CH, Nickisch H, Harmeling S (2009) Learning to detect unseen object classes by between-class attribute transfer. In: 2009 IEEE conference on Computer Vision and Pattern Recognition 951–958

  32. Leclerc M, Tharmarasa R, Florea MC, Boury-Brisset AC, Kirubarajan T, Duclos-Hindie N (2018) Ship classification using deep learning techniques for maritime target tracking. In: 2018 21st International Conference on Information Fusion (FUSION), pp 737–744

  33. Liu H, Nie J, Liu Y, Wu Y, Wang H, Qu F, Liu W, Li Y (2020) A multi-modality sensor system for unmanned surface vehicle. Neural Process Lett 52(2):977–992

    Article  Google Scholar 

  34. Maresca S, Greco M, Gini F, Grasso R, Coraluppi S, Horstmann J (2010) Vessel detection and classification: an integrated maritime surveillance system in the Tyrrhenian Sea. In 2010 2nd International Workshop on Cognitive Information Processing 40–45

  35. Maritime RobotX challenge (2017) https://www.robotx.org/. Accessed 21 April 2020

  36. MaritimeTraffic (2018) Vessel photos. http://www.marinetraffic.com/en/photos/of/ships. Accessed 21 April 2020

  37. Moreira RDS, Ebecken NFF, Alves AS, Livernet F, Campillo-Navetti A (2014) A survey on video detection and tracking of maritime vessels. Int J Recent Res Appl Stud 20(1):37–50

    Google Scholar 

  38. Patino L, Cane T, Vallee A, Ferryman J (2016) Pets 2016: dataset and challenge. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 1–8

  39. Pires N, Guinet J, Dusch E (2010) ASV: an innovative automatic system for maritime surveillance. Navigation 58(232):1–20

    Google Scholar 

  40. Prasad DK, Prasath CK, Rajan D, Rachmawati L, Rajabaly E, Quek C (2016) Challenges in video based object detection in maritime scenario using computer vision. In: International Conference on Connected Vehicles

  41. Prasad DK, Rajan D, Rachmawati L, Rajabally E, Quek C (2017) Video processing from electro-optical sensors for object detection and tracking in a maritime environment: a survey. IEEE Trans Intell Transp Syst 18(8):1993–2016

    Article  Google Scholar 

  42. Qiaoqiao S, Wei L, Fan Z, Wei H, Xu S, Lianru G (2018) Deep cnn with multi-scale rotation invariance features for ship classification. IEEE Access 6:38656–38668

  43. Quigley M, Conley K, Gerkey B, Faust J, Foote T, Leibs J, Ng AY (2009) ROS: an open-source robot operating system. In: ICRA workshop on open source software 3(2):5–5

  44. Ribeiro R, Cruz G, Matos J, Bernardino A (2019) A data set for airborne maritime surveillance environments. IEEE Trans Circuits Syst Video Technol 29(9):2720–2732

    Article  Google Scholar 

  45. Shao Z, Wu W, Wang Z, Du W, Li C (2018) Seaships: a large-scale precisely-annotated dataset for ship detection. IEEE Trans Multimedia 20(10):1–1

  46. Shipspotting (2015) Ship Photos and Shipping News. http://www.shipspotting.com. Accessed 21 April 2020

  47. Smith TF, Waterman MS (1981) Identification of Common Molecular Subsequences. J Mol Biol 147:195–197

  48. Stanislas L, Dunbabin M (2018) Multimodal sensor fusion for robust obstacle detection and classification in the maritime RobotX challenge. IEEE J Ocean Eng 44(2):343–351

    Article  Google Scholar 

  49. Sun Y, Bo L, Fox D (2013) Attribute based object identification. In: 2013 IEEE international conference on robotics and automation. 2096–2103

  50. Sun Y, Su L, Luo Y, Meng H, Li W, Zhang Z, Wang P, Zhang W (2022) Global mask R-CNN for marine ship instance segmentation. Neurocomputing. 257-270:257–270

    Article  Google Scholar 

  51. Sun Y, Su L, Luo Y, Meng H, Zhang Z, Zhang W, Yuan S (2022) IRDCLNet: instance segmentation of ship images based on interference reduction and dynamic contour learning in foggy scenes. IEEE Trans Circuits Syst Video Technol 32:6029–6043

    Article  Google Scholar 

  52. Teutsch M, Krüger W (2010) Classification of small boats in infrared images for maritime surveillance. In: 2010 International WaterSide Security Conference. 1–7

  53. Thompson DJ (2017) Maritime object detection, tracking, and classification using lidar and vision-based sensor fusion. Dissertations and Theses, pp 377. https://commons.erau.edu/edt/377

  54. Vessel Finder (2018) User contributed ship photos and vessels gallery. https://www.vesselfinder.com/gallery. Accessed 21 April 2020

  55. Wang Q, Wan J, Yuan Y (2018) Locality constraint distance metric learning for traffic congestion detection. Pattern Recogn 75:272–281

    Article  Google Scholar 

  56. Wang Q, Gao J, Lin W, Yuan Y (2019) Learning from synthetic data for crowd counting in the wild, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 8190–8199

  57. Wang Q, Gao J, Lin W, Yuan Y (2021) Pixel-wise crowd understanding via synthetic data. Int J Comput Vis 129(1):225–245

    Article  Google Scholar 

  58. Withagen PJ, Schutte K, Vossepoel AM, Breuers MGJ (1999) Automatic classification of ships from infrared (flir) images. Proc SPIE - Int Soc Opt Eng 3720:180–187

    Google Scholar 

  59. Yaman C, Asari V (2007, June) Long-range target classification in a cluttered environment using multi-sensor image sequences. In: 2007 3rd international conference on Recent Advances in Space Technologies 304-308

  60. Yang CS, Jeong J (2016) Integrated ship monitoring system for realtime maritime surveillance. IGARSS 2016–2016 IEEE International Geoscience and Remote Sensing Symposium

  61. Yu C, Wang J, Peng C, Gao C, Yu G, Sang N (2018) Bisenet: bilateral segmentation network for real-time semantic segmentation. In: Proceedings of the European conference on computer vision (ECCV). 325–341

  62. Zhang MM, Choi J, Daniilidis K, Wolf MT, Kanan C (2015) VAIS: a dataset for recognizing maritime imagery in the visible and infrared spectrums. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 10–16

  63. Zhang X, Zhou F, Lin Y, Zhang S (2016) Embedding Label Structures for Fine-Grained Feature Representation. 2016 IEEE conference on computer vision and pattern recognition (CVPR). IEEE

  64. Zhang W, He X, Li W, Zhang Z, Luo Y, Su L, Wang P (2020) An integrated ship segmentation method based on discriminator and extractor. Image Vis Comput 93:103824

    Article  Google Scholar 

  65. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2881–2890

Download references

Acknowledgments

This work was supported in part by the Project of Intelligent Situation Awareness System for Smart Ship (MC-201920-X01), in part by the Project of Research on Intelligent Ship Testing and Verification (MC-201905-C03), in part by the Special Funds for Basic Scientific Research Business Expenses of Central Universities - Doctoral Research Innovation Fund Project Funding, and in part by the National Natural Science Foundation of China under Grants (U1613213, 91748131, 61771471). Thanks to Dr. Yuxin Sun. The revision of the paper was mainly done by Dr. Yuxin Sun, who made outstanding contributions in the revision process, including additional experiments and paper writing.

Author information

Authors and Affiliations

  1. College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin, 150001, China

    Li Su, Yusheng Chen & Hao Song

  2. Key Laboratory of Intelligent Technology and Application of Marine Equipment, Ministry of Education, Harbin Engineering University, Harbin, 150001, China

    Li Su

  3. Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Wanyi Li

Authors
  1. Li Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yusheng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hao Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wanyi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yusheng Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Su, L., Chen, Y., Song, H. et al. A survey of maritime vision datasets. Multimed Tools Appl 82, 28873–28893 (2023). https://doi.org/10.1007/s11042-023-14756-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14756-9

Keywords

Search

Navigation