Skip to main content
SpringerLink
Log in

Visual monitoring of High-Sea fishing activities using deep learning-based image processing

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Due to the increasing shortage of fish in the seas, Vessel Monitoring Systems (VMS) play a very important role in fishing activity monitoring, control and surveillance. In this context, the detection of fishing activities in prohibited zones is a critical task. Although position, speed and other information are provided by the VMS system, detecting fishing activities using only this type of information may be rather ineffective. The purpose of this paper is to propose an intelligent video-surveillance system that is able to automatically detect if (and when) fishing activities occur by automatically visually monitoring the vessel activities. After detection, the Control Center would be informed of relevant events and receive visual data through a low-bandwidth, low-cost satellite link to check if illegal activities are effectively being performed. To achieve this goal, the proposed visual monitoring solution adopts a deep-learning approach twice: First, to detect the fishing activities at the vessel, a well-known convolutional neural network designed for image classification is exploited. Second, to enhance at the Control Center the heavily compressed decoded images, transmitted with a low-bandwidth satellite channel, a convolutional neural network suitable for image restoration is also used. The proposed monitoring system follows a new approach for the problem addressed based on image data and can significantly improve the VMS even detection and validation efficiency by exploiting advanced image processing tools, thus helping to ensure the availability and sustainability of fish stocks for generations to come.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Amabdiyil S, Thomas D, Pillai VPM (2016) “Marine vessel detection comparing GPRS and satellite images for security applications”, international conference on wireless communications. Signal Processing and Networking, Chennai

    Google Scholar 

  2. Bloisi D, Iocchi L, Fiorini M, Graziano G (2011) Automatic maritime surveillance with visual target detection. International Defense and Homeland Security Simulation Workshop, Rome

    Google Scholar 

  3. "FFmpeg - Post Processing Filters," [Online]. Available: https://trac.ffmpeg.org/wiki/Postprocessing. [Accessed 19 February 2018]

  4. Food and Agricultural Organization of the United Nations, “2018 The State of the World Fisheries and Aquaculture” [Online]. Available at: http://www.fao.org/state-of-fisheries-aquaculture [Accessed 06 November 2018]

  5. He K, Zhang X, Ren S, Sun J (2016) “Deep residual learning for image recognition”, IEEE conference on computer vision and pattern recognition. Las Vegas, NV

    Google Scholar 

  6. Hensman P, Masko D (2015) “The impact of imbalanced training data for convolutional neural networks”, in first level degree project in computer science. KTH Royal Institute of Technology, Stockholm

    Google Scholar 

  7. X. Jiang, D. L. Silver, B. Hu, E. N. de Souza, and S. Matwin “Fishing activity Detection from AIS data using autoencoders”, Canadian conference on artificial intelligence, Victoria, Canada, May 2016.

  8. Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Li F-F (2014) “Large-scale video classification with convolutional neural networks”, in computer vision and pattern recognition. Columbus, OH

    Google Scholar 

  9. Krizhevsky A, Sutskever I, Hinton GE (2012) “ImageNet classification with deep convolutional neural networks”, in neural information processing systems. Lake Tahoe, NV

    Google Scholar 

  10. Lucas A, Iliadis M, Molina R, Katsaggelos A (2018) Using deep neural networks for inverse problems in imaging. IEEE Signal Process Mag 35(1):20–36

    Article  Google Scholar 

  11. Marzuki M, Gaspar P, Garello R, Kerbaol V (July 2018) Fishing gear identification from vessel-monitoring-system-based fishing vessel trajectories. IEEE J Ocean Eng 43(3):689–699

    Article  Google Scholar 

  12. P. Perdigão, “Ultra-narrow band satellite visual monitoring of high-sea fishing vessels”, M.Sc. Thesis, Instituto Superior Técnico, Lisbon, Portugal, June 2018.

  13. Rawat W, Wang Z (2017) Deep convolutional neural networks for image classification: a comprehensive review. in Neural Computation 9(8):1735–1780

    MATH  Google Scholar 

  14. Richter T (2016) “JPEG on steroids: common optimization techniques for JPEG image compression”, international conference on image processing. Phoenix, AZ

    Google Scholar 

  15. Russakovsky O et al (April 2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115:211–252

    Article  MathSciNet  Google Scholar 

  16. C. Szegedy and S. Ioffe “Batch normalization: accelerating deep network training by reducing internal covariate shift”, international conference on machine learning, Lille, France, 2015.

  17. M. Tello, C. López-Martínez, and J. J. Mallorqui “A Novel Algorithm for Ship Detection in SAR Imagery Based on the Wavelet Transform”, IEEE Geoscience and Remote Sensing Letters, vol. 2, no. 2, 2005.

  18. "The Common Fisheries Policy," [Online]. Available: https://ec.europa.eu/fisheries/cfp_en [Accessed 23 March 2017].

  19. Wallace GK (1992) The JPEG still picture compression standard. IEEE Trans Consum Electron 38(1):17–34

    Article  MathSciNet  Google Scholar 

  20. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a Gaussian Denoiser: residual learning of deep CNN for image Denoising. IEEE Trans Image Process 26(7):3142–3155

    Article  MathSciNet  MATH  Google Scholar 

  21. A. Zisserman, K. Simonyan, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv preprint arXiv:1409.1556, September 2014.

Download references

Author information

Authors and Affiliations

  1. Instituto Superior Técnico - Universidade de Lisboa, Instituto de Telecomunicações, Lisbon, Portugal

    Pedro Perdigão, João Ascenso & Fernando Pereira

  2. Xsealance, Queluz, Portugal

    Pedro Lousã

Authors
  1. Pedro Perdigão
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pedro Lousã
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. João Ascenso
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fernando Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to João Ascenso.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Perdigão, P., Lousã, P., Ascenso, J. et al. Visual monitoring of High-Sea fishing activities using deep learning-based image processing. Multimed Tools Appl 79, 22131–22156 (2020). https://doi.org/10.1007/s11042-020-08949-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08949-9

Keywords

Search

Navigation