Skip to main content

Advertisement

SpringerLink
Log in

JF-YOLO: the jellyfish bloom detector based on deep learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The unmonitored jellyfish boom inevitably destroys coastal biodiversities as a type of planktons with extremely high fecundity. It even seriously endangers people’s economic and social activities, such as clogging the water intake system of hydropower plants and hindering coastal tourism development. In the past, underwater video monitoring tended to be time-consuming and costly. This paper proposes JF-YOLO: an automatic jellyfish blooms detection model based on deep learning. We collecte many jellyfish videos in real environments to form a dataset for model training. JF-YOLO uses the improved YOLO-V4 detection model to ensure detection accuracy and speed. The experimental results show that the detection accuracy of the JF-YOLO network is better than that of the YOLO-V4 network, with the average detection accuracy increasing from 85.35% to 92.67% and the recall rate increasing from 72.32% to 85.74%. As a promising solution, JF-YOLO can effectively monitor the number or density of jellyfish and provide early warning when they appear abnormal, bringing convenience to ocean governance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data Availability

The datasets generated during the current study are not publicly available due Additional research required but are available from the corresponding author on reasonable request.

References

  1. Ambati LS, El-Gayar O (2021) Human activity recognition: a comparison of machine learning approaches. J Midwest Assoc Inform Syst (JMWAIS) 2021:1. https://doi.org/10.17705/3jmwa.000065

    Article  Google Scholar 

  2. An L, Wang L, Ou D, Jia C, Li W, Ding Y, You C, Liao J, Huang H (2021) The ecological mechanisms of Acetes blooms as a threat to the security of cooling systems in coastal nuclear power plants. J Coast Conserv 25 (6):1–10

    Article  Google Scholar 

  3. Baliarsingh S K, Lotliker A A, Srichandan S, Samanta A, Kumar N, Nair TM B (2020) A review of jellyfish aggregations, focusing on India’s coastal waters. Ecol Process 9(1):1–9

    Article  Google Scholar 

  4. Barrado C, Fuentes J A, Salamí E, Royo P, Olariaga A D, López J, Fuentes V L, Gili J M, Pastor E (2014) Jellyfish monitoring on coastlines using remote piloted aircraft. In: IOP conference series: Earth and environmental science, vol 17. IOP Publishing, p 012195

  5. Bochkovskiy A, Wang C-Y, Liao H-Y M (2020) Yolov4: optimal speed and accuracy of object detection. arXiv:2004.10934

  6. Bosch-Belmar M, Milisenda G, Basso L, Doyle T K, Leone A, Piraino S (2020) Jellyfish impacts on marine aquaculture and fisheries. Rev Fisheries Sci Aquacult 29(2):242–259

    Article  Google Scholar 

  7. Chen Q, Wang Y, Yang T, Zhang X, Cheng J, Sun J (2021) You only look one-level feature. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13039–13048

  8. El-Gayar O F, Ambati L S, Nawar N (2020) Wearables, artificial intelligence, and the future of healthcare. IGI Global. https://www.igi-global.com/chapter/wearables-artificial-intelligence-and-the-future-of-healthcare/www.igi-global.com/chapter/wearables-artificial-intelligence-and-the-future-of-healthcare/236337

  9. Ghermandi A, Galil B, Gowdy J, Nunes PALD (2015) Jellyfish outbreak impacts on recreation in the Mediterranean Sea: Welfare estimates from a socioeconomic pilot survey in Israel. Ecosyst Serv 11 :140–147

    Article  Google Scholar 

  10. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587

  11. Goutte C, Gaussier E (2005) A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. In: European conference on information retrieval. Springer, pp 345–359

  12. Graham W M, Martin D L, Martin J C (2003) In situ quantification and analysis of large jellyfish using a novel video profiler. Mar Ecol Prog Ser 254:129–140. https://doi.org/10.3354/meps254129

    Article  Google Scholar 

  13. Han Y, Chang Q, Ding S, Gao M, Zhang B, Li S (2022) Research on multiple jellyfish classification and detection based on deep learning. Multimed Tools Applic 81(14):19429–19444. https://doi.org/10.1007/s11042-021-11307-y

    Article  Google Scholar 

  14. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  15. Influence of the digital divide and socio-economic factors on prevalence of diabetes. Issues In Information Systems (2020) https://doi.org/10.48009/4_iis_2020_103-113

  16. Kamili A, Fatima I, Hassan M, Parah S A, Vijaya Kumar V, Ambati L S (2020) Embedding information reversibly in medical images for e-health. J Intell Fuzzy Syst 39(6):8389–8398. https://doi.org/10.3233/JIFS-189157

    Article  Google Scholar 

  17. Kim D, Shin J-, Kim H, Lee D, Lee S-M, Myung H (2013) Experimental tests of autonomous jellyfish removal robot system JEROS. In: Kim J-H, Matson E T, Myung H, Xu P (eds) Robot intelligence technology and applications 2012, vol 208. Springer, Berlin, pp 395–403. https://doi.org/10.1007/978-3-642-37374-9_38

  18. Kim D, Kim H, Jung S, Koo J, Kim J, Myung H (2015) A vision-based detection algorithm for moving jellyfish in underwater environment. In: 2015 12th International conference on ubiquitous robots and ambient intelligence (URAI). IEEE, pp 144–145

  19. Kim H, Kim D, Jung S, Koo J, Shin J-U, Myung H (2015) Development of a UAV-type jellyfish monitoring system using deep learning. In: 2015 12th International conference on ubiquitous robots and ambient intelligence (URAI). IEEE, Goyang, pp 495–497. https://doi.org/10.1109/URAI.2015.7358813

  20. Kim H, Koo J, Kim D, Jung S, Shin J-U, Lee S, Myung H (2016) Image-based monitoring of jellyfish using deep learning architecture. IEEE Sens J 16(8):2215–2216. https://doi.org/10.1109/JSEN.2016.2517823

    Article  Google Scholar 

  21. Labao A B, Naval P C Jr (2019) Cascaded deep network systems with linked ensemble components for underwater fish detection in the wild. Eco Inform 52:103–121

    Article  Google Scholar 

  22. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  23. Li J, Su Z, Geng J, Yin Y (2018) Real-time detection of steel strip surface defects based on improved yolo detection network. IFAC-PapersOnLine 51(21):76–81

    Article  Google Scholar 

  24. Liang T, Chu X, Liu Y, Wang Y, Tang Z, Chu W, Chen J, Ling H (2021) Cbnetv2: a composite backbone network architecture for object detection. arXiv:2107.00420

  25. Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125

  26. Liu J, Yu C, Hu Z, Zhao Y, Bai Y, Xie M, Luo J (2020) Accurate prediction scheme of water quality in smart mariculture with deep Bi-S-SRU learning network. IEEE Access: Practical Innovations, Open Solutions 8:24784–24798

    Article  Google Scholar 

  27. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg A C (2016) Ssd: single shot multibox detector. In: European conference on computer vision. Springer, pp 21–37

  28. Liu Y, Wang Y, Wang S, Liang T, Zhao Q, Tang Z, Ling H (2020) Cbnet: a novel composite backbone network architecture for object detection. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 11653–11660

  29. Martin-Abadal M, Ruiz-Frau A, Hinz H, Gonzalez-Cid Y (2020) Jellytoring: real-time jellyfish monitoring based on deep learning object detection. Sensors 20(6):1708. https://doi.org/10.3390/s20061708

    Article  Google Scholar 

  30. Mcilwaine B, Rivas Casado M (2021) JellyNet: the convolutional neural network jellyfish bloom detector. Int J Appl Earth Obs Geoinf 97:102279. https://doi.org/10.1016/j.jag.2020.102279

    Article  Google Scholar 

  31. Misra D (2019) Mish: a self regularized non-monotonic activation function. arXiv:1908.08681

  32. Miyao Y, Isobe A, et al. (2014) An application of low-altitude remote sensing using a vessel-towed balloon for monitoring jellyfish patchiness in coastal waters. J Remote Sens Soc Jpn 34(2):113–120

    Google Scholar 

  33. Peng F, Miao Z, Li F, Li Z (2021) S-FPN: a shortcut feature pyramid network for sea cucumber detection in underwater images. Expert Syst Appl, 115306

  34. Pierce J (2009) Prediction, location, collection and transport of jellyfish (Cnidaria) and their polyps. Zoo Biol 28(2):163–176. https://doi.org/10.1002/zoo.20218

    Article  Google Scholar 

  35. Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv:1804.02767

  36. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. Adv Neural Inform Process Syst 28:91–99

    Google Scholar 

  37. Rife J, Rock SM (2003) Segmentation methods for visual tracking of deep-ocean jellyfish using a conventional camera. IEEE J Oceanic Eng 28(4):595–608. https://doi.org/10.1109/JOE.2003.819315

    Article  Google Scholar 

  38. Roohi A, Kideys A E, Sajjadi A, Hashemian A, Pourgholam R, Fazli H, Khanari A G, Eker-Develi E (2010) Changes in biodiversity of phytoplankton, zooplankton, fishes and macrobenthos in the Southern Caspian Sea after the invasion of the ctenophore Mnemiopsis leidyi. Biol Invasions 12(7):2343–2361

    Article  Google Scholar 

  39. Samsuri SS, Arshad MR, AManaf A, Yaacob MIH (2017) Detection of jellyfish using acoustic sensor. Methods (San Diego, Calif.) 10:0–0049

    Google Scholar 

  40. Tian Y, Yang G, Wang Z, Wang H, Li E, Liang Z (2019) Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Comput Electron Agricul 157:417–426

    Article  Google Scholar 

  41. Wang C-Y, Liao H-Y M, Wu Y-H, Chen P-Y, Hsieh J-W, Yeh I-H (2020) CSPNet: a new backbone that can enhance learning capability of CNN. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 390–391

  42. Wang K, Liew J H, Zou Y, Zhou D, Feng J (2019) Panet: few-shot image semantic segmentation with prototype alignment. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9197–9206

  43. Woo S, Park J, Lee J-Y, Kweon I S (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  44. Xiufen Wang, Huiyuan Wang, Song Wang (2011) Jellyfish detection based on K-FOE residual map and ring segmentation. In: 2011 IEEE 13th international conference on communication technology. IEEE, Jinan, pp 762–766. https://doi.org/10.1109/ICCT.2011.6157979

  45. Yi Z, Yongliang S, Jun Z (2019) An improved tiny-yolov3 pedestrian detection algorithm. Optik 183 :17–23

    Article  Google Scholar 

  46. Zhang J, Zhang X (2019) Automatic recognition and counting method of deep-sea jellyfish based on image multi-feature matching. In: 2019 11th International conference on intelligent human-machine systems and cybernetics (IHMSC), vol 1. IEEE, pp 233–236

  47. Zheng Z, Wang P, Liu W, Li J, Ye R, Ren D (2020) Distance-IoU loss: faster and better learning for bounding box regression. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 12993–13000

  48. Zi W, Yong T, Yuanyuan F, Wei M, Shuai W, Xiaolin L (2020) Monitoring of biomass at cooling water system of hongyanhe nuclear power plant by using acoustic methods. In: E3S web of conferences, vol 194. EDP Sciences, p 01007

Download references

Acknowledgments

This work was supported in part by the Natural Science Foundation of Hebei Province, Research on 3D dense reconstruction of underwater vision based on deep learning and point cloud quadratic determination (Grant numbers F2019203195) and National Natural Science Foundation of China, Research on occlusion perception, repair and reliability evaluation method for occlusion face recognition (Grant numbers 62106214).

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Yanshan University, Qinhuangdao, 066004, China

    Wengming Zhang, Feng Rui, Cunjun Xiao, Haibin Li & Yaqian Li

  2. Key Laboratory of Industrial Computer Control Engineering of Hebei Province, Qinhuangdao, 066004, China

    Wengming Zhang, Feng Rui, Cunjun Xiao, Haibin Li & Yaqian Li

Authors
  1. Wengming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feng Rui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cunjun Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haibin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yaqian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cunjun Xiao.

Ethics declarations

Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, W., Rui, F., Xiao, C. et al. JF-YOLO: the jellyfish bloom detector based on deep learning. Multimed Tools Appl 83, 7097–7117 (2024). https://doi.org/10.1007/s11042-023-15465-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15465-z

Keywords

Search

Navigation