Skip to main content
SpringerLink
Log in

Development of outdoor swimmers detection system with small object detection method based on deep learning

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Wild swimming, or swimming in prohibited outdoor places, is a major source of drowning occurrences and a key problem in outdoor water safety management. Currently, manual patrol and warning signs are the basic methods adopted by the local government for outdoor water safety management to inspect drowning accidents. However, they are inefficient, costly, and of little avail. To this goal, a novel object detector for outdoor swimmers was developed via transfer learning utilizing the Microsoft Common Objects in Context (MS COCO) dataset as a training starting point. The model was then evaluated and retrained to possess the capacity to classify swimmers, suspected swimmers, and pedestrians. The total precision and detection time of our proposed swimmer detection with small object detection approach are 99.45% and 43.44 ms, respectively, which are greater than those of existing methods and traditional data augmentation methods. We verified the effectiveness of the proposed method on small target detection and designed two prototypes of hardware systems (fixed monitoring device and drone monitoring device) to meet the requirements of stationary and movable detection scenarios that can identify and warn of the possible phenomenon of wild swimming efficiently. This scheme can provide a more comprehensive reference for other innovative city applications that rely on cameras and can be valuable for society.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Chatterjee, R., Chatterjee, A., Islam, S., Khan, M.K.: An object detection-based few-shot learning approach for multimedia quality assessment. Multimed. Syst. 1–14 (2022). https://doi.org/10.1007/s00530-021-00881-8. Accessed 29 Jan 2022

  2. Chua, S.D., Lim, S., Lai, S., Chang, T.: Development of a child detection system with artificial intelligence using object detection method. J. Electr. Eng. Technol. 14(6), 2523–2529 (2019)

    Article  Google Scholar 

  3. Zou, Z., Shi, Z., Guo, Y., Ye J.: Object detection in 20 years: a survey. (2019). arXiv preprint arXiv:1905.05055

  4. Vahab, A., Naik, M.S., Raikar, P.G.: Applications of object detection system. Int. Res. J. Eng. Technol. 6(4), 4186–4192 (2019)

    Google Scholar 

  5. Pranav, M V., Shreyas Madhav, A V., Meena, J.: DeepRecog: threefold underwater image deblurring and object recognition framework for AUV vision systems. Multimed. Syst. 28(2), 583–593 (2021)

  6. Heo, S., Cho, S., Kim, Y.: Real-time object detection system with multi-path neural networks. In: 2020 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS), pp. 174–187 (2020)

  7. Liu, Y., Sun, P., Wergeles, N.: A survey and performance evaluation of deep learning methods for small object detection. Expert Syst. Appl. 172, 114602 (2021)

    Article  Google Scholar 

  8. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)

    Article  Google Scholar 

  9. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), pp. 886–893 (2005)

  10. Sun, Z., Bebis, G., Miller, R.: On-road vehicle detection: a review. IEEE Trans. Pattern Anal. Mach. Intell. 28(5), 694–711 (2006)

    Article  Google Scholar 

  11. Eng, H.-L., Wang, J., Kam, A. H., Yau, W.-Y.: Novel region-based modeling for human detection within highly dynamic aquatic environment. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’ 04), p. II (2004)

  12. Eng, H.-L., Wang, J., Wah, A.S., Yau, W.-Y.: Robust human detection within a highly dynamic aquatic environment in real time. IEEE Trans. Image Process. 15(6), 1583–1600 (2006)

    Article  Google Scholar 

  13. Zecha, D., Greif, T., Lienhart, R.: Swimmer detection and pose estimation for continuous stroke-rate determination. In: Multimedia on Mobile Devices 2012; and Multimedia Content Access: Algorithms and Systems VI, p. 830410 (2012)

  14. Eng, H.-L., Toh, K.-A., Yau, W.-Y., Wang, J.: DEWS: a live visual surveillance system for early drowning detection at pool. IEEE Trans. Circuits Syst. Video Technol. 18(2), 196–210 (2008)

    Article  Google Scholar 

  15. Chan, K.L.: Detection of swimmer using dense optical flow motion map and intensity information. Mach. Vis. Appl. 24(1), 75–101 (2013)

    Article  Google Scholar 

  16. Hong, D., Kim, Y.: Efficient swimmer detection algorithm using CNN-based SVM. J. Korean Inf. Sci. Soc. 22(12), 79–85 (2017)

    Google Scholar 

  17. Tong, K., Wu, Y., Zhou, F.: Recent advances in small object detection based on deep learning: a review. Image Vis. Comput. 97, 103910 (2020)

    Article  Google Scholar 

  18. Shrestha, A., Mahmood, A.: Review of deep learning algorithms and architectures. IEEE Access 7, 53040–53065 (2019)

    Article  Google Scholar 

  19. Shrivastava, A., Sukthankar, R., Malik, J., Gupta, A.: Beyond skip connections: top-down modulation for object detection. (2016). arXiv preprint arXiv:06851

  20. Bell, S., Zitnick, C. L., Bala, K., Girshick, R.: Inside–outside net: detecting objects in context with skip pooling and recurrent neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2874–2883 (2016)

  21. Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)

  22. Dai, J., Li, Y., He, K., Sun, J.: R-fcn: object detection via region-based fully convolutional networks. (2016). arXiv preprint arXiv:06409

  23. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the Ieee International Conference on Computer Vision, pp. 2961–2969 (2017)

  24. Ren, S., He, K., Girshick, R., Zhang, X., Sun, J.: Object detection networks on convolutional feature maps. IEEE Trans. Pattern Anal. Mach. Intell. 39(7), 1476–1481 (2016)

    Article  Google Scholar 

  25. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. (2015). arXiv preprint arXiv:01497

  26. Yang, F., Choi, W., Lin, Y.: Exploit all the layers: fast and accurate cnn object detector with scale dependent pooling and cascaded rejection classifiers. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2129–2137 (2016)

  27. Deng, H., Ergu, D., Liu, F., Ma, B., Cai, Y.: An embeddable algorithm for automatic garbage detection based on complex marine environment. Sensors 21(19), 6391 (2021)

    Article  Google Scholar 

  28. Li, X., Shang, M., Qin, H., Chen, L.: Fast accurate fish detection and recognition of underwater images with fast r-cnn. In: OCEANS 2015-MTS/IEEE Washington, pp. 1–5 (2015)

  29. Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)

  30. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)

  31. Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. (2018). arXiv preprint arXiv:02767

  32. Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y. M.: Yolov4: optimal speed and accuracy of object detection. (2020). arXiv preprint arXiv:10934

  33. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  34. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A. C.: Ssd: single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37 (2016)

  35. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: common objects in context. In: European Conference on Computer Vision, pp. 740–755 (2014)

  36. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  37. He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9), 1904–1916 (2015)

    Article  Google Scholar 

  38. Liu, S., Qi, L., Qin, H., Shi, J., Jia, J.: Path aggregation network for instance segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8759–8768 (2018)

  39. Misra, D.: Mish: a self regularized non-monotonic neural activation function. (2019). https://doi.org/10.48550/arXiv.1908.08681. arXiv:1605.06409

  40. Xu, J., Li, Z., Du, B.: Reluplex made more practical: leaky ReLU. In: 2020 IEEE Symposium on Computers and Communications (ISCC), pp. 1–7 (2020)

  41. Ghiasi, G., Lin, T.-Y., Le, Q. V.: Dropblock: a regularization method for convolutional networks. (2018). https://doi.org/10.48550/arXiv.1810.12890. arXiv:1810.12890

  42. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  43. Li, J., Liang, X., Shen, S., Xu, T., Feng, J., Yan, S.: Scale-aware fast R-CNN for pedestrian detection. IEEE Trans. Multimed. 20(4), 985–996 (2017)

    Google Scholar 

  44. Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

  45. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

  46. Yu, J., Jiang, Y., Wang, Z., Cao, Z., Huang, T.: Unitbox: an advanced object detection network. In: Proceedings of the 24th ACM International Conference on Multimedia, pp. 516–520 (2016)

  47. Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., Ren, D.: Distance-IoU loss: faster and better learning for bounding box regression. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 12993–13000 (2020)

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant nos. 61971078, 61501070) and Chongqing Municipal Education Commission (Grant no. CYS21478).

Author information

Author notes

  1. Yuewei Li, Yu Xiu and Qingling Xia contributed equally to this work.

Authors and Affiliations

  1. School of Artificial Intelligence, Chongqing University of Technology, Chongqing, 401135, China

    Hanguang Xiao, Yuewei Li & Qingling Xia

  2. School of Information and Computer, Anhui Polytechnic University, Anhui, 241000, China

    Yu Xiu

Authors
  1. Hanguang Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuewei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Xiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qingling Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hanguang Xiao.

Additional information

Communicated by I. Bartolini.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiao, H., Li, Y., Xiu, Y. et al. Development of outdoor swimmers detection system with small object detection method based on deep learning. Multimedia Systems 29, 323–332 (2023). https://doi.org/10.1007/s00530-022-00995-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-00995-7

Keywords

Search

Navigation