Skip to main content
SpringerLink
Log in

Multi-modal Obstacle Avoidance in USVs via Anomaly Detection and Cascaded Datasets

  • Conference paper
  • First Online:
Advanced Concepts for Intelligent Vision Systems (ACIVS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14124))

  • 209 Accesses

Abstract

We introduce a novel strategy for obstacle avoidance in aquatic settings, using anomaly detection for quick deployment of autonomous water vehicles in limited geographic areas. The unmanned surface vehicle (USV) is initially manually navigated to collect training data. The learning phase involves three steps: learning imaging modality specifics, learning the obstacle-free environment using collected data, and setting obstacle detector sensitivity with images containing water obstacles. This approach, which we call cascaded datasets, works with different image modalities and environments without extensive marine-specific data. Results are demonstrated with LWIR and RGB images from river missions.

This work was financed by the Slovenian Research Agency (ARRS), research projects [J2-2506] and [J2-2501 (A)] and research programs [P2-0095] and [P2-0250 (B)].

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Stereolabs ZED stereo camera (only the left frame) and Device A-lab SmartIR384L thermal camera.

  2. 2.

    Data was sampled from a section between 46.0402\(^{\circ }\)N, 14.5125\(^{\circ }\)E and 46.0234\(^{\circ }\)N, 14.5079\(^{\circ }\)E.

References

  1. Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: GANomaly: semi-supervised anomaly detection via adversarial training. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11363, pp. 622–637. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20893-6_39

    Chapter  Google Scholar 

  2. Bergman, L., Hoshen, Y.: Classification-based anomaly detection for general data. In: International Conference on Learning Representations (ICLR) (2020)

    Google Scholar 

  3. Bergmann, P., Fauser, M., Sattlegger, D., Steger, C.: MVTec AD – a comprehensive real-world dataset for unsupervised anomaly detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9584–9592 (2019)

    Google Scholar 

  4. Bergmann, P., Fauser, M., Sattlegger, D., Steger, C.: Uninformed students: student-teacher anomaly detection with discriminative latent embeddings. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4183–4192 (2020)

    Google Scholar 

  5. Bovcon, B., Kristan, M.: A water-obstacle separation and refinement network for unmanned surface vehicles. In: IEEE International Conference on Robotics and Automation (ICRA) (2020)

    Google Scholar 

  6. Bovcon, B., Kristan, M.: WaSR—a water segmentation and refinement maritime obstacle detection network. IEEE Trans. Cybern. 52, 12661–12674 (2021)

    Article  Google Scholar 

  7. Bovcon, B., Muhovic, J., Perš, J., Kristan, M.: The MaSTr1325 dataset for training deep USV obstacle detection models. In: IEEE International Conference on Intelligent Robots and Systems (IROS), pp. 3431–3438 (2019)

    Google Scholar 

  8. Bovcon, B., Muhovič, J., Vranac, D., Mozetič, D., Perš, J., Kristan, M.: MODS-A USV-oriented object detection and obstacle segmentation benchmark. IEEE Trans. Intell. Transp. Syst. 23, 13403–13418 (2021)

    Article  Google Scholar 

  9. Caesar, H., et al.: NuScenes: a multimodal dataset for autonomous driving. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11621–11631 (2020)

    Google Scholar 

  10. Cane, T., Ferryman, J.: Evaluating deep semantic segmentation networks for object detection in maritime surveillance. In: IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6 (2018)

    Google Scholar 

  11. Cohen, N., Hoshen, Y.: Sub-image anomaly detection with deep pyramid correspondences. arXiv preprint arXiv:2005.02357 (2020)

  12. Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3213–3223 (2016)

    Google Scholar 

  13. Defard, T., Setkov, A., Loesch, A., Audigier, R.: PaDiM: a patch distribution modeling framework for anomaly detection and localization. In: International Conference on Pattern Recognition (ICPR), vol. 12664, pp. 475–489 (2020)

    Google Scholar 

  14. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 248–255 (2009)

    Google Scholar 

  15. Fei, Y., Huang, C., Jinkun, C., Li, M., Zhang, Y., Lu, C.: Attribute restoration framework for anomaly detection. IEEE Trans. Multimed. 24, 116–127 (2021)

    Google Scholar 

  16. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3354–3361 (2012)

    Google Scholar 

  17. González, A., et al.: Pedestrian detection at day/night time with visible and FIR cameras: a comparison. Sensors 16(6), 820 (2016)

    Article  Google Scholar 

  18. Gudovskiy, D., Ishizaka, S., Kozuka, K.: CFLOW-AD: real-time unsupervised anomaly detection with localization via conditional normalizing flows. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1819–1828 (2022)

    Google Scholar 

  19. Haselmann, M., Gruber, D.P., Tabatabai, P.: Anomaly detection using deep learning based image completion. In: International Conference on Machine Learning and Applications (ICMLA) (2018)

    Google Scholar 

  20. Heidarsson, H.K., Sukhatme, G.S.: Obstacle detection and avoidance for an autonomous surface vehicle using a profiling sonar. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 731–736 (2011)

    Google Scholar 

  21. Hendrycks, D., Mazeika, M., Kadavath, S., Song, D.: Using self-supervised learning can improve model robustness and uncertainty. In: Advances in Neural Information Processing Systems (NIPS) (2019)

    Google Scholar 

  22. Hermann, D., Galeazzi, R., Andersen, J., Blanke, M.: Smart sensor based obstacle detection for high-speed unmanned surface vehicle. In: IFAC Conference on Manoeuvring and Control of Marine Craft (MCMC), vol. 48, pp. 190–197 (2015)

    Google Scholar 

  23. Jeong, M., Li, A.Q.: Efficient LiDAR-based in-water obstacle detection and segmentation by autonomous surface vehicles in aquatic environments. In: IEEE International Conference on Intelligent Robots and Systems (IROS), pp. 5387–5394 (2021)

    Google Scholar 

  24. Jia, X., Zhu, C., Li, M., Tang, W., Zhou, W.: LLVIP: a visible-infrared paired dataset for low-light vision. In: IEEE International Conference on Computer Vision (ICCV), pp. 3496–3504 (2021)

    Google Scholar 

  25. Kiefer, B., et al.: 1st workshop on maritime computer vision (MaCVi) 2023: challenge results. In: IEEE Winter Conference on Applications of Computer Vision Workshops (WACVW), pp. 265–302 (2023)

    Google Scholar 

  26. Kim, H., et al.: Vision-based real-time obstacle segmentation algorithm for autonomous surface vehicle. IEEE Access 7, 179420–179428 (2019)

    Article  Google Scholar 

  27. Kristan, M., Kenk, V.S., Kovačič, S., Perš, J.: Fast image-based obstacle detection from unmanned surface vehicles. IEEE Trans. Cybern. 46(3), 641–654 (2016)

    Article  Google Scholar 

  28. Lee, S.J., Roh, M.I., Lee, H.W., Ha, J.S., Woo, I.G.: Image-based ship detection and classification for unmanned surface vehicle using real-time object detection neural networks. In: International Ocean and Polar Engineering Conference (ISOPE) (2018)

    Google Scholar 

  29. Ma, L., Xie, W., Huang, H.: Convolutional neural network based obstacle detection for unmanned surface vehicle. Math. Biosci. Eng. 17, 845–861 (2020)

    Article  MathSciNet  Google Scholar 

  30. Mai, K.T., Davies, T., Griffin, L.D.: Brittle features may help anomaly detection. In: Women in Computer Vision Workshop at CVPR (2021)

    Google Scholar 

  31. Moosbauer, S., Konig, D., Jakel, J., Teutsch, M.: A benchmark for deep learning based object detection in maritime environments. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2019)

    Google Scholar 

  32. Muhovič, J., Mandeljc, R., Bovcon, B., Kristan, M., Perš, J.: Obstacle tracking for unmanned surface vessels using 3-D point cloud. IEEE J. Oceanic Eng. 45, 786–798 (2019)

    Article  Google Scholar 

  33. Nirgudkar, S., Robinette, P.: Beyond visible light: usage of long wave infrared for object detection in maritime environment. In: International Conference on Advanced Robotics (ICAR), pp. 1093–1100 (2021)

    Google Scholar 

  34. Noroozi, M., Favaro, P.: Unsupervised learning of visual representations by solving jigsaw puzzles. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 69–84. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_5

    Chapter  Google Scholar 

  35. Onunka, C., Bright, G.: Autonomous marine craft navigation: on the study of radar obstacle detection. In: International Conference on Control Automation Robotics & Vision (ICARCV), pp. 567–572 (2010)

    Google Scholar 

  36. Perš, J., et al.: Modular multi-sensor system for unmanned surface vehicles. In: International Electrotechnical and Computer Science Conference (ERK) (2021)

    Google Scholar 

  37. Qiao, D., Liu, G., Li, W., Lyu, T., Zhang, J.: Automated full scene parsing for marine ASVs using monocular vision. J. Intell. Robot. Syst. 104(2) (2022)

    Google Scholar 

  38. Rippel, O., Mertens, P., Merhof, D.: Modeling the distribution of normal data in pre-trained deep features for anomaly detection. In: International Conference on Pattern Recognition (ICPR) (2020)

    Google Scholar 

  39. Rudolph, M., Wehrbein, T., Rosenhahn, B., Wandt, B.: Fully convolutional cross-scale-flows for image-based defect detection. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1829–1838 (2022)

    Google Scholar 

  40. Ruff, L., et al.: Deep one-class classification. In: International Conference on Machine Learning (ICML), pp. 4393–4402 (2018)

    Google Scholar 

  41. Ruiz, A.R.J., Granja, F.S.: A short-range ship navigation system based on ladar imaging and target tracking for improved safety and efficiency. IEEE Trans. Intell. Transp. Syst. 10(1), 186–197 (2009)

    Article  Google Scholar 

  42. Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: f-AnoGAN: fast unsupervised anomaly detection with generative adversarial networks. Med. Image Anal. 54, 30–44 (2019)

    Article  Google Scholar 

  43. Steccanella, L., Bloisi, D., Castellini, A., Farinelli, A.: Waterline and obstacle detection in images from low-cost autonomous boats for environmental monitoring. Robot. Auton. Syst. 124, 103346 (2020)

    Article  Google Scholar 

  44. Teledyne: Teledyne FLIR thermal dataset for algorithm training. https://www.flir.eu/oem/adas/adas-dataset-form/

  45. Tsai, C.C., Wu, T.H., Lai, S.H.: Multi-scale patch-based representation learning for image anomaly detection and segmentation. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 3992–4000 (2022)

    Google Scholar 

  46. Žust, L., Kristan, M.: Learning maritime obstacle detection from weak annotations by scaffolding. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 955–964 (2022)

    Google Scholar 

  47. Wang, G., Han, S., Ding, E., Huang, D.: Student-teacher feature pyramid matching for unsupervised anomaly detection. In: British Machine Vision Conference (BMVC) (2021)

    Google Scholar 

  48. Wang, M., Wang, Q., Hong, D., Roy, S.K., Chanussot, J.: Learning tensor low-rank representation for hyperspectral anomaly detection. IEEE Trans. Cybern. 53(1), 679–691 (2022)

    Article  Google Scholar 

  49. Wang, W., Gheneti, B., Mateos, L.A., Duarte, F., Ratti, C., Rus, D.: Roboat: an autonomous surface vehicle for urban waterways. In: IEEE International Conference on Intelligent Robots and Systems (IROS), pp. 6340–6347 (2019)

    Google Scholar 

  50. Waymo: An autonomous driving dataset (2019)

    Google Scholar 

  51. Yang, J., Li, Y., Zhang, Q., Ren, Y.: Surface vehicle detection and tracking with deep learning and appearance feature. In: International Conference on Control, Automation and Robotics (ICCAR), pp. 276–280 (2019)

    Google Scholar 

  52. Yao, L., Kanoulas, D., Ji, Z., Liu, Y.: ShorelineNet: an efficient deep learning approach for shoreline semantic segmentation for unmanned surface vehicles. In: IEEE International Conference on Intelligent Robots and Systems (IROS), pp. 5403–5409 (2021)

    Google Scholar 

  53. Yu, J., et al.: FastFlow: unsupervised anomaly detection and localization via 2D normalizing flows. arXiv preprint arXiv:2111.07677 (2021)

  54. Zavrtanik, V., Kristan, M., Skočaj, D.: Reconstruction by inpainting for visual anomaly detection. Pattern Recogn. 112, 1–9 (2021)

    Article  Google Scholar 

  55. Zhan, W., et al.: Autonomous visual perception for unmanned surface vehicle navigation in an unknown environment. Sensors 19(10) (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering, University of Ljubljana, Tržaska 25, Ljubljana, Slovenia

    Tilen Cvenkel, Marija Ivanovska, Jon Muhovič & Janez Perš

Authors
  1. Tilen Cvenkel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marija Ivanovska
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jon Muhovič
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Janez Perš
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Janez Perš .

Editor information

Editors and Affiliations

  1. DGA TA, Toulouse, France

    Jaques Blanc-Talon

  2. University of Auckland, Auckland, New Zealand

    Patrice Delmas

  3. Ghent University, Ghent, Belgium

    Wilfried Philips

  4. University of Antwerp, Wilrijk, Belgium

    Paul Scheunders

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cvenkel, T., Ivanovska, M., Muhovič, J., Perš, J. (2023). Multi-modal Obstacle Avoidance in USVs via Anomaly Detection and Cascaded Datasets. In: Blanc-Talon, J., Delmas, P., Philips, W., Scheunders, P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2023. Lecture Notes in Computer Science, vol 14124. Springer, Cham. https://doi.org/10.1007/978-3-031-45382-3_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-45382-3_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-45381-6

  • Online ISBN: 978-3-031-45382-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation