Abstract
Obstacle detection addresses the detection of an object, of any kind, that interferes with the canonical trajectory of a subject, such as a human or an autonomous robotic agent. Prompt obstacle detection can become critical for the safety of visually impaired individuals (VII). In this context, we propose a novel methodology for obstacle detection, which is based on a Generative Adversarial Network (GAN) model, trained with human eye fixations to predict saliency, and the depth information provided by an RGB-D sensor. A method based on fuzzy sets are used to translate the 3D spatial information into linguistic values easily comprehensible by VII. Fuzzy operators are applied to fuse the spatial information with the saliency information for the purpose of detecting and determining if an object may interfere with the safe navigation of the VII. For the evaluation of our method we captured outdoor video sequences of 10,170 frames in total, with obstacles including rocks, trees and pedestrians. The results showed that the use of fuzzy representations results in enhanced obstacle detection accuracy, reaching 88.1%.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Rodríguez, A., Bergasa, L.M., Alcantarilla, P.F., Yebes, J., Cela, A.: Obstacle avoidance system for assisting visually impaired people. In: Proceedings of the IEEE Intelligent Vehicles Symposium Workshops, Madrid, Spain, p. 16 (2012)
Iakovidis, D.K., Diamantis, D., Dimas, G., Ntakolia, C., Spyrou, E.: Digital enhancement of cultural experience and accessibility for the visually impaired. In: Paiva, S. (ed.) Improved Mobility for the Visually Impaired. Springer (2019, to appear)
Brassai, S.T., Iantovics, B., Enachescu, C.: Optimization of robotic mobile agent navigation. Stud. Inform. Control 21, 403–412 (2012)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)
Kaur, B., Bhattacharya, J.: A scene perception system for visually impaired based on object detection and classification using multi-modal DCNN. arXiv preprint arXiv:1805.08798 (2018)
Tapu, R., Mocanu, B., Zaharia, T.: DEEP-SEE: joint object detection, tracking and recognition with application to visually impaired navigational assistance. Sensors 17, 2473 (2017)
Suresh, A., Arora, C., Laha, D., Gaba, D., Bhambri, S.: Intelligent smart glass for visually impaired using deep learning machine vision techniques and robot operating system (ROS). In: Kim, J.-H., Myung, H., Kim, J., Xu, W., Matson, E.T., Jung, J.-W., Choi, H.-L. (eds.) RiTA 2017. AISC, vol. 751, pp. 99–112. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-78452-6_10
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Poggi, M., Mattoccia, S.: A wearable mobility aid for the visually impaired based on embedded 3D vision and deep learning. In: 2016 IEEE Symposium on Computers and Communication (ISCC), pp. 208–213 (2016)
Lee, C.-H., Su, Y.-C., Chen, L.-G.: An intelligent depth-based obstacle detection system for visually-impaired aid applications. In: 2012 13th International Workshop on Image Analysis for Multimedia Interactive Services, pp. 1–4. IEEE (2012)
Song, H., Liu, Z., Du, H., Sun, G.: Depth-aware saliency detection using discriminative saliency fusion. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1626–1630. IEEE (2016)
Mancini, M., Costante, G., Valigi, P., Ciarfuglia, T.A.: J-MOD 2: joint monocular obstacle detection and depth estimation. IEEE Robot. Autom. Lett. 3, 1490–1497 (2018)
Heinrich, S.: Fast obstacle detection using flow/depth constraint. In: 2002 Intelligent Vehicle Symposium, pp. 658–665. IEEE (2002)
Chen, L., Guo, B., Sun, W.: Obstacle detection system for visually impaired people based on stereo vision. In: 2010 Fourth International Conference on Genetic and Evolutionary Computing, pp. 723–726. IEEE (2010)
Pan, J., et al.: Salgan: visual saliency prediction with generative adversarial networks. arXiv preprint arXiv:1701.01081 (2017)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Bylinskii, Z., Recasens, A., Borji, A., Oliva, A., Torralba, A., Durand, F.: Where should saliency models look next? In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 809–824. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_49
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database (2009)
Nguyen, H.T., Walker, C.L., Walker, E.A.: A First Course in Fuzzy Logic. CRC Press, Boca Raton (2018)
Acknowledgments
This research has been co-financed by the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH – CREATE – INNOVATE (project code:Τ1EDK-02070). The Titan X used for this research was donated by the NVIDIA Corporation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Dimas, G., Ntakolia, C., Iakovidis, D.K. (2019). Obstacle Detection Based on Generative Adversarial Networks and Fuzzy Sets for Computer-Assisted Navigation. In: Macintyre, J., Iliadis, L., Maglogiannis, I., Jayne, C. (eds) Engineering Applications of Neural Networks. EANN 2019. Communications in Computer and Information Science, vol 1000. Springer, Cham. https://doi.org/10.1007/978-3-030-20257-6_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-20257-6_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20256-9
Online ISBN: 978-3-030-20257-6
eBook Packages: Computer ScienceComputer Science (R0)