Abstract
Many visually impaired individuals often use upward staircases to move to other floors, but it is difficult for them to find distant upward staircases. Several assistive systems have been proposed in the past, and the recent trends are smartphone-based systems. This paper described a CNN-based recognition method of upward staircases. The recognition method was a key technology for our smartphone-based assistive system. In the method, GoogLeNet models were used as CNNs. Two types of image data augmentation were used beforehand. One was the data augmentation based on the Affine transformation, and the other was the data augmentation based on the Cutout technique, where square- and human-type masks were arranged in the grid positions of images. These masks were able to emulate situations where upward staircases were partially occluded by persons. These data augmentation produced four image datasets and therefore four GoogLeNet models, which were applied to the 560 images of actual 28 environments. There were upward staircases in the 14 environments, and not in the other 14 environments. The recognition accuracy was evaluated by F-measures. When the square-type masks were used, the maximum F-measure was 0.95.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
WHO: World health organization, media centre, visual impairment and blindness, fact sheet n\(^o\) 282 (2017). http://www.who.int/mediacentre/factsheets/fs282/en/. Accessed 15 Jan 2018
Kaur, P., Kaur, S.: Proposed hybrid color histogram based obstacle detection technique. In: Proceedings of the Third International Symposium on Computer Vision and the Internet, VisionNet 20116, pp. 88–97. Association for Computing Machinery, New York (2016)
Yasumuro, Y., Murakami, M., Imura, M., Kuroda, T., Manabe, Y., Chihara, K.: E-cane with situation presumption for the visually impaired. In: Carbonell, N., Stephanidis, C. (eds.) UI4ALL 2002. LNCS, vol. 2615, pp. 409–421. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-36572-9_32
Filipe, V., Fernandes, F., Fernandes, H., Sousa, A., Paredes, H., Barroso, J.: Blind navigation support system based on Microsoft Kinect. In: Proceedings of the 4th International Conference on Software Development for Enhancing Accessibility and Fighting Info-exclusion (DSAI 2012), Douro Region, Portugal, pp. 94–101 (2012)
Wang, S., Pan, H., Zhang, C., Tian, Y.: RGB-D image-based detection of stairs, pedestrian crosswalks and traffic signs. J. Vis. Commun. Image Represent. 25(2), 263–272 (2014)
Takizawa, H., Yamaguchi, S., Aoyagi, M., Ezaki, N., Mizuno, S.: Kinect cane: an assistive system for the visually impaired based on the concept of object recognition aid. Pers. Ubiquit. Comput. 19(5), 955–965 (2015). https://doi.org/10.1007/s00779-015-0841-4
Nakamura, D., Takizawa, H., Aoyagi, M., Ezaki, N., Mizuno, S.: Smartphone-based escalator recognition for the visually impaired. Sensors 17(5), 1057 (2017)
Iwamoto, T., Takizawa, H., Aoyagi, M.: A preliminary study on recognition of restroom signs by use of SVM and CNN (in Japanese). IEICE Technical reports (WIT), No. 180, pp. 11–14. IEICE (2018)
Szegedy, C., et al.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)
Devries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. CoRR, Vol. abs/1708.04552 (2017)
Acknowledgment
This work was supported in part by the JSPS KAKENHI Grant Number 19H04500.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Takizawa, H., Sekita, G., Kobayashi, M., Ohya, A., Aoyagi, M. (2021). Smartphone-Based Recognition Aid of Upward Staircases with People for the Visually Impaired. In: Stephanidis, C., Antona, M., Ntoa, S. (eds) HCI International 2021 - Posters. HCII 2021. Communications in Computer and Information Science, vol 1420. Springer, Cham. https://doi.org/10.1007/978-3-030-78642-7_75
Download citation
DOI: https://doi.org/10.1007/978-3-030-78642-7_75
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78641-0
Online ISBN: 978-3-030-78642-7
eBook Packages: Computer ScienceComputer Science (R0)