Abstract
Product recognition continues to be a major access barrier for visually impaired (VI) and blind individuals in modern supermarkets. R&D approaches to this problem in the assistive technology (AT) literature vary from automated vision-based solutions to crowdsourcing applications where VI clients send image identification requests to web services. The former struggle with run-time failures and scalability while the latter must cope with concerns about trust, privacy, and quality of service. In this paper, we investigate a mobile cloud computing framework for remote caregiving that may help VI and blind clients with product recognition in supermarkets. This framework emphasizes remote teleassistance and assumes that clients work with dedicated caregivers (helpers). Clients tap on their smartphones’ touchscreens to send images of products they examine to the cloud where the SURF algorithm matches incoming image against its image database. Images along with the names of the top 5 matches are sent to remote sighted helpers via push notification services. A helper confirms the product’s name, if it is in the top 5 matches, or speaks or types the product’s name, if it is not. Basic quality of service is ensured through human eyesight sharing even when image matching does not work well. We implemented this framework in a module called EyeShare on two Android 2.3.3/2.3.6 smartphones. EyeShare was tested in three experiments with one blindfolded subject: one lab study and two experiments in Fresh Market, a supermarket in Logan, Utah. The results of our experiments show that the proposed framework may be used as a product identification solution in supermarkets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bujacz, M., Baranski, P., Moranski, M., Strumillo, P., Materka, A.: Remote Guidance for the Blind - A Proposed Teleassistance System and Navigation Trials. In: Proceedings of the Conference on Human System Interactions, pp. 888–892. IEEE, Krakow (2008)
Peake, P., Leonard, J.: The Use of Heart-Rate as an Index of Stress in Blind Pedestrians. Ergonomics (1971)
Kutiyanawala, A., Kulyukin, V., Nicholson, J.: Teleassistance in Accessible Shopping for the Blind. In: Proceedings of the 2011 International Conference on Internet Computing, July 18-21, pp. 190–193. ICOMP Press, Las Vegas (2011)
Kulyukin, V., Kutiyanawala, A.: Accessible Shopping Systems for Blind and Visually Impaired Individuals: Design Requirements and the State of the Art. The Open Rehabilitation Journal 2, 158–168 (2010), 10.2174/1874943701003010158, ISSN: 1874-9437
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded Up Robust Features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)
Sam, S., Tsai, S., Chen, D., Chandrasekhar, V., Takacs, G., Ngai-Man, C.:
Vedantham, R., Grzeszczuk, R., Girod, B.: Mobile Product Recognition. In: Proceedings of the International Conference on Multimedia (MM 2010), pp. 1587–1590. ACM, New York (1873), http://doi.acm.org/10.1145/1873951.1874293 , doi:10.1145/1873951.1874293
Girod, B., Chandrasekhar, V., Chen, D.M., Ngai-Man, C., Grzeszczuk, R., Reznik, Y., Takacs, G., Tsai, S.S., Vedantham, R.: Mobile Visual Search. IEEE Signal Processing Magazine 28(4), 61–76 (2011), doi:10.1109/MSP.2011.940881
von Reischach, F., Michahelles, F., Guinard, D., Adelmann, R., Fleisch, E., Schmidt, A.: An Evaluation of Product Identification Techniques for Mobile Phones. In: Gross, T., Gulliksen, J., Kotzé, P., Oestreicher, L., Palanque, P., Prates, R.O., Winckler, M. (eds.) INTERACT 2009. LNCS, vol. 5726, pp. 804–816. Springer, Heidelberg (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kulyukin, V., Zaman, T., Andhavarapu, A., Kutiyanawala, A. (2012). Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Cloud Computing. In: Miesenberger, K., Karshmer, A., Penaz, P., Zagler, W. (eds) Computers Helping People with Special Needs. ICCHP 2012. Lecture Notes in Computer Science, vol 7383. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31534-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-31534-3_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31533-6
Online ISBN: 978-3-642-31534-3
eBook Packages: Computer ScienceComputer Science (R0)