skip to main content
10.1145/2971648.2971730acmconferencesArticle/Chapter ViewAbstractPublication PagesubicompConference Proceedingsconference-collections
research-article

CueSee: exploring visual cues for people with low vision to facilitate a visual search task

Authors Info & Claims
Published:12 September 2016Publication History

ABSTRACT

Visual search is a major challenge for low vision people. Conventional vision enhancements like magnification help low vision people see more details, but cannot indicate the location of a target in a visual search task. In this paper, we explore visual cues---a new approach to facilitate visual search tasks for low vision people. We focus on product search and present CueSee, an augmented reality application on a head-mounted display (HMD) that facilitates product search by recognizing the product automatically and using visual cues to direct the user's attention to the product. We designed five visual cues that users can combine to suit their visual condition. We evaluated the visual cues with 12 low vision participants and found that participants preferred using our cues to conventional enhancements for product search. We also found that CueSee outperformed participants' best-corrected vision in both time and accuracy.

Skip Supplemental Material Section

Supplemental Material

References

  1. Alexander Andreopoulos and John K. Tsotsos. 2013. 50 Years of object recognition: Directions forward. Computer Vision and Image Understanding 117, 8: 827--891. http://doi.org/10.1016/j.cviu.2013.04.005Google ScholarGoogle ScholarCross RefCross Ref
  2. AOA (American Optometric Association). Common Types of Low Vision. Retrieved July 7, 2015 from http://www.aoa.org/patients-and-public/caring-for-your-vision/low-vision/common-types-of-low-vision?sso=yGoogle ScholarGoogle Scholar
  3. AOA. What Causes Low Vision? Retrieved July 7, 2015 from http://www.aoa.org/patients-and-public/caring-for-your-vision/low-vision/what-causes-low-vision?sso=yGoogle ScholarGoogle Scholar
  4. Apple. Apple - Accessibility - iOS. Retrieved July 7, 2015 from https://www.apple.com/accessibility/ios/Google ScholarGoogle Scholar
  5. Jeffrey P Bigham. 2014. Making the Web Easier to See with Opportunistic Accessibility Improvement. 117--122. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Jeffrey P. Bigham, Chandrika Jayant, Andrew Miller, Brandyn White, and Tom Yeh. 2010. VizWiz::LocateIt - enabling blind people to locate objects in their environment. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, IEEE, 65--72. http://doi.org/10.1109/CVPRW.2010.5543821Google ScholarGoogle ScholarCross RefCross Ref
  7. P Burnard. 1991. A method of analysing interview transcripts in qualitative research. Nurse education today 11, 6: 461--466. http://doi.org/10.1016/0260-6917(91)90009-YGoogle ScholarGoogle Scholar
  8. CDC (Center for Disease Control). 2004. Summary Health Statistics for the U.S. Population: National Health Interview Survey, 2004. Retrieved May 3, 2015 from http://www.cdc.gov/nchs/data/series/sr_10/sr10_229.pdfGoogle ScholarGoogle Scholar
  9. CHILI. 2013. Chilitags, Robust Fiducial Markers for Augmented Reality. Retrieved September 16, 2015 from http://chili.epfl.ch/softwareGoogle ScholarGoogle Scholar
  10. V. R. Cimarolli, K. Boerner, M. Brennan-Ing, J. P. Reinhardt, and a. Horowitz. 2012. Challenges faced by older adults with vision loss: a qualitative study with implications for rehabilitation. Clinical Rehabilitation 26: 748--757. http://doi.org/10.1177/0269215511429162Google ScholarGoogle ScholarCross RefCross Ref
  11. eSight. eSight - Home Page. Retrieved July 7, 2015 from http://esighteyewear.com/Google ScholarGoogle Scholar
  12. Steven Feiner. 2014. ParaFrustum : Visualization Techniques for Guiding a User to a Constrained Set of Viewing Positions and Orientations. Proceedings of the Annual ACM Symposium on User Interface Software and Technology (UIST): 331--340. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Grace Sze-en Foo. 2009. Grocery Shopping Assistant for the Blind / Visually Impaired.Google ScholarGoogle Scholar
  14. Chaitanya P. Gharpure and Vladimir a. Kulyukin. 2008. Robot-assisted shopping for the blind: Issues in spatial cognition and product selection. Intelligent Service Robotics 1, 3: 237--251. http://doi.org/10.1007/s11370-008-0020-9Google ScholarGoogle ScholarCross RefCross Ref
  15. CP Gharpure and VA Kulyukin. 2008. Robot-assisted shopping for the blind: issues in spatial cognition and product selection. Intelligent Service Robotics. Retrieved July 7, 2015 from http://link.springer.com/article/10.1007/s11370-008-0020-9Google ScholarGoogle Scholar
  16. Google. Android accessibility features - Android Accessibility Help.Google ScholarGoogle Scholar
  17. David S. Hayden, Michael Astrauskas, Qian Yan, Liqing Zhou, and John a. Black. 2011. Note-taker 3.0. Assets (Xiii): 269. http://doi.org/10.1145/2049536.2049601Google ScholarGoogle Scholar
  18. Bill Holton. 2014. A Review of iOS Access for All: Your Comprehensive Guide to Accessibility for iPad, iPhone, and iPod Touch, by Shelly Brisbin.Google ScholarGoogle Scholar
  19. K. Holtzblatt and S. Jones. 1993. Contextual inquiry: A participatory technique for system design. In Participatory design: Principles and practice, D. Schuler and A. Namioka (eds.). Lawrence Earlbaum, Hillsdale, NJ, 180--193.Google ScholarGoogle Scholar
  20. Weiyin Hong, James Y. L. Thong, and Kar Yan Tam. 2004. Does Animation Attract Online Users' Attention? The Effects of Flash on Information Search Performance and Perceptions. Information Systems Research. Retrieved September 16, 2015 from http://pubsonline.informs.org/doi/abs/10.1287/isre.1040.0017 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. D Hwang and E Peli. 2014. An augmented-reality edge enhancement application for Google glass. Optometry and Vision Science 91, 8: 1021--1030. http://doi.org/10.1097/OPX.0000000000000326Google ScholarGoogle ScholarCross RefCross Ref
  22. Rabia Jafri, Syed Abid Ali, and Hamid R. Arabnia. 2013. Computer Vision-based Object Recognition for the Visually Impaired Using Visual Tags. International Conference on Image Processing, Computer Vision, and Pattern Recognition: 400--406.Google ScholarGoogle Scholar
  23. Richard L. Kline and Ephraim P. Glinert. 1995. Improving GUI accessibility for people with low vision. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '95: 114--121. http://doi.org/10.1145/223904.223919 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Vladimir Kulyukin and Aliasgar Kutiyanawala. 2010. From ShopTalk to ShopMobile: vision-based barcode scanning with mobile phones for independent blind grocery shopping. Proceedings of the 2010 Rehabilitation Engineering and Assistive Technology Society of North America Conference (RESNA 2010), Las Vegas, NV, 703: 1--5.Google ScholarGoogle Scholar
  25. Raja S. Kushalnagar, Stephanie a. Ludi, and Poorna Kushalnagar. 2011. Multi-View Platform: An Accessible Live Classroom Viewing Approach for Low Vision Students. Assets (Xiii): 267. http://doi.org/10.1145/2049536.2049600 Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Patrick E. Lanigan, Aaron M. Paulos, Andrew W. Williams, Dan Rossi, and Priya Narasimhan. 2007. Trinetra: Assistive technologies for grocery shopping for the blind. Proceedings - International Symposium on Wearable Computers, ISWC: 147--148. http://doi.org/10.1109/ISWC.2006.286369Google ScholarGoogle Scholar
  27. Mary Leach. Peripheral Prism Glasses Are a Simple, Inexpensive Rehabilitation Tool for Patients with Hemianopia | Massachusetts Eye and Ear. Retrieved March 29, 2016 from http://www.masseyeandear.org/news/press-releases/2013/11/2013-prism-glassesGoogle ScholarGoogle Scholar
  28. GE Legge, GS Rubin, DG Pelli, and MM Schleske. 1985. Psychophysics of reading---II. Low vision. Vision research.Google ScholarGoogle Scholar
  29. Gang Luo and Eli Peli. 2006. Use of an augmented-vision device for visual search by patients with tunnel vision. Investigative Ophthalmology and Visual Science 47, 9: 4152--4159. http://doi.org/10.1167/iovs.05-1672Google ScholarGoogle ScholarCross RefCross Ref
  30. MAGic. The best screen magnifier - MAGic. Retrieved July 7, 2015 from http://www.freedomscientific.com/Products/LowVision/MAGicGoogle ScholarGoogle Scholar
  31. Peter McLeod, Jon Driver, and Jennie Crisp. 1988. Visual Search for a Conjunction of Movement and Form is parallel. Nature 336: 403--405.Google ScholarGoogle Scholar
  32. Michele Merler, Carolina Galleguillos, and Serge Belongie. 2007. Recognizing groceries in situ using in vitro training data. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. http://doi.org/10.1109/CVPR.2007.383486Google ScholarGoogle ScholarCross RefCross Ref
  33. J. Elton Moore and B.J. LeJeune. Low Vision | International Encyclopedia of Rehabilitation. Retrieved July 7, 2015 from http://cirrie.buffalo.edu/encyclopedia/en/article/17/Google ScholarGoogle Scholar
  34. Nhung Xuan Nguyen, Malte Weismann, and Susanne Trauzettel-Klosinski. 2009. Improvement of reading speed after providing of low vision aids in patients with age-related macular degeneration. Acta Ophthalmologica 87, 8: 849--853. http://doi.org/10.1111/j.1755-3768.2008.01423.xGoogle ScholarGoogle ScholarCross RefCross Ref
  35. John Nicholson, Vladimir Kulyukin, and Daniel Coster. 2009. ShopTalk: Independent Blind Shopping Through Verbal Route Directions and Barcode Scans. The Open Rehabilitation Journal 2, 1: 11--23. http://doi.org/10.2174/1874943700902010011Google ScholarGoogle ScholarCross RefCross Ref
  36. E L I Peli. 2001. Vision Multiplexing : an Engineering Approach. Optometry & Vision Science 78, 5: 304--315.Google ScholarGoogle ScholarCross RefCross Ref
  37. E Peli and T Peli. 1984. Image enhancement for the visually impaired. Optical Engineering 23, 1: 47--51. Retrieved from http://opticalengineering.spiedigitallibrary.org/article.aspx?articleid=1244416Google ScholarGoogle ScholarCross RefCross Ref
  38. Eli Peli, Gang Luo, Alex Bowers, and Noa Rensing. 2007. Applications of augmented-vision head-mounted systems in vision rehabilitation. Journal of the Society for Information Display 15: 1037--1045. http://doi.org/10.1889/1.2825088Google ScholarGoogle ScholarCross RefCross Ref
  39. Denis G Pelli. 2008. Crowding: a cortical constraint on object recognition. Current opinion in neurobiology 18, 4: 445--51. http://doi.org/10.1016/j.conb.2008.09.008Google ScholarGoogle Scholar
  40. B Phillips and H Zhao. 1993. Predictors of assistive technology abandonment. Assistive technology : the official journal of RESNA 5, 1: 36--45. http://doi.org/10.1080/10400435.1993.10132205Google ScholarGoogle Scholar
  41. GS Rubin and GE Legge. 1989. Psychophysics of reading. VI---The role of contrast in low vision. Vision Research.Google ScholarGoogle Scholar
  42. PremNandhini Satgunam, Russell L. Woods, Gang Luo, et al. 2012. Effects of Contour Enhancement on Low-Vision Preference and Visual Search. Optometry and Vision Science 89, 9: E1364--E1373. http://doi.org/10.1097/OPX.0b013e318266f92fGoogle ScholarGoogle ScholarCross RefCross Ref
  43. Liz Segre and Gary Heiting. 2014. Low Vision Aids for Reading.Google ScholarGoogle Scholar
  44. Kristen Shinohara and Jacob O. Wobbrock. 2011. In the shadow of misperception. Proceedings of the 2011 annual conference on Human factors in computing systems - CHI '11: 705. http://doi.org/10.1145/1978942.1979044Google ScholarGoogle Scholar
  45. Liz Simon. 2008. Low vision and rehabilitation for older people: integrating services into the health care system. 21, 66: 28--30.Google ScholarGoogle Scholar
  46. AI Squared. Ai Squared - We've got accessibility covered. Retrieved July 7, 2015 from http://www.aisquared.com/zoomtextGoogle ScholarGoogle Scholar
  47. Sarit Szprio, Yuhang Zhao, and Shiri Azenkot. 2016. Finding a Store, Searching for a Product: A Study of Daily Challenges of Low Vision People. Ubicomp 2016.Google ScholarGoogle Scholar
  48. Fernando Vargas-Martín and Eli Peli. 2001. P-16: Augmented View for Tunnel Vision: Device Testing by Patients in Real Environments. SID Symposium Digest of Technical Papers 32, 1: 602. http://doi.org/10.1889/1.1831932Google ScholarGoogle ScholarCross RefCross Ref
  49. VisionAware. What Are Low Vision Optical Devices? Retrieved October 11, 2015 from http://www.visionaware.org/info/your-eye-condition/eye-health/low-vision/low-vision-optical-devices/1235Google ScholarGoogle Scholar
  50. Tess Winlock, Eric Christiansen, and Serge Belongie. 2010. Toward real-time grocery detection for the visually impaired. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, CVPRW 2010: 49--56. http://doi.org/10.1109/CVPRW.2010.5543576Google ScholarGoogle ScholarCross RefCross Ref
  51. Molly Wood. 2015. Video Feature: Signs That Virtual Reality Is on the Verge of Taking Off - The New York Times. The New York Times. Retrieved September 24, 2015 from http://www.nytimes.com/2015/01/29/technology/personaltech/video-feature-signs-that-virtual-reality-is-on-the-verge-of-taking-off.html?_r=0Google ScholarGoogle Scholar
  52. S Yantis and J Jonides. 1984. Abrupt visual onsets and selective attention: evidence from visual search. Journal of Experimental Psychology: Human Perception and Performance 10, 5: 601--621. http://doi.org/10.1037/0096-1523.10.5.601Google ScholarGoogle ScholarCross RefCross Ref
  53. Steven Yantis and John Jonides. 1990. Abrupt visual onsets and selective attention: Voluntary versus automatic allocation. Journal of Experimental Psychology: Human Perception and Performance 16, 1: 121--134.Google ScholarGoogle ScholarCross RefCross Ref
  54. Yuhang Zhao, Sarit Szpiro, and Shiri Azenkot. 2015. ForeSee: A Customizable Head-Mounted Vision Enhancement System for People with Low Vision. ASSETS '15 The 17th International ACM SIGACCESS Conference on Computers and Accessibility: 239--249. http://doi.org/10.1145/2700648.2809865 Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Low Vision | International Encyclopedia of Rehabilitation. Retrieved July 7, 2015 from http://cirrie.buffalo.edu/encyclopedia/en/article/17/Google ScholarGoogle Scholar
  56. LVC - Closed Circuit Television. Retrieved July 6, 2015 from http://www.lowvisioninfo.org/hightech.htmGoogle ScholarGoogle Scholar

Index Terms

  1. CueSee: exploring visual cues for people with low vision to facilitate a visual search task

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing
        September 2016
        1288 pages
        ISBN:9781450344616
        DOI:10.1145/2971648

        Copyright © 2016 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 12 September 2016

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

        Acceptance Rates

        UbiComp '16 Paper Acceptance Rate101of389submissions,26%Overall Acceptance Rate764of2,912submissions,26%

        Upcoming Conference

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader