Skip to main content

Advertisement

Springer Nature Link
Log in

ARbility: re-inviting older wheelchair users to in-store shopping via wearable augmented reality

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

Engaging in outdoor daily activities such as shopping is an essential, yet challenging activity for older wheelchair users (OWU). However, little is investigated on how to augment the OWU’s independence during their in-person shopping experiences, specifically by addressing their physical conditions. We first conducted semi-structured interviews and a large-scale survey with 77 people in total to discover OWU’s needs and pain points in comparison with those of general older adults or wheelchair population. Based on these findings, we propose ARbility, a wearable AR-based shopping system for OWU which supports product recognition from seated positions on wheelchairs and one-stop shopping functionality for minimizing physical loads. In our user evaluation with 13 OWU in a real-world environment, ARbility demonstrated 33% decrease in arm movement, with the participants validating its efficacy and usability in qualitative interviews. We conclude with implications on how a wearable AR-based shopping system supports the active aging and inclusion of OWU.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Ahmed MS, Matsumura B, Cristian A (2005) Age-related changes in muscles and joints. Phys Med Rehabil Clin 16(1):19–39

    Article  Google Scholar 

  • Ahn J, Williamson J, Gartrell M, Han R, Lv Q, Mishra S (2015) Supporting healthy grocery shopping via mobile augmented reality. ACM Trans Multimed Comput Commun Appl. https://doi.org/10.1145/2808207

    Article  Google Scholar 

  • Andersen JH, Kaergaard A, Frost P, Thomsen JF, Bonde JP, Fallentin N, Borg V, Mikkelsen S (2002) Physical, psychosocial, and individual risk factors for neck/shoulder pain with pressure tenderness in the muscles among workers performing monotonous, repetitive work. Spine 27(6):660–667

    Article  Google Scholar 

  • Arshad H, Shukri S, Obeidy W, Zainal Abidin R (2017) An interactive application for halal products identification based on augmented reality. Int J Adv Sci Eng Inf Technol 7:139. https://doi.org/10.18517/ijaseit.7.1.1793

    Article  Google Scholar 

  • Becker SA (2004) E-government visual accessibility for older adult users. Soc Sci Comput Rev 22(1):11–23

    Article  Google Scholar 

  • Boldu R, Matthies DJC, Zhang H, Nanayakkara S (2020) Aisee: an assistive wearable device to support visually impaired grocery shoppers. ACM Interact Mob Wearable Ubiquitous Technol. https://doi.org/10.1145/3432196

    Article  Google Scholar 

  • Boninger ML (2007) Assessing mobility characteristics and activity levels of manual. J Rehabil Res Develop 44(4–7):561

    Google Scholar 

  • Boyarski D, Neuwirth C, Forlizzi J, Regli SH (1998) A study of fonts designed for screen display. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI ’98, pp. 87–94. ACM Press/Addison-Wesley Publishing Co., USA. https://doi.org/10.1145/274644.274658. https://doi.org/10.1145/274644.274658

  • Boyarski D, Neuwirth C, Forlizzi J, Regli SH (1998) A study of fonts designed for screen display. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI ’98, pp. 87–94. ACM Press/Addison-Wesley Publishing Co., USA. https://doi.org/10.1145/274644.274658. https://doi.org/10.1145/274644.274658

  • Bright AK, Coventry L (2013) Assistive technology for older adults: Psychological and socio-emotional design requirements. In: Proceedings of the 6th International Conference on PErvasive Technologies Related to Assistive Environments. PETRA ’13. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/2504335.2504344. https://doi.org/10.1145/2504335.2504344

  • Bromley R, Matthews D, Thomas C (2007) City centre accessibility for wheelchair users: the consumer perspective and the planning implications. Cities 24:229–241. https://doi.org/10.1016/j.cities.2007.01.009

    Article  Google Scholar 

  • Bureau UC (2021) National population totals and components of change: 2020-2021. https://www.census.gov/data/tables/time-series/demo/popest/2020s-national-total.html

  • Capoor J, Stein AB (2005) Aging with spinal cord injury. Phys Med Rehabil Clinics 16(1):129–161

    Article  Google Scholar 

  • Cohen RB, Williams GR Jr (1998) Impingement syndrome and rotator cuff disease as repetitive motion disorders. Clin Orthop Relat Res 351:95–101

    Article  Google Scholar 

  • Cornman JC, Freedman VA, Agree EM (2005) Measurement of assistive device use: implications for estimates of device use and disability in late life. The Gerontologist 45(3):347–358

    Article  Google Scholar 

  • Dalton NS, Collins E, Marshall P (2015) Display blindness? looking again at the visibility of situated displays using eye-tracking. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 3889–3898

  • Electronics L (2018) LG G7+ ThinQ. https://www.lg.com/hk_en/mobile-phone/lg-LMG710EAW-128GB. Accessed 2 Nov 2021

  • Electronics S (2021) Samsung Bixby: your personal voice assistant: Samsung US. https://www.samsung.com/us/explore/bixby/. Accessed 2 Nov 2021

  • Fernández-Ballesteros R (2008) Active Aging: The Contribution of Psychology, pp. 7–18. Hogrefe Publishing

  • Frost P, Bonde JPE, Mikkelsen S, Andersen JH, Fallentin N, Kaergaard A, Thomsen JF (2002) Risk of shoulder tendinitis in relation to shoulder loads in monotonous repetitive work. Am J Indus Med 41(1):11–18

    Article  Google Scholar 

  • Geng W, Han F, Lin J, Zhu L, Bai J, Wang S, He L, Xiao Q, Lai Z (2018) Fine-grained grocery product recognition by one-shot learning. In: Proceedings of the 26th ACM International Conference on Multimedia. MM ’18, pp. 1706–1714. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3240508.3240522. https://doi.org/10.1145/3240508.3240522

  • Gerling KM, Mandryk RL, Kalyn MR (2013) Wheelchair-based game design for older adults. In: Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 1–8

  • Gilleard C (1996) Consumption and identity in later life: toward a cultural gerontology. Age Soc 16(4):489–498

    Article  Google Scholar 

  • Google (2021) Google Pixel (2nd Gen) Hardware , Google Android. (Accessed on 02/11/2021). https://www.android.com/intl/en_uk/phones/google-pixel-2/

  • Guo HJ, Sapra A (2020) Instrumental activity of daily living (iadl)

  • Gutierrez F, Verbert K, Htun NN (2018) Phara: an augmented reality grocery store assistant, pp. 339–345. https://doi.org/10.1145/3236112.3236161

  • Hoenig H (2004) Assistive technology and mobility aids for the older patient with disability. Annal Long Term Care 12:12–19

    Google Scholar 

  • Hoenig H, Taylor DH Jr, Sloan FA (2003) Does assistive technology substitute for personal assistance among the disabled elderly? Am J Pub Health 93(2):330–337

    Article  Google Scholar 

  • inc A (2020) Apple Siri virtual assistant. https://www.apple.com/siri/. Accessed 25 Aug 2020

  • Karmarkar AM, Dicianno BE, Cooper R, Collins DM, Matthews JT, Koontz A, Teodorski EE, Cooper RA (2011) Demographic profile of older adults using wheeled mobility devices. Journal of aging research 2011

  • Kaye HS, Kang T, Laplante MP (2002) Wheelchair use in the united states. In: Disability Statistics Abstract, vol. 23, pp. 1–4

  • Kemp B, Mosqueda LA (2004) Aging with a disability: what the clinician needs to Know. JHU

  • Kline DW, Schieber F (1985) Vision and aging

  • Kuoppamäki S, Tuncer S, Eriksson S, McMillan D (2021) Designing kitchen technologies for ageing in place: a video study of older adults’ cooking at home. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 5(2):1–19

    Article  Google Scholar 

  • Lawton MP, Brody EM (1969) Assessment of older people: self-maintaining and instrumental activities of daily living. The gerontologist 9(3_Part_1), 179–186

  • Lee B, Chun J (2009) Manipulation of virtual objects in marker-less ar system by fingertip tracking and hand gesture recognition. In: Proceedings of the 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human. ICIS ’09, pp. 1110–1115. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1655925.1656127. https://doi.org/10.1145/1655925.1656127

  • Li FM, Chen DL, Fan M, Truong KN (2019) Fmt: a wearable camera-based object tracking memory aid for older adults. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3(3):1–25

    Article  Google Scholar 

  • Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: European Conference on Computer Vision, pp. 740–755 . Springer

  • Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60:91. https://doi.org/10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  • Mason L, Gerling K, Dickinson P, De Angeli A (2019) Design goals for playful technology to support physical activity among wheelchair users. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–12

  • Microsoft: Cortana - Your Personal Productivity Assistant (2021). https://www.microsoft.com/en-us/cortana

  • Microsoft: HoloLens (1st Gen) Hardware, Microsoft Docs (2020). https://docs.microsoft.com/en-us/hololens/hololens1-hardware. Accessed 25 Aug 2020

  • MicroSoft: MRTK V2 (2020). https://docs.microsoft.com/ko-kr/windows/mixed-reality/develop/unity/mrtk-getting-started. Accessed 25 Aug 2020

  • Niwa H, Nagata K, Ohta M, Yamashita K (2016) A product identification method for a mixed-reality web shopping system. In: 2016 IEEE 5th Global Conference on Consumer Electronics, pp. 1–2. https://doi.org/10.1109/GCCE.2016.7800481

  • Organization WH (2002) Active ageing : a policy framework. World Health Organization

  • Pentland W, McColl M, Rosenthal C (1995) The effect of aging and duration of disability on long term health outcomes following spinal cord injury. Spinal Cord 33(7):367–373

    Article  Google Scholar 

  • Qiao S, Shen W, Qiu W, Liu C, Yuille A (2017) Scalenet: Guiding object proposal generation in supermarkets and beyond

  • Rashid Z (2016) Cricking implementation with augmented reality and rfid; towards independent living of people with motor disabilities. PhD thesis. https://doi.org/10.13140/RG.2.2.13769.60004

  • Rashid Z, Meliá-Seguì J, Pous R (2015) Bridging offline (shopping) and online (shopping) world through augmentable smart glass interfaces, vol. 9254. https://doi.org/10.1007/978-3-319-22888-4_31

  • Rashid Z, Meliá-Seguì J, Pous R, Peig E (2016) Using augmented reality and internet of things to improve accessibility of people with motor disabilities in the context of smart cities. Fut Gener Comput Syst. https://doi.org/10.1016/j.future.2016.11.030

    Article  Google Scholar 

  • Rashid Z, Pous R, Norrie C (2017) An independent shopping experience for wheelchair users through augmented reality and rfid. Assist Technol Off J RESNA. https://doi.org/10.1080/10400435.2017.1329240

    Article  Google Scholar 

  • Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767

  • Rello L, Pielot M, Marcos M-C (2016) Make it big! the effect of font size and line spacing on online readability. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. CHI ’16, pp. 3637–3648. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/2858036.2858204. https://doi.org/10.1145/2858036.2858204

  • Rodger S, Jackson D, Vines J, McLaughlin J, Wright P (2019) Journeycam: Exploring experiences of accessibility and mobility among powered wheelchair users through video and data. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–15

  • Salisbury S, Nitz J, Souvlis T (2006) Shoulder pain following tetraplegia: a follow-up study 2–4 years after injury. Spinal Cord 44(12):723–728

    Article  Google Scholar 

  • Sekuler R, Hutman LP, Owsley CJ (1980) Human aging and spatial vision. Science 209(4462):1255–1256

    Article  Google Scholar 

  • Smith GC (1988) The spatial shopping behaviour of the urban elderly: a review of the literature. Geoforum 19(2):189–200

    Article  Google Scholar 

  • Tonioni A, Serra E, Di Stefano L. A deep learning pipeline for Product recognition in store shelves

  • Trefler E, Fitzgerald SG, Hobson DA, Bursick T, Joseph R (2004) Outcomes of wheelchair systems intervention with residents of long-term care facilities. Assist Technol 16(1):18–27

    Article  Google Scholar 

  • Wang Q, Markopoulos P, Yu B, Chen W, Timmermans A (2017) Interactive wearable systems for upper body rehabilitation: a systematic review. J Neuroeng Rehabil 14(1):1–21

    Article  Google Scholar 

  • Winlock T, Christiansen E, Belongie S (2010). Toward real-time grocery detection for the visually impaired. https://doi.org/10.1109/CVPRW.2010.5543576

    Article  Google Scholar 

  • Wobbrock JO, Kientz JA (2016) Research contributions in human-computer interaction. interactions 23(3):38–44

    Google Scholar 

Download references

Author information

Author notes

  1. Cholmin Kang and Inhwa Yeom have contributed equally to this work.

Authors and Affiliations

  1. Graduate School of Culture Technology, Korea Advanced Institute of Science and Technology, Daehak-ro, Daejeon, 34141, Republic of Korea

    Cholmin Kang, Inhwa Yeom, Amirsaman Ashtari, Woontack Woo &  Junyong Noh

Authors
  1. Cholmin Kang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Inhwa Yeom
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Amirsaman Ashtari
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Woontack Woo
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Junyong Noh
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Woontack Woo or Junyong Noh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, C., Yeom, I., Ashtari, A. et al. ARbility: re-inviting older wheelchair users to in-store shopping via wearable augmented reality. Virtual Reality 27, 1919–1936 (2023). https://doi.org/10.1007/s10055-023-00769-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-023-00769-0

Keywords