Skip to main content

Vision-Based Cooking Assistance System for Visually Impaired People

  • Conference paper
  • First Online:
HCI International 2022 Posters (HCII 2022)

Abstract

Cooking simple meals is an essential skill for independent living. Our user research showed that visually impaired people felt strong anxiety and danger about cooking, especially when they used induction cooktops due to fear of burns from fire, difficulty in locating high temperature objects, and absence of cookware adapted for them. This study proposes a vision sensing system to assist visually impaired people so that they can use an induction cooktop safely and easily. Our system is designed to first detect the induction cooking zone, cookware, and the cook’s hands using a vision-based deep-learning algorithm of Python with a YOLO v5 model, and then to give real-time feedback on the relative position of cookware to users. The algorithm is composed of (1) a hob position notification that guides the direction to move pots or pans to the accurate position on the hob, and (2) a hand position notification that guides the relative position of the users’ hands. The feedback is given to the user using sound effects and voice guidance designed based on user preference. Our usability assessment indicates that the proposed system increased the accessibility to stove-like appliances for visually impaired people and encourages their confidence to try cooking. Our final prototype includes light feedback along with audible guides to cover various types of visual impairments. We expect that this system will encourage visually impaired people to get a clearer sense of location information around hobs, and have higher confidence in cooking on their own.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Li, F.M., Dorst, J., Cederberg, P., Carrington, P.: Non-visual cooking: exploring practices and challenges of meal preparation by people with visual impairments. In: International ACM SIGACCESS Conference on Computers and Accessibility, vol. 23, pp. 1–11 (2021)

    Google Scholar 

  2. Korean Ministry of Health and Welfare: National Survey on Persons with disabilities (2020)

    Google Scholar 

  3. Jones, N., Bartlett, H.E., Cooke, R.: An analysis of the impact of visual impairment on activities of daily living and vision-related quality of life in a visually impaired adult population. Br. J. Vis. Impair. 37(1), 50–63 (2019)

    Article  Google Scholar 

  4. Bilyk, M.C., Sontrop, J.M., Chapman, G.E., Barr, S.I., Mamer, L.: Food experiences and eating patterns of visually impaired and blind people. Can J. Diet. Pract. Res. 70(1), 13–18 (2009)

    Article  Google Scholar 

  5. Nam, J.H., Jeon, J.H., Oh, C.W.: A study on the perceptions and experiences of safety accidents in residential areas of visually impaired living alone: focusing on rental apartments in Seoul. Korean J. Soc. Welf. Stud. 21(4), 101–132 (2020)

    Google Scholar 

  6. Kim, T.S.: Difficulties experienced by graduates of schools for the visually impaired during independent living and problem solving in adulthood. Korea Assoc. Educ. Rehabil. Visually Impaired 37(1), 83–112 (2021)

    Google Scholar 

  7. Oh, S.R., Bae, Y.C.: Braille block recognition algorithm for the visually impaired based on YOLO V3. J. Korean Inst. Intell. Syst. 31(1), 60–67 (2021)

    Google Scholar 

  8. Abdolrahmani, A., Kuber, R., Branham, S.M.: “Siri Talks at You”: An Empirical Investigation of Voice-Activated Personal Assistant (VAPA) usage by individuals who are blind. In: International ACM SIGACCESS Conference on Computers and Accessibility, vol. 20, pp. 249–258 (2018)

    Google Scholar 

  9. Azenkot, S., Rector, K., Ladner, R.E., Wobbrock, J.O.: PassChords: secure multi-touch authentication for blind people. In: International ACM SIGACCESS Conference on Computers and Accessibility, vol. 20, pp. 159–166 (2012)

    Google Scholar 

Download references

Acknowledgement

This work was supported by University Innovation. (No. 2021 University Innovation-132, University Innovation (Hongik University)).

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Youkeun Oh or Dokshin Lim .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kim, M., Hwang, S., Choi, K., Oh, Y., Lim, D. (2022). Vision-Based Cooking Assistance System for Visually Impaired People. In: Stephanidis, C., Antona, M., Ntoa, S. (eds) HCI International 2022 Posters. HCII 2022. Communications in Computer and Information Science, vol 1580. Springer, Cham. https://doi.org/10.1007/978-3-031-06417-3_72

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06417-3_72

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06416-6

  • Online ISBN: 978-3-031-06417-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics