Abstract
The present work explores the potential offered by the integration of various technologies such as augmented reality and artificial intelligence in the field of home-based care for dependent individuals. The proposed system focuses on providing assistance for everyday questions posed by individuals with certain levels of dependency or by new caregivers. This will start from the user’s interaction with a query about how to perform a specific action, accompanied by an image of their visual context and a manual detailing the routine, arrangement of the dependant’s belongings, and general information about the dependant. This will return a step-by-step guided response on what should be done, along with a list of objects to use that appear in the image of their environment. These objects will subsequently be highlighted in augmented reality for more comprehensive and understandable information.
This solution aims to improve and prolong the independence at home of individuals in the early stages of dependency or dementia, as well as to enhance the adaptation of caregivers to new home environments, thereby improving the quality of life for patients and the work of caregivers.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Achiam, J., et al.: GPT-4 Technical report. arXiv preprint arXiv:2303.08774 (2023)
Cheng, T., Song, L., Ge, Y., Liu, W., Wang, X., Shan, Y.: YOLO-world: real-time open-vocabulary object detection. arXiv preprint arXiv:2401.17270 (1 2024)
Eloy, S., Dias, L., Ourique, L., Dias, M.S.: Home mobility hazards detected via object recognition in augmented reality, pp. 415–422 (2019)
Kasowski, J., Johnson, B.A., Neydavood, R., Akkaraju, A., Beyeler, M.: Furthering visual accessibility with extended reality (XR): a systematic. arXiv preprint arXiv:2109.04995 (2021). https://doi.org/10.1145/nnnnnnn.nnnnnnn
Lewis, P., et al.: Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Advances in Neural Information Processing Systems, vol. 33, pp. 9459–9474 (2020). https://github.com/huggingface/transformers/blob/master/
Park, Y.J., Ro, H., Lee, N.K., Han, T.D.: Deep-cARe: projection-based home care augmented reality system with deep learning for elderly. Appl. Sci. (Switzerland) 9(18) (2019). https://doi.org/10.3390/app9183897
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016). https://doi.org/10.1109/CVPR.2016.91
Vona, F., et al.: Combining virtual and augmented reality to improve daily autonomy for people with autism spectrum disorder. In: Proceedings of the 2022 International Conference on Advanced Visual Interfaces, pp. 1–3. Association for Computing Machinery (2022). https://doi.org/10.1145/3531073.3534499
Wang, L.L., Jia, L.Q., Chu, F.Q., Li, M.X.: Design of home care system for rural elderly based on artificial intelligence. J. Phys. Conf. Ser. 1757 (2021). https://doi.org/10.1088/1742-6596/1757/1/012057
Watson, J., Mac Aodha, O., Prisacariu, V., Brostow, G., Firman, M.: The temporal opportunist: self-supervised multi-frame monocular depth. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1164–1174 (2021). http://arxiv.org/abs/2104.14540
Zeng, D., et al.: SHECS: a local smart hands-free elderly care support system on smart AR glasses with AI technology. In: 2021 IEEE International Symposium on Multimedia (ISM), pp. 66–74. Institute of Electrical and Electronics Engineers Inc. (2021). https://doi.org/10.1109/ISM52913.2021.00019
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ferri-Molla, I., Izquierdo-Domenech, J., Aliaga-Torro, C., Linares-Pellicer, J. (2024). Multimodal Assistance System for the Care of Individuals in Early Stages of Dependency Using Augmented Reality and Artificial Intelligence. In: De Paolis, L.T., Arpaia, P., Sacco, M. (eds) Extended Reality. XR Salento 2024. Lecture Notes in Computer Science, vol 15028. Springer, Cham. https://doi.org/10.1007/978-3-031-71704-8_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-71704-8_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-71703-1
Online ISBN: 978-3-031-71704-8
eBook Packages: Computer ScienceComputer Science (R0)