Abstract
Serious games for mental health is seen as the groundwork for assistive technology to maintain and improve mental health. We present a technical system layout we partly implemented for demonstration purposes and highlight vision-based perception and manipulation capabilities. These include physical interactions employing artificial general intelligence in virtual reality applications. We employ hand gesture tracking, as well as an Oculus Rift integrated gaze and eye tracking system. The resulting serious games should eventually cover daily life activities, which we additionally monitor. The dynamic and contextual modelling of obstacles are central issues, and capabilities required for serious games include knowledge about the 3D world. Such knowledge include gaze and hand sensors interpretations for multimedia information extraction in causal relationships. Towards this goal, we envision to make use of virtual reality with a physics engine (rigid and soft body dynamics including collision detection) for the observed objects. We also exploit semantic networks to enable the machine to filter information and infer ongoing complex events including hidden BDI (beliefs, desires, intentions) variables. We see this combination of employed technology as the relevant groundwork for reaching human-level general intelligence and to enable real-world applications. Future applications and user groups we target on include dementia patients.
A. Lőrincz—This research was supported by EIT Digital in the CPS for Smart Factories activity and Kognit, http://kognit.dfki.de.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Antol, S., Zitnick, C.L., Parikh, D.: Zero-shot learning via visual abstraction. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 401–416. Springer, Heidelberg (2014)
Borji, A., Lennartz, A., Pomplun, M.: What do eyes reveal about the mind?: algorithmic inference of search targets from fixations. Neurocomputing 149, 788–799 (2015)
Cirillo, A.: Using validation techniques with people living with dementia, June 2013. http://assistedliving.about.com/od/familycaregivercommunication/a/Using-Validation-Techniques-With-People-Living-With-Dementia.htm
Cleary, J.G., Witten, I.H.: Data compression using adaptive coding and partial string matching. IEEE Trans. Commun. 32(4), 396–402 (1984)
Cohene, T., Baecker, R., Marziali, E., Mindy, S.: Memories of a life: a design case study for alzheimer’s disease. In: Lazar, J. (ed.) Universal Usability, pp. 357–387. Wiley, Chichester (2007)
Dai, P., Lin, C.H., Weld, D.S., et al.: Pomdp-based control of workflows for crowdsourcing. Artif. Intell. 202, 52–85 (2013)
Douglas, S., James, I., Ballard, C.: Non-pharmacological interventions in dementia. Adv. Psychiatr. Treat. 10(3), 171–177 (2004)
Gabrilovich, E., Markovitch, S.: Computing semantic relatedness using wikipedia-based explicit semantic analysis. In: IJCAI, vol. 7, pp. 1606–1611 (2007)
Gharsellaoui, S., Selouani, S.A., Dahmane, A.O.: Automatic emotion recognition using auditory and prosodic indicative features. In: 2015 IEEE 28th Canadian Conference on Electrical and Computer Engineering (CCECE), pp. 1265–1270. IEEE (2015)
Howe, J.: The rise of crowdsourcing. Wired Mag. 14(6), 1–4 (2006)
Jeni, L.A., Lőrincz, A., Szabó, Z., Cohn, J.F., Kanade, T.: Spatio-temporal event classification using time-series kernel based structured sparsity. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 135–150. Springer, Heidelberg (2014)
Jonson, R.: Picture Communication Symbols. Mayer-Johnson, Solana Beach (1985)
Lin, X., Parikh, D.: Don’t just listen, use your imagination: leveraging visual common sense for non-visual tasks. arXiv preprint arXiv:1502.06108 (2015)
Liu, H., Singh, P.: Conceptnet – a practical commonsense reasoning tool-kit. BT Technol. J. 22(4), 211–226 (2004)
Lőrincz, A.: Revolution in health and wellbeing. KI-Künstliche Intelligenz 29(2), 219–222 (2015)
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Oberweger, M., Wohlhart, P., Lepetit, V.: Hands deep in deep learning for hand pose estimation. arXiv preprint arXiv:1502.06807 (2015)
Pintér, B., Vörös, G., Palotai, Z., Szabó, Z., Lőrincz, A.: Determining unintelligible words from their textual contexts. Procedia Soc. Behav. Sci. 73, 101–108 (2013)
Pintér, B., Vörös, G., Szabó, Z., Lőrincz, A.: Wikifying novel words to mixtures of wikipedia senses by structured sparse coding. In: Fred, A., De Marsico, M. (eds.) Pattern Recogn. Appl. Methods, pp. 241–255. Springer, Heidelberg (2015)
Song, C., Qu, Z., Blumm, N., Barabási, A.L.: Limits of predictability in human mobility. Science 327(5968), 1018–1021 (2010)
Speer, R., Havasi, C.: Representing general relational knowledge in conceptnet 5. In: LREC, pp. 3679–3686 (2012)
Foundation, S.: Computer based technology and caring for older adults (2003). http://www.spry.org/pdf/cbtcoa_english.pdf
Taulbee, L.R., Folsom, J.C.: Reality orientation for geriatric patients. Psychiatr. Serv. 17(5), 133–135 (1966)
Toyama, T., Sonntag, D.: Towards episodic memory support for dementia patients by recognizing objects, faces and text in eye gaze. In: Hölldobler, S., Krötzsch, M., Peñaloza, R., Rudolph, S. (eds.) KI 2015. LNCS, vol. 9324, pp. 316–323. Springer, Heidelberg (2015). doi:10.1007/978-3-319-24489-1_29
Vörös, G., Verő, A., Pintér, B., Miksztai-Réthey, B., Toyama, T., Lőrincz, A., Sonntag, D.: Towards a smart wearable tool to enable people with SSPI to communicate by sentence fragments. In: Cipresso, P., Matic, A., Lopez, G. (eds.) MindCare 2014. LNICST, vol. 100, pp. 90–99. Springer, Heidelberg (2014)
Woods, B., Aguirre, E., Spector, A.E., Orrell, M.: Cognitive stimulation to improve cognitive functioning in people with dementia. Cochrane Database Syst Rev 2 (2012)
Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: Appearance-based gaze estimation in the wild. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society (2015)
Zitnick, C.L., Parikh, D., Tech, V.: Bringing semantics into focus using visual abstraction. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2013)
Zygouris, S., Giakoumis, D., Votis, K., Doumpoulakis, S., Ntovas, K., Segkouli, S., Karagiannidis, C., Tzovaras, D., Tsolaki, M.: Can a virtual reality cognitive training application fulfill a dual role? Using the virtual supermarket cognitive training application as a screening tool for mild cognitive impairment. J. Alzheimers Dis. 40, 1–10 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Sárkány, A. et al. (2016). Maintain and Improve Mental Health by Smart Virtual Reality Serious Games. In: Serino, S., Matic, A., Giakoumis, D., Lopez, G., Cipresso, P. (eds) Pervasive Computing Paradigms for Mental Health. MindCare 2015. Communications in Computer and Information Science, vol 604. Springer, Cham. https://doi.org/10.1007/978-3-319-32270-4_22
Download citation
DOI: https://doi.org/10.1007/978-3-319-32270-4_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-32269-8
Online ISBN: 978-3-319-32270-4
eBook Packages: Computer ScienceComputer Science (R0)