Abstract
The design of physiologically adaptive systems entails several complex steps from acquiring human body signals to create responsive adaptive behaviors that can be used to enhance conventional communication pathways between human and technological systems. Categorizing and classifying the computing techniques used to create intelligent adaptation via physiological metrics is an important step towards creating a body of knowledge that allows the field to develop and mature accordingly. This paper proposes the creation of a taxonomy that groups several physiologically adaptive (also called biocybernetic) systems that have been previously designed and reported. The taxonomy proposes two subcategories of adaptive techniques: control theoretics and machine learning, which have multiple sub-categories that we illustrate with systems created in the last decades. Based on the proposed taxonomy, we also propose a design framework that considers four fundamental aspects that should be defined when designing physiologically adaptive systems: the medium, the application area, the psychophysiological target state, and the adaptation technique. We conclude the paper by discussing the importance of the proposed taxonomy and design framework as well as suggesting research areas and applications where we envision biocybernetic systems will evolve in the following years.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alpaydin, E.: Introduction to Machine Learning. Adaptive Computation and Machine Learning Series. The MIT Press, Cambridge (2009)
Amores, J., Richer, R., Zhao, N., Maes, P., Eskofier, B.M.: Promoting relaxation using virtual reality, olfactory interfaces and wearable EEG. In: 2018 IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN), pp. 98–101. IEEE (2018)
Barzilay, O., Wolf, A.: Adaptive rehabilitation games. J. Electromyogr. Kinesiol. 23(1), 182–189 (2013)
Boucsein, W.: Electrodermal Activity. Springer, Boston (2012). https://doi.org/10.1007/978-1-4614-1126-0
Cacioppo, J.T., Tassinary, L.G., Berntson, G.: Handbook of Psychophysiology. Cambridge University Press, Cambridge (2007)
Chanel, G., Rebetez, C., Betrancourt, M., Pun, T.: Emotion assessment from physiological signals for adaptation of game difficulty. Syst. Man Cybern. 41(6), 1052–1063 (2011)
Dobrovsky, A., Borghoff, U.M., Hofmann, M.: Improving adaptive gameplay in serious games through interactive deep reinforcement learning. In: Klempous, R., Nikodem, J., Baranyi, P.Z. (eds.) Cognitive Infocommunications, Theory and Applications. TIEI, vol. 13, pp. 411–432. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-95996-2_19
Eberlin, P., Mulholland, T.: Bilateral differences in parietal-occipital EEG induced by contingent visual feedback. Psychophysiology 13(3), 212–218 (1976)
Ernst, G.: Heart-rate variability-more than heart beats? Front. Public Health 5, 240 (2017)
Ewing, K.C., Fairclough, S.H., Gilleade, K.: Evaluation of an adaptive game that uses EEG measures validated during the design process as inputs to a biocybernetic loop. Front. Hum. Neurosci. 10, 223 (2016)
Fairclough, S., Gilleade, K.: Construction of the biocybernetic loop: a case study. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, pp. 571–578 (2012)
Fairclough, S.H.: Physiological computing and intelligent adaptation. In: Emotions and Affect in Human Factors and Human-Computer Interaction, pp. 539–556. Elsevier, Amsterdam (2017)
Fortin-Côté, A., et al.: Predicting video game players’ fun from physiological and behavioural data. In: Arai, K., Kapoor, S., Bhatia, R. (eds.) FICC 2018. AISC, vol. 886, pp. 479–495. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-03402-3_33
Fuchs, S.: Session overview: adaptation strategies and adaptation management. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2018. LNCS (LNAI), vol. 10915, pp. 3–8. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91470-1_1
Fuchs, S., Schwarz, J.: Towards a dynamic selection and configuration of adaptation strategies in augmented cognition. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2017. LNCS (LNAI), vol. 10285, pp. 101–115. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58625-0_7
Hardy, S., Dutz, T., Wiemeyer, J., Göbel, S., Steinmetz, R.: Framework for personalized and adaptive game-based training programs in health sport. Multimedia Tools Appl. 74(14), 5289–5311 (2015)
Hou, M., Banbury, S., Burns, C.: Intelligent Adaptive Systems: An Interaction-Centered Design Perspective. CRC Press, Boca Raton (2014)
Jacob, R.J., Leggett, J.J., Myers, B.A., Pausch, R.: Interaction styles and input/output devices. Behav. Inf. Technol. 12(2), 69–79 (1993)
Jacucci, G., Fairclough, S., Solovey, E.T.: Physiological computing. Computer 48(10), 12–16 (2015)
Jagodnik, K.M., Thomas, P.S., van den Bogert, A.J., Branicky, M.S., Kirsch, R.F.: Training an actor-critic reinforcement learning controller for arm movement using human-generated rewards. IEEE Trans. Neural Syst. Rehabil. Eng. 25(10), 1892–1905 (2017)
Kosunen, I., et al.: Exploring the dynamics of the biocybernetic loop in physiological computing. Series of publications A/Department of Computer Science, University of Helsinki (2018)
Kramer, A.F., Weber, T.: Applications of psychophysiology to human factors. In: Handbook of Psychophysiology, vol. 2, pp. 794–814 (2000)
Labonte-Lemoyne, E., Courtemanche, F., Louis, V., Fredette, M., Sénécal, S., Léger, P.M.: Dynamic threshold selection for a biocybernetic loop in an adaptive video game context. Front. Hum. Neurosci. 12, 282 (2018)
Loewe, N., Nadj, M.: Physio-adaptive systems-a state-of-the-art review and future research directions. In: ECIS (2020)
Luong, T., Martin, N., Raison, A., Argelaguet, F., Diverrez, J.M., Lécuyer, A.: Towards real-time recognition of users mental workload using integrated physiological sensors into a VR HMD. In: 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 425–437 (2020)
Marín-Morales, J., et al.: Affective computing in virtual reality: emotion recognition from brain and heartbeat dynamics using wearable sensors. Sci. Rep. 8(1), 1–15 (2018)
Montoya, M.F., Muñoz, J.E., Henao, O.A.: Enhancing virtual rehabilitation in upper limbs with biocybernetic adaptation: the effects of virtual reality on perceived muscle fatigue, game performance and user experience. IEEE Trans. Neural Syst. Rehabil. Eng. 28(3), 740–747 (2020)
Mulholland, T.: Biofeedback as scientific method. In: Biofeedback: Theory and Research, pp. 9–28 (1977)
Muñoz, J., Gonçalves, A., Vieira, T., Cró, D., Chisik, Y., i Badia, S.B.: Space connection-a multiplayer collaborative biofeedback game to promote empathy in teenagers: a feasibility study. In: International Conference on Physiological Computing Systems, vol. 2, pp. 88–97. SciTePress (2016)
Muñoz, J., Gouveia, E., Cameirao, M., Bermudez, I., Badia, S.: The biocybernetic loop engine: an integrated tool for creating physiologically adaptive videogames. In: Proceedings of the 4th International Conference on Physiological Computing Systems, pp. 45–54 (2017)
Muñoz, J.E., Cameirão, M., Bermúdez i Badia, S., Gouveia, E.R.: Closing the loop in exergaming-health benefits of biocybernetic adaptation in senior adults. In: Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play, pp. 329–339 (2018)
Muñoz, J.E., Pope, A.T., Velez, L.E.: Integrating biocybernetic adaptation in virtual reality training concentration and calmness in target shooting. In: Holzinger, A., Pope, A., Plácido da Silva, H. (eds.) PhyCS 2016-2018. LNCS, vol. 10057, pp. 218–237. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-27950-9_12
Muñoz, J.E., Quintero, L., Stephens, C.L., Pope, A.T.: A psychophysiological model of firearms training in police officers: a virtual reality experiment for biocybernetic adaptation. Front. Psychol. 11, (2020)
Nikzad-Khasmakhi, N., Balafar, M., Reza Feizi-Derakhshi, M.: The state-of-the-art in expert recommendation systems. Eng. Appl. Artif. Intell. 82, 126–147 (2019)
Oskoei, M.A., Hu, H.: Adaptive myoelectric control applied to video game. Biomed. Signal Process. Control 18, 153–160 (2015)
Palsson, O.S., Harris Sr., R.L., Pope, A.T.: Method and apparatus for encouraging physiological self-regulation through modulation of an operator’s control input to a video game or training simulator. US Patent 6,450,820, 17 Sep 2002
Palsson, O., Pope, A.: Stress counter response training of pilots via instrument functionality feedback. Abstract. In: Proceedings of the 1999 Association for Applied Psychophysiology and Biofeedback Meeting (1999)
Parent, M., et al.: PASS: a multimodal database of physical activity and stress for mobile passive body/brain-computer interface research. Front. Neurosci. 14, 1274 (2020)
Parnandi, A., Gutierrez-Osuna, R.: A comparative study of game mechanics and control laws for an adaptive physiological game. J. Multimodal User Interfaces 9(1), 31–42 (2014). https://doi.org/10.1007/s12193-014-0159-y
Pope, A.T., Bogart, E.H., Bartolome, D.S.: Biocybernetic system evaluates indices of operator engagement in automated task. Biol. Psychol. 40(1–2), 187–195 (1995)
Pope, A.T., Stephens, C.L., Blanson, N.M.: Physiologically modulating videogames or simulations which use motion-sensing input devices. US Patent 8,827,717, 9 Sep 2014
Pope, A.T., Stephens, C.L., Gilleade, K.: Biocybernetic adaptation as biofeedback training method. In: Fairclough, S.H., Gilleade, K. (eds.) Advances in Physiological Computing. HIS, pp. 91–115. Springer, London (2014). https://doi.org/10.1007/978-1-4471-6392-3_5
Pope, A.T., Stephens, C.L., Jones, C.A.: Method and system for physiologically modulating action role-playing open world video games and simulations which use gesture and body image sensing control input devices. US Patent 9,084,933, 21 Jul 2015
Prinzel III, L.J., Pope, A.T., Palsson, O.S., Turner, M.J.: Method and apparatus for performance optimization through physical perturbation of task elements. US Patent 8,628,333, 14 Jan 2014
Raol, J.R., Ayyagari, R.: Control systems: classical, modern, and AI-based approaches. CRC Press, Boca Raton (2019)
Rodriguez-Guerrero, C., Knaepen, K., Fraile-Marinero, J.C., Perez-Turiel, J., Gonzalez-de Garibay, V., Lefeber, D.: Improving challenge/skill ratio in a multimodal interface by simultaneously adapting game difficulty and haptic assistance through psychophysiological and performance feedback. Front. Neurosci. 11, 242 (2017)
Ros, T., J Baars, B., Lanius, R.A., Vuilleumier, P.: Tuning pathological brain oscillations with neurofeedback: a systems neuroscience framework. Front. Hum. Neurosci. 8, 1008 (2014)
Roy, R.N., Drougard, N., Gateau, T., Dehais, F., Chanel, C.P.: How can physiological computing benefit human-robot interaction? Robotics 9(4), 100 (2020)
Saeed, A., Ozcelebi, T., Lukkien, J., van Erp, J.B.F., Trajanovski, S.: Model adaptation and personalization for physiological stress detection. In: 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), pp. 209–216 (2018)
Schwartz, M.S., Andrasik, F.: Biofeedback: A Practitioner’s Guide. Guilford Publications, New York (2017)
Shirzad, N., der Loos, H.F.M.V.: Adaptation of task difficulty in rehabilitation exercises based on the user’s motor performance and physiological responses. In: 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR), vol. 2013, pp. 1–6 (2013)
Shu, L., et al.: A review of emotion recognition using physiological signals. Sensors 18(7), 2074 (2018)
Sinclair, J., Hingston, P., Masek, M., Nosaka, K.: Testing an exergame for effectiveness and attractiveness. In: 2010 2nd International IEEE Consumer Electronics Society’s Games Innovations Conference, pp. 1–8. IEEE (2010)
Stephens, C., et al.: Biocybernetic adaptation strategies: machine awareness of human engagement for improved operational performance. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2018. LNCS (LNAI), vol. 10915, pp. 89–98. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91470-1_9
Stephens, C., et al.: Crew state monitoring and line-oriented flight training for attention management. In: 19th International Symposium on Aviation Psychology, p. 196 (2017)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. The MIT Press, Cambridge (2018)
Treacy Solovey, E., Afergan, D., Peck, E.M., Hincks, S.W., Jacob, R.J.: Designing implicit interfaces for physiological computing: guidelines and lessons learned using fNIRS. ACM Trans. Comput. Hum. Interact. (TOCHI) 21(6), 1–27 (2015)
Verhulst, A., Yamaguchi, T., Richard, P.: Physiological-based dynamic difficulty adaptation in a theragame for children with cerebral palsy. In: PhyCS, pp. 164–171 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Muñoz, J.E., Quintero, L., Stephens, C.L., Pope, A. (2021). Taxonomy of Physiologically Adaptive Systems and Design Framework. In: Sottilare, R.A., Schwarz, J. (eds) Adaptive Instructional Systems. Design and Evaluation. HCII 2021. Lecture Notes in Computer Science(), vol 12792. Springer, Cham. https://doi.org/10.1007/978-3-030-77857-6_40
Download citation
DOI: https://doi.org/10.1007/978-3-030-77857-6_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77856-9
Online ISBN: 978-3-030-77857-6
eBook Packages: Computer ScienceComputer Science (R0)