Abstract
When looking at the way in which humans choose to participate in and interact with cyber physical-embedded environments such as Internet of Things (IoT), one could assume that such environments are permeated with ‘ad hoc’, ‘heterogeneous’ and ‘dynamic’ interactions in them. Existing literature on human to computer interactions, their types and definitions fail to provide a concrete understanding of these dimensions in cyber physical-embedded environments. Therefore, this paper presents the results of investigating existing categories of Human Computer Interaction (HCI) to make sense of these interactions and the inherent heterogeneity they carry. An integrative literature review using the PRISMA model to locate, select, and include 120 relevant articles has been carried out. The main finding of this review is a semantic classification of possible Human-to-Environment Interactions (HEI). The classification plays an important role as a starting point when looking at the current and future offerings of the HEI in the IoT. The classification also serves input as formal knowledge representations, such as Ontology Web Language (OWL) ontologies, which could assist in creating explicit representations of interaction.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
where objects denote sensors, actuators, phones, devices, tablets, computers and smart wearables, to name but a few.
- 2.
which may be needed for their implementation and/or acceptance.
References
Weiser, M.: The computer for the 21st century. Sci. Am. 265(3), 94–104 (1991)
Frauenberger, C.: Entanglement HCI the next wave? ACM Trans. Comput. -Hum. Interact. 27(1), 2:1–2:27 (2019)
Stephanidis, C., et al.: Seven HCI grand challenges. Int. J. Hum. Comput. Interact. 35(14), 1229–1269 (2019)
Harper, R.H.R.: The role of HCI in the age of AI. Int. J. Hum. Comput. Interact. 35(15), 1331–1344 (2019)
Dix, A.: Human-computer interaction, foundations and new paradigms. J. Vis. Lang. Comput. 42, 122–134 (2017)
Earnshaw, R.A., De Silva, M., Excell, P.S.: Ten unsolved problems with the internet of things. In: International Conference on Cyber Worlds 2015 Proceedings, pp. 1–7. IEEE, Sweden (2015)
Juric, R., McClenaghan, K.M.: Towards the semantic classification of constituent parts of the internet-of-vehicles. In: 24th International Conference for the Society for Design and Process Science Transformative Research and Education through Transdisciplinary Means Proceedings, pp. 17–22. SDPS, Taiwan (2019)
Gheisar, M., Wang, G., Chen, S.: An edge computing-enhanced internet of things framework for privacy-preserving in Smart City. J. Comput. Electr. Eng. 81, (2020)
Gulati, N., Kaur, P.D.: Towards socially enabled internet of industrial things: architecture, semantic model and relationship management. J. Ad Hoc Netw. 91, (2019)
Zhu, T., Dhelim, S., Zhou, Z., Yang, S., Ning, H.: An architecture for aggregating information from distributed data nodes for industrial internet of things. J. Comput. Electr. Eng. 58, 337–349 (2017)
Kataria, P., Juric, R., Mandani, K.: Go-CID: generic ontology for context-aware, interoperable and data sharing applications. In: Smith, J.E, (eds.), SEA 2007, pp. 439–444. ACTA Press (2007)
Gomez, J., Oviedo, B., Fernandez, A., Sanchez, M.A.Z., Viteri, J.T.M., Leon, A.R.E.: Semantic representation models of sensor data for monitoring agricultural crops. In: Botto-Tobar, M., León-Acurio, J., Díaz Cadena, A., Montiel Díaz, P. (eds.) Advances in Emerging Trends and Technologies (ICAETT 2019). Advances in Intelligent Systems and Computing, vol. 1066, pp. 33–41. Springer, Cham (2019)
Ning, H., Shi, F., Zhu, T., Li, Q., Chen, L.: A novel ontology consistent with acknowledged standards in smart homes. J. Comput. Netw. 148, 101–107 (2019)
Pahal, N., Mallik, A., Chaudhury, S.: An ontology- based context-aware IoT framework for smart surveillance. In: Mohamed, B., Abdelhakim, B.A., Ali, Y. (eds.) International Conference on Smart City applications 2018, pp. 1–7. Association for Computer Machinery (2018)
Veiga, E.F., Arruda, M.K., Neto, J.A.B., Bulcão-Neto, R.: An ontology-based representation service of context information for the internet of things. In: Roesler, V., Valdeni de Lima, J., (eds.) Webmedia 2017. 23rd Brazillian Symposium on Multimedia and the Web, 2017, pp. 301–308. Association for Computer Machinery (2017)
Horbaek, K., Oulasvirta, A.: What is interaction? In: 2017 CHI Conference of Human Factors in Computing Systems Proceedings, pp. 5040–5052. Association for Computer Machinery, New York (2017)
Kim, K.J.: Interacting socially with the internet of things (IoT): effects of source attribution and specialisation in human - IoT interaction. J. Comput. Mediated Commun. 21(6), 420–435 (2016)
Bakker, S., Niemantsverdriet, K.: The interaction-attention continuum: considering various levels of human attention in interaction design. Int. J. Des. 10(2), 1–14 (2016)
Kostakos, V., Musolesi, M.: Introduction to the special issue on social networks and ubiquitous interactions. Int. J. Hum. Comput Stud. 71(9), 859–861 (2013)
Reeves, D., Serafin, S.: Sonic interaction design. Int. J. Hum. Comput. Interact. Stud. 6(9), 905–906 (2009)
Reeves, S., Beck, J.: Talking about interaction. Int. J. Hum. Comput. Interact. Stud. 131, 144–151 (2019)
Mechant, P., All, A., De Marez, L.: Evaluating user experience in smart home contexts: a methodological framework. In: Streitz, N., Konomi, S. (eds.) Distributed, Ambient and Pervasive Interactions: Understanding Humans (DAPI 2018). LNCS, vol. 10921, pp. 91–102. Springer, Cham (2018)
Pallot, M. Pawar, K.: A holistic model of user experience for living lab experimental design. In: Katzy, B., Holzmann, T., Sailer, K., Thobens, K.D. (eds.) 18th International Conference on Engineering, Technology and Innovation, pp. 1–15. IEEE (2012)
Jordan, P.: Designing Pleasurable Products. An Introduction to the New Human Factors, 1st edn. Taylor and Francis, London, New York (2000)
Gaver, W.W., Martin, H.: Alternatives: exploring information appliances through conceptual design proposals. In: SIGCHI 2000 Conference on Human factors in Computing Systems, pp. 209–216. Association for Computer Machinery, New York (2000)
Stumpf, T., Califf, C.B., Frye, J.J.: The conceptualisation and uses of technological metaworlds in travel. In: 52nd Hawaii International Conference on Systems Sciences Proceedings, pp. 6937–6946, HICSS, Hawaii (2019)
Janlert, L.E., Stolterman, E.: Faceless interaction - a conceptual examination of the notion of interface: past, present, and future. J. Hum. Comput. Interact. 30(6), 507–539 (2015)
Bibri, S.E.: The human face of ambient intelligence: cognitive, emotional, affective, behavioral and conversational aspects. In: Atlantis Ambient and Pervasive Intelligence, vol. 9. Atlantis Press, Paris (2015)
Harper, R.H.R.: The role of HCI in the age of AI. Int. J. Hum. Comput. Interact. 35(15), 1331–1344 (2019)
Patterson, R.E.: Intuitive cognition and models of human-automation interaction. J. Hum. –Autom. Interact. 59(1), 101–115 (2017)
Glodek, M., et al.: Fusion paradigms in cognitive technical systems for human-computer interaction. J. Neurocomput. 161, 17–32 (2015)
Dorsemaine, B., Gaulier, J.P., Wary, J.P., Kheir, K., Urien, P.: Internet of things: a definition & taxonomy. In: 9th International Conference on Next Generation Mobile Applications, Services and Technologies Proceedings, pp. 9–11, NGMAST, Cambridge (2015)
Barker, L., et al.: Taxonomy for internet of things: tools for monitoring personal effects. In: International Conference on Pervasive and Embedded Computing and Communication Systems Proceedings (PECCS 2014), Portugal, vol. 1 (2014)
Fortino, G., Rovella, A., Russo, W., Savaglio, C.: On the classification of cyber physical smart objects in the internet of things. In: International Workshop on Networks of Cooperating Objects for Smart Cities, vol. 1156, pp. 86–94, UBICITEC, Germany (2014)
Mamo, K., Nieto, J.I., Leon, M.D.C., Vazquez, M., López, J.D.S., Buenrostro, R.: Major existing classification matrices and future directions for internet of things. J. Adv. Internet of Things 07(04), 112–120 (2017)
Liberati, A., et al.: The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 6(7), (2009)
Popovici, I., Schipor, O.A., Vatavu, R.D.: Hover: exploring cognitive maps and mid-air pointing for television control. Int. J. Hum. Comput. Stud. 129, 95–107 (2019)
Ardito, C., Buono, P., Desolda, G., Matera, M.: From Smart objects to smart experiences: an end user development approach. Int. J. Hum. Comput. Stud. 114, 52–68 (2018)
Xiang, A.C., Li, Y.: Improv: an input framework for improvising cross-device interaction by demonstration. ACM Trans. Comput. Hum. Interact. 24(2), 15 (2017)
Sutcliffe, A., Hart, J.: Analysing the role of interactivity in user experience. Int. J. Hum. Comput. Interact. 33(3), 229–240 (2017)
Loke, L., Robertson, T.: Moving and making strange: an embodied approach to movement-based interaction design. ACM Trans. Comput. Hum. Interact. 20(1), 7 (2013)
Bilandzic, M., Foth, M.: A review of locative media, mobile and embodied spatial interaction. Int. J. Hum. Comput. Stud. 70(1), 66–71 (2012)
McGrenere, J., Baecker, R.M., Booth, K.S.: A field evaluation of an adaptable two-interface design of feature-rich software. ACM Trans. Comput. Hum. Interact. 14(1), 3 (2007)
Benford, S., et al.: Expected, sensed and desired: a framework for designing sensing-based interaction. ACM Trans. Comput. Hum. Interact. 12(1), 3–30 (2005)
Zhai, S., Bellotti, V.: Introduction to sensing-based interaction. ACM Trans. Comput. Hum. Interact. 12(1), 1–2 (2005)
Quek, F., et al.: Multimodal human discourse: gesture and speech. ACM Trans. Comput. Hum. Interact. 9(3), 171–193 (2002)
Benford, S., Dourish, P., Rodden, T.: Introduction to the special issue on human-computer interaction and collaborative virtual environments. ACM Trans. Comput. Hum. Interact. 7(4), 439–441 (2000)
Velloso, E., Carter, M., Newn, J., Esteves, A., Clarke, C., Gellersen, H.: Motion correlation: selecting objects by matching their movement. ACM Trans. Comput. Hum. Interact. 24(3), 22 (2017)
Hornbaek, K., Mottelson, A., Knibbe, J., Vogel, D.: What do we mean by “Interaction”? an analysis of 35 years of CHI. ACM Trans. Comput. Hum. Interact. 26(4), 27 (2019)
Oviatt, S., Seneff, S.: Introduction to mobile and adaptive conversational interfaces. ACM Trans. Comput. Hum. Interact. 11(3), 237–240 (2004)
Truschin, S., Schermann, M., Goswami, S., Krcmar, H.: Designing interfaces for multiple-goal environments: experimental insights from in-vehicle speech interfaces. ACM Trans. Comput. Hum. Interact. 21(1), 7 (2014)
Nansen, B., Vetere, F., Robertson, T., Downs, J., Brereton, M., Durick, J.: Reciprocal habituation: a study of older people and the Kinect. ACM Trans. Comput. Hum. Interact. 21(3), 18 (2014)
Kosmyna, N., Tarpin-Bernard, F., Rivet, B.: Adding human learning in brain-computer interfaces (BCIs): towards a practical control modality. ACM Trans. Comput. Hum. Interact. 22(3), 12 (2015)
Solovey, E.T., Afergan, D., Peck, E.M., Hincks, S.W., Jacob, R.J.K.: Designing Implicit Interfaces for psychological computing: guidelines and lessons learned using fNIRS. ACM Trans. Comput. Hum. Interact. 21(6), 35 (2015)
Kirsh, D.: Embodied cognition and the magical future of interaction design. ACM Trans. Comput. Hum. Interact. 20(1), 3 (2013)
Benford, S., Calder, M., Rodden, T., Sevegnani, M.: On Lions, Impala, and Bigraphs: modelling interactions and physical virtual spaces. ACM Trans. Comput. Hum. Interact. 23(2), 9 (2016)
Vasquez-Alverez, Y., Aylett, M.P., Brewster, R., Von Jungenfeld, S.A., Virolainen, A.: Designing interactions with multilevel auditory displays in mobile audio-augmented reality. ACM Trans. Comput. Hum. Interact. 23(1), 3 (2015)
Mehra, S., Werkhoven, P., Worring, M.: Navigating on handheld displays: dynamic versus static peephole navigation. ACM Trans. Comput. Hum. Interact. 13(4), 448–457 (2006)
Benford, S., GreenHalgh, C., Reynard, G., Brown, C., Koleva, B.: Understanding and constructing shared spaces with mixed reality boundaries. ACM Trans. Comput. Hum. Interact. 5(3), 185–223 (1998)
Hornbaek, K., Hertzum, M.: Technology acceptance and user experience: a review of experiential components in HCI. ACM Trans. Comput. Hum. Interact. 24(5), 33 (2017)
Paik, J., Kim, J.W., Ritter, F.E., Reitter, D.: Predicting user performance and learning in human-computer interaction with the herbal compiler. ACM Trans. Comput. Hum. Interact. 22(5), 25 (2015)
Borsci, S., Macredie, R.D., Barnett, J., Martin, J., Kuljis, J., Young, T.: Reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation. ACM Trans. Comput. Hum. Interact. 20(5), 29 (2013)
Rapp, A., Tirassa, M., Tirabeni, L.: Rethinking technologies for behaviour change: a view from the inside of human change. ACM Trans. Comput. Hum. Interact. 26(4), 22 (2019)
Rozendaal, M.C., Boon, B., Kaptelinin, V.: Objects with intent: designing everyday things as collaborative partners. ACM Trans. Comput. Hum. Interact. 26(4), 26 (2019)
Dalsgaard, P., Hansen, L.K.: Performing perception - staging aesthetics of interaction. ACM Trans. Comput. Hum. Interact. 15(3), 13 (2008)
Raisamo, R., Rakkolainen, I., Majaranta, P., Salminen, K., Rantala, J., Farooq, A.: Human augmentation: past, present and future. Int. J. Hum Comput Stud. 131, 131–143 (2019)
Rhiu, I., Hwan Yun, M.: Exploring user experience of smartphones in social media: a mixed-method analysis. Int. J. Hum. Comput. Interact. 34(10), 960–969 (2018)
Mealla Cincuegrani, S., Jorda, S., Valjamae, A.: Physiopucks: increasing user motivation by combining tangible and implicit physiological interaction. ACM Trans. Comput. Hum. Interact. 23(1), 4 (2016)
Poor, G.M., et al.: Applying the Norman 1986 user-centered model to post-WIMP UIs: theoretical predictions and empirical outcomes. ACM Trans. Comput. Hum. Interact. 23(5), 30 (2016)
Bilandzic, M., Foth, M.: Embedded, embodied and situated contexts in interaction with technologies. Int. J. Hum. Comput. Stud. 70(1), 66–71 (2012)
Kjeldskov, J., Paay, J.: Indexicality: understanding mobile human-computer interaction in context. ACM Trans. Comput. Hum. Interact. 17(4), 14 (2010)
Blackwell, A.F., Rode, J.A., Toye, E.F.: How do we program the home? Gender, attention investment, and the psychology of programming at home. Int. J. Hum. Comput. Stud. 67(4), 324–341 (2009)
Hinckley, K., Pierce, J., Horvitz, E., Sinclair, M.: Foreground and background interaction with sensor-enhanced mobile devices. ACM Trans. Comput. Hum. Interact. 12(1), 31–52 (2005)
Liu, Y., Feyen, R., Tsimhoni, O.: Queueing network-model human processor (QN-MHP): a computational architecture for multitask performance in human-machine systems. ACM Trans. Comput. Hum. Interact. 13(1), 37–90 (2006)
Ruddle, R.A., Savage, J.C.D., Jones, D.M.: Symmetric and asymmetric action integration during cooperative object manipulation in virtual environments. ACM Trans. Comput. Hum. Interact. 9(4), 285–308 (2002)
Zanden, B.V., Myers, B.A.: Demonstrational and constraint-based techniques for pictorially specifying application objects and behaviours. ACM Trans. Comput. Hum. Interact. 2(4), 308–356 (1995)
Jacob, R.J.K., Sibert, L.E., McFarlane, D.C., Mullen, M.P.: Integrality and separability of input devices. ACM Trans. Comput. Hum. Interact. 1(1), 3–26 (1994)
Kang, H., Kim, K.J.: Feeling connected to smart objects? A moderated mediation model of locus of agency, anthropomorphism, and sense of connectedness. Int. J. Hum. Comput. Stud. 133, 45–55 (2020)
Janssen, C.P., Boyle, L.N., Kun, A.L., Ju, W., Chuang, L.L.: A hidden Markov framework to capture human machine interaction in automated vehicles. Int. J. Hum. Comput. Interact. 35(11), 947–955 (2019)
Victorelli, E.Z., Reis, J.C.D., Hornung, H., Prado, A.B.: Understanding human-data interaction: literature review and recommendations for design. Int. J. Hum. Comput. Stud. 134, 13–21 (2019)
Alan, A.T., Costanza, E., Ramchurn, S.D., Fischer, J., Rodden, T., Jennings, N.R.: Tariff agent: interacting with a future smart energy system at home. ACM Trans. Comput. Hum. Interact. 23(4), 25 (2016)
Baljko, M., Tenhaaf, N.: The Aesthetics of emergence: co-constructed interactions. ACM Trans. Comput. Hum. Interact. 15(3), 11 (2008)
McGuffic, M.J., Balakrishnan, R.: Fitts’ law and expanding targets: experimental studies and designs for user interfaces. ACM Trans. Comput. Hum. Interact. 12(4), 388–422 (2005)
Abowd, G.D., Mynatt, E.D.: Charting past, present, and future research in ubiquitous computing. ACM Trans. Comput. Hum. Interact. 7(1), 3–28 (2000)
Kieras, D.E., Wood, S.D., Meyer, D.E.: Predictive engineering models based on the EPIC architecture for a multimodal high-performance human computer interaction task. ACM Trans. Comput. Hum. Interact. 4(3), 230–275 (1997)
Dix, A., Rodden, T., Davies, N., Trevor, J., Friday, A., Palfreyman, K.: Exploiting space and location as a design framework for interactive mobile systems. ACM Trans. Comput. Hum. Interact. 7(3), 285–321 (2000)
Lee, M.: An empirical study of home IoT services in South Korea: the moderating effect of the usage experience. Int. J. Hum. Comput. Interact. 35(7), 535–547 (2019)
Bickmore, T.W., Picard, R.W.: Establishing and maintaining long-term human-computer relationships. ACM Trans. Comput. Hum. Interact. 12(2), 293–327 (2005)
Bermudez-Edo, M., Elsaleh, T., Barnaghi, P., Taylor, K.: IoT-Lite: a lightweight semantic model for the internet of things. J. Pers. Ubiquit. Comput. 21(3), 475–487 (2017)
Compton, M., et al.: The SSN ontology of the W3C semantic sensor network incubator group. J. Web Semant. Sci. Serv. Agents World Wide Web 17(C), 25–32 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kataria, P. (2021). Towards a Semantic Classification of Possible Human-to-Environment Interactions in IoT. In: Streitz, N., Konomi, S. (eds) Distributed, Ambient and Pervasive Interactions. HCII 2021. Lecture Notes in Computer Science(), vol 12782. Springer, Cham. https://doi.org/10.1007/978-3-030-77015-0_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-77015-0_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77014-3
Online ISBN: 978-3-030-77015-0
eBook Packages: Computer ScienceComputer Science (R0)