Exploring the cognitive process for service task in smart home: A robot service mechanism
Introduction
The aging of the worldwide population is rising, resulting in the overstretched healthcare resources and an unbalanced demographic composition [1]. The technological advances in internet of things (IoT) have provided some promising opportunities for mitigating this phenomenon [2]. Within the context of continuous development of IoT, the new paradigm in conjunction with cognitive computing has received increasing attention, which is also known as cognitive IoT (CIoT) [3]. Over time, the CIoT is expected to play an active role in advancement of various domains such as industrial, healthcare and home applications [3], [4], [5], to contribute to the socio-economic development. As a home application of CIoT paradigm, the smart home (SH) devotes to improving the quality of life for residents, such as monitoring energy consumption of appliances [6], assessing user health status according to their activities of daily living (ADLs) [7], predicting future activity occurrence times based on sensory devices [8], and performing everyday manipulation tasks by service robot [9]. Therefore, SH system has brought significant benefits for residents to maintain a comfortable and healthful home environment as well as provide appropriate services.
Generally, SH is a home environment embedded with the diversity of sensors, smart devices, and infrastructures in conjunction with information and communication technology [10]. The basic premise of SH is that the smart devices and infrastructures are capable of working together directly without intervention from the residents. In such SH, the needs of residents can be inferred automatically by monitoring and analyzing the contextual information, thereby providing the reasonable context-aware services. With regard to contextual information monitoring, it can be roughly divided into three categories. The first is the environmental monitoring. According to the measured results (e.g., air quality, energy consumption, etc.) by wireless sensors, smart actuators can take corresponding measures to SH ensure environmental quality for inhabitants [11]. The second category is the human activity monitoring. The monitoring facilities can be the wireless sensor network in SH or the wearable device on inhabitants [5]. For example, based upon non-intrusive sensors, an activity recognition system was developed to monitor ADLs in SH [10]. The last category can be described as health monitoring, which mainly uses wearable devices for long-term monitoring of health informatics, such as limb motions, blood pressures, heart rates, etc. [12]. These systems (or solutions) have contributed to the development of SH, but the everyday manipulation tasks are not performed. As an example, when you come back and sit on the sofa, the application devices can be adjusted to the desired temperature and humidity in term of your preferences. However, it is impossible to perform the service “offering you a cup of coffee”. In this case, the quality of life may not be substantially enhanced.
To remedy this situation, service robot should be introduced, which plays an important role in SH. As the study of Borja et al. [13], the robot Rovio was integrated into SH. However, only a few simple services such as the verbal communication, surveillance and telepresence could be offered. To enable service robot to understand and perform everyday manipulation tasks in home environment, cognition-based service robot should be developed. Although some significant achievements have been made [9], it is not enough to meet the requirements of users (i.e., active service and personalized service) in SH. Basically, the major challenges can be summarized as follows.
- •
How to actively recognize the user’s intention by service robot based on its limited equipments?
- •
How to understand and perform the complex or vaguer service tasks by service robot?
In order to cope with these problems, in this paper, the intelligent space (IS) [14] and service robot are integrated to develop the SH system architecture, where the overall performance of service robot can be improved under the premise of simplifying robot system function. In addition to being able to make decisions based on perceived contextual information, the developed SH system can perform complex service tasks. Besides, an ontology-based conceptual context model is investigated to address the data heterogeneity of SH system, in which nine elements to represent any information of home environment are defined, namely, user, object, agent, temporal, spatial, event, case, environment and map. Furthermore, we present a robot service mechanism by system cooperation. Specifically, the robot service mechanism of how to actively discover and provide the service task in the right way is holistically explored, including task-driven robot mechanism, execution-oriented task cognition, knowledge-based task inference, JSHOP2 (a Java implementation of simple hierarchical ordered planner 2)-based task planning, quality-oriented task assessment and safety-oriented autonomous navigation. In summary, this paper aims to propose the overall perspective on robot service mechanism that can actively discover and perform the service task in the right way, in order to facilitate future research of SH. The major contributions are outlined below.
- (1)
SH system architecture with the features of adaptability and scalability is designed through the integration of IS and robot. In contrast to the traditional architecture, the proposed SH system architecture can effectively improve the environmental perception and service execution abilities. Besides, it not only overcomes the limitations of stand-alone robot, but also enhances the task cognition capability of robot.
- (2)
In order to support SH system architecture and service task, a knowledge-based context model is proposed to address the heterogeneity issue. In this context, the heterogeneous data can be automatically interpreted as uniform and meaningful semantic knowledge. Also, the knowledge sharing and reuse can be supported.
- (3)
A comprehensive robot service mechanism is presented by system cooperation in SH, which can bridge the gap between the complex tasks and the executable task descriptions by robot. In this case, the robot is endowed with high cognition and inference abilities, and is able to perform the complex and vaguer service task in the right way. Additionally, the active and personalized services can also be provided in terms of individual preference.
- (4)
Comprehensive experiments and evaluations are cond-ucted in the real-world to verify the proposed schemes. The results are provided to demonstrate the feasibility and effectiveness of our proposals.
The remainder of this paper is organized as follows: Section 2 provides the related work. In Section 3, the system architecture and its unique features are described. Section 4 introduces the ontology-based context model for addressing the data heterogeneity issue. Then, a robot service mechanism based on system cooperation, including overall system cooperation, cognitive system cooperation and autonomous navigation system (ANS) cooperation, is fully outlined in Section 5, while the comprehensive experiments are implemented to demonstrate its effectiveness. Section 6 concludes this paper.
Section snippets
From IS to SH
IS is the embodiment of pervasive computing in local physical space. Since 1996, the concept of IS was presented by Hashimoto from the University of Tokyo [14]. In the beginning, IS consisted of two sets of vision cameras and computers, and later a large-sized video projector was added. Along with development of IS, its basic concept has extended to a wider range of applications, such as smart classroom [15], smart meeting room [16], SH [5], etc.
As a derivative of IS, the core idea of SH is
System architecture
This paper concentrates on the service-oriented SH, in which the service robot is introduced to provide appropriate services for users. In the following, the developed system architecture and its unique features will be described.
Context model
Context model, as a key accessor for supporting context-aware applications, has been received significant attention in the last two decades [25]. A successful context model should include all the information and the relationships between them. The simplified, generalized, and most widely accepted definition of context was coined first by Dey [41]: “Context is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered
Robot service mechanism and implement
In general, the service tasks are complex and vague, such as “give me a glass of water” or “make coffee”. Performing such service tasks requires that the service robot has the high understanding and reasoning (or inference) abilities. To bridge this gap, in the following sections, we holistically explore a robot service mechanism by system cooperation in SH. In this case, service robot is able to discover timely and provide the active services appropriately.
Conclusion
In this paper, we have presented a fresh perspective on the complete robot service mechanism for cognition process of service task in SH. For this purpose, the system architecture of SH with the unique features of adaptability and scalability has been exploited by combining IS and service robot, which not only improves the environmental perception but also enhances task execution capability of robot. To solve the data heterogeneity issue, we have analyzed and presented an ontology-based context
Declaration of Competing Interest
Authors declare that they do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.
Acknowledgments
This work was supported by National Natural Science Foundation of China [grant numbers U1813215, 61773239]; and the Special fund for Taishan Scholars Program of Shandong Province, China .
Ying Zhang received the B.S. degree in automatic control from Heze University, Heze, China, in 2014 and the M.S. degree in Control Theory and Control Engineering from Shandong Jianzhu University, Jinan, China, in 2017. He is currently pursuing the Ph.D. degree in Control Theory and Control Engineering at Shandong University, Jinan, China. His current research interests include intelligent robot system, context modeling, knowledge representation, semantic map.
References (53)
- et al.
Towards fog-driven IoT eHealth: Promises and challenges of IoT in medicine and healthcare
Future Gener. Comput. Syst.
(2018) - et al.
Internet of things: A survey on enabling technologies, protocols, and applications
IEEE Commun. Surveys Tutor.
(2015) - et al.
Distributed behavior model orchestration in cognitive internet of things solution
Enterp. Inf. Syst.
(2018) - et al.
A novel big data analytics and intelligent technique to predict driver’s intent
Comput. Ind.
(2018) - et al.
Major requirements for building smart homes in smart cities based on internet of things technologies
Future Gener. Comput. Syst.
(2017) - et al.
Appliance flexibility analysis considering user behavior in home energy management system using smart plugs
IEEE Trans. Ind. Electron.
(2019) - et al.
A knowledge-driven approach for activity recognition in smart homes based on activity profiling
Future Gener. Comput. Syst.
(2017) - et al.
Learning activity predictors from sensor data: Algorithms, evaluation, and applications
IEEE Trans. Knowl. Data Eng.
(2017) - et al.
Cognition-enabled robot manipulation in human environments: requirements, recent work, and open problems
IEEE Robot. Autom. Mag.
(2017) - et al.
A knowledge-based approach for multiagent collaboration in smart home: from activity recognition to guidance service
IEEE Trans. Instrum. Meas.
(2019)
Implementation of an intelligent indoor environmental monitoring and management system in cloud
Future Gener. Comput. Syst.
Wearable heading estimation for motion tracking in health Care by adaptive fusion of visual–inertial measurements
IEEE J. Biomed. Health Informat.
Integration of service robots in the smart home by means of UPnP: A surveillance robot case study
Robot. Auton. Syst.
Intelligent space-concept and contents
Adv. Robot.
The smart classroom: Merging technologies for seamless tele-education
IEEE Pervasive Comput.
A survey on smart meeting rooms and open issues
Int. J. Smart Home
Possibilistic activity recognition with uncertain observations to support medication adherence in an assisted ambient living setting
Knowl.-Based Syst.
A posture recognition-based fall detection system for monitoring an elderly person in a smart home environment
IEEE Trans. Inf. Technol. Biomed.
Unsupervised visit detection in smart homes
Pervasive Mobile Comput.
Controlling mobile robots in distributed intelligent sensor network
IEEE Trans. Ind. Electron.
Field-programmable system-on-chip for localization of UGVs in an indoor ispace
IEEE Trans. Ind. Inf.
Human-following mobile robot in a distributed intelligent sensor network
IEEE Trans. Ind. Electron.
Pedestrian-behavior-based mobile agent control in intelligent space
IEEE Trans. Instrum. Meas.
From activity recognition to intention recognition for assisted living within smart homes
IEEE Trans. Human-Mach. Syst.
Context-aware computing, learning, and big data in Internet of Things: a survey
IEEE Internet Things J.
An information provision system to promote energy conservation and maintain indoor comfort in smart homes using sensed data by IoT sensors
Future Gener. Comput. Syst.
Cited by (20)
Modular interactive computation scheme for the internet of things assisted robotic services
2022, Swarm and Evolutionary ComputationCitation Excerpt :In this paper, a knowledge-based context model was designed to enhance life quality in a smart home. Zhang et al. [27] presented a robot service mechanism in a smart home that increases living quality by determining context-aware services. Interference and planning of tasks were derived to navigate the robot effectively.
Automatic stroke generation for style-oriented robotic Chinese calligraphy
2021, Future Generation Computer SystemsCitation Excerpt :As a typical Cyber–Physical system, intelligent robots perceive information from the physical world with various sensors, and incorporate the perceived information with human commands to robotic learning systems in the cyber world to instruct the robots. Intelligent robots based on such cyber and physical world interaction can efficiently provide services to human users based on the users’ requirements in a certain environment [1,2], which are playing an increasingly important role in the application fields of domestic appliances and education, amongst others. In particular, robots with calligraphy abilities are considered as a key solution for cultural inheritance and education [3,4].
User preference-aware navigation for mobile robot in domestic via defined virtual area
2021, Journal of Network and Computer ApplicationsCitation Excerpt :Yet achieving this scenario is arduous for the mobile robot due to several uncertainties such as One straightforward solution to address these uncertainties is to add the physical boundaries (Billard and Kragic, 2019) or deploy the additional detection devices (Zhang et al., 2020a) in the environment. But for users, this is intrusive.
Topic-based crossing-workflow fragment discovery
2020, Future Generation Computer SystemsCitation Excerpt :This technique is to promote the reuse or repurposing of workflows as the whole, and it is inappropriate when crossing workflow fragments are to be discovered in order to satisfy novel requirements. The discovery of workflow fragments is closely related to service discovery, whose approaches mostly focus on WSDL-based keyword search [30], and semantic matching based on domain knowledge or ontologies [31], context-awareness [32–34], or QoS-based discovery [35,36]. Machine learning techniques are usually adopted to examine service relevance [37].
Ying Zhang received the B.S. degree in automatic control from Heze University, Heze, China, in 2014 and the M.S. degree in Control Theory and Control Engineering from Shandong Jianzhu University, Jinan, China, in 2017. He is currently pursuing the Ph.D. degree in Control Theory and Control Engineering at Shandong University, Jinan, China. His current research interests include intelligent robot system, context modeling, knowledge representation, semantic map.
Guohui Tian was born on August 1969 in Hebei province, P.R. China. He received the B.S. degree from Department of Mathematics, Shandong University, Jinan, China, in 1990, the M.S. degree from the Department of Automation, Shandong University of Technology, Jinan, China, in 1993, and the Doctor degree from School of Automation, Northeastern University, Shenyang, China, in 1997. He studied as a Post-Doctorial researcher in School of Mechanical Engineering of Shandong University from 1999 to 2001, and studied as a visiting professor in Graduate School of Engineering of Tokyo University of Japan from 2003 to 2005. He was a Lecturer from 1997 to 1998 and an associate professor from 1998 to 2002 in Shandong University. At present, he is professor in the School of Control Science and Engineering, Shandong University. And also he is the Vice Director of the Intelligence Robot Specialized Committee of Chinese Association for Artificial Intelligence, the Vice Director of the Intelligent Manufacturing System Specialized Committee of Chinese Association for Automation, and the member of the IEEE Robotics and Automation Society. His research interests include service robot, intelligent space, cloud robotics, brain-inspired intelligent robotics, et al..
Huanzhao Chen received the B.S. degree in automatic control from Binzhou University, Binzhou, China, in 2011, and M.S. degree in College of Electronic and Control Engineering from Beijing University of Technology, Beijing, China. in 2014. He is currently pursuing the Ph.D. degree in Control Theory and Control Engineering at Shandong University, Jinan, China. His research interests include robot recognition, semantic knowledge processing and reasoning in intelligent space.