Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

According to recent tendencies toward aging societies and nuclear families, there are a large number of elderly people and children who act alone in their daily lives. They sometimes experience social withdrawal because of various difficulties such as anxiety regarding outings without any attendant or the psychological burden of appealing to others for assistance. Their isolated lives make them lack communication.

On the other hand, there are caregivers to help elderly adults and children. The caregivers talk and touch their clients. Even though the main purpose of caregivers is to support their care receivers’ lives, the communication with caregivers also provides social stimuli for the care receivers. However, few care receivers can experience such communication throughout their whole lives because of the lack of caregivers. Thus, we proposed that anthropomorphic robots [1] could help such people if the robots are designed like a caregiver for each user.

Accordingly, we have developed multiple prototypes of a stuffed-toy robot that cuddles up to the user’s upper arm [2,3,4,5,6]. Among various modalities for communication, our system focuses on haptic stimuli. The system especially targets outing scenes, and we have proposed direction indication [5] as one application. However, the features of anthropomorphic media are not just intelligibility in such indications or notification but also include empathetic and emotional communication.

Here, we focused on empathetic and emotional touches in care. A concept proposed by Gineste called “humanitude [7]” is a breakthrough method for caregivers, especially for people with dementia, using communication steps: (1) looking at the patient in the patient’s field of view to create eye contact, (2) touching the patient’s body, (3) talking to the patient, and (4) letting him/her walk. The effectiveness of the humanitude method has been verified [8]. Similarly, “validation [9]” by Feil is another approach. The definition and techniques of validation are a little more complicated than those of humanitude; however, the concepts of both humanitude and validation are similar, that is, empathy and respect for the patients.

With these communication methods in care, not only for people with dementia but also for children or other people who need support, empathetic attitude is considered the most important characteristic in order to actualize warm support like human or living beings. However, while physical contact in humanitude or validation is expected to create such a positive effect, some people do not like to be touched by others. It is assumed that there is not a sufficient relationship of trust between the caregiver and the care receiver in such cases.

In this paper we propose a framework for bidirectional haptic communication with a wearable robot based on anthropomorphism for familiar engagement. From the viewpoint of the relationship development, we consider the first or earliest contacts should be a weak stimulus. Accordingly, in order to design a wearable robot not only as a temporary partner for an outing but also for much more familiar relationships with trust, we discuss the design of the bidirectional physical contact by considering the user’s situation and habituation to the robot.

2 Related Research

2.1 Interaction Design for a Communication Robot

Giannopulu proposed enrobotment [10] as a method for the development of children, especially those with autism, who cannot mirror the relationship of object-self-other. Here, a robot has the potential to be interpreted as an artificial object or communicative presence as the other. We have previously discussed the proposed robot system with haptic interaction from the viewpoint of enrobotment [11]. The acceptability of the robot system should meet some requirements, stepwise levels of communication, though not just for autism children. Before basic communication, the robot should show the potential for communication. To maintain the anthropomorphism for sustainability, the robot should behave as though it has a human-like internal state that is changed by the interaction. In this paper we discuss a framework for the internal states of both the robot and the user to be suitable for the stepwise habituation in human-robot interactions.

There are researches on multimodal expressions of robots, such as gaze [12, 13], gesture [14], personal space [15], and facial expression [16]. Among various modalities, the appropriate use of physical contact helps with warm and intelligible communication, as the concept of humanitude has shown [7]. In order to show an empathetic attitude to the user using our proposed robot, we especially focused on both the haptic modality and the stepwise change in the habituation. In various communication modalities, haptic communication with physical contact is recognized as an intimate or familiar expression. Therefore, the proposed robot system presupposes the user’s attention to the robot while establishing a better relationship.

2.2 Social Touches

Social touches [17] have been discussed as a human-human communication channel in psychology for decades. In recent years, research has been conducted on tactile human-robot interaction [18, 19]. Recognition of the user’s touch [20] as well as expression of the robot’s social touch [21, 22, etc.] have been discussed from the viewpoint of both the mechanical structure and the design for haptic stimuli.

We have developed a haptic-sensible stuffed-toy device [23] as an outlet target for the user’s mental suffering. In order to support indirect communication with the user and other people, the stuffed toy contains touch sensors to detect the user’s input, recognize the type of touch, and then post a comment on an SNS corresponding to the user’s touch. This would be effective for the people who do not have the chance to express their personal emotions. In addition to the input from the user, the artificial physical contact is also expected to become an emotional vent for people who have difficulty in their minds. In this paper we propose a framework for emotional and bidirectional physical contact between the robot and the user that is applicable for our wearable robot prototypes [2,3,4,5,6].

3 Hardware of Previously Developed Robots

Before considering the interactional design of bidirectional physical contact, we introduce three hardware configurations for the robots [2,3,4,5,6], as shown in Figs. 2, 3, 4, 5 and 6. The purpose of the proposed robots is to support elderly people during outings, so the robots are designed to be wearable.

3.1 Prototype Robot with Vibration Motor and Pressure Actuator

The first prototype has three servomotors; one motor is attached to the one-degree-of-freedom (DOF) left arm, and two motors are attached to the two-DOF head of the robot. The cuff of the blood-pressure monitor is used not only for loading the robot but also for creating a haptic pressure and for attaching to the user’s upper arm. A small vibration motor is attached on the inner side. The robot also contains Peltier devices to create temperature stimuli. Figure 1 shows the hardware equipment in the robot.

The robot makes anthropomorphic physical contact by (1) the motion of the robot and (2) the haptic stimuli from the actuators. Figure 2 shows a view of the first robot prototype. Subjective evaluation demonstrated the effectiveness of both the haptic stimuli and the anthropomorphic motion using this robot [2, 3]. The robot can not only notify the user various signals corresponding to the alert or message but also express affection as an emotional element. For example, the robot expresses warm hugs by a pressure stimulus on the arm from the cuff and the Peltier device at 40 \(^{\circ }\)C. The robot also expresses its fear by a strong vibration with a cold temperature at 5 \(^{\circ }\)C.

Fig. 1.
figure 1

Structures of the first prototpe robot [2, 3]

Full size image
Fig. 2.
figure 2

View of the first prototpe robot [2, 3]

Full size image
Fig. 3.
figure 3

Configuration of the second prototpe robot [4, 29]

Full size image
Fig. 4.
figure 4

View of the second prototpe robot [4, 29]

Full size image
Fig. 5.
figure 5

Structures of the third prototpe robot [5]

Full size image
Fig. 6.
figure 6

View of the third prototpe robot [5]

Full size image

3.2 Prototype Robot in Simplified Structures

As a simplified robot system, we have also developed a smartphone-based system as a second prototype [4, 5]. The system was developed as a wearable messaging robot but without any servomotors. The versatility of the system should be improved to be light weight and easy to adopt for common devices. We tried to maintain the anthropomorphism as well as to simplify the system by removing kinetic motion mechanisms.

Figure 3 shows the simplified configuration of the wearable message system. The internal smartphone in the stuffed toy provides multiple-sensors’ signals, such as those of an accelerometer, a geomagnetic sensor, and GPS. The sensors in the smartphone can be used as an ambient device to estimate the user’s activities. A small board PC communicating with the smartphone in the robot is also connected to a board PC (Raspberry PI) via Wi-Fi to generate both a vibration through a vibration motor and the robot’s voice through a small speaker. The total weight of the second prototype is 150 g, which succeeded at reducing the first testbed by 100 g.

Figure 4 shows a view of the wearable stuffed toy. While the purpose of the simplification in this prototype system was focused on the daily support of information acquisition when the user is out of the house, multiple applications adopting the system should be investigated. For instance, the service application on mobile devices, the smartphone’s GPS could be used to provide a walking navigation system [24]. The system could also be applied for our developed notification system regarding the user’s toilet timings, as suggested from the user’s activities [25]. Thus, the second prototype is assumed to be practical but limited.

The notifications from the stuffed toy are also considered to be softer compared to the first prototype; however, the trusting relationship is not engaged in this structure. From the viewpoint of the trusting relationship, affective expression should also be investigated.

3.3 Prototype Robot with Pneumatic Actuators

Based on the previous prototype robot, we developed a wearable robot with multiple pneumatic actuators and touch-pressure sensors (piezo). The system produces haptic stimuli as though it pats, seizes, or pulls the user’s clothes on her/his arm. The robot includes servomotors to generate kinetic motions of the robot, and multiple pulling devices with multiple vibration motors to create the illusion of physical contact from the robot. In order to enable two-way haptic communication, the robot also includes multiple touch sensors in its head and back.

The robot was mainly developed to indicate directions. The multiple pneumatic actuators work in predetermined orders to create directional sensations on the user’s upper arm [28], where the array of pneumatic actuators is lined up on the periphery. For example, when it is necessary to notify the user of directions, the robot gazes in the direction and simulates directional touch by generating a motion of its head with pneumatic stimuli from the opposite direction to the indicatied direction with a time lag. Figure 5 shows the structure of the third prototype robot, and Fig. 6 shows a view of the robot. Different from these approaches to signal notification, we developed anthropomorphic touch from the viewpoint of affective and emotional expression from the robot.

4 System Design

In order to develop the interaction design for bidirectional physical contact, we investigated the basic parameters and expressions for each robot.

4.1 Interaction Types in Physical Contact

First, we describe interaction types of physical contact between humans from the viewpoint of active/passive states.

Active and Passive Physical Contact: There are several modalities for communication. Humans display nonverbal expressions such as gaze, gesture, and intonation. Touches are used for transmitting information to another person, such as beginning communication and expressing emotions. From the viewpoint of spontaneous motivation, the touch from a person is considered to be an active touch. On the other hand, passive touch is considered to be categorized into a reaction to another person. A touch can send a signal to another person without making a sound and can also be a push-type communication. The way of touching another person is affected by one’s internal state.

Thus, the internal state of the robot should reflect human-like social touch. In fact, there are robots related to the intrinsic motivation [26, 27]; however, they are not currently related to emotional activities. In this paper our bidirectional physical contact design includes the internal state of the robot itself.

Types of Interaction Directions: Next we consider one-way and bidirectional touches. In various communications we sometimes use touch while another person talks or displays facial expressions. One-way touches are often used in personal communication to draw another person’s attention and interest. Reactions to another’s active touch appear especially in gestures, gazes, or verbal expressions. Compositive communication using one-way physical contact will enable (a) the intelligible expression as seen in the engagement of the personal message from the robot even in low volume voice, and (b) the sensitive expression such that the robot can stop saying something as though the robot were taken its breath when the user emotionally touches on the robot.

On the other hand, there are few cases of bidirectional touch without any other modality. These cases are limited to familiar, intimate, or emotional communications. Of course, they include animal-like, instinctive interactions. Here, we focused on bidirectional physical contact to design a flow for trusting relationship and support affection for a familiar relationship. Although there are multiple combinations of modalities with touches in real-life communications, we restricted other modalities in order to focus solely on the touch.

4.2 The First Demonstration System: Mimic Interaction

Mimicry is one of the basic, elementary methods in human communication, and several researches have designed the interaction with virtual agents using paralinguistic intonation [30], facial expression [31], subconscious self-touching gesture [32], and so on. ELIZA is one of the oldest interactive systems, using text chat repeating the user’s word [33].

Based upon these simple but effective systems for virtual communication, the first prototype of our interaction design using the third hardware prototype adopted mimicry in haptic interaction. The robot contains two touch sensors on the top-forward and back sides of its head, which are allocated to each pair of pneumatic actuators 1–2 and 3–4 in Fig. 5. The strength and length of the user’s touch to the robot’s head are converted into actions by the pairs of pneumatic actuators with a two-second delay.

In accordance with the hardware configuration of the third prototype, the robot mimics a haptic expression. For instance, the robot shortly fastens its arms around the user’s upper arm in a short-term stimulus when the user slaps the head of the robot. When the user strokes the robot’s head, the robot reacts to the user’s input by the operations of the two pairs of pneumatic actuators in turns as though it is stroking the user’s arm. When the user tightly cuddles up to the robot’s head, the robot gives the user’s arm a hug at a strength corresponding to the user’s input.

4.3 Human-Robot Interaction Design Policy for Physical Contact

Here, we discuss the policy for human-robot interaction design with anthropomorphic physical contact from the viewpoint of human-like communication.

Unsurprisingly, the system requires both the recognition of the user’s expressive touch and the robot’s expression of touch for the interaction. A short-term, simple interaction is easily established by the recognition of touch input and the expression of touch output; however, long-term, verisimilar communication is based on the internal state of one’s mind.

In order to consider long-term communication with physical contact, the robot should (1) realize stepwise communication from the viewpoint of habituation, (2) contain intrinsic motivation for communication with an aspect of satisfaction, and (3) be affected by the trusting relationship with the user. The configuration aims to become the comprehensive design for interactive physical contact, including active or passive touch and one-way or bidirectional touch, with heterogeneous robots.

Figure 7 shows the basic flow of the interaction design with physical contact. The system has a mechanism for detecting physical contact from the user, internal-state data of the robot, and a physical contact expression mechanism. The user’s emotional estimation is based on the data analysis of haptic inputs. We classified the types of touch from the data for emotional estimation. Figure 8 shows our simple expected categories of physical contact corresponding to the emotional state by strengths and lengths of touch.

When it is necessary to notify the user of something without the robot’s internal desire, the robot does so through a simple touch that is in short term with a relatively strong intensity in order to be intelligible. In the case of urgent notifications, the touch should be repeated according to the degree of the urgency. On the other hand, when the robot has an internal desire to touch the user, the robot makes affective and emotional physical contact, such as embracing the user’s arm. The enfolding expression should be a long-lasting touch with a soft intensity. The user’s simple reaction to the robot as notification would become simple as described in the above example, for the robot’s expression. On the other hand, when the user has an internal desire for communication, he or she is expected to use a relatively long-lasting touch with soft intensity, such as a stroke or embrace.

In order to satisfy the internal desire of the robot, the user’s reactions are ranked by the level of satisfaction: (1) the positive reactions in the physical contact to the robot’s emotional touch, (2) the simple reactions to the robot’s emotional touch as the user’s confirmation, or (3) no reaction.

Fig. 7.
figure 7

Basic flow of communication including physical contact

Full size image
Fig. 8.
figure 8

Expected categories of physical contact corresponding to emotional state

Full size image

From the basic configuration of the bidirectional physical contact system, it is expected to create a robot that cultivates its warm mind corresponding to the haptic interaction with the user and touches the user actively, corresponding to the current internal state, such as the emotional state or satisfaction. The reactive expressions of physical contact can be changed by each-time interaction with the user.

Stepwise Communication Design: We have proposed a stepwise interaction model for enrobotment [11], a procedure to make robots familiar to children [10]. In the early interaction between the robot and the user, the system should consider the level of the user’s nervousness regarding the robot. Our previous investigation contained four levels of tactile interactions between a robot and a child in accordance with the acquisition of the trusting relationship, as shown in Table 1: the user’s interest without touch (level 0), the short touch interaction started by the user as confirmation of safety and interactivity (level 1), long-lasting physical contact such as strokes given by both the robot and the user as emotional or expressive communication (level 2), and continuous physical contact, such as a hug when the robot is accepted as a reliable and persistent presence with a trusting relationship for affective interaction (level 3).

The table should also describe each direction of the communication. For example, level 1 currently involves simple touch. It is better to adopt mimicry or shortened mimicry of the user’s touch for the simplest engagement when the robot uses passive touch.

Table 1. Pre-designed stepwise communication in physical contact for enrobotment
Full size table

Here, we should consider a parameterization of the habituation to calculate the internal state and to design the expressions and reactions with physical contact. Habituation is considered to reduce nervousness [34]. In the situation with the user’s high nervousness, the robot should provide a weak stimulus to the user, while the given stimulus should be perceived as strongly corresponding to the nervousness. In order to realize the appropriate social touch from the robot corresponding to the user’s internal state, the robot should have internal models of both the robot and the user.

Internal Models for Bidirectional Physical Contact: As with human-human communication, the robot should assume not only its internal state but also the user’s internal state as a “mental model of others [35].” In addition to the internal states, we consider that the strength of the robot’s motivation for the communication needs to be prepared.

The preliminary prototype for the tactile interaction model used the robot’s internal satisfaction value corresponding to the robot’s desire for the physical contact and the reaction from the user. On the other hand, the emotional circumplex model [36] has two axes: valence and arousal. The PAD model [37] has three: pleasure (which is similar to valence), arousal, and dominance. These models do not merely demonstrate momentary emotions but also everything from long-term emotion to temperament and personality [38]. The satisfaction is regarded to be related to the arousal and pleasure. Accordingly, our robots with physical contact adopt these segmentalized parameters as the internal state model, which reflect the results of the interaction instead of the simple satisfaction value.

The motivation for communication is to be relieved of nervousness about the relationship. The first state of the relationship, level 0 in the previously explained stepwise interaction, is in a highly nervous state, as shown in Fig. 9-I. When there is a positive interaction with the user, the internal state of the robot changes to Fig. 9-II in Fig. 9. The goal of the motivation is to reach Fig. 9-III. If there is some negative interaction, the state moves to Fig. 9-IV. When there is no reaction from the user, the robot’s state gradually moves to Fig. 9-V. \(\theta \) in Fig. 9 is the change of the emotional state with a basic route from Fig. 9-I to Fig. 9-III.

Fig. 9.
figure 9

Internal states and motivation for communication

Full size image

Next, we discuss the conversion of the touch input corresponding to the receiver’s internal state. For example, when the user is in the internal state at Fig. 9-I, a strong stimulus would be converted into a negative impression in the sensitive situation. On the other hand, the variation of the interaction would enable the growth of a trusting relationship in a favorable situation. Therefore, it is important to convert the external stimulus into the interpreted emotion of the receiver based on the receiver’s internal state, not only to create action and reaction by a simple mechanism. Figure 10 shows the proposed conversion into the emotion for both the robot and the user. There should also be a relation between the internal states of both the robot and user via physical contact. Both internal state models allow the robot to have aspects of (a) honest expressions corresponding to its internal state and (b) the robot’s concern for the user corresponding to her/his internal state. The long-term internal state as emotion toward each other should be considered based on the assumed relationship between them in the future.

Fig. 10.
figure 10

The conversion of the external touch input into the internal state

Full size image

Common Expressions of Physical Contact for Different Robots: Finally, the common expressions of physical contact from the robot should be investigated for heterogeneous robots. The expressive design of the physical contact involves the problem of the acquisition of the sensor data for automatic emotional estimation. In our preliminary test using a stuffed toy containing a touch sensor, the patterns of physical contact and the user’s emotional expression observed by the sensor were classified by the length and the strength of the touch as we expected. The robot’s physical contact corresponding to the robot’s internal state should be based on the patterns of the user’s expressive touches.

From the viewpoint of the compatibility of the design, we assigned the strength of physical contact to the actuators in each robot we developed corresponding to the power of the haptic stimuli. The vibration motors in the first and second prototype are considered to provide narrow and weak stimuli. On the other hand, the pneumatic actuators in the third prototype are considered to generate large and strong stimuli, even when only one actuator is used.

According to the characteristics of stimuli from each actuator, the patterns of physical contact are converted. In Table 2, there are different types of physical contact for the four pneumatic actuators in the third prototype. The notifying touch without an emotional element is generated through simple conversion of the strength and width. The affective touch varies among several patterns. The embracing expression, which is a robot behavior where the robot embraces the user’s arm, is implemented by shortening all of the actuators. The stroking expression, which is a robot behavior where the robot strokes the user’s arm, is implemented by alternatively activating actuators 1 and 2 and actuators 3 and 4. Clinging, which is a robot behavior where the robot pulls the user’s arm or clothing while hanging on to the user, is implemented by sequentially activating actuators (1, 4)–(1, 2, 3, 4)–(2, 3). The strength, width, and length of the touch are interpreted through (1) the internal states of both the robot and the user and (2) characteristics of the actuators.

For the other prototypes, the placement and the characteristics of the actuators, such as vibration motors, should be used for the conversion of the stimuli. If there is no substitute actuator for the robot, the strength and timing of the stimuli should be designed in order to complement the missing actuator.

Table 2. Patterns of physical contact for the third prototype
Full size table

5 Conclusion

In this paper, we discussed the framework for a bidirectional haptic interaction system for our wearable robots. In order to involve various interactions, we classified types of physical contact (active or passive touch and one-way or bidirectional touch) and designed policies for the haptic communication. Based on the configurations of the wearable robots, we proposed an interactive physical contact mechanism and the flow of the communication with the internal states of both the robot and the user. There are three methods for communication with bidirectional physical touch: the stepwise change of the levels, the emotional effect on the interpretation of the touch, and the generation of the physical touch that transforms from the commonly designed expression into the appropriate physical contact by changing the strength and width corresponding to each actuator in each robot.

The previously proposed prototypes are configured for elderly people during outings. If the bidirectional physical contact grows the trusting relationship between the user and the robot, both the accessibility and the acceptability of the wearable robot will be improved, which will lead to the continuous use of the robot. A deep understanding through emotional interaction with physical contact is expected to reduce the user’s anxieties, which stop the user from taking further action.

The system requires the evaluation of the basic interaction with stepwise habituation and the internal state. In future work, the compositive design of the physical contact with other modalities should be further considered. From the viewpoint of the application for care receivers, a demonstration experiment in an elderly care house or nursery school should be held to clarify the effectiveness of the proposed design for elderly people or children.