Keywords

1 Introduction

Telecommunication including virtual tours and web conferences is increasing. For example, the global web conferencing market will be around 10 billion dollars [1]. Today, many video conference systems such as Skype, appear.in and other research systems [2,3,4] help people communicate from remote places. One of the challenges in current telecommunication is lack of physical interaction, which can cause lack of connection for remote users. For example, those telecommunication systems cannot allow remote users to move pieces of a board game while playing with a local user. Thus, remote users can feel lack of connection because they cannot realize what they want to do [5]. We propose that installing a robot arm and teleoperating it can address the problem by enabling remote physical interaction (See Fig. 1).

Fig. 1.
figure 1

Installing a robot arm to realize physical interaction

Full size image

Teleoperation is found to be challenging and can induce excessive mental workload [6]. For example, teleoperation degrades work efficiency more than 50% compared with boarding operation in the construction machinery case [7]. Even for operators of construction machineries who have sophisticated skills to manipulate them, teleoperation can be challenging. Thus, some autonomy can be important because autonomy can realize easier teleoperation [8], and many researchers have achieved high efficiency teleoperation by increasing autonomy level. For example, master-slave interfaces have been proposed to allow teleoperators to input only hand position [9]. Additionally, AR based interfaces have been developed which allow teleoperators to input only the goal position [10].

However, higher levels of autonomy lead to humans feeling less sense of agency, which refers to the ownership of the actions [11]. There is a tradeoff between sense of agency and level of autonomy. Lack of sense of agency can cause lack of connection with local area and people, because a remote person does not feel they caused actions. Numerous research addresses levels of robot autonomy [12,13,14], and some research examines using robots to realize physical interaction [15, 16]. However, they focus on efficiency. There is limited prior work examining how sense of connection relates to agency and different autonomy levels. Therefore, we investigate the relationship between autonomy level and sense of connection of a remote person with local area and people. Figure 2 shows our hypothesis about the relationship between autonomy level and sense of connection. Lower levels of autonomy can lead to higher levels of connection as the green dotted curve in Fig. 2 because lower levels of autonomy lead to high sense of agency. The position of the green dotted curve in Fig. 2 can depend on mental workload to teleoperate because low mental workload can lead to ease teleoperation, which can lead to high levels of connection. Among many possible physical interactions, in this paper we focus on a pushing task for two reasons. First, pushing is one of the major functions in hand and arm use [17], and second, pushing something including buttons activates some exhibits in many places such as museums and theme parks.

Fig. 2.
figure 2

Tradeoff between connection and autonomy level

Full size image

2 Research Question

The purpose of this study is to investigate the relationship between autonomy level and sense of connection. Both autonomy level and sense of connection can be related to sense of agency as mentioned in the Sect. 1. Thus, in this Section, we explain sense of agency and decide autonomy levels to investigate.

2.1 Sense of Agency

Sense of agency can be categorized into the Feeling of agency (FOA) which is not conceptual and the Judgement of agency (JOA) which is conceptual [18]. For example, when people try to push buttons, they do not consciously consider all the joint angles of their upper limbs, associated with FOA, although they think about which buttons to push, which impacts JOA. In the pushing case, humans must decide which objects to push, mentally plan trajectories, calculate each joint angle for the planned trajectories [19].

2.2 Autonomy Levels to Investigate

We should investigate whether FOA associated with control of trajectories and joint angles affects the sense of connection that subjects feel with local area and people. If FOA affects the sense of connection, the interface should allow remote people to control each joint angle or trajectories. If FOA does not affect the sense of connection the robot arm control system should recognize the environment and manipulate the robot arm such as [10]. Thus, we investigate and compare the autonomy levels of three different interfaces. The first one is to input all the joint angles, the second one is to input trajectories (hand positions and pitch angles), and the last one is to input only the goal hand position.

3 Experiments

We conducted experiments to investigate the affect of FOA on the sense of connection. In this chapter, we explain the experimental tasks, the three developed interfaces, experimental procedure, results, and discussion.

3.1 Experimental Tasks

The experimental tasks, as shown in Fig. 3, are for a local person to select which buttons will light by clicking the keyboard (1–6), and for a remote person to control the robot to push as many illuminated buttons as possible in 1 min. A remote person can recognize the local area from watching the view captured from the camera in the local area. We allow only one button to light, and when the robot pushes the illuminated button, that button turns off. We used “PhantomX Pincher Robot Arm” with 4 degrees of freedom, as shown in Fig. 3 because pushing buttons does not require roll and yaw.

Fig. 3.
figure 3

Experimental setup

Full size image

3.2 Development of Three Interfaces

We should investigate whether FOA affects the sense of connection as explained in Sect. 2.2. Therefore, we developed three interfaces to control the robot arm. The first one is to input all the joint angles by moving another matching robot arm (JA; Joint Angles) as shown in Fig. 4(a). The second one is to input the hand position and pitch angle tracked by using a leap motion (PP; Position and Pitch) as shown in Fig. 4(b). The last one is to input only the goal hand position of the robot by clicking the circle in the web page (GP; Goal Position) as shown in Fig. 4(c).

Fig. 4.
figure 4

Experimental comparison

Full size image

3.3 Experimental Procedure

14 volunteers (12 male and 2 female) were involved as subjects in this experiment. Subjects tried 2 sets, and each set includes 6 trials with the task explained in Sect. 3.1. Half of the subjects (7 subjects) performed the tasks as remote persons in the first set, and then performed as local persons in the second set, and the other half of the subjects (7 subjects) performed the tasks as local persons in the first set, and then performed as local persons in the second set. All the subjects as remote persons performed in the order of JA, PP, GP, GP, PP, and JA in a set. We measured the number of buttons pushed as work efficiency, sense of agency by the questionnaire referred to [20], sense of connection by three factors including telepresence, which is defined as feeling of being there, by the questionnaire referred to [21], social presence, which is defined as being perceived as real, by the questionnaire referred to [22], and social connection by the questionnaire referred to [23], and mental workload by NASA-TLX [24] (see Appendix for details).

3.4 Results and Discussion

Sense of Agency.

Figure 5 shows the results of sense of agency. No significant differences are observed among the three interfaces by the Bonferroni method. This result is different from the previous research which indicates that higher levels of autonomy lead to humans feeling less sense of agency [11], so we discuss below. Figure 6 shows the results of the numbers of buttons pushed correctly. The Bonferroni method indicates that GP pushed more correct (illuminated) buttons than JA and PP by a significant amount. Therefore, the results suggest that autonomy works well. Moreover, subjects can have enough JOA because they selected which buttons to push. Those can lead to high sense of agency in GP.

Fig. 5.
figure 5

Sense of agency

Full size image
Fig. 6.
figure 6

The numbers of buttons pushed correctly

Full size image

Sense of Connection

Telepresence. Figure 7 shows the results of telepresence. The Bonferroni method indicates that JA provides significantly higher telepresence than GP and a significant trend is observed between PP and GP. Therefore, higher level of autonomy can lead to lack of telepresence.

Fig. 7.
figure 7

Telepresence

Full size image

Social Presence for Remote Person.

Figure 8 shows the results of social presence for remote people. The Bonferroni method indicates that PP has significantly lower social presence than JA and GP, so we discuss why PP has lower social presence. Figure 9 shows the results of incorrect pushes which are the number of non-illuminated buttons pushed. The Bonferroni method indicates that PP has more incorrect pushes than JA and GP. Subjects could push incorrectly due to difficulties in teleoperation because they can recognize which buttons are on. Figure 10 shows the results of mental workload. Bonferroni method indicates that PP has higher mental workload than GP significantly and a significant trend is observed between JA and PP. Thus, PP can require high cognitive load, which may lead to difficulties in teleoperation and many incorrect pushes.

Fig. 8.
figure 8

Social presence for remote people

Full size image
Fig. 9.
figure 9

The numbers of buttons pushed incorrectly

Full size image
Fig. 10.
figure 10

Mental workload

Full size image

Social Presence for Local People.

Figure 11 shows the results of social presence for local people. No significant differences are observed among the three interfaces by Bonferroni method. This result suggests that autonomy levels may have little effect on social presence for local people.

Fig. 11.
figure 11

Social presence for local people

Full size image

Social Connection.

Figure 12 shows the results of social connection. PP has lower score, but no significant differences are observed among the three interfaces by the Bonferroni method. This result suggests that autonomy levels may have little effect on social connection, but we need more subjects.

Fig. 12.
figure 12

Social connection

Full size image

Discussion.

Here, we discuss the best interface. 6 subjects preferred JA, 3 subjects preferred PP, 4 subjects preferred GP, and 1 subject preferred JA for fun and GP for performance from the questionnaire. The reason given why subjects who preferred JA was for fun, and one for GP was for performance from the questionnaire. Moreover, the results suggested that JA had high telepresence and GP had high work efficiency. Therefore, people who would like to have a fun experience may prefer JA and those who would like to have high performance work may prefer GP.

4 Conclusion

The purpose of this study was to investigate the relationship between autonomy level and sense of connection for remote social physical interaction. In the study we investigated whether or not FOA is affected by autonomy. The results suggested that higher autonomy could decrease telepresence. Furthermore, the results suggested that PP has lower social presence. Overall, remote people could prefer JA for fun, and GP for performance. We plan to conduct future experiments with different tasks including grasping and transporting.