Keywords

1 Introduction

In recent years, many personal mobility operated by weight shift like Segway (Segway, Inc.) [1] and UNI-CUB (Honda Motor Co., Ltd.) [2] have been developed and attracted. Segway is introduced for security staff’s transportation in Haneda airport, Japan. It is expected to expand the rider’s view by raising his/her line of sights and to improve maneuverability. UNI-CUB appears in a music video of OK Go, a music artist from USA. They can use both hands freely during their ride on UNI-CUB, and they show fresh and good performance in the music video [3]. As mentioned above, a lot of such kinds of vehicles on land are developed [4] and commercialized. However, how about on water?

At the present time, the transportation on water is mainly ship. There are many works around water such as river cleaning and water quality survey by using ships. Ships are often propelled by the outboard motor at the rear of the ships and can move at steady speed while it’s difficulty to turn on the spot and maneuver in a narrow space. Therefore, we think that the efficiency of water activities can be improved by implementing maneuvering method with weight shift that is introduced to the mobility on land. We have developed Multidirectional INtuitive Aquatic MObility (MINAMO, which means “water surface” in Japanese) as shown in Fig. 1, which is maneuvered with weight shift [5]. The MINAMO enables the rider to freely move on water with hands-free maneuvering and it is expected that the efficiency of tasks on water is improved.

On the other hand, in Japan, Superhuman Sports Society chants slogan of ‘unification of person and machine’ to create the future where everyone can sport equally by the fusion of people and technology [6]. In Switzerland, a competition called “Cybathlon” was held in 2016 [7]. In this competition, disables who wear “motorized leg prosthesis” or “powered exoskeleton” perform tasks. Not only the performance of the competitor but also the technology contributes to the result. Thus, the relationship between people and machines is getting close, and various UIs are being developed.

In this study, we have implemented various new UIs to maneuver the MINAMO to investigate the appropriate UIs for water activities experimentally. We will summarize the evaluation of each UI on usability and develop MINAMO into more comfortable water vehicle.

Fig. 1.
figure 1

The MINAMO

Full size image

2 Principle of Multidirectional Movement of MINAMO

Many mobile mechanisms that can move multidirectionally have been reported such as multidirectional personal mobility [2, 4] and omnidirectional mobile robot [8]. In [4, 8], the mechanism arrangements are explained. Propulsion arrangements in an underwater vehicle are introduced in [9].

The MINAMO realizes multidirectional movement on the water using four thrusters fixed as in Fig. 2. The x-y axes and the number of thrusters are defined as shown in the same figure. Each thruster can generate the propulsive force forward and backward. The multidirectional movement and turn are accomplished by adjusting the propulsion of these thrusters.

A method to calculate the output command to the thrusters (Fig. 3) is described below:

  1. 1.

    The rider commands the propulsive direction \(\varvec{f}_{\text {M}}=[f_{\text {Mx}},\,f_{\text {My}},\,\tau _{\text {Mz}} ]^\text {T}\) to MINAMO through the UI.

  2. 2.

    The MINAMO’s propulsive command \(\varvec{f}_{\text {M}}\) is converted to the thrusters’ output command \(\varvec{f}_{\text {T}}=[f_{\text {T1}},\,f_{\text {T2}},\,f_{\text {T3}},\,f_{\text {T4}}]^\text {T}\) by using the transformation matrix determined by the thruster arrangement as shown in Eq. (1).

  3. 3.

    The thrusters’ output command \(\varvec{f}_{\text {T}}\) is given to the motor drivers.

    $$\begin{aligned} \left[ \begin{array}{c} f_{\text {T1}} \\ f_{\text {T2}} \\ f_{\text {T3}} \\ f_{\text {T4}} \\ \end{array} \right]= & {} \frac{1}{4r} \left[ \begin{array}{rrr} -\sqrt{2}r &{} \sqrt{2}r &{} 1 \\ \sqrt{2}r &{} \sqrt{2}r &{} -1 \\ \sqrt{2}r &{} -\sqrt{2}r &{} 1 \\ -\sqrt{2}r &{} -\sqrt{2}r &{} -1 \\ \end{array} \right] \left[ \begin{array}{c} f_{\text {Mx}} \\ f_{\text {My}} \\ \tau _{\text {Mz}} \\ \end{array} \right] \end{aligned}$$
    (1)

where, r is the radius of the thruster arrangement.

Fig. 2.
figure 2

Coordinate of MINAMO and arrangement of thrusters

Full size image
Fig. 3.
figure 3

Derivation of propulsion

Full size image

3 User Interfaces of MINAMO

In this paper, five UIs of MINAMO, a force plate, a joystick, a gamepad, an inertial measurement unit (IMU) and Myo are evaluated. In this section, these UIs are described in detail.

3.1 Force Plate

Figure 4 shows a force plate used as an UI. It is made of the acrylic plate and four load cells that support the plate. The load cells are placed at the four corners of the force plate as shown in Fig. 5. The sensor values are expressed in Fig. 5 and as follows:

$$\begin{aligned} \varvec{w}_\text {FP}= \left[ \begin{array}{cccc} w_{\text {LF}}, &{} w_{\text {LB}}, &{} w_{\text {RB}}, &{} w_{\text {RF}}\\ \end{array} \right] ^\text {T}\end{aligned}$$
(2)

In addition, the total value \(w_{\text {sum}}=w_{\text {LF}}+w_{\text {LB}}+w_{\text {RB}}+w_{\text {RF}}\) corresponds to the weight of the rider.

Based on the above values, COG of the rider on the force plate is calculated and the propulsive direction vector \(\varvec{f}_{\text {Md}}=\left[ \begin{array}{ccc} f_{\text {Mxd}}, &{} f_{\text {Myd}}, &{} \tau _{\text {Mzd}} \\ \end{array} \right] ^\text {T}\) is set as follows:

$$\begin{aligned} f_{\text {Mxd}}= & {} \,k_{{\mathrm{FP}}\mathrm{x}} \frac{(w_{\text {LF}}+w_{\text {RF}})-(w_{\text {LB}}+w_{\text {RB}})}{w_{\text {sum}}} \end{aligned}$$
(3)
$$\begin{aligned} f_{\text {Myd}}= & {} \,k_{{\mathrm{FP}}\mathrm{y}} \frac{(w_{\text {LF}}+w_{\text {LB}})-(w_{\text {RF}}+w_{\text {RB}})}{w_{\text {sum}}} \end{aligned}$$
(4)
$$\begin{aligned} \tau _{\text {Mzd}}= & {} \,k_{{\mathrm{FP}}\mathrm{z}} \frac{(w_{\text {RF}}+w_{\text {LB}})-(w_{\text {LF}}+w_{\text {RB}})}{w_{\text {sum}}} \end{aligned}$$
(5)

They can be rewritten as a vector expression below:

$$\begin{aligned} \varvec{f}_{\text {Md}}= & {} \,\frac{1}{w_{\text {sum}}} \varvec{K}_\text {FP}\,\varvec{S}\,\varvec{w}_\text {FP}\end{aligned}$$
(6)

where,

$$\begin{aligned} \varvec{K}_\text {FP}= \left[ \begin{array}{ccc} k_{{\mathrm{FP}}\mathrm{x}} &{} 0 &{} 0 \\ 0 &{} k_{{\mathrm{FP}}\mathrm{y}} &{} 0 \\ 0 &{} 0 &{} k_{{\mathrm{FP}}\mathrm{z}} \\ \end{array} \right] ,\qquad \varvec{S} = \left[ \begin{array}{rrrr} 1 &{} -1 &{} -1 &{} 1 \\ 1 &{} 1 &{} -1 &{} -1 \\ -1 &{} 1 &{} -1 &{} 1 \\ \end{array} \right] \end{aligned}$$
(7)

As a result, the rider can move in the direction with the COG and turn by providing weight on the diagonal line on the force plate such as “\(w_{\text {RF}}\) and \(w_{\text {LB}}\)” or “\(w_{\text {LF}}\) and \(w_{\text {RB}}\)”.

The rider has to keep standing during the operation of this UI to move the COG of rider.

Fig. 4.
figure 4

Force plate

Full size image
Fig. 5.
figure 5

Arrangement of load cells

Full size image

3.2 Joystick

A joystick (Fig. 6) that can detect the knob’s inclination in the x-y axes and the rotation in the z axis is used as the second UI. Maneuvering the joystick corresponds with the propulsive direction of MINAMO as shown in Fig. 7. Therefore, multiplying the inclination ratio of the joystick knob by the gain is used for the MINAMO’s propulsive command as follows:

$$\begin{aligned} \varvec{f}_{\text {Md}}= & {} \,\varvec{K}_\text {JS}\,\varvec{\theta }_\text {JS}\end{aligned}$$
(8)

where,

$$\begin{aligned} \varvec{K}_\text {JS}= \left[ \begin{array}{ccc} k_{{\mathrm{JS}}\mathrm{x}} &{} 0 &{} 0 \\ 0 &{} k_{{\mathrm{JS}}\mathrm{y}} &{} 0 \\ 0 &{} 0 &{} k_{{\mathrm{JS}}\mathrm{z}} \\ \end{array} \right] ,\quad \varvec{\theta }_\text {JS}= \left[ \begin{array}{ccc} \displaystyle \frac{\theta _{\mathrm{JSx}}}{\theta _\text {JSx, max}} &{} \displaystyle \frac{\theta _\text {JSy}}{\theta _\text {JSy, max}} &{} \displaystyle \frac{\theta _\text {JSz}}{\theta _\text {JSz, max}} \\ \end{array} \right] ^\text {T}\end{aligned}$$
(9)

The rider can operate this UI in a sitting state since the rider does not have to move the COG. On the other hand, the rider cannot operate the MINAMO without the hands.

Fig. 6.
figure 6

Joystick

Full size image
Fig. 7.
figure 7

Joystick operation

Full size image

3.3 Gamepad

We constructed a remote operation system using a gamepad shown in Fig. 8. The analog stick in the gamepad is used for the maneuvering. The left and right analog sticks are used for translation and turning, respectively. In addition, the arrow pad and LR buttons have the function to change the maneuvering mode and the gain for the propulsion. The gamepad makes it possible to maneuver the MINAMO without boarding it and to maneuver with a bird’s-eye view as RC cars or ships.

Fig. 8.
figure 8

Gamepad

Full size image

The MINAMO’s propulsive command by the gamepad is calculated as follows:

$$\begin{aligned} \varvec{f}_{\text {Md}}= & {} \,\varvec{K}_\text {GP}\,\varvec{\theta }_\text {GP}\end{aligned}$$
(10)

where,

$$\begin{aligned} \varvec{K}_\text {GP}= \left[ \begin{array}{ccc} k_{{\mathrm{GP}}\mathrm{x}}&{} 0 &{} 0 \\ 0 &{} k_{{\mathrm{GP}}\mathrm{y}} &{} 0 \\ 0 &{} 0 &{} k_{{\mathrm{GP}}\mathrm{z}} \\ \end{array} \right] ,\,\varvec{\theta }_\text {GP}= \left[ \begin{array}{ccc} \displaystyle \frac{\theta _\text {GPx}}{\theta _\text {GPx, max}} &{} \displaystyle \frac{\theta _\text {GPy}}{\theta _\text {GPy, max}} &{} \displaystyle \frac{\theta _\text {GPz}}{\theta _\text {GPz, max}} \\ \end{array} \right] ^\text {T} \end{aligned}$$
(11)

\(\theta _\text {GPx}\), \(\theta _\text {GPy}\), \(\theta _\text {GPz}\) are the angles of the analog sticks as shown in Fig. 8.

3.4 Inertial Measurement Unit (IMU)

We implemented hands-free maneuvering using MINAMO’s inclination on the water surface [10]. A high performance IMU is used for the inclination detection. This unit has a 3-axis gyroscope (angular velocity sensor), a 3-axis accelerometer, and a 32-bit microprocessor. The inertial information can be obtained by the serial communication of USB connection with PC, and the roll, pitch and yaw angles are calculated based on the inertial information received on the PC. In the touch display fixed to the front of MINAMO, the calculated roll and pitch angles and propulsive direction are displayed. At the same time, the rider can instantly recognize the inclination of 3D MINAMO model as illustrated in Fig. 9.

$$\begin{aligned} \varvec{f}_{\text {Md}}= & {} \,\varvec{K}_\text {IMU}\,\varvec{\theta }_\text {IMU}\end{aligned}$$
(12)

where,

$$\begin{aligned} \varvec{K}_\text {IMU}= \left[ \begin{array}{ccc} k_{{\mathrm{IMU}}\mathrm{x}} &{} 0 &{} 0 \\ 0 &{} k_{{\mathrm{IMU}}\mathrm{y}} &{} 0 \\ 0 &{} 0 &{} k_{{\mathrm{IMU}}\mathrm{z}} \\ \end{array} \right] ,\qquad \varvec{\theta }_\text {IMU}= \left[ \begin{array}{ccc} \displaystyle \theta _\text {pitch}, &{} \displaystyle \theta _\text {roll}, &{} \displaystyle \theta _\text {yaw}\\ \end{array} \right] ^\text {T} \end{aligned}$$
(13)

In this study, \(k_{{\mathrm{IMU}}\mathrm{z}}\) was set to zero since \(\varvec{\theta }_\text {IMU}\) cannot be adjusted arbitrarily by the rider himself/herself.

Fig. 9.
figure 9

GUI for IMU

Full size image

3.5 Myo

Figure 10 shows Myo which is an armband type wearable device with myoelectric sensors, a gyroscope sensor, and an accelerometer [11]. Myo can easily detect gestures of arms, wrists and fingers. We built new hands-free maneuvering system using it.

In this study, we made gestures of ‘spreading out hands’, ‘bending the wrist inward’ and ‘bending the wrist outward’ correspond to ‘moving forward’, ‘turning counterclockwise’ and ‘turning clockwise’, respectively, as shown Fig. 11.

Since maneuvering by the hand gesture does not interfere with the posture of the rider, he or she can move with hands-free maneuvering without a physical burden.

Fig. 10.
figure 10

Myo

Full size image
Fig. 11.
figure 11

Hand gesture on Myo

Full size image

4 Experiments

In this section, the comparative experiments of UIs are described in detail.

4.1 Purpose of Experiments

Using the five types of UIs which were described in the previous section, the difference in the operation performance and the influence to the rider are evaluated experimentally. The purpose of the experiments is to examine the suitable UI for MINAMO.

figure a

4.2 Expermental Procedure

The experimental environment is an outdoor swimming pool without flow as shown in Fig. 12. We set the following three procedures as evaluation criteria.

figure b

Procedures 1 to 3 are performed using each UI and the period of time spent performing each procedure is recorded.

In the case of maneuvering by the inclination (IMU), since it does not have a turning function, the rider shall turn it by using the spoon net like an oar.

Fig. 12.
figure 12

Experiment environment

Full size image

4.3 Experimental Results

The overlapped photos during the experiments are shown in Figs. 13, 14, 15, 16 and 17. The experimental results of time are summarized in Table 1.

As seen from Figs. 13, 14, 15, 16 and 17, in the cases of the force plate and Myo, the rider was standing on the MINAMO. On the other hand, in the cases of the joystick, the gamepad and IMU, the rider was sitting on the MINAMO.

Table 1. Experimental results of time
Full size table

4.4 Considerations

The differences in time for ‘Tasks on the water’ between UIs are smaller than those in time for ‘Straight Running’ and ‘Cornering’. This may have been caused by the influences of the position where the ball was thrown in.

The periods of time for ‘Straight Running’ and ‘Cornering’, especially, in the cases of the joystick and the gamepad, were very short. The movements in the cases were also stable. Two reasons can be considered.

One of the reasons is that the maneuverability is not easily affected by the condition of the rider. In the cases of the force plate and Myo, the internal variation of the rider himself/herself is used as the command value. Therefore, it is considered that the maneuvering accuracy varies depending on the rider’s physical condition. Meanwhile, the operation amounts in the joystick and the gamepad are generated by the inclination and the rotation of the potentiometer. They are easily adjusted in combination with the visual feedback. As a result, the riders can maneuver regardless of their own condition.

Fig. 13.
figure 13

Experiment in case of force plate

Full size image
Fig. 14.
figure 14

Experiment in case of joystick

Full size image
Fig. 15.
figure 15

Experiment in case of gamepad

Full size image
Fig. 16.
figure 16

Experiment in case of IMU

Full size image
Fig. 17.
figure 17

Experiment in case of Myo

Full size image

The other reason is the ease of turning. It can be said that the joystick and the gamepad have the translational and the turning commands independent from each other, and it is possible to command intendedly without confusing the respective commands.

As described above, while the UI which has the operation amount visualized like a joystick is suitable for the movement, the hands-free UI is suitable for the tasks on the water. Furthermore, it is expected that MINAMO will become a more comfortable water vehicle by constructing a system that the rider can more easily select the UI.

5 Conclusions

Experimental results suggest that it is important to select the appropriate UI for the specified purpose in order to perform tasks on the water efficiently.

The maneuvering with the force plate or Myo often causes wrong instructions due to human errors. As a result, it took a relatively long time to accomplish the tasks. The reason may be that it is difficult for the rider to control the muscles of the arms and legs perfectly and the command includes noises sometimes. On the other hand, the UIs allow the rider hands-free operation.

As future works, it is necessary to construct a comfortable switching and combining system of UIs to improve the maneuverability.