A saliency-driven robotic head with bio-inspired saccadic behaviors for social robotics

Abstract

This paper presents a robotic head for social robots to attend to scene saliency with bio-inspired saccadic behaviors. Scene saliency is determined by measuring low-level static scene information, motion, and object prior knowledge. Towards the extracted saliency spots, the designed robotic head is able to turn gazes in a saccadic manner while obeying eye–head coordination laws with the proposed control scheme. The results of the simulation study and actual applications show the effectiveness of the proposed method in discovering of scene saliency and human-like head motion. The proposed techniques could possibly be applied to social robots to improve social sense and user experience in human–robot interaction.

Appendix

1.1 Computation of linear projections using corresponding points

Given one corresponding position, the projective transformation maps is

$$\begin{aligned} \left[ \begin{array}{c} r_{k}^{i}\\ c_{k}^{i}\\ 1 \end{array}\right] =\left[ \begin{array}{ccc} P_{11}^{ij} &{} P_{12}^{ij} &{} P_{13}^{ij}\\ P_{21}^{ij} &{} P_{22}^{ij} &{} P_{23}^{ij}\\ P_{31}^{ij} &{} P_{32}^{ij} &{} P_{33}^{ij} \end{array}\right] \left[ \begin{array}{c} r_{k}^{j}\\ c_{k}^{j}\\ 1 \end{array}\right] \end{aligned}$$

(28)

To solve the optimal linear projection, the linear projection (28) can be rewritten in the matrix form Hartley and Zisserman (2000),

$$\begin{aligned} (\mathbf v _{k}^{x})^{T}\mathbf p ^{ij}&= 0\end{aligned}$$

(29)

$$\begin{aligned} (\mathbf v _{k}^{y})^{T}\mathbf p ^{ij}&= 0 \end{aligned}$$

(30)

where

$$\begin{aligned} \mathbf p ^{ij}&= [P_{11}^{ij},P_{12}^{ij},P_{13}^{ij},P_{21}^{ij},P_{22}^{ij},P_{23}^{ij},P_{31}^{ij},P_{32}^{ij},P_{33}^{ij}]^{T}\end{aligned}$$

(31)

$$\begin{aligned} \mathbf v _{k}^{x}&= [-r_{k}^{i},-c_{k}^{i},-1,0,0,0,r_{k}^{j}r_{k}^{i},r_{k}^{j}c_{k}^{i},r_{k}^{j}]\end{aligned}$$

(32)

$$\begin{aligned} \mathbf v _{k}^{y}&= [0,0,0,-r_{k}^{i},-c_{k}^{i},-1,c_{k}^{j}r_{k}^{i},c_{k}^{j}c_{k}^{i},c_{k}^{j}]. \end{aligned}$$

(33)

Assuming that corresponding points in both images can be identified and correspondence can be approximated with linear maps within a small movement of the camera, the projection matrix can be computed by

$$\begin{aligned} V_{ij}\mathbf p ^{ij}=0 \end{aligned}$$

(34)

where $V_{ij}=[\mathbf v _{1}^{x},\mathbf v _{1}^{y}, \mathbf v _{2}^{x},\mathbf v _{2}^{y},\ldots , \mathbf v _{k}^{x},\mathbf v _{k}^{y}]$ with $k$ corresponding points between the two images.

1.2 Quaternion element projection

Property 1

(Quaternion element projection) Let a quaternion be $x=o\mathsf 1 +a\mathsf i +b\mathsf j +c\mathsf k$ with $1$, $\mathsf i$, $\mathsf j$ and $\mathsf k$ as the element basis. If $a=0$, then $x*\mathsf i =\mathsf i *x^{+}$ where $x^{+}$ is the conjugate of $x$. The rest for $b$ and $c$ may be deduced by analogy.

Proof

The proof is straightforward by expanding the left and right sides of $x*\mathsf i =\mathsf i *x^{+}$ with the quaternion definition. $\square$