A hand shape instruction recognition and learning system using growing SOM with asymmetric neighborhood function

Abstract

For Human–Machine Interaction systems, it is a convenient method to send user׳s instructions to robots, TV sets, and other electronic equipments by showing different shapes of a hand of user. In our previous works, we proposed to use improved Kohonen׳s Self-Organizing Maps (SOMs), i.e., Transient-SOM (T-SOM) and Parameterless Growing SOM (PL-G-SOM) to recognize different patterns of hand shapes given by different bendings of five fingers of a hand. Recently, an asymmetric neighborhood function was proposed and introduced into the conventional SOM to improve the learning performance by Aoki and Aoyagi. In this paper, we propose to employ their asymmetric neighborhood function into Growing SOM (GSOM), which is an improved SOM to deal with additional online learning for input data. Furthermore, the improved GSOM is applied to a hand shape recognition and instruction learning system, and the results of experiments with eight kinds of instructions showed the effectiveness of the proposed system.

Introduction

Theories and techniques of Human–Machine Interaction (HMI) are becoming more and more important nowadays, for the rapid developments of partner robots such as housework robots, entertainment robots, pet robots, etc. Users of robots or other intelligent machines are not only limited to experts and engineers, but also ordinary people and even children. For an intelligent autonomous robot, visual or auditory information should be acquired and analyzed to estimate the intention or instructions of human. So approaches such as gesture or posture recognition [1], [2], [3], [4], [5], [6], voice discrimination [7], [8], [9], face recognition [10], [11], [12], etc., have attracted many researchers.

Generally, for autonomous systems, there are two common issues to deal with:

• What is the input?

• How to get the output, or adapt the input for an unknown input?

The first issue seems to be more important, because the second issue it is directly related to the former. The input data needs to be classified, recognized and recorded by the system. So it is the major task of pattern recognition.

In [2], [3], [7], [8], [9], Kohonen׳s Self-Organizing Map (SOM) [13], [14], [15], [16], [17] is used to solve the first issue. SOM is a powerful unsupervised machine learning method to deal with clustering of high-dimensional data. As being a type of artificial neural networks (ANNs) which represents competitive learning in the primary sensory cortex, SOM has been extensively applied to data mining, pattern recognition, and visualized clustering of complex data sets.

In [4,5], hidden Markov model (HMM) is used to recognize hand gestures. HMM is a stochastic model which supposes than an unknown system belongs to a Markov processing with unknown parameters and estimates the parameters according to the observable information. For its high cost of computation, HMM is not spread in practical use.

In [6], Kuremoto et al. used a simplified retina-V1 model and one-pass dynamic programming method to estimate the different gestures of an arm of user.

Zhao, Huang and Sun [10] proposed a neural networks committee (NNC) machine to tackle face recognition based on multi-features.

In [11], Chen et al. proposed a kernel machine-based one-parameter regularized fisher discriminant method for face recognition, and showed that it is superior to conventional linear discriminant analysis (LDA) methods.

Li, Zheng and Huang in [12] gave an improved LDA named “locally linear discriminant embedding (LLDE)”.

Additionally, advanced ANNs [18], [19], [20], support vector machines (SVMs) [21] are also powerful tools for pattern recognition and HMI problems.

In this paper, we concentrate to develop a visual recognition system using an improved SOM. The system for HMI is supposed to classify input images consisting different shapes of a user’s hand, and learn how to output correct actions according to the judgment of the user (instructor).

Theoretically, there are some defects existed in the original SOM such as:

• Exhaustion of units.

• Difficulty of adjustment of parameters.

• Disorder of topology of map.

To overcome these problems, different improved SOM variants have been proposed.

For example, to deal with the exhaustion of units, Growing SOM (GSOM) is proposed in [22], [23], [24], [25], and Transient-SOM (T-SOM) is proposed in our previous works [26], [27].

To keep stable learning convergence and use less parameters, Parameterless SOM (PL-SOM) is proposed by Berglund & Sitte in [28] and we adopted its idea to GSOM as a Parameterless Growing SOM (PL-G-SOM) [7], [8], [9], [29].

To avoid defects of map׳s topology in learning process, an asymmetric neighborhood function is considered and adapted to the original SOM by Aoki and Aoyagi in [30], [31], [32] recently. However, this method is not used in other SOM variants such as GSOM, and PL-G-SOM.

In this paper, we propose to introduce Aoki and Aoyagi׳s asymmetric neighborhood function [30], [31], [32] into GSOM as two novel variation of SOM, one is named “AGSOM” and other one is “IAGSOM”. In AGSOM, the direction of asymmetric neighborhood is fixed during learning process. In IAGSOM, opposite directions are used iteratively to solve the disorder of topology asymptotically. AGSOM and IAGSOM are used in a hand shape instruction recognition and learning system [26], [27], [29] for HMI. Experiments including image processing were performed to confirm the proposed system.

In the rest of this paper, we will describe the original SOM [13], [14], [15], PL-G-SOM [7], [8], [9], [29], AGSOM (proposed), and IAGSOM (proposed) in Section 2, and employ them into a hand shape instruction recognition and learning system in Section 3. Hand image instruction learning experiments were reported in Section 4. Conclusions are in Section 5 at last.