Elsevier

Applied Soft Computing

Volume 11, Issue 1, January 2011, Pages 927-935
Applied Soft Computing

Fuzzy clustering of human motor motion

https://doi.org/10.1016/j.asoc.2010.01.013Get rights and content

Abstract

Acquisition of the behavioural skills of a human operator and recreating them in an intelligent autonomous system has been a critical but rather challenging step in the development of complex intelligent autonomous systems. Development of a systematic and generic method for realising this process by acquiring human postural and motor movements is explored. This is achieved by breaking down the human motion into a number of segments called motion or skill primitives. The proposed methodology is developed based on studying the movement of the human hand. The motion is measured by a dual-axis accelerometer and a gyroscope mounted on the hand. The gyroscope locates the position and configuration of the hand, whereas the accelerometer measures the kinematics parameters of the movement. The covariance and the mean of the data produced by the sensors are used as features in the clustering process. A fuzzy clustering method is developed and applied to identify different movements of the human hand. The proposed clustering approach identifies the sequence of the motion primitives embedded in the data produced from the human wrist movement. A review of the previous work in the area is carried out and the developed methodology is described. An overview of the experimental setup and procedures to validate the approach is given. The results of the validation are analysed critically and some conclusions are drawn.

Introduction

Acquisition of the behavioural skills of a human operator and recreating them in an intelligent autonomous system has been a critical but rather challenging step in the development of complex intelligent autonomous systems. A systematic and generic method for realising this process will greatly simplify the development, commissioning and maintenance of autonomous systems.

Human skills are robust and reactive strategies for executing recurring tasks in a particular domain. Based on proficiency in these skills, people can develop and execute higher level plans. Efficient acquisition and modeling of these skills, and implementing them in an autonomous system are quite challenging. It requires simultaneous reduction of robot programming complexity and increasing sensor integration, which are competing and contradictory goals [1].

A human operator automatically employs tacit skills to perform a dynamic real-time task and is typically unable to provide an accurate and complete description of the employed skills and their sequence [2]. Hence, conventional methods used in knowledge acquisition are not sufficient to identify and acquire the skills deployed by the operator.

Teaching of the human psychomotor behaviour to a robotics manipulator through demonstration has become a popular area of research, and has led to an expansion in the development of relevant products and applications. According to Smith and Smith [3], there are three types of human psychomotor movements:

  • The postural movement which regulates body positioning.

  • The locomotor movement, which translates and rotates the body.

  • The manipulative movements through which the environment is manipulated.

This work has its focus on the first two movements and explores how the human postural and locomotor movements can be transferred to a biped robot through demonstration. Since the human motion is inherently stochastic, complicated and unpredictable, tracking the multi-paths of the human movement does not fully describe the human skills deployed to perform the manipulation. In this work a more effective approach is developed in which the human motion is broken down into a number of segments called motion primitives.

The approach deploys the concept of skill primitives in formulating its hypothesis and methodologies. In the context of robotics, a skill primitive can be defined as an instruction which drives a robotics system along a specific trajectory relative to a reference coordinate frame towards performing a particular task [4]. Complex non-linear behaviour is achieved through piecewise integration of skill primitives. A skill, as stated by James [1], represents an innate capability to perform a specific task. This definition implies that a skill cannot be considered in isolation from the task. Hence, a robotics task can be considered as the synthesis of the skills required to carry it out. A valid skill should have the ability to deal with uncertainty and variations in the task.

In the study reported in this paper, the proposed methodology is developed based on studying the movement of the human hand. The motion is measured by a dual-axis accelerometer and a gyroscope mounted on the hand. The gyroscope locates the position and configuration of the hand, whereas the accelerometer measures the kinematics parameters of the movement. The covariance and the mean of the data produced by the sensors are used as features in the clustering process.

A fuzzy clustering method is developed and applied to identify different movements of the human hand. The proposed clustering approach identifies the sequence of the motion primitives embedded in the data produced from the human wrist movement.

The remainder of the paper is organized as follows. A review of the background work is carried out in Section 2. The concept of prototype-based fuzzy clustering is introduced in Section 3. The application of feed-forward neural network (FFNN) in constructing the boundaries of motion clusters is described in Section 4. An overview of the experimental setup and pre-processing of the data produced by the experimental work is provided in Section 5. The application of the fuzzy clustering to the motion data in validating the approach is described in Section 6. In this section, the progress made and the results obtained are reported and a critical review of the outcomes is carried out. Finally, Section 7 analyses the strengths and constraints of the work and makes some recommendations for future work.

Section snippets

Background

Perceiving and understanding the human behavioural motions has gained increasing interest in a number of areas including robotics, homecare and video processing. Inamura et al. [5] have developed a robot mimesis loop capable of identifying human movements. The sequential joint angles of the robot have been encoded as 11 basic self-motion elements. Variant human behaviours have been abstracted by proto-symbols generated from the self-motion elements using Hidden Markov Model.

The concept of skill

Prototype-based fuzzy clustering

It is a challenging process to break the natural human motions into distinctive primitives due to blurry boundaries between them. This makes fuzzy clustering an ideal approach in the clustering process. In this work a prototype-based fuzzy clustering method is deployed.

The prototype-based fuzzy clustering methods which include Fuzzy-C-Mean algorithm [17], G-K algorithm [18], AFC (adaptive fuzzy clustering) algorithm [19] and EM (electromagnetic) algorithm [20] are widely used in pattern

Cluster boundary construction

The boundaries of the clusters identified in the process are constructed using feed-forward neural network (FFNN). The FFNN is composed of a hierarchy of processing nodes, organized in a series of two or more mutually exclusive sets or layers of neurons. The first layer is the input layer which serves as a holding site for the inputs. The output layer is the point at which the overall mapping of the network input is available. Between the two extremes lies one or more layers called hidden

Experimental setup and data

The performance of the proposed algorithm has been evaluated based on a series of experiments focused on modeling and perception of different hand movements. The motion is measured by a dual-axis accelerometer and a gyroscope mounted on the hand. The gyroscope locates the position and configuration of the hand, whereas the accelerometer measures the kinematics parameters of the movement. In this section the experimental rig and the characteristics of the data produced on the rig are studied.

Application of fuzzy clustering

Different steps in application of the fuzzy clustering algorithm are illustrated in this section. The process is initiated by selecting and normalising appropriate features. Based on the inputs of the fuzzy clustering as a sequence of vectors F¯j, the first fuzzy clustering process provides the natural distribution of variant motion primitives. This step can be illustrated by the following mathematical relationship:F={Fj¯=fj1fj2fj3fj4|j=1N}FuzzyClusteringU=uij.where fj1 represents the

Validation

The methodology proposed in this study is validated in this section by applying it to two data streams obtained from different human hand motion sequences. In the first motion sequence, both of the human antebrachium and wrist movements are recorded four times. The second motion sequence consists only of the human wrist movements repeated four times. All these movements are recorded by the sensors mounted on the hand. The motion data is stored as eight different data sets representing two

Conclusion

The work reported in this paper has been conducted in the context of developing a generic approach to teach human postural and locomotor movements to a humanoid robot through observation. While the study has had a limited scope, it well demonstrates the concept and the potential of such an approach for programming complex autonomous systems.

The research was carried out based on a set of motion primitives defined based on the human wrist motion. While the defined primitives were basic and rather

Acknowledgment

Fazel Naghdy thanks Xucheng Zhang for conducting the experimental work validating this work.

References (28)

  • D.M. James, Sensorimotor primitives for programming robotic assembly skills, Ph.D. Thesis, Carnegie Mellon University,...
  • A. Isaac et al.

    Goal directed learning to fly

  • K.U. Smith et al.

    Perception and Motor

    (1962)
  • Open PC-based Robot Control Systems [Online], cited at:...
  • T. Inamura et al.

    Advanced intelligent mechatronics, mimesis embodiment and proto-symbol acquisition for humanoids

  • G.E. Hovland et al.

    Skill acquisition from human demonstration using a hidden Markov model

  • J. Eisenstein et al.

    Device independence and extensibility in gesture recognition

  • M.M. Rahman et al.

    A tuned Eignespace technique for articulated motion

    Lecture Notes on Computer Science

    (2006)
  • R. Amit et al.

    Learning movement sequences from demonstration

  • C. Juang et al.

    Human body posture classification by a neural fuzzy network and home care system application

    IEEE Transactions on Systems, and Cybernetics – Part A: Systems and Humans

    (2007)
  • Q.C. Nguyven et al.

    Real-time human tracer based on location and motion recognition of user for smart home

  • G. Shi et al.

    Towards HMM based human motion recognition using MEMS inertial sensors

  • S. Nomm et al.

    Hybrid approach to detection of the surgeon's hand motions during endoscope surgery

  • J. Hoey et al.

    Value-directed human behaviour analysis from video using partially observable Markov decision processes

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (July, 2007)
  • Cited by (7)

    • Fuzzy granular gravitational clustering algorithm for multivariate data

      2014, Information Sciences
      Citation Excerpt :

      One way to find relationships in data is to use clustering algorithms, these algorithms look through each piece of given data and find similarities between them, grouping them into clusters, serving to identify relationships which could be impossible for a human to find. The areas of application for these types of algorithms are vast; such as, business failure prediction [10], grouping similar opinions in the analysis of judicial prose [16], medical ECG beat type identification [18], find relations in the data between multidatabase systems [20], control de desulphurization process of steel [23], image segmentation [36], analysis of human motion for recreation in autonomous systems [43], pattern recognition [49], diagnosis decision support for Human Papilloma virus identification [51], sales forecasting of new apparel items [62], and molecular sequence analysis and taxonomy analysis [65]. There are distinct differences between existing clustering algorithms, mainly focused in how clusters are found, such as cluster accumulation [4], multi-objective nature-inspired [6], cluster estimation [11], density-based [21], fuzzy neural networks [33], general Type-2 fuzzy c-means [35], ant colonies [37], weight clustering [45], condition fuzzy c-means [46], simultaneous clustering [50], and constrained clustering with active query selection [63].

    • A tool for linguistic assessment of rehabilitation exercises

      2014, Applied Soft Computing Journal
      Citation Excerpt :

      Human motion tracking and analysis systems are particularly useful for medical diagnosis and monitoring [1]. These systems are able to acquire a big amount of information regarding the poses or movements of people [23]. One of the oldest yoga exercises, the Sun Salutation (Fig. 1), is taken as an example of a rehabilitation exercise.

    View all citing articles on Scopus
    View full text