Extreme learning machine for time sequence classification
Introduction
Time sequence is ubiquitous in many fields. For instance, the human–robot-interface may require to classify the gaits, gestures, or actions, all of which are representative time sequences. Especially, human activity recognition has become an important emerging field of research within context-aware systems [4], [5]. Reference [6] presented a wearable activity sensor system and a systematic activity classification scheme for the classification of human daily physical activities. The wearable activity sensor system, consisting of two activity sensor modules worn on dominant hand wrists and ankles of the users, is used for collecting activity acceleration signals. Other similar studies focused on how one can use a variety of accelerometers to identify a range of user activities.
The Dynamic Time Warping (DTW) distance has been extensively utilizedfor time sequence classification. It allows a measure of the similarity invariant to certain nonlinear variations in the time dimension and attempts to compensate for possible time translations/dilations between patterns. However, for long sequence, it is more approporiate to measure similarity from higher level structure but not point-to-point local comparisons. In [2], a Bag-Of-Features (BoF) approach in which complex objects are characterized by feature vectors of subobjects is proposed to tackle the problem of time sequence classification. The BoF representation allows one to integrate local information from segments of the time series in an efficient way. But this work is still based on the shape-based features such as the slope and variance. In [31], the Linear Dynamic Systems (LDS) model is used to construct a Bag-of-Systems (BoS) framework to classify visual dynamic texture. LDS is a powerful tool to model the rich time sequence. In [11] it was used to model the visual dynamic texture, and in a recent literature [22] the authors used such a model to discuss the intrinsic relation between control and machine learning. All of the above-mentioned methods transform the original time sequence into histogram representation and use the popular Support Vector Machine (SVM) to design the classifier.
On the other hand, Extreme Learning Machine (ELM) [14], [15] has attracted more and more researchers’ attention for its better performance than traditional parameters learning algorithm such as gradient descent algorithm in generalized single hidden layer feed-forward neural networks (SFLNs). In [16], the authors have proved that ELM tends to have better scalability and achieve similar (for regression and binary class cases) or much better (for multi-class cases) generalization performance at much faster learning speed (up to thousands times) than traditional SVM. ELM has been used in several domains ranging from human action recognition [17], [8], [25], face recognition [32], [26], visual tracking [20] and so on.
Motivated by the advantage of ELM and LDS, we regard time sequences as the output of an intrinsic dynamic system shown in Fig. 1. To obtain more complete representation for the time sequence, we use un-ordered multiple local LDSs to represent the whole time series. As soon as the features of each time series are obtained, we can train a classifier for recognition. The main contributions are summarized as follows:
- 1.
The whole time sequence is divided into several smaller sub-sequence by means of the sliding time window technique. The sub-sequence is reasonably modeled as LDS by appropriate dimension reduction. Further, the whole time sequence is represented as a BoS, which is a bag of LDS patches. Such a model is very flexible to describe time sequence originated from different sensor sources.
- 2.
To model the BoS, a codebook is proposed, which utilizes the Martin distance between LDSs and avoid the problem that LDS lies in non-Euclidean manifold.
- 3.
The obtained feature vector of time sequence is classified by an ELM, which provides strong generality and parameter insensitivity.
The rest of this paper is organized as follows. In Section 2, the overall architecture is illustrated. Section 3 reviews LDS and the metric for LDSs. In Section 4 we classify time sequences using proposed framework. Section 5 provides some experimental results. Finally, the conclusion is given in Section 6.
Section snippets
Architecture
The framework for time sequence recognition is inspired by the BoF approach to classify time series [1] and the bag of dynamical systems [27] in categorizing dynamic textures. The main steps in our framework are as follows:
- 1.
Extract LDS models from the training set.
- 2.
Form codebook using K-medoids clustering algorithm.
- 3.
Represent time sequence using the formed codebook.
- 4.
Train ELM using the representation vectors and corresponding labels.
- 5.
Given a new time sequence, infer which class it belongs to using
LDS representation for time series
Assume that a time series is a realization of a second-order stationary stochastic process [11]. This means that the joint statistics between two time instances is shift-invariant. In our paper, we assume that there exists symmetric positive-definite matrices and such thatwhere is the hidden state at time t with initial condition η0, models the dynamics of the hidden state, maps the hidden state to the
Recognizing time series using a bag of linear dynamical systems
The LDS introduced in the above section can be used to characterize the dynamics of the time sequence. However, its representative capability is weak since it is a simple linear model, while practical time sequence always contain complicated dynamics. To tackle this problem, we extract multiple subsequences from the time sequence and use the BoS method to construct features for classification. In this section we first introduce the feature extraction, then the codebook design and the time
Experiments and results
In this section we provide two sets of experimental evaluation. One is related to human activity recognition and the other is about tactile sequence classification. The two tasks are different but we regarded them as time series classification in the proposed unified framework. To the best knowledge of the authors, this is the first time for such a method to be used for such tasks.
Conclusions
This paper proposes a novel framework for time series recognition. The main difference between our method and previous work is that LDSs parameters are selected as features to represent time series. In order to describe time series more accurately, a codebook is formed by a set of LDSs parameters. Finally, we obtain a set of distribution vectors. A great advantage of this method is that the complicated feature design procedure is avoided and the LDSs can well capture the dynamics of the time
Acknowledgments
This work was supported in part by the National Key Project for Basic Research of China under Grant 2013CB329403; in part by the National Natural Science Foundation of China under Grant 61210013; and in part by the Tsinghua University Initiative Scientific Research Program under Grant 20131089295.
Huaping Liu received the Ph.D. degree from the Department of Computer Science and Technology, Tsinghua University, Beijing, China, in 2004. He is currently an Associate Professor in the Department of Computer Science and Technology at Tsinghua University. His research interests include intelligent control and robotics.
References (32)
- et al.
Sparsity induced similarity measure and its applications
IEEE Trans. Circuits Syst. Video Technol.
(2012) - et al.
Design of a flexible tactile sensor for classification of rigid and deformable objects
Robot. Auton. Syst.
(2014) - et al.
Extreme learning machinetheory and applications
Neurocomputing
(2006) - et al.
Dynamic action recognition based on denemes and extreme learning machine
Pattern Recognit. Lett.
(2013) - et al.
Human action recognition using extreme learning machine based on visual vocabularies
Neurocomputing
(2010) - et al.
A bag-of-features framework to classify time series
IEEE Trans. Pattern Anal. Mach. Intell.
(2013) - Y. Bekiroglu, D. Kragic, V. Kyrki, Learning grasp stability based on tactile data and HMMs, in: Proceedings of RO-MAN,...
- et al.
Assessing grasp stability based on learning and haptic data
IEEE Trans. Robot.
(2011) - et al.
Real world activity summary for senior home monitoring
Multimedia Tools Appl.
(2011) - F.C. Chuang, Y.T. Yang, T.P. Kao, A wearable activity sensor system and its physical activity classification scheme,...
Subspace angles between ARMA models
Syst. Control Lett.
Dynamic textures
Int. J. Comput. Vis.
A flexible capacitive tactile sensor array with micro structure for robotic application
Sci. China Inf. Sci.
Cited by (16)
Human-in-the-loop optimization of wearable robots to reduce the human metabolic energy cost in physical movements
2020, Robotics and Autonomous SystemsCitation Excerpt :The optimization problem with such objective functions could not be solved effectively with common optimization algorithms, such as sequential quadratic programming, genetic algorithm and particle swarm optimization and other iterative algorithm [24,25]. Bayesian optimization which is commonly used to select optimal hyperparameters in machine learning may be an effective tool to solve the optimization problem with the complex objective function charactering the human metabolic energy cost during movement [11,23,26–28]. OpenSim is an open-source software platform which could create and edit models of a broad range of human musculoskeletal structures and wearable robots [29,30].
Evaluation of extreme learning machine for classification of individual and combined finger movements using electromyography on amputees and non-amputees
2017, Neural NetworksCitation Excerpt :In this paper, ELM is applied to a time sequence case. EMG signal will be segmented to a window that can be overlapped each other using sliding window technique or not (Englehart & Hudgins, 2003; Liu, Yu, Wang, & Sun, 2016). ELM will process the signal from the window and then produce the output.
Effect of variation in gesticulation pattern in dynamic hand gesture recognition system
2016, NeurocomputingCitation Excerpt :After the training phase, the testing was performed using 5-fold cross validation process. Generally, the feedforward neural networks are ideal classifiers for nonlinear mappings that utilize gradient descent approach for weights and bias optimization [32–34]. The important factors that influence the performance of a traditional neural learning algorithm include:
ASELM: Adaptive semi-supervised ELM with application in question subjectivity identification
2016, NeurocomputingCitation Excerpt :Now the problem transforms to train and test the classification model. Machine learning methods have been widely used in classification problems in recent years and get well performances [18–20]. However, the problems of data imbalance and the lack of labelled data mentioned above make it difficult to train a good classifier that suitable for the real CQA problem.
Self co-articulation removal and hybrid classifier-feature combination for dynamic hand gesture recognition
2023, Multimedia Tools and ApplicationsDynamic hand gesture tracking and recognition: Survey of different phases
2023, International Journal of Systematic Innovation
Huaping Liu received the Ph.D. degree from the Department of Computer Science and Technology, Tsinghua University, Beijing, China, in 2004. He is currently an Associate Professor in the Department of Computer Science and Technology at Tsinghua University. His research interests include intelligent control and robotics.
Lianzhi Yu received the Ph.D. degree from Shanghai Jiao Tong University. She is currently an Associate Professor. Her research interests include micro-robot control and pattern recognition.
Wen Wang received Bachelor degree from the University of South China, in 2012, and now is a master candidate student. His research interests include computer vision and activity recognition.
Fuchun Sun received the Ph.D. degree from the Department of Computer Science and Technology, Tsinghua University, Beijing, China, in 1998. Now he is a full Professor in this department. He serves as an Associated Editor for IEEE Transactions on Fuzzy Systems and Mechatronics, and a member of the Editorial Board of the International Journal of Robotics and Autonomous Systems, International Journal of Control, Automation, and Systems, Science in China Series F: Information Science and Acta Automatica Sinica. His research interests include intelligent control, neural networks, fuzzy systems, and robot teleoperation.