Classification of multivariate time series using locality preserving projections

Abstract

Multivariate time series (MTS) are used in very broad areas such as multimedia, medicine, finance and speech recognition. A new approach for MTS classification using locality preserving projections (LPP) is proposed. By using LPP, the MTS samples can be projected into a lower-dimensional space in which the MTS samples related to the same class are close to each other, the MTS samples in testing set can be identified by one-nearest-neighbor classifier in the lower-dimensional space. Experimental results performed on five real-world datasets demonstrate the effectiveness of our proposed approach for MTS classification.

Introduction

Time series classification is an important problem in time series data mining. Many algorithms have been proposed for univariate time series classification [1], [2], [3], [4], [5], however, few papers are found about Multivariate time series (MTS) classification in the literature. MTS are used in very broad areas such as multimedia, medicine, finance and speech recognition. MTS classification is difficult for traditional machine learning algorithms mainly because of dozens of variables and different lengths of MTS samples.

MTS is a series of observations, x_i(t)[i = 1, 2, … , n; t = 1, 2, … , m], where m is the number of observations and n is the number of variables (n is greater than, or equal to 2) [21]. MTS sample is stored by using m × n matrix. Classifying MTS samples poses two challenges [22]:

• MTS sample has dozens of variables. There are usually important correlations among the variables. If an MTS sample is broken into multiple univariate time series and each processed separately, the correlations among the variables in MTS sample will be lost.

• MTS samples can be of different lengths, even for similar MTS samples. For instance, the length of MTS sample in Japanese Vowels dataset [16] is between 7 and 29, the average length of MTS sample in Australian Sign Language dataset [16] is around 60.

Li et al. [6], [22] proposed two different feature vector selection approaches from MTS sample using the singular vector decomposition (SVD). The first approach takes into account the first singular vector and the normalized singular values, and the second approach considers the first two dominating singular vectors weighted by their associated singular values. The correlations among the variables can be extracted by Li’s two approaches. However, the class labels are not taken into consideration when feature vectors are extracted from MTS samples, and the dimensionality of these feature vectors is relatively high.

In this paper, a new approach to classifying MTS using locality preserving projections (LPP) [8], [9] is proposed. LPP is linear projective map that optimally preserves the neighborhood structure of the dataset. Our approach first extracts feature vectors from MTS samples using Li’s two different approaches [6], [22]; then by using LPP, these feature vectors are projected into a lower-dimensional space in which the MTS samples related to the same class are close to each other; finally, the MTS samples in testing set can be identified by one-nearest-neighbor (1NN) classifier in the lower-dimensional space. To the best of our knowledge, LPP has not been previously investigated for MTS classification.

In comparison with our proposed approach (Li’s two different feature vector selection approaches + LPP + 1NN), we also adopt two-dimensional singular value decomposition (2dSVD) [15] for feature extraction. A feature matrix is obtained for each MTS sample by using 2dSVD.

The contributions of this work include:

• Projecting feature vectors extracted from MTS samples into a lower-dimensional space by using LPP, in which the MTS samples related to the same class are close to each other.

• Applying one-nearest-neighbor classifier to the feature vectors of the MTS samples in the lower-dimensional space and classifying the MTS samples.

• Using 2dSVD to extract feature matrix from each MTS sample and comparing the 2dSVD + 1NN with our proposed approach.

Several experiments are performed on five real-world datasets. The results demonstrate the effectiveness of our approach in terms of accuracy and CPU time. Our approach significantly outperform Li’s two approaches + 1NN, outperform or perform comparably to 2dSVD + 1NN in terms of classification error rate and CPU time on five real-world datasets. The advantage of our approach is that our approach can be used with MTS samples of different lengths. However, 2dSVD assumes that the MTS samples have the equal length, which is very difficult to satisfy in real-world dataset.

The remainder of this paper is organized as follows. Section 2 gives a brief review of related work and background. Section 3 proposes a new approach for MTS classification using LPP. Section 4 experimentally demonstrates the effectiveness of our proposed approach. Conclusion is presented in Section 5.