Sparse functional principal component analysis in a new regression framework

Abstract

The functional principal component analysis is widely used to explore major sources of variation in a sample of random curves. These major sources of variation are represented by functional principal components (FPCs). The FPCs from the conventional FPCA method are often nonzero in the whole domain, and are hard to interpret in practice. The main focus is to estimate functional principal components (FPCs), which are only nonzero in subregions and are referred to as sparse FPCs. These sparse FPCs not only represent the major variation sources but also can be used to identify the subregions where those major variations exist. The current methods obtain sparse FPCs by adding a penalty term on the length of nonzero regions of FPCs in the conventional eigendecomposition framework. However, these methods become an NP-hard optimization problem. To overcome this issue, a novel regression framework is proposed to estimate FPCs and the corresponding optimization is not NP-hard. The FPCs estimated using the proposed sparse FPCA method is shown to be equivalent to the FPCs using the conventional FPCA method when the sparsity parameter is zero. Simulation studies illustrate that the proposed sparse FPCA method can provide more accurate estimates for FPCs than other available methods when those FPCs are only nonzero in subregions. The proposed method is demonstrated by exploring the major variations among the acceleration rate curves of 107 diesel trucks, where the nonzero regions of the estimated sparse FPCs are found well separated.

Introduction

Functional principal component analysis (FPCA) is a crucial dimension reduction tool in functional data analysis. FPCA explores major sources of variability in a sample of random curves by finding functional principal components (FPCs) that maximize the curve variation. Consequently, the top few FPCs explain most of the variability in the random curves. Besides, each random curve can be approximated by a linear combination of the top few FPCs. Therefore, the infinite-dimensional curves are projected to a low-dimensional space defined by the top FPCs. This powerful dimensional reduction feature also promotes the popularity of FPCA.

The theoretical properties of FPCA have been carefully studied at length. For example, Dauxois et al. (1982) first studied the asymptotic properties of PCA estimators for the infinite-dimensional data from a linear operator viewpoint. Following this point of view, Mas (2002) and Bosq (2000) utilized functional analysis to study FPCA theoretically. On the other hand, Hall and Horowitz, 2007, Hall et al., 2006 and Yao et al. (2005a) studied FPCA from the kernel perspective. Sang et al. (2017) proposed a parametric approach for estimating FPCs to enhance their interpretability for users. Nie et al. (2018) propose a supervised version of FPCA by considering the correlation of the functional predictor and response variable. In addition, FPCA has been widely and successfully applied in many applications such as functional linear regression (Yao et al., 2005b), classification and clustering of functional data (Ramsay and Silverman, 2005, Yao et al., 2005b, Müller, 2005, Müller and Stadtmüller, 2005, Peng and Müller, 2008, Dong et al., 2018). All these applications assume the functional data are densely and regularly observed. When it comes to sparse and irregularly observed data, (Yao et al., 2005a) proposed to estimate the FPC score using conditional expectation, which allows recovering the individual trajectory by borrowing information across all the subjects. The smooth version of functional principal component analysis is carefully studied by Rice and Silverman, 1991, Pezzulli, 1993, Silverman, 1996, and Yao et al. (2005a). There are mainly three methods to achieve smoothness. The first method smooths the functional data in the first step and conducts the regular FPCA on the sample covariance function. The second method smooths the covariance function first and then eigendecomposes the resulting smoothed covariance function to estimate the smoothed FPCs. The last method directly adds a roughness penalty in the optimization criterion for estimating the FPCs.

The conventional FPCA aims to estimate FPCs that maximize the curve variation. These FPCs represent the source or direction of maximum variations among curves, and the curves are projected to the low-dimensional space defined by these FPCs. Therefore, it is essential to interpret them. However, these FPCs are usually nonzero in the whole observed domain, and users often find it hard to interpret these FPCs. On the other hand, if the estimated FPC is only nonzero in a subregion of the entire domain, we can easily use them to identify the subregions from which the major variation of the curves exhibits. In this paper, our goal is to propose a method to estimate the sparse functional principal components, which are only nonzero in a subregion and, at the same time, account for an almost maximum amount of variation within the curves.

Several methods have been proposed to enhance the interpretability of functional principal components. The first method is the interpretable functional principal components analysis (iFPCA) proposed by Lin et al. (2016). This method adds an ${\ell }_0$-penalty on the length of the nonzero region of FPCs and obtains FPCs, which are only nonzero in subregions. However, the optimization in their framework is an NP-hard problem because of the use of the ${\ell }_0$-penalty. A greedy backward elimination algorithm is proposed to solve this optimization problem approximately. The second method is called a localized functional principal components analysis (LFPCA) method proposed by Chen and Lei (2015). This method adds an ${\ell }_1$ penalty to the original eigendecomposition problem of smoothed FPCs, which is also not a convex optimization problem. They approximate this non-convex problem through a Deflated Fantope Localization method and propose a novel estimation procedure in a sequential manner. In addition, Di et al. (2014) considered the functional principal component analysis on sparsely sampled multilevel functional data. The sparsity in their work refers to the situations when the functional data are not fully observed rather than the sparsity of the FPCs. Li et al. (2016) studied the problem when the low-rank structure of the functional data was related to multivariate supervision data. The resulting supervised FPCs incorporate the information carried within the response data. In comparison, our work needs no supervision information and assumes the underlying FPCs are sparse on their own.

This paper has three major contributions. Firstly, we propose a new regression-type framework for the sparse functional principal component analysis. The estimated sparse FPCs can not only account for a reasonable variation within the functional data but also be sparse on the whole domain. We also show that the FPCs estimated with our proposed sparse FPCA method is equivalent to the FPCs with the conventional FPCA method when the sparsity parameter is zero. Secondly, our approach is not an NP-hard optimization problem, and the computation is very efficient. Lastly, our method estimates the top sparse FPCs simultaneously rather than sequentially estimating each FPC. Sequentially estimating the FPCs often leads to a quadratic optimization problem with multiple linear constraints. The numerical complexity increases as the rank of FPCs increases. Besides, the sequential manner does not allow parallel computing because the $K$th FPC can only be estimated after obtaining the first $K-1$ FPCs. In our regression framework, the regression step of our algorithm only involves individual FPC such that it can be solved in a parallel way. An R package “sparseFPCA” is developed to implement our proposed sparse FPC (SFPCA) method. The computing scripts for our simulation study can be downloaded at https://github.com/caojiguo/sparseFPCA.

The rest of the paper is organized as follows. In Section 2, we introduce our SFPCA method and show its connection with the conventional FPCA. Details of our approach and the computation algorithm are described in Section 3. In Section 4, we apply our proposed method in a real-data application to explore major sources of variation among the acceleration rates of 107 diesel trucks. In Section 5, two carefully-designed simulations are conducted to evaluate the finite sample performance of our proposed method in comparison with other alternative methods in different settings. Section 6 provides concluding remarks.