Elsevier

Pattern Recognition

Volume 134, February 2023, 109100
Pattern Recognition

Simultaneous Robust Matching Pursuit for Multi-view Learning

https://doi.org/10.1016/j.patcog.2022.109100Get rights and content

Highlights

  • We propose a simultaneous robust matching pursuit (SRMP) method.

  • We devise an efficient half-quadratic optimization algorithm for SRMP.

  • We put forward a SRMP based multi-view subspace clustering approach.

  • A SRMP based classifier is developed for multi-view pattern classification.

Abstract

Joint sparse representation (JSR) has attracted massive attention with many successful applications in pattern recognition recently. In this paper, we propose a novel robust multi-view JSR method referred to as Simultaneous Robust Matching Pursuit (SRMP) based on the outlier-resistant M-estimator originating from robust statistics. Because of the complexity of the objective function, we design an efficient optimization algorithm to implement SRMP based on the half-quadratic theory. In addition, we have also extended the proposed method for the problems of multi-view subspace clustering and multi-view pattern classification, respectively. The experimental results corroborate the efficacy and robustness of SRMP for multi-view data recovery, subspace clustering and classification.

Introduction

Joint sparse representation (JSR) has drawn intensive interest with successful applications ranging from feature selection [1], [2], to subspace clustering [3], [4], visual tracking [5], and multimodal biometric recognition [6]. Depending on the field, this problem is also referred to as multiple measurement vector (MMV) [7] and simultaneous sparse approximation (SSA) [8]. The goal of JSR is to jointly recover multiple sparse signals with the same sparsity pattern given multiple measurement vectors and the dictionary. Compared with the standard sparse representation with single measurement vector (SMV), JSR can take advantage of the common knowledge of multiple sparse signals to further enhance the recovery performance.

A variety of JSR approaches have been proposed in the field of pattern recognition in recent years. For instance, Kang et al. [5] proposed a JSR based visual tracking method by exploring the group similarity in the multi-feature space. To exploit the correlation among distinct modalities, Shekhar et al. [6] put forward a JSR based method termed SMBR (Sparse Multimodal Biometrics Recognition) for multimodal biometrics recognition. Peng and Du [9] advanced a correntropy based JSR method termed as RJSRP for hyperspectral image classification. To harness the nonlocal self-similarity of images, Zha et al. [10] developed a joint patch-group based sparse representation approach for image restoration. Recently, Wang et al. [11] proposed the modal regression based JSR method with application to multi-view face recognition.

Besides in pattern recognition, various JSR methods [6] have also been developed in many other fields such as signal processing. Tropp et al. [8] extended the OMP (Orthogonal Matching Pursuit) from SMV to MMV and proposed the SOMP (Simultaneous OMP) algorithm. Blanchard et al. [12] generalized other three greedy algorithms to MMV, which are originally designed in the SMV setting. They are CoSaMP (Compressive Sampling Matching Pursuit) [13], IHT (Iterative Hard Thresholding) [14], and HTP (Hard Thresholding Pursuit) [15], respectively. Their extensions to MMV are referred to as SCoSaMP, SIHT, and SHTP [12], respectively.

Due to the simplicity and analytical tractability, the squared Frobenius norm has been widely employed to measure the reconstruction loss in the objective of JSR. The resulting loss function is referred to as mean square error (MSE) criterion, which is known to be sensitive to outlier and non-Gaussian noises [16]. To alleviate such limitation, we develop a robust multi-view JSR approach. The contributions of this paper are summarized as follows:

  • 1.

    We propose a SRMP (Simultaneous Robust Matching Pursuit) method for robust multi-view JSR in the presence of outliers and heavy noise. Unlike previous JSR methods employing MSE and being sensitive to outliers, SRMP is grounded in the M-estimator from robust statistics and exhibits stronger robustness against outliers and gross corruption.

  • 2.

    Another contribution is to propose an efficient half-quadratic iterative algorithm for implementing SRMP. We have also proved the convergence of the sequence of the objective function values induced by the iterative algorithm.

  • 3.

    Based on SRMP, we develop a robust multi-view subspace clustering method for unsupervised learning. It generalizes single-view sparse subspace clustering and aims to cluster multi-view data. We have also leveraged SRMP to advance a robust multi-view classification method, which leverages the correlation information among multiple views to further enhance the classification performance.

The remainder of the paper is arranged as follows. Section 2 briefly introduces several important related works. In Section 3, we depict the proposed approach as well as the optimization algorithm. Section 4 devotes to developing a robust multi-view subspace clustering method while Section 5 proposes a robust multi-view pattern classification approach. Section 6 presents the experiments for multi-view data recovery, clustering and classification. Finally, Section 7 concludes the paper.

To enhance the readability of the paper, we first explain the key notations used in this work. Scalars are written in italic letters (e.g., x), vectors are expressed using boldface lowercase letters (e.g., x), and matrices are written in boldface capital letters (e.g., X). X(i,:), X(:,j) and Xij denote the ith row, jth column and (i,j) entry of the matrix X, respectively. Xrow,0 and rowsupp(X) denote the number and the index set of nonzero rows of X, respectively. The Frobenius norm and the 1,2 of X are defined as XF=i,jXij2 and X1,2=ijXij2, respectively. Im×m and 0m×n denote the identity and zero matrix, respectively. We omit the subscripts for simplicity if the dimensions are clear from the context. For a vector vRd, diag(v) is a diagonal matrix with the coordinates of v on the main diagonal. Table 1 summarizes the key notations and acronyms used in this work.

Section snippets

Related work

Let DRd×m(d<m) and C0Rm×n be the dictionary matrix and the row sparse coefficient matrix, respectively. The measurement matrix SRd×n is formulated asS=DC0+N,where NRd×n denotes the noise matrix. The JSR problem (also called the MMV problem [12]) is to recover the unknown row sparse signal C0 given the input matrix S and the dictionary D. Most existing greedy algorithms for JSR adopt the following row-sparsity constrained model [8], [12]minCRm×nSDCF2s.t.Crow,0T,where Crow,0 is

Proposed method

For multi-view signals, the data in different views may have distinct dimensions. Let dv be the dimension of the measurement data in the vth view for v=1,,V, where V is the number of views. Let DvRdv×m and c0vRm×1 be the dictionary matrix and the sparse coefficient vector in the vth view, respectively. The vth view measurement vector svRdv×1 is formulated assv=Dvc0v+nv,v=1,,V,where nvRdv denotes the noise vector in the vth view. For simplicity, let C0=[c01,,c0V]. The locations of nonzero

SRMP for multi-view subspace clustering

In this section, we develop a SRMP based multi-view subspace clustering (SRMP-MSC) approach. Consider a collection of N multi-view data samples, where the ith data sample is represented by {yiv}v=1V and yiv denotes the vth view of the ith sample. Let Yv=[y1v,,yNV]Rdv×N be the matrix containing all data samples in the vth view, which are drawn from a union of subspaces S1v,,SKv. Here K denotes the number of subspaces in each view.

Assume that data samples in different views admit the same

SRMP for multi-view pattern classification

This section aims to devise a SRMP based multi-view pattern classification approach. Before introducing the proposed algorithm, we first formulate the problem of multi-view pattern classification (MPC). Let Av=[a1v,,aNv]Rdv×M be the matrix containing training samples from c classes in the vth view where v{1,,V} and M denotes the number of training samples. The class label of jth multi-view sample {ajv}v=1V belongs to {1,,c}. Given a new test sample {yv}v=1V where yvRdv, the goal of MPC is

Multi-view joint sparse recovery

This part devotes to evaluating the effectiveness of SRMP for multi-view joint sparse signal recovery. For comparison, we consider other five competing JSR algorithms: SOMP [8], SGOMP [26], SHTP [12], RJSRP [9] and MLEJSR [27]. Since the five algorithms above are originally developed for single-view data, we also extend them for multi-view data using the analogous strategy as SRMP in Algorithm 1.

Conclusion

In this section, we will discuss the strengths and weakness of this work and some possible future works along this line of research.

Strengths Compared with previous joint sparse representation (JSR) methods, a key advantage of the proposed SRMP method is that it can exploit the correlation information among multiple views and handle gross data corruption simultaneously. This is achieved by enforcing the representation vectors of different views to share the same sparsity pattern and leveraging

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant nos. 62076041, 11671161 and 62276111, in part by the Research Grants of University of Macau under Grant nos. MYRG2019-00039-FST and MYRG2022-00108-FST, in part by Science and Technology Development Fund, Macao S.A.R under Grant no. FDCT/0036/2021/AGJ, in part by the Fundamental Research Funds for the Central Universities under Grant no. 2662022JC004, and in part by the Research Grants of Huazhong

Yulong Wang received the B.Sc. and M.Sc. degrees from Hubei University,Wuhan, China, and the Ph.D. degree from the University of Macau, Macau, China. He is a Professor with the College of Informatics, Huazhong Agricultural University, Wuhan 430070, China. His current research interests include pattern recognition and computer vision.

References (37)

  • S. Shekhar et al.

    Joint sparse representation for robust multimodal biometrics recognition

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2014)
  • J. Chen et al.

    Theoretical results on sparse representations of multiple-measurement vectors

    IEEE Trans. Signal Process.

    (2006)
  • J. Peng et al.

    Robust joint sparse representation based on maximum correntropy criterion for hyperspectral image classification

    IEEE Trans. Geosci. Remote Sens.

    (2017)
  • Z. Zha et al.

    Image restoration using joint patch-group-based sparse representation

    IEEE Trans. Image Process.

    (2020)
  • Y. Wang et al.

    Modal regression based atomic representation for robust face recognition and reconstruction

    IEEE Trans. Cybern.

    (2020)
  • J. Blanchard et al.

    Greedy algorithms for joint sparse recovery

    IEEE Trans. Signal Process.

    (2014)
  • S. Foucart

    Hard thresholding pursuit: an algorithm for compressive sensing

    SIAM J. Numer. Anal.

    (2011)
  • P.J. Huber

    Robust Statistics

    (1981)
  • Cited by (4)

    • Anti-Noise Muiti-View Feature Selection With Sample Constraints

      2023, Proceedings - IEEE International Conference on Data Mining, ICDM

    Yulong Wang received the B.Sc. and M.Sc. degrees from Hubei University,Wuhan, China, and the Ph.D. degree from the University of Macau, Macau, China. He is a Professor with the College of Informatics, Huazhong Agricultural University, Wuhan 430070, China. His current research interests include pattern recognition and computer vision.

    Kit Ian Kou received the M.Sc. and Ph.D. degree, both in Mathematics, from University of Macau, Macao, China. She is a Life Member of Clare Hall, University of Cambridge, United Kingdom. She is an Associated Professor of Department of Mathematics, Faculty of Science and Technology, University of Macau. Her current research interests include pattern recognition, Quaternion analysis in image processing, and and Fourier analysis. Prof. Kou received the third prize in the Natural Science Award of 2018 Macao Science and Technology Awards.

    Hong Chen received the B.Sc. and Ph.D. degrees from Hubei University, Wuhan, China, in 2003 and 2009, respectively. He worked as a postdoc researcher at University of Texas at Arlington during 2016.2–2017.8. He is currently a Professor with the College of Science, Huazhong Agricultural University, Wuhan, China. His current research interests include machine learning, statistical learning theory, and approximation theory.

    Yuan Yan Tang is a Chair Professor in Faculty of Science and Technology at University of Macau and Professor/Adjunct Professor/Honorary Professor at several institutes including Chongqing University in China, Concordia University in Canada, and Hong Kong Baptist University in Hong Kong. His current interests include wavelets, pattern recognition, and image processing. He has published more than 400 academic papers and is the author/coauthor of over 25 monographs/books/bookchapters. He is the Founder and Editor-in-Chief of International Journal on Wavelets, Multiresolution, and Information Processing (IJWMIP), and Associate Editors of several international journals. He is the Founder and Chair of pattern recognition committee in IEEE SMC. He has serviced as general chair, program chair, and committee member for many international conferences. Dr. Tang is the Founder and General Chair of the series International Conferences on Wavelets Analysis and Pattern Recognition (ICWAPRs). He is the Founder and Chair of the Macau Branch of International Associate of Pattern Recognition (IAPR). Dr. Y. Y. Tang is the IEEE Fellow and IAPR Fellow.

    Luoqing Li received the B.Sc. degree from Hubei University, Wuhan, China, the M.Sc. degree from Wuhan University, Wuhan, and the Ph.D. degree from Beijing Normal University, Beijing, China. He is a Professor with the Faculty of Mathematics and Statistics, Hubei University, Wuhan, China. Dr. Li is the Managing Editor of the International Journal on Wavelets, Multiresolution, and Information Processing.

    View full text