Xiaodong Feng
Skip Abstract Section
Abstract
Sparse representation has been a powerful technique for modeling high-dimensional data. As an unsupervised technique to extract sparse representations, sparse coding encodes the original data into a new sparse code space and simultaneously learns a dictionary representing high-level semantics. Existing methods have considered local manifold within high-dimensional data using graph/hypergraph Laplacian regularization, and more from the manifold could be utilized to improve the performance. In this article, we propose to further regulate the sparse coding so that the learned sparse codes can well reconstruct the hypergraph structure. In particular, we add a novel hypergraph consistency regularization term (HC) by minimizing the reconstruction error of the hypergraph incidence or weight matrix. Moreover, we extend the HC term to multi-hypergraph consistent sparse coding (MultiCSC) and automatically select the optimal manifold structure under the multi-hypergraph learning framework. We show that the optimization of MultiCSC can be solved efficiently, and that several existing sparse coding methods can fit into the general framework of MultiCSC as special cases. As a case study, hypergraph incidence consistent sparse coding is applied to perform semi-auto image tagging, demonstrating the effectiveness of hypergraph consistency regulation. We perform further experiments using MultiCSC for image clustering, which outperforms a number of baselines.
- M. Aharon, M. Elad, and A. Bruckstein. 2006. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Sign. Process. 54 (2006), 4311--4322. Google Scholar
Digital Library
- Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. 2009. NUS-WIDE: A real-world web image database from national university of singapore. In Proceedings of the ACM International Conference on Image and Video Retrieval. ACM, 48.Google ScholarDigital Library
- W. Dong, L. Zhang, G. Shi, and X. Li. 2013. Nonlocally centralized sparse representation for image restoration. IEEE Trans. Image Process. 22 (2013), 1620--1630. Google ScholarDigital Library
- E. Elhamifar and R. Vidal. 2009. Sparse subspace clustering. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’09). 2790--2797. Google ScholarCross Ref
- Xiaodong Feng, Sen Wu, Wenjun Zhou, and Zhiwei Tang. 2016. Multi-hypergraph incidence consistent sparse coding for image data clustering. In Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 79--91. Google ScholarDigital Library
- S. Gao, I. Tsang, and L. Chia. 2013. Laplacian sparse coding, hypergraph laplacian sparse coding, and applications. IEEE Trans. Patt. Anal. Mach. Intell. 35 (2013), 92--104. Google ScholarDigital Library
- Shenghua Gao, Zhengxiang Wang, Liang-Tien Chia, and Ivor Wai-Hung Tsang. 2010. Automatic image tagging via category label and web data. In Proceedings of the International Conference on Multimedia (MM’10). ACM, 1115--1118. Google ScholarDigital Library
- B. Geng, D. Tao, C. Xu, Y. Yang, and X. Hua. 2012. Ensemble manifold regularization. IEEE Trans. Patt. Anal. Mach. Intell. 34 (2012), 1227--1233. Google ScholarDigital Library
- M. Grant and S. Boyd. 2014. CVX: MATLAB software for disciplined convex programming. version 2.1. (2014). Retrieved from http://www.stanford.edu/∼boyd/cvx/.Google Scholar
- C Hong, J Yu, D Tao, and M Wang. 2015. Image-based 3D human pose recovery by multi-view locality sensitive sparse retrieval. IEEE Trans. Industr. Electron. 62, 6 (2015), 3742--3751.Google ScholarCross Ref
- Y. Huang, Q. Liu, S. Zhang, and D. N. Metaxas. 2010. Image retrieval via probabilistic hypergraph ranking. In Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’10). 3376--3383. Google ScholarCross Ref
- T. Jin, J. Yu, J. You, K. Zeng, C. Li, and Z. Yu. 2015. Low-rank matrix factorization with multiple hypergraph regularizer. Pattern Recogn. 48 (2015), 1011--1022. Google ScholarDigital Library
- A. Krause and V. Cevher. 2010. Submodular dictionary selection for sparse representation. In Proceedings of the 27th International Conference on Machine Learning (ICML’10). 567--574.Google Scholar
- Daniel D. Lee and Seung H. Sebastian. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 6755 (1999), 788--791. Google ScholarCross Ref
- Daniel D. Lee and Seung H. Sebastian. 2001. Algorithms for nonnegative matrix factorization. Adv. Neural Info. Process. Syst. 13, 6 (2001), 556--562.Google Scholar
- H. Lee, A. Battle, R. Raina, and A. Y. Ng. 2006. Efficient sparse coding algorithms. Adv. Neural Info. Process. Syst. (NIPS’16) (2006), 801--808.Google Scholar
- S. Li, H. Yin, and L. Fang. 2012. Group-sparse representation with dictionary learning for medical image denoising and fusion. IEEE Trans. Biomed. Eng. 59 (2012), 3450--3459. Google ScholarCross Ref
- Fan Liu, Jinhui Tang, Yan Song, Liyan Zhang, and Zhenmin Tang. 2015. Local structure-based sparse representation for face recognition. ACM Trans. Intell. Syst. Technol. 7, 1 (2015), 1--20. Google ScholarDigital Library
- X. Liu, M. Song, D. Tao, X. Zhou, C. Chen, and J. Bu. 2014. Semi-supervised coupled dictionary learning for person re-identification. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). 3550--3557. Google ScholarDigital Library
- D. Zhang M. Yang and X. Feng. 2011. Fisher discrimination dictionary learning for sparse representation. In Proceedings of the 2011 IEEE International Conference on Computer Vision (ICCV’11). 543--550.Google Scholar
- J. Mairal, F. Bach, J. Ponce, and G. Sapiro. 2009. Online dictionary learning for sparse coding. In Proceedings of the 26th Annual International Conference on Machine Learning (ICML’09). 689--696. Google ScholarDigital Library
- J. Mairal, M. Leordeanu, F. Bach, M. Hebert, and J. Ponce. 2008. Discriminative sparse image models for class-specific edge detection and image interpretation. In Proceedings of the 10th European Conference on Computer Vision (ECCV’08). 43--56. Google ScholarDigital Library
- Ameesh Makadia, Vladimir Pavlovic, and Sanjiv Kumar. 2008. A new baseline for image annotation. In Proceedings of the 10th European Conference on Computer Vision (ECCV’08). Springer, 316--329. Google ScholarDigital Library
- Tood K. Moon. 1996. The expectation-maximization algorithm. IEEE Sign. Process. Mag. 13, 6 (1996), 47--60. Google ScholarCross Ref
- Bruno A. Olshausen and others. 1996. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 6583 (1996), 607--609. Google ScholarCross Ref
- Bruno A. Olshausen and David J. Field. 1997. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Res. 37, 23 (1997), 3311--3325. Google ScholarCross Ref
- I. Ramirez, P. Sprechmann, and G. Sapiro. 2010. Classification and clustering via dictionary learning with structured incoherence and shared features. In Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’10). 3501--3508. Google ScholarCross Ref
- Gang Wang, Derek Hoiem, and David Forsyth. 2009. Building text features for object image classification. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’09). IEEE, 1367--1374.Google ScholarCross Ref
- J. J. Wang, H. Bensmail, and X. Gao. 2013a. Multiple graph regularized nonnegative matrix factorization. Pattern Recogn. 46 (2013), 2840--2847. Google ScholarDigital Library
- Jing Yan Wang, Halima Bensmail, and Xin. Gao. 2013b. Feature selection and multi-kernel learning for sparse representation on a manifold. Neural Networks 51C, 3 (2013), 9--16.Google Scholar
- Jing Yan Wang, Halima Bensmail, Nan Yao, and Xin Gao. 2013c. Discriminative sparse coding on multi-manifolds. Knowledge-Based Systems 54, 4 (2013), 199--206.Google ScholarDigital Library
- Jing Yan Wang and Xin Gao. 2013. Semi-supervised sparse coding. In Proceedings of the International Joint Conference on Neural Networks. 1630--1637.Google Scholar
- Jing Yan Wang, Yijun Sun, and Xin Gao. 2015b. Sparse structure regularized ranking. Multimed. Tools Appl. 74, 2 (2015), 1--20.Google ScholarDigital Library
- Meng Wang, Xueliang Liu, and Xindong Wu. 2015a. Visual classification by l 1 -hypergraph modeling. IEEE Trans. Knowl. Data Eng. 9 (2015), 2564--2574. Google ScholarDigital Library
- J. Wright, Y. Ma, J. Mairal, G. Sapiro, T. S. Huang, and S. Yan. 2010. Sparse representation for computer vision and pattern recognition. Proc. IEEE 98, 6 (2010), 1031--1044. Google ScholarCross Ref
- J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma. 2009. Robust face recognition via sparse representation. IEEE Trans. Patt. Anal. Mach. Intell. 31 (2009), 210--227. Google ScholarDigital Library
- S. Wu, X. Feng, and W. Zhou. 2014. Spectral clustering of high-dimensional data exploiting sparse representation vectors. Neurocomputing 135 (2014), 229--239. Google ScholarDigital Library
- J. Yu, Y. Rui, Y. Y. Tang, and D. Tao. 2014. High-order distance-based multiview stochastic learning in image classification. IEEE Transactions on Cybernetics 44, 12 (2014), 2431--2442. Google ScholarCross Ref
- J. Yu, X. Yang, F. Gao, and D. Tao. 2016. Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans. Cybernet. (2016), 1--11.Google Scholar
- G. Zhang Z. Jiang and L. S. Davis. 2012. Submodular dictionary learning for sparse coding. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). 3418--3425.Google Scholar
- Z. Lin Z. Jiang and L. S. Davis. 2013. Label consistent k-svd: Learning a discriminative dictionary for recognition. IEEE Trans. Patt. Anal. Mach. Intell. 35 (2013), 2651--2664. Google ScholarDigital Library
- G. Zhang, Z. Jiang, and L. S. Davis. 2013. Online semi-supervised discriminative dictionary learning for sparse representation. In Proceedings of the 11th Asian Conference on Computer Vision (ACCV’12). 259--273. Google ScholarDigital Library
- Q. Zhang and B. Li. 2010. Discriminative K-SVD for dictionary learning in face recognition. In Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’10). 2691--2698. Google ScholarCross Ref
- X. Zhang, L. Zhao, L. Zong, X. Liu, and H. Yu. 2014. Multi-view clustering via multi-manifold regularized nonnegative matrix factorization. In Proceedings of the 2014 IEEE International Conference on Data Mining (ICDM’14). 1103--1108. Google ScholarDigital Library
- Haixia Zheng and Horace H. S. Ip. 2015. Iterative semi-supervised sparse coding model for image classification. J. Sign. Process. Syst. 81, 1 (2015), 1--12. Google ScholarDigital Library
- H. Zheng and D. Tao. 2015. Discriminative dictionary learning via fisher discrimination K-SVD algorithm. Neurocomputing 162 (2015), 9--15. Google ScholarDigital Library
- M. Zheng, J. Bu, C. Chen, C. Wang, L. Zhang, G. Qiu, and D. Cai. 2011. Graph regularized sparse coding for image representation. IEEE Trans. Image Process. 20 (2011), 1327--1336. Google ScholarDigital Library
Index Terms
- Multi-Hypergraph Consistent Sparse Coding
Recommendations
Multi-hypergraph Incidence Consistent Sparse Coding for Image Data Clustering
PAKDD 2016: Proceedings, Part II, of the 20th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining - Volume 9652Sparse representation has been a powerful technique for modeling image data and thus enhance the performance of image clustering. Sparse coding, as an unsupervised way to extract sparse representation, learns a dictionary that represents high-level ...
Multiple graph regularized sparse coding and multiple hypergraph regularized sparse coding for image representation
Manifold regularized sparse coding shows promising performance for various applications. The key issue that must be considered in the application is how to adaptively select the suitable graph hyper-parameters in manifold learning for the sparse coding ...
Noise-robust semi-supervised learning via fast sparse coding
This paper presents a novel noise-robust graph-based semi-supervised learning algorithm to deal with the challenging problem of semi-supervised learning with noisy initial labels. Inspired by the successful use of sparse coding for noise reduction, we ...
Comments