Abstract
We propose adaptive constraint propagation (ACP) for semi-supervised kernel matrix learning (SS-KML). SS-KML aims to learn a kernel matrix from the given samples which contains just a little supervised information such as class labels or pairwise constraints. Recently, for effective SS-KML, constraint propagation methods by semi-definite programming are being actively studied, and the representative works are pairwise constraint propagation (PCP) and kernel propagation (KP). They have used hard constraints in their frameworks for constraint propagation and achieved outstanding classification performance. However, a small set of the hard constraints sometimes cannot cover all the samples of the full data set, and thus lead to large distortions of global discriminative data structure in the learned kernel matrix. It has a negative influence on the classification performance. To deal with this problem, we provide two adaptive fidelity terms to satisfy the requirement that two must-link samples become close and two cannot-link samples become far apart. Then, we build a new framework based on them to adaptively propagate the constraints. Experimental results demonstrate that ACP outperforms state-of-the-art SS-KML methods such as PCP and KP in terms of both effectiveness and efficiency.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aiolli F, Martino GDS, Hagenbuchner M, Sperduti A (2009) Learning nonsparse kernels by self-organizing maps for structured data. IEEE Trans Neural Netw 20(12):1938–1949
Cozman FG, Cohen I, Cirelo MC et al (2007) Semi-supervised learning of mixture models. In: Proceedings of international conference on machine learning, pp 361–368
Fowlkes C, Belongie S, Malik J (2004) Spectral grouping using the Nystrom method. IEEE Trans Pattern Anal Mach Intell 26(2):214–225
Fowlkes C, Belongie S, Malik J (2001) Efficient spatiotemporal grouping using the nystrom method. In: Proceedings of IEEE international conference on computer vision and pattern recognition, pp 231–238
Gilorami M (2002) Mercer kernel-based clustering in feature space. IEEE Trans Neural Netw 13(3):780–784
Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update. ACM SIGKDD Explor Newsl 11(1):10–18
Hoi SCH, Jin R (2008) Active kernel learning. In: Proceedings of international conference on machine learning, pp 400–407
Hoi SCH, Jin R, Lyu MR (2007) Learning nonparametric kernel matrices from pairwise constraints. In: Proceedings of international conference on machine learning, pp 361–368
Hu E, Chen S, Zhang D, Yin X (2010) Semisupervised kernel matrix learning by kernel propagation. IEEE Trans Neural Netw 21(11):1831–1841
Huiskes MJ, Lew MS (2008), The MIR flickr retrieval evaluation. In: Proceedings of ACM international conference on multimedia, information retrieval, pp 39–43
Kulis B, Basu S, Dhillon I, Mooney R (2005) Semi-supervised graph clustering: a kernel approach. In: Proceedings of international conference on machine learning, pp 457–464
Lazebnik S, Schmid C (2006) Ponce J. Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of IEEE international conference on computer vision and pattern recognition 2:2169–2178
Likas A, Vlassis N, Verbeek JJ (2003) The global k-means clustering algorithm. Pattern Recognit 36(2):451–461
Li Z, Liu J, Tang X (2008) Pairwise constraint propagation by semidefinite programming for semi-supervised classification. In: Proceedings of international conference on machine learning, pp 576–583
Liu G, Lin Z, Yu Y (2010) Robust subspace segmentation by low-rank representation. In: Proceedings of international conference on machine learning, pp 663–670
Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of IEEE international conference on computer vision, pp 416–423
Ning J, Zhang L, Zhang D, Wu C (2010) Interactive image segmentation by maximal similarity based region merging. Pattern Recognit 43(2):445–456
Shang F, Liu Y, Wang F (2011) Learning spectral embedding for semi-supervised clustering. In: Proceedings of IEEE ICDM, pp 597–606
Sindhwani V, Keerthi SS (2009) Newton methods for fast solution of semi-supervised linear SVMs. large scale kernel machines, Princeton, pp 155–174
Sturm JF, Romanko O, Polik I (2006) SeDuMi version 1.1R3. http://sedumi.mcmaster.ca/
Tao W, Jin H, Zhang Y (2007) Color image segmentation based on mean shift and normalized cuts. IEEE Trans Syst Man Cybern Part B 37(5):1382–1389
Vapnik VN (1998) Statistical learning theory. Wiley, New York
Verbeek J, Guillaumin M, Mensink T, Schmid C (2010) Image annotation with tagprop on the MIRFLICKR set. In: Proceedings of ACM international conference on multimedia, information retrieval, pp 537–546
Wagsaff K, Basu S, Davidson I (2006) When is constrained clustering beneficial, and why? In: Proceedings of american association for artificial intelligence
Wagsaff K, Cardie C, Rogers S, Schroedl S (2001) Constrained K-means clustering with background knowledge. In: Proceedings of international conference on machine learning, pp 577–584
Wang F, Zhang C (2006) Label propagation through linear neighborhoods. In: Proceedings of international conference on machine learning, pp 985–992
Yan B, Domeniconi C (2006) An adaptive kernel method for semi-supervised clustering. In: Proceedings of european conference on machine learning, pp 521–532
Zelnik-Manor L, Perona P (2004) Self-tuning spectral clustering. In: Proceedings of advances in neural information processing
Zhang D, Chen S, Zhou ZH, Yang Q (2008) Constraint projections for ensemble learning. In: Proceedings of AAAI, pp 758–763
Zhou D, Bousquet O, Lal TN, Weston J, Schölkopf B (2004) Learning with local and global consistency. In: Proceedings of, advances in neural information processing systems, pp 321–328
Acknowledgments
The authors would like to thank the anonymous reviewers for their valuable comments that have led to improvements in the quality and presentation of the paper. This work was supported by the National Basic Research Program (973 Program) of China (No. 2013CB329402), the National Natural Science Foundation of China (Nos. 61271298 and 61050110144), the Fund for Foreign Scholars in University Research and Teaching Programs (the 111 Project) (No. B07048), and the Program for Cheung Kong Scholars and Innovative Research Team in University (No. IRT1170).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jian, M., Jung, C., Shen, Y. et al. Adaptive Constraint Propagation for Semi-Supervised Kernel Matrix Learning. Neural Process Lett 41, 107–123 (2015). https://doi.org/10.1007/s11063-013-9337-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-013-9337-2