ABSTRACT
Multiple Kernel K-means (MKKM) uses various kernels from different sources to improve clustering performance. However, most of the existing models are non-convex, which is prone to be stuck into bad local optimum, especially with noise and outliers. To address the issue, we propose a novel Self-Paced and Discrete Multiple Kernel K-Means (SPD-MKKM). It learns the MKKM model in a meaningful order by progressing both samples and kernels from easy to complex, which is beneficial to avoid bad local optimum. In addition, whereas existing methods optimize in two stages: learning the relaxation matrix and then finding the discrete one by extra discretization, our work can directly gain the discrete cluster indicator matrix without extra process. What's more, a well-designed alternative optimization is employed to reduce the overall computational complexity via using the coordinate descent technique. Finally, thorough experiments performed on real-world datasets illustrated the excellence and efficacy of our method.
- Liang Du, Peng Zhou, Lei Shi, Hanmo Wang, Mingyu Fan, Wenjian Wang, and Yi-Dong Shen. 2015. Robust multiple kernel k-means using $ell_2,1$-norm. In Proc. IJCAI. 3476--3482.Google Scholar
- Mark Girolami. 2002. Mercer kernel-based clustering in feature space. IEEE Trans. Neural Netw. Learn. Syst., Vol. 13, 3 (2002), 780--784.Google ScholarDigital Library
- Mehmet Gönen and Ethem Alpaydin. 2011. Multiple kernel learning algorithms. J. Mach. Learn. Res., Vol. 12 (2011), 2211--2268.Google ScholarDigital Library
- Mehmet Gönen and Adam A Margolin. 2014. Localized data fusion for kernel k-means clustering with application to cancer biology. In Proc. NeurIPS. 1305--1313.Google Scholar
- J. A. Hartigan and M. A. Wong. 1979. Algorithm AS 136: A k-means clustering algorithm. J. Roy. Stat. Soc., Vol. 28, 1 (1979), 100--108.Google Scholar
- Zhao Kang, Xiao Lu, Jinfeng Yi, and Zenglin Xu. 2018. Self-weighted multiple kernel learning for graph-based clustering and semi-supervised classification. In Proc. IJCAI. 2312--2318.Google ScholarCross Ref
- M. Pawan Kumar, Benjamin Packer, and Daphne Koller. 2010. Self-Paced Learning for Latent Variable Models. In Proc. NeurIPS. 1189--1197.Google Scholar
- M. Pawan Kumar, Haithem Turki, Dan Preston, and Daphne Koller. 2011. Learning specific-class segmentation from diverse data. In Proc. ICCV. 1800--1807.Google ScholarDigital Library
- Jiyuan Liu, Xinwang Liu, Jian Xiong, Qing Liao, Sihang Zhou, Siwei Wang, and Yuexiang Yang. 2022. Optimal Neighborhood Multiple Kernel Clustering With Adaptive Local Kernels. IEEE Trans. Knowl. Data Eng., Vol. 34, 6 (2022), 2872--2885.Google Scholar
- Xinwang Liu, Yong Dou, Jianping Yin, Lei Wang, and En Zhu. 2016. Multiple kernel k-means clustering with matrix-induced regularization. In Proc. AAAI. 1888--1894.Google ScholarCross Ref
- X. Liu, M. Li, C. Tang, J. Xia, J. Xiong, L. Liu, M. Kloft, and E. Zhu. 2020. Efficient and Effective Regularized Incomplete Multi-view Clustering. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 43, 8 (2020), 2634 -- 2646.Google Scholar
- Jitao Lu, Yihang Lu, Rong Wang, Feiping Nie, and Xuelong Li. 2022b. Multiple Kernel K-Means Clustering with Simultaneous Spectral Rotation. In Proc. ICASSP. 4143--4147.Google ScholarCross Ref
- Yihang Lu, Jitao Lu, Rong Wang, and Feiping Nie. 2022a. Discrete Multi-Kernel K-Means with Diverse and Optimal Kernel Learning. In Proc. ICASSP. 4153--4157.Google ScholarCross Ref
- Fan Ma, Deyu Meng, Qi Xie, Zina Li, and Xuanyi Dong. 2017. Self-Paced Co-training. In Proc. ICML. 2275--2284.Google Scholar
- Feiping Nie, Cheng-Long Wang, and Xuelong Li. 2019. K-multiple-means: A multiple-means clustering method with specified k clusters. In Proc. SIGKDD. 959--967.Google ScholarDigital Library
- Yazhou Ren, Shudong Huang, Peng Zhao, Minghao Han, and Zenglin Xu. 2020. Self-paced and auto-weighted multi-view clustering. Neural Comput.,, Vol. 383 (2020), 248--256.Google Scholar
- Yazhou Ren, Peng Zhao, Zenglin Xu, and Dezhong Yao. 2017. Balanced self-paced learning with feature corruption. In Proc. IJCNN. 2064--2071.Google ScholarCross Ref
- Ingo Steinwart, Don R. Hush, and Clint Scovel. 2006. An Explicit Description of the Reproducing Kernel Hilbert Spaces of Gaussian RBF Kernels. IEEE Trans. Inf. Theory, Vol. 52, 10 (2006), 4635--4643.Google ScholarDigital Library
- James Steven Supancic and Deva Ramanan. 2013. Self-Paced Learning for Long-Term Tracking. In Proc. CVPR. 2379--2386.Google ScholarDigital Library
- Kevin D. Tang, Vignesh Ramanathan, Fei-Fei Li, and Daphne Koller. 2012. Shifting Weights: Adapting Object Detectors from Image to Video. In Proc. NeurIPS. 647--655.Google Scholar
- Rong Wang, Jitao Lu, Yihang Lu, Feiping Nie, and Xuelong Li. 2021. Discrete Multiple Kernel k-means. In Proc. IJCAI. 3111--3117.Google ScholarCross Ref
- Rong Wang, Jitao Lu, Yihang Lu, Feiping Nie, and Xuelong Li. 2022. Discrete and Parameter-Free Multiple Kernel k-Means. IEEE Trans. Image Process., Vol. 31 (2022), 2796--2808.Google ScholarCross Ref
- Stephen J Wright. 2015. Coordinate descent algorithms. Math. Program., Vol. 151, 1 (2015), 3--34.Google ScholarDigital Library
- Shuyin Xia, Daowan Peng, Deyu Meng, Changqing Zhang, Guoyin WANG, Elisabeth Giem, Wei Wei, and Zizhong Chen. 2020. Ball k-Means: Fast Adaptive Clustering With No Bounds. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 44 (2020), 87 -- 99.Google Scholar
- Chang Xu and Dacheng Tao. 2015. Multi-view Self-Paced Learning for Clustering. In Proc. IJCAI. 3974--3980.Google Scholar
- Yaqiang Yao, Yang Li, Bingbing Jiang, and Huanhuan Chen. 2021. Multiple kernel k-means clustering by selecting representative kernels. IEEE Trans. Neural Netw. Learn. Syst., Vol. 32 (2021), 4983 -- 4996.Google ScholarCross Ref
- Shi Yu, Leon Tranchevent, Xinhai Liu, Wolfgang Glanzel, Johan AK Suykens, Bart De Moor, and Yves Moreau. 2011. Optimized data fusion for kernel k-means clustering. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 34, 5 (2011), 1031--1039.Google Scholar
- Bin Zhao, James T Kwok, and Changshui Zhang. 2009. Multiple kernel clustering. In Proc. SDM. 638--649.Google ScholarCross Ref
- Qian Zhao, Deyu Meng, Lu Jiang, Qi Xie, Zongben Xu, and Alexander G. Hauptmann. 2015. Self-Paced Learning for Matrix Factorization. In Proc. AAAI. 3196--3202.Google Scholar
- Peng Zhou, Liang Du, Xinwang Liu, Yi-Dong Shen, Mingyu Fan, and Xuejun Li. 2021. Self-Paced Clustering Ensemble. IEEE Trans. Neural Netw. Learn. Syst., Vol. 32, 4 (2021), 1497--1511. nGoogle ScholarCross Ref
Index Terms
Self-Paced and Discrete Multiple Kernel k-Means
Recommendations
Scalable Multiple Kernel k-means Clustering
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge ManagementWith its simplicity and effectiveness, k-means is immensely popular, but it cannot perform well on complex nonlinear datasets. Multiple kernel k-means (MKKM) demonstrates the ability to describe highly complex nonlinear separable data structures. ...
Comments