skip to main content
10.1145/3511808.3557696acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
short-paper

Self-Paced and Discrete Multiple Kernel k-Means

Authors Info & Claims
Published:17 October 2022Publication History

ABSTRACT

Multiple Kernel K-means (MKKM) uses various kernels from different sources to improve clustering performance. However, most of the existing models are non-convex, which is prone to be stuck into bad local optimum, especially with noise and outliers. To address the issue, we propose a novel Self-Paced and Discrete Multiple Kernel K-Means (SPD-MKKM). It learns the MKKM model in a meaningful order by progressing both samples and kernels from easy to complex, which is beneficial to avoid bad local optimum. In addition, whereas existing methods optimize in two stages: learning the relaxation matrix and then finding the discrete one by extra discretization, our work can directly gain the discrete cluster indicator matrix without extra process. What's more, a well-designed alternative optimization is employed to reduce the overall computational complexity via using the coordinate descent technique. Finally, thorough experiments performed on real-world datasets illustrated the excellence and efficacy of our method.

References

  1. Liang Du, Peng Zhou, Lei Shi, Hanmo Wang, Mingyu Fan, Wenjian Wang, and Yi-Dong Shen. 2015. Robust multiple kernel k-means using $ell_2,1$-norm. In Proc. IJCAI. 3476--3482.Google ScholarGoogle Scholar
  2. Mark Girolami. 2002. Mercer kernel-based clustering in feature space. IEEE Trans. Neural Netw. Learn. Syst., Vol. 13, 3 (2002), 780--784.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Mehmet Gönen and Ethem Alpaydin. 2011. Multiple kernel learning algorithms. J. Mach. Learn. Res., Vol. 12 (2011), 2211--2268.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Mehmet Gönen and Adam A Margolin. 2014. Localized data fusion for kernel k-means clustering with application to cancer biology. In Proc. NeurIPS. 1305--1313.Google ScholarGoogle Scholar
  5. J. A. Hartigan and M. A. Wong. 1979. Algorithm AS 136: A k-means clustering algorithm. J. Roy. Stat. Soc., Vol. 28, 1 (1979), 100--108.Google ScholarGoogle Scholar
  6. Zhao Kang, Xiao Lu, Jinfeng Yi, and Zenglin Xu. 2018. Self-weighted multiple kernel learning for graph-based clustering and semi-supervised classification. In Proc. IJCAI. 2312--2318.Google ScholarGoogle ScholarCross RefCross Ref
  7. M. Pawan Kumar, Benjamin Packer, and Daphne Koller. 2010. Self-Paced Learning for Latent Variable Models. In Proc. NeurIPS. 1189--1197.Google ScholarGoogle Scholar
  8. M. Pawan Kumar, Haithem Turki, Dan Preston, and Daphne Koller. 2011. Learning specific-class segmentation from diverse data. In Proc. ICCV. 1800--1807.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Jiyuan Liu, Xinwang Liu, Jian Xiong, Qing Liao, Sihang Zhou, Siwei Wang, and Yuexiang Yang. 2022. Optimal Neighborhood Multiple Kernel Clustering With Adaptive Local Kernels. IEEE Trans. Knowl. Data Eng., Vol. 34, 6 (2022), 2872--2885.Google ScholarGoogle Scholar
  10. Xinwang Liu, Yong Dou, Jianping Yin, Lei Wang, and En Zhu. 2016. Multiple kernel k-means clustering with matrix-induced regularization. In Proc. AAAI. 1888--1894.Google ScholarGoogle ScholarCross RefCross Ref
  11. X. Liu, M. Li, C. Tang, J. Xia, J. Xiong, L. Liu, M. Kloft, and E. Zhu. 2020. Efficient and Effective Regularized Incomplete Multi-view Clustering. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 43, 8 (2020), 2634 -- 2646.Google ScholarGoogle Scholar
  12. Jitao Lu, Yihang Lu, Rong Wang, Feiping Nie, and Xuelong Li. 2022b. Multiple Kernel K-Means Clustering with Simultaneous Spectral Rotation. In Proc. ICASSP. 4143--4147.Google ScholarGoogle ScholarCross RefCross Ref
  13. Yihang Lu, Jitao Lu, Rong Wang, and Feiping Nie. 2022a. Discrete Multi-Kernel K-Means with Diverse and Optimal Kernel Learning. In Proc. ICASSP. 4153--4157.Google ScholarGoogle ScholarCross RefCross Ref
  14. Fan Ma, Deyu Meng, Qi Xie, Zina Li, and Xuanyi Dong. 2017. Self-Paced Co-training. In Proc. ICML. 2275--2284.Google ScholarGoogle Scholar
  15. Feiping Nie, Cheng-Long Wang, and Xuelong Li. 2019. K-multiple-means: A multiple-means clustering method with specified k clusters. In Proc. SIGKDD. 959--967.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Yazhou Ren, Shudong Huang, Peng Zhao, Minghao Han, and Zenglin Xu. 2020. Self-paced and auto-weighted multi-view clustering. Neural Comput.,, Vol. 383 (2020), 248--256.Google ScholarGoogle Scholar
  17. Yazhou Ren, Peng Zhao, Zenglin Xu, and Dezhong Yao. 2017. Balanced self-paced learning with feature corruption. In Proc. IJCNN. 2064--2071.Google ScholarGoogle ScholarCross RefCross Ref
  18. Ingo Steinwart, Don R. Hush, and Clint Scovel. 2006. An Explicit Description of the Reproducing Kernel Hilbert Spaces of Gaussian RBF Kernels. IEEE Trans. Inf. Theory, Vol. 52, 10 (2006), 4635--4643.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. James Steven Supancic and Deva Ramanan. 2013. Self-Paced Learning for Long-Term Tracking. In Proc. CVPR. 2379--2386.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Kevin D. Tang, Vignesh Ramanathan, Fei-Fei Li, and Daphne Koller. 2012. Shifting Weights: Adapting Object Detectors from Image to Video. In Proc. NeurIPS. 647--655.Google ScholarGoogle Scholar
  21. Rong Wang, Jitao Lu, Yihang Lu, Feiping Nie, and Xuelong Li. 2021. Discrete Multiple Kernel k-means. In Proc. IJCAI. 3111--3117.Google ScholarGoogle ScholarCross RefCross Ref
  22. Rong Wang, Jitao Lu, Yihang Lu, Feiping Nie, and Xuelong Li. 2022. Discrete and Parameter-Free Multiple Kernel k-Means. IEEE Trans. Image Process., Vol. 31 (2022), 2796--2808.Google ScholarGoogle ScholarCross RefCross Ref
  23. Stephen J Wright. 2015. Coordinate descent algorithms. Math. Program., Vol. 151, 1 (2015), 3--34.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Shuyin Xia, Daowan Peng, Deyu Meng, Changqing Zhang, Guoyin WANG, Elisabeth Giem, Wei Wei, and Zizhong Chen. 2020. Ball k-Means: Fast Adaptive Clustering With No Bounds. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 44 (2020), 87 -- 99.Google ScholarGoogle Scholar
  25. Chang Xu and Dacheng Tao. 2015. Multi-view Self-Paced Learning for Clustering. In Proc. IJCAI. 3974--3980.Google ScholarGoogle Scholar
  26. Yaqiang Yao, Yang Li, Bingbing Jiang, and Huanhuan Chen. 2021. Multiple kernel k-means clustering by selecting representative kernels. IEEE Trans. Neural Netw. Learn. Syst., Vol. 32 (2021), 4983 -- 4996.Google ScholarGoogle ScholarCross RefCross Ref
  27. Shi Yu, Leon Tranchevent, Xinhai Liu, Wolfgang Glanzel, Johan AK Suykens, Bart De Moor, and Yves Moreau. 2011. Optimized data fusion for kernel k-means clustering. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 34, 5 (2011), 1031--1039.Google ScholarGoogle Scholar
  28. Bin Zhao, James T Kwok, and Changshui Zhang. 2009. Multiple kernel clustering. In Proc. SDM. 638--649.Google ScholarGoogle ScholarCross RefCross Ref
  29. Qian Zhao, Deyu Meng, Lu Jiang, Qi Xie, Zongben Xu, and Alexander G. Hauptmann. 2015. Self-Paced Learning for Matrix Factorization. In Proc. AAAI. 3196--3202.Google ScholarGoogle Scholar
  30. Peng Zhou, Liang Du, Xinwang Liu, Yi-Dong Shen, Mingyu Fan, and Xuejun Li. 2021. Self-Paced Clustering Ensemble. IEEE Trans. Neural Netw. Learn. Syst., Vol. 32, 4 (2021), 1497--1511. nGoogle ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Self-Paced and Discrete Multiple Kernel k-Means

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management
        October 2022
        5274 pages
        ISBN:9781450392365
        DOI:10.1145/3511808
        • General Chairs:
        • Mohammad Al Hasan,
        • Li Xiong

        Copyright © 2022 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 17 October 2022

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • short-paper

        Acceptance Rates

        CIKM '22 Paper Acceptance Rate621of2,257submissions,28%Overall Acceptance Rate1,861of8,427submissions,22%

        Upcoming Conference

      • Article Metrics

        • Downloads (Last 12 months)100
        • Downloads (Last 6 weeks)2

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader