short-paper

Self-Paced and Discrete Multiple Kernel k-Means

Authors:

Xuelong LiAuthors Info & Claims

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Pages 4284 - 4288

https://doi.org/10.1145/3511808.3557696

Published: 17 October 2022 Publication History

Abstract

Multiple Kernel K-means (MKKM) uses various kernels from different sources to improve clustering performance. However, most of the existing models are non-convex, which is prone to be stuck into bad local optimum, especially with noise and outliers. To address the issue, we propose a novel Self-Paced and Discrete Multiple Kernel K-Means (SPD-MKKM). It learns the MKKM model in a meaningful order by progressing both samples and kernels from easy to complex, which is beneficial to avoid bad local optimum. In addition, whereas existing methods optimize in two stages: learning the relaxation matrix and then finding the discrete one by extra discretization, our work can directly gain the discrete cluster indicator matrix without extra process. What's more, a well-designed alternative optimization is employed to reduce the overall computational complexity via using the coordinate descent technique. Finally, thorough experiments performed on real-world datasets illustrated the excellence and efficacy of our method.

References

[1]

Liang Du, Peng Zhou, Lei Shi, Hanmo Wang, Mingyu Fan, Wenjian Wang, and Yi-Dong Shen. 2015. Robust multiple kernel k-means using $ell_2,1$-norm. In Proc. IJCAI. 3476--3482.

[2]

Mark Girolami. 2002. Mercer kernel-based clustering in feature space. IEEE Trans. Neural Netw. Learn. Syst., Vol. 13, 3 (2002), 780--784.

Digital Library

[3]

Mehmet Gönen and Ethem Alpaydin. 2011. Multiple kernel learning algorithms. J. Mach. Learn. Res., Vol. 12 (2011), 2211--2268.

Digital Library

[4]

Mehmet Gönen and Adam A Margolin. 2014. Localized data fusion for kernel k-means clustering with application to cancer biology. In Proc. NeurIPS. 1305--1313.

[5]

J. A. Hartigan and M. A. Wong. 1979. Algorithm AS 136: A k-means clustering algorithm. J. Roy. Stat. Soc., Vol. 28, 1 (1979), 100--108.

[6]

Zhao Kang, Xiao Lu, Jinfeng Yi, and Zenglin Xu. 2018. Self-weighted multiple kernel learning for graph-based clustering and semi-supervised classification. In Proc. IJCAI. 2312--2318.

[7]

M. Pawan Kumar, Benjamin Packer, and Daphne Koller. 2010. Self-Paced Learning for Latent Variable Models. In Proc. NeurIPS. 1189--1197.

[8]

M. Pawan Kumar, Haithem Turki, Dan Preston, and Daphne Koller. 2011. Learning specific-class segmentation from diverse data. In Proc. ICCV. 1800--1807.

Digital Library

[9]

Jiyuan Liu, Xinwang Liu, Jian Xiong, Qing Liao, Sihang Zhou, Siwei Wang, and Yuexiang Yang. 2022. Optimal Neighborhood Multiple Kernel Clustering With Adaptive Local Kernels. IEEE Trans. Knowl. Data Eng., Vol. 34, 6 (2022), 2872--2885.

[10]

Xinwang Liu, Yong Dou, Jianping Yin, Lei Wang, and En Zhu. 2016. Multiple kernel k-means clustering with matrix-induced regularization. In Proc. AAAI. 1888--1894.

[11]

X. Liu, M. Li, C. Tang, J. Xia, J. Xiong, L. Liu, M. Kloft, and E. Zhu. 2020. Efficient and Effective Regularized Incomplete Multi-view Clustering. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 43, 8 (2020), 2634 -- 2646.

[12]

Jitao Lu, Yihang Lu, Rong Wang, Feiping Nie, and Xuelong Li. 2022b. Multiple Kernel K-Means Clustering with Simultaneous Spectral Rotation. In Proc. ICASSP. 4143--4147.

[13]

Yihang Lu, Jitao Lu, Rong Wang, and Feiping Nie. 2022a. Discrete Multi-Kernel K-Means with Diverse and Optimal Kernel Learning. In Proc. ICASSP. 4153--4157.

[14]

Fan Ma, Deyu Meng, Qi Xie, Zina Li, and Xuanyi Dong. 2017. Self-Paced Co-training. In Proc. ICML. 2275--2284.

[15]

Feiping Nie, Cheng-Long Wang, and Xuelong Li. 2019. K-multiple-means: A multiple-means clustering method with specified k clusters. In Proc. SIGKDD. 959--967.

Digital Library

[16]

Yazhou Ren, Shudong Huang, Peng Zhao, Minghao Han, and Zenglin Xu. 2020. Self-paced and auto-weighted multi-view clustering. Neural Comput., Vol. 383 (2020), 248--256.

[17]

Yazhou Ren, Peng Zhao, Zenglin Xu, and Dezhong Yao. 2017. Balanced self-paced learning with feature corruption. In Proc. IJCNN. 2064--2071.

[18]

Ingo Steinwart, Don R. Hush, and Clint Scovel. 2006. An Explicit Description of the Reproducing Kernel Hilbert Spaces of Gaussian RBF Kernels. IEEE Trans. Inf. Theory, Vol. 52, 10 (2006), 4635--4643.

Digital Library

[19]

James Steven Supancic and Deva Ramanan. 2013. Self-Paced Learning for Long-Term Tracking. In Proc. CVPR. 2379--2386.

Digital Library

[20]

Kevin D. Tang, Vignesh Ramanathan, Fei-Fei Li, and Daphne Koller. 2012. Shifting Weights: Adapting Object Detectors from Image to Video. In Proc. NeurIPS. 647--655.

[21]

Rong Wang, Jitao Lu, Yihang Lu, Feiping Nie, and Xuelong Li. 2021. Discrete Multiple Kernel k-means. In Proc. IJCAI. 3111--3117.

[22]

Rong Wang, Jitao Lu, Yihang Lu, Feiping Nie, and Xuelong Li. 2022. Discrete and Parameter-Free Multiple Kernel k-Means. IEEE Trans. Image Process., Vol. 31 (2022), 2796--2808.

[23]

Stephen J Wright. 2015. Coordinate descent algorithms. Math. Program., Vol. 151, 1 (2015), 3--34.

Digital Library

[24]

Shuyin Xia, Daowan Peng, Deyu Meng, Changqing Zhang, Guoyin WANG, Elisabeth Giem, Wei Wei, and Zizhong Chen. 2020. Ball k-Means: Fast Adaptive Clustering With No Bounds. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 44 (2020), 87 -- 99.

[25]

Chang Xu and Dacheng Tao. 2015. Multi-view Self-Paced Learning for Clustering. In Proc. IJCAI. 3974--3980.

[26]

Yaqiang Yao, Yang Li, Bingbing Jiang, and Huanhuan Chen. 2021. Multiple kernel k-means clustering by selecting representative kernels. IEEE Trans. Neural Netw. Learn. Syst., Vol. 32 (2021), 4983 -- 4996.

[27]

Shi Yu, Leon Tranchevent, Xinhai Liu, Wolfgang Glanzel, Johan AK Suykens, Bart De Moor, and Yves Moreau. 2011. Optimized data fusion for kernel k-means clustering. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 34, 5 (2011), 1031--1039.

[28]

Bin Zhao, James T Kwok, and Changshui Zhang. 2009. Multiple kernel clustering. In Proc. SDM. 638--649.

[29]

Qian Zhao, Deyu Meng, Lu Jiang, Qi Xie, Zongben Xu, and Alexander G. Hauptmann. 2015. Self-Paced Learning for Matrix Factorization. In Proc. AAAI. 3196--3202.

[30]

Peng Zhou, Liang Du, Xinwang Liu, Yi-Dong Shen, Mingyu Fan, and Xuejun Li. 2021. Self-Paced Clustering Ensemble. IEEE Trans. Neural Netw. Learn. Syst., Vol. 32, 4 (2021), 1497--1511. n

Cited By

Yang GZou JChen YDu LZhou P(2024)Heat Kernel Diffusion for Enhanced Late Fusion Multi-View ClusteringIEEE Signal Processing Letters10.1109/LSP.2024.344922931(2310-2314)Online publication date: 2024
https://doi.org/10.1109/LSP.2024.3449229
Xin HLu YTang HWang RNie F(2023)Self-Weighted Euler $k$-Means ClusteringIEEE Signal Processing Letters10.1109/LSP.2023.330590930(1127-1131)Online publication date: 2023
https://doi.org/10.1109/LSP.2023.3305909

Index Terms

Self-Paced and Discrete Multiple Kernel k-Means
1. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Kernel methods
      2. Unsupervised learning and clustering

Recommendations

Scalable Multiple Kernel k-means Clustering
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

With its simplicity and effectiveness, k-means is immensely popular, but it cannot perform well on complex nonlinear datasets. Multiple kernel k-means (MKKM) demonstrates the ability to describe highly complex nonlinear separable data structures. ...
MCSA Self-Paced Training Kit: Microsoft Windows 2000 Core Requirements: Exams 70-210, 70-215, 70-216, and 70-218, Second Edition
MCSA Self-Paced Training Kit: Microsoft Windows 2000 Core Requirements;Exams 70-210,70-215,70-216,and 70-218

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

October 2022

5274 pages

ISBN:9781450392365

DOI:10.1145/3511808

General Chairs:
Mohammad Al Hasan
Indiana University Purdue University, Indianapolis, USA
,
Li Xiong
Emory University, Atlanta, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Conference

CIKM '22

Sponsor:

CIKM '22: The 31st ACM International Conference on Information and Knowledge Management

October 17 - 21, 2022

GA, Atlanta, USA

Acceptance Rates

CIKM '22 Paper Acceptance Rate 621 of 2,257 submissions, 28%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
236
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang GZou JChen YDu LZhou P(2024)Heat Kernel Diffusion for Enhanced Late Fusion Multi-View ClusteringIEEE Signal Processing Letters10.1109/LSP.2024.344922931(2310-2314)Online publication date: 2024
https://doi.org/10.1109/LSP.2024.3449229
Xin HLu YTang HWang RNie F(2023)Self-Weighted Euler $k$-Means ClusteringIEEE Signal Processing Letters10.1109/LSP.2023.330590930(1127-1131)Online publication date: 2023
https://doi.org/10.1109/LSP.2023.3305909

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten