Article

An objective evaluation criterion for clustering

Authors:

Arindam Banerjee,

John LangfordAuthors Info & Claims

KDD '04: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 515 - 520

https://doi.org/10.1145/1014052.1014112

Published: 22 August 2004 Publication History

Get Access

Abstract

We propose and test an objective criterion for evaluation of clustering performance: How well does a clustering algorithm run on unlabeled data aid a classification algorithm? The accuracy is quantified using the PAC-MDL bound [3] in a semisupervised setting. Clustering algorithms which naturally separate the data according to (hidden) labels with a small number of clusters perform well. A simple extension of the argument leads to an objective model selection method. Experimental results on text analysis datasets demonstrate that this approach empirically results in very competitive bounds on test set performance on natural datasets.

References

[1]

A. Banerjee, I. Dhillon, J. Ghosh, and S. Sra. Generative model-based clustering of directional data. In KDD, pages 19--28, 2003.

Digital Library

Google Scholar

[2]

A. Banerjee and J. Ghosh. Frequency sensitive competitive learning for balanced clustering on high-dimensional hyperspheres. In IEEE Transactions on Neural Networks, May 2004.

Digital Library

Google Scholar

[3]

A. Blum and J. Langford. PAC-MDL bounds. In COLT, 2003.

Google Scholar

[4]

T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley-Interscience, 1991.

Digital Library

Google Scholar

[5]

I. Dhillon, S. Mallela, and R. Kumar. A divisive information-theoretic feature clustering algorithm for text classification. Journal of Machine Learning Research, 3(4):1265--1287, 2003.

Digital Library

Google Scholar

[6]

I. S. Dhillon and D. S. Modha. Concept decompositions for large sparse text data using clustering. Machine Learning, 42(1):143--175, 2001.

Digital Library

Google Scholar

[7]

B. E. Dom. An information-theoretic external cluster-validity measure. Technical Report RJ 10219, IBM Research Report, 2001.

Google Scholar

[8]

J. Ghosh. Scalable clustering. In Nong Ye, editor, The Handbook of Data Mining, pages 247--277. Lawrence Erlbaum Assoc., 2003.

Google Scholar

[9]

A. K. Jain and R. C. Dubes. Algorithms for Clustering Data. Prentice Hall, New Jersey, 1988.

Digital Library

Google Scholar

[10]

M. Meilua. Comparing clusterings by the variation of information. In COLT, 2003.

Google Scholar

Cited By

View all

Varde A(2022)Computational Estimation by Scientific Data Mining with Classical Methods to Automate Learning Strategies of ScientistsACM Transactions on Knowledge Discovery from Data10.1145/350273616:5(1-52)Online publication date: 9-Mar-2022
https://doi.org/10.1145/3502736
Xie XZhang ZChen TLiu YPoon PXu B(2020)METTLE: A METamorphic Testing Approach to Assessing and Validating Unsupervised Machine Learning SystemsIEEE Transactions on Reliability10.1109/TR.2020.297226669:4(1293-1322)Online publication date: Dec-2020
https://doi.org/10.1109/TR.2020.2972266
Mercioni MHolban S(2018)Evaluating hierarchical and non-hierarchical grouping for develop a smart system2018 International Symposium on Electronics and Telecommunications (ISETC)10.1109/ISETC.2018.8583997(1-4)Online publication date: Nov-2018
https://doi.org/10.1109/ISETC.2018.8583997
Show More Cited By

Index Terms

An objective evaluation criterion for clustering
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Cluster analysis

Recommendations

F-statistics algorithm for gene clustering evaluation
BCB '10: Proceedings of the First ACM International Conference on Bioinformatics and Computational Biology

An enormous amount of microarray data has been generated and archived for a large variety of biological studies such as gene expression. In order to analyze gene expression data, many clustering algorithms have been proposed, but very few techniques ...
Many-objective fuzzy centroids clustering algorithm for categorical data

We propose a novel many-objective clustering algorithm for categorical data.Our method can take advantage of different cluster validity indices simultaneously.Two versions of the proposed algorithm are presented with and without cluster number.The ...
Design and evaluation of clustering criterion for optimal hierarchical agglomerative clustering

Comments

Information & Contributors

Information

Published In

KDD '04: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining

August 2004

874 pages

ISBN:1581138881

DOI:10.1145/1014052

General Chairs:
Won Kim
Cyber Database Solutions
,
Ronny Kohavi
Amazon.com
,
Program Chairs:
Johannes Gehrke
Cornell University
,
William DuMouchel
AT&T Labs Research

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 August 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

KDD04

Sponsor:

KDD04: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 22 - 25, 2004

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
842
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Varde A(2022)Computational Estimation by Scientific Data Mining with Classical Methods to Automate Learning Strategies of ScientistsACM Transactions on Knowledge Discovery from Data10.1145/350273616:5(1-52)Online publication date: 9-Mar-2022
https://doi.org/10.1145/3502736
Xie XZhang ZChen TLiu YPoon PXu B(2020)METTLE: A METamorphic Testing Approach to Assessing and Validating Unsupervised Machine Learning SystemsIEEE Transactions on Reliability10.1109/TR.2020.297226669:4(1293-1322)Online publication date: Dec-2020
https://doi.org/10.1109/TR.2020.2972266
Mercioni MHolban S(2018)Evaluating hierarchical and non-hierarchical grouping for develop a smart system2018 International Symposium on Electronics and Telecommunications (ISETC)10.1109/ISETC.2018.8583997(1-4)Online publication date: Nov-2018
https://doi.org/10.1109/ISETC.2018.8583997
Xu HYue ZWang CDong KPang HHan Z(2017)Multi-source data fusion study in scientometricsScientometrics10.1007/s11192-017-2290-5111:2(773-792)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1007/s11192-017-2290-5
Fan WBouguila N(2016)Model-Based Clustering Based on Variational Learning of Hierarchical Infinite Beta-Liouville Mixture ModelsNeural Processing Letters10.1007/s11063-015-9466-x44:2(431-449)Online publication date: 1-Oct-2016
https://dl.acm.org/doi/10.1007/s11063-015-9466-x
Huang FNiranjan UHakeem MAnandkumar A(2015)Online tensor methods for learning latent variable modelsThe Journal of Machine Learning Research10.5555/2789272.291208816:1(2797-2835)Online publication date: 1-Jan-2015
https://dl.acm.org/doi/10.5555/2789272.2912088
Ando SThanomphongphan TSeki YSuzuki E(2015)Ensemble anomaly detection from multi-resolution trajectory featuresData Mining and Knowledge Discovery10.1007/s10618-013-0334-x29:1(39-83)Online publication date: 1-Jan-2015
https://dl.acm.org/doi/10.1007/s10618-013-0334-x
Ghosh JGupta G(2012)Bregman Bubble Clustering: A Robust Framework for Mining Dense ClustersData Mining: Foundations and Intelligent Paradigms10.1007/978-3-642-23166-7_7(157-208)Online publication date: 2012
https://doi.org/10.1007/978-3-642-23166-7_7
Georgieva OTschumitschew KKlawonn F(2011)Cluster Validity Measures Based on the Minimum Description Length PrincipleKnowledge-Based and Intelligent Information and Engineering Systems10.1007/978-3-642-23851-2_9(82-89)Online publication date: 2011
https://doi.org/10.1007/978-3-642-23851-2_9
Mallapragada PJin RJain A(2010)Non-parametric mixture models for clusteringProceedings of the 2010 joint IAPR international conference on Structural, syntactic, and statistical pattern recognition10.5555/1887003.1887041(334-343)Online publication date: 18-Aug-2010
https://dl.acm.org/doi/10.5555/1887003.1887041
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

F-statistics algorithm for gene clustering evaluation

Many-objective fuzzy centroids clustering algorithm for categorical data

Design and evaluation of clustering criterion for optimal hierarchical agglomerative clustering

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations