Abstract
Learning an appropriate distance measure under supervision of side information has become a topic of significant interest within machine learning community. In this paper, we address the problem of metric learning for constrained clustering by considering three important issues: (1) considering importance degree for constraints, (2) preserving the topological structure of data, and (3) preserving some natural distribution properties in the data. This work provides a unified way to handle different issues in constrained clustering by learning an appropriate distance measure. It has modeled the first issue by injecting the importance degree of constraints directly into an objective function. The topological structure of data is preserved by minimizing the reconstruction error of data in the target space. Finally we addressed the issue of preserving natural distribution properties in the data by using the proximity information of data. We have proposed two different methods to address the above mentioned issues. The first approach learns a linear transformation of data into a target space (linear-model) and the second one uses kernel functions to learn an appropriate distance measure (non-linear-model). Experiments show that considering these issues significantly improves clustering accuracy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abin AA (2016a) Clustering with side information: further efforts to improve efficiency. Pattern Recognit Lett 84:252–258
Abin AA (2016b) Querying beneficial constraints before clustering using facility location analysis. IEEE Trans Cybern 99:1–12
Abin AA, Beigy H (2014) Active selection of clustering constraints: a sequential approach. Pattern Recognit 47(3):1443–1458
Abin AA, Beigy H (2015) Active constrained fuzzy clustering: a multiple kernels learning approach. Pattern Recognit 48(3):953–967
Arenas A, Diaz-Guilera A, Kurths J, Moreno Y, Zhou C (2008) Synchronization in complex networks. Phys Rep 469(3):93–153
Baghshah MS, Afsari F, Shouraki SB, Eslami E (2014) Scalable semi-supervised clustering by spectral kernel learning. Pattern Recognit Lett 45:161–171
Bar-Hillel A, Hertz T, Shental N, Weinshall D (2005) Learning a Mahalanobis metric from equivalence constraints. J Mach Learn Res 6:937–965
Basu S, Banerjee A, Mooney RJ (2004) Active semi-supervision for pairwise constrained clustering. In: Proceedings of the 5th SIAM international conference on data mining, ICDM ’04, pp 333–344
Basu S, Davidson I, Wagstaff KL (2008) Constrained clustering: advances in algorithms, theory, and applications. Chapman and Hall/CRC, Boca Raton
Bilenko M, Basu S, Mooney RJ (2004a) Integrating constraints and metric learning in semi-supervised clustering. In: Proceedings of the 21th international conference on Machine learning, ICML ’04, pp 11–18
Bilenko M, Basu S, Mooney RJ (2004b) Integrating constraints and metric learning in semi-supervised clustering. In: Proceedings of the twenty-first international conference on Machine learning, ACM, p 11
Chang H, yan Yeung D (2004) Locally linear metric adaptation for semi-supervised clustering. In: Proceedings of the 21th international conference on machine learning, ICML ’04, pp 153–160
Cheng H, Hua KA, Vu K (2008) Constrained locally weighted clustering. Proc VLDB Endow 1(1):90–101
Davidson I, Wagstaff KL, Basu S (2006) Measuring constraint-set utility for partitional clustering algorithms. In: Proceedings of the 10th European conference on Principle and practice of knowledge discovery in databases, PKDD ’06, pp 115–126
Ding S, Jia H, Zhang L, Jin F (2014) Research of semi-supervised spectral clustering algorithm based on pairwise constraints. Neural Comput Appl 24(1):211–219
Gong C, Fu K, Wu Q, Tu E, Yang J (2014) Semi-supervised classification with pairwise constraints. Neurocomputing 139:130–137
He P, Xu X, Hu K, Chen L (2014) Semi-supervised clustering via multi-level random walk. Pattern Recognit 47(2):820–832
Hertz T, Bar-hillel A, Weinshall D (2004) Boosting margin based distance functions for clustering. In: Proceedings of the 21th international conference on machine learning, ICML ’04, pp 393–400
Hoi SCH, Jin R, Lyu MR (2007) Learning non-parametric kernel matrices from pairwise constraints. In: Proceedings of the 24th international conference on Machine learning, ICML ’07, pp 361–368
Hubert L, Arabie P (1985) Comparing Partitions. J Classif 2(1):193–218
Jain P, Kulis B, Davis JV, Dhillon IS (2012) Metric and kernel learning using a linear transformation. J Mach Learn Res 13:519–547
Kalakech M, Biela P, Macaire L, Hamad D (2011) Constraint scores for semi-supervised feature selection: a comparative study. Pattern Recognit Lett 32(5):656–665
Khoreva A, Galasso F, Hein M, Schiele B (2014) Learning must-link constraints for video segmentation based on spectral clustering. In: Pattern recognition, Springer, pp 701–712
Korinna Bade AN (2014) Hierarchical constraints. Mach Learn 94(3):371–399
Kulis B, Basu S, Dhillon I, Mooney R (2009) Semi-supervised graph clustering: a kernel approach. Mach Learn 74(1):1–22
LeCun Y, Cortes C (2010) MNIST handwritten digit database http://yann.lecun.com/exdb/mnist/
Liu H, Wu Z, Li X, Cai D, Huang T (2012) Constrained non-negative matrix factorization for image representation. IEEE Trans Pattern Anal Mach Intell 34(7):1299–1311
Melnykov V, Melnykov I, Michael S (2016) Semi-supervised model-based clustering with positive and negative constraints. Adv Data Anal Classif 10(3):327–349
Milligan G, Cooper M (1986) A study of the comparability of external criteria for hierarchical cluster analysis. Multivariate Behav Res 21(4):441–458
Okabe M, Yamada S (2009) Clustering with constrained similarity learning. In: Proceedings of the 2009 IEEE/WIC/ACM international conference on web intelligence and international conference on intelligent agent technology, pp 30–33
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323–2326
Sinkkonen J, Kaski S (2002) Clustering based on conditional distributions in an auxiliary space. Neural Comput 14:217–239
Smieja M, Wiercioch M (2017) Constrained clustering with a complex cluster structure. Adv Data Anal Classif 11(3):493–518
Soleymani Baghshah M, Bagheri Shouraki S (2010) Non-linear metric learning using pairwise similarity and dissimilarity constraints and the geometrical structure of data. Pattern Recognit 43:2982–2992
Truong D, Battiti R (2013) A flexible cluster-oriented alternative clustering algorithm for choosing from the pareto front of solutions. Mach Learn 98(1):57–91
Vu VV, Labroche N, Bouchon-Meunier B (2012) Improving constrained clustering with active query selection. Pattern Recognit 45(4):1749–1758
Wagstaff K, Cardie C (2000) Clustering with instance-level constraints. In: Proceedings of the seventeenth national conference on artificial intelligence and twelfth conference on on innovative applications of artificial intelligence, July 30–August 3, 2000, Austin, Texas, USA, p 1097
Wagstaff K, Cardie C, Rogers S, Schrödl S (2001) Constrained k-means clustering with background knowledge. In: Proceedings of the 18th international conference on machine learning, ICML ’01, pp 577–584
Wang Q, Yuen PC, Feng G (2013) Semi-supervised metric learning via topology preserving multiple semi-supervised assumptions. Pattern Recognit 46(9):2576–2587
Wu S, Feng X, Zhou W (2014) Spectral clustering of high-dimensional data exploiting sparse representation vectors. Neurocomputing 135:229–239
Xiang S, Nie F, Zhang C (2008) Learning a Mahalanobis distance metric for data clustering and classification. Pattern Recognit 41(12):3600–3612
Ye J, Zhao Z, Liu H (2007) Adaptive distance metric learning for clustering. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR 2007), pp 1–7
Yin X, Chen S, Hu E, Zhang D (2010) Semi-supervised clustering with metric learning: an adaptive kernel method. Pattern Recognit 43(4):1320–1333
Zhang Z, Zhao M, Chow TWS (2012) Marginal semi-supervised sub-manifold projections with informative constraints for dimensionality reduction and recognition. Neural Netw 36:97–111
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Abin, A.A., Bashiri, M.A. & Beigy, H. Learning a metric when clustering data points in the presence of constraints. Adv Data Anal Classif 14, 29–56 (2020). https://doi.org/10.1007/s11634-019-00359-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11634-019-00359-6