Abstract
K-Means clustering still plays an important role in many computer vision problems. While the conventional Lloyd method, which alternates between centroid update and cluster assignment, is primarily used in practice, it may converge to solutions with empty clusters. Furthermore, some applications may require the clusters to satisfy a specific set of constraints, e.g., cluster sizes, must-link/cannot-link. Several methods have been introduced to solve constrained K-Means clustering. Due to the non-convex nature of K-Means, however, existing approaches may result in sub-optimal solutions that poorly approximate the true clusters. In this work, we provide a new perspective to tackle this problem by considering constrained K-Means as a special instance of Binary Optimization. We then propose a novel optimization scheme to search for feasible solutions in the binary domain. This approach allows us to solve constrained K-Means clustering in such a way that multiple types of constraints can be simultaneously enforced. Experimental results on synthetic and real datasets show that our method provides better clustering accuracy with faster run time compared to several existing techniques.
This work was supported by an Asian Office of Aerospace Research and Development Grant FA2386-16-1-4027 and an ARC Future Fellowship FT140101229 to MM. Eriksson was supported by FT170100072.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Althoff, T., Ulges, A., Dengel, A.: Balanced clustering for content-based image browsing. Series of the Gesellschaft fur Informatik, pp. 27–30 (2011)
Bertacco, L., Fischetti, M., Lodi, A.: A feasibility pump heuristic for general mixed-integer problems. Discret. Optim. 4(1), 63–76 (2007)
Bradley, P., Bennett, K., Demiriz, A.: Constrained K-Means Clustering, pp. 1–8. Microsoft Research, Redmond (2000)
Fard, M.M., Thonet, T., Gaussier, E.: Deep \(k\)-means: jointly clustering with \(k\)-means and learning representations. arXiv preprint arXiv:1806.10069 (2018)
Fischetti, M., Glover, F., Lodi, A.: The feasibility pump. Math. Program. 104(1), 91–104 (2005)
Ge, T., He, K., Ke, Q., Sun, J.: Optimized product quantization for approximate nearest neighbor search. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2946–2953. IEEE (2013)
Geißler, B., Morsi, A., Schewe, L., Schmidt, M.: Penalty alternating direction methods for mixed-integer optimization: a new view on feasibility pumps. SIAM J. Optim. 27(3), 1611–1636 (2017)
Gersho, A., Gray, R.M.: Vector Quantization and Signal Compression, vol. 159. Springer, Heidelberg (2012). https://doi.org/10.1007/978-1-4615-3626-0
Jegou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 117–128 (2011)
Johnson, S.C.: Hierarchical clustering schemes. Psychometrika 32(3), 241–254 (1967)
Kalantidis, Y., Avrithis, Y.: Locally optimized product quantization for approximate nearest neighbor search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2321–2328 (2014)
Khan, S.S., Ahmad, A.: Cluster center initialization algorithm for k-means clustering. Pattern Recogn. Lett. 25(11), 1293–1302 (2004)
Le Tan, D.K., Le, H., Hoang, T., Do, T.T., Cheung, N.M.: DeepVQ: a deep network architecture for vector quantization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2579–2582 (2018)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Li, Z., Liu, J.: Constrained clustering by spectral kernel learning. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 421–427. IEEE (2009)
Liu, H., Han, J., Nie, F., Li, X.: Balanced clustering with least square regression (2017)
MacQueen, J., et al.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Oakland, CA, USA, vol. 1, pp. 281–297 (1967)
Mahajan, M., Nimbhorkar, P., Varadarajan, K.: The planar k-means problem is NP-hard. Theor. Comput. Sci. 442, 13–21 (2012)
Malinen, M.I., Fränti, P.: Balanced K-means for clustering. In: Fränti, P., Brown, G., Loog, M., Escolano, F., Pelillo, M. (eds.) S+SSPR 2014. LNCS, vol. 8621, pp. 32–41. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44415-3_4
Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12(Oct), 2825–2830 (2011)
Pena, J.M., Lozano, J.A., Larranaga, P.: An empirical comparison of four initialization methods for the k-means algorithm. Pattern Recogn. Lett. 20(10), 1027–1040 (1999)
Rifkin, R.M., Lippert, R.A.: Notes on regularized least squares (2007)
Wagstaff, K., Cardie, C., Rogers, S., Schroedl, S.: Constrained k-means clustering with background knowledge. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 577–584. Citeseer (2001)
Wright, S., Nocedal, J.: Numerical optimization. Springer Sci. 35(67–68), 7 (1999)
Yang, B., Fu, X., Sidiropoulos, N.D., Hong, M.: Towards k-means-friendly spaces: simultaneous deep learning and clustering. arXiv preprint arXiv:1610.04794 (2016)
Zhu, S., Wang, D., Li, T.: Data clustering with size constraints. Knowl.-Based Syst. 23(8), 883–889 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Le, H.M., Eriksson, A., Do, TT., Milford, M. (2019). A Binary Optimization Approach for Constrained K-Means Clustering. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11364. Springer, Cham. https://doi.org/10.1007/978-3-030-20870-7_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-20870-7_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20869-1
Online ISBN: 978-3-030-20870-7
eBook Packages: Computer ScienceComputer Science (R0)