Abstract
Although the k-means algorithm has been parallelized into different platforms, it has not yet been explored on multi-GPU architecture thoroughly. This paper presents a study of parallelizing k-means on a novel MGPUSim architecture, including its parallel execution mechanism, architecture design, etc. In addition, it proposes an optimization method “O-kmeans” to initialize the selection of clustering centers by first finding the centroids of the samples and then dividing the initialized clustering centers with centroids, thus solving the problem of poor clustering effect of the k-means algorithm when the data size is large. The performance of this algorithm is tested with both real and synthetic datasets. The experimental results show that:(1) The proposed O-kmeans algorithm performs well on the MGPUSim. It can achieve a 26.74×–62.92× speedup for real data sets, which is better than the CUDA implementation of kernel k-means. (2) In synthetic datasets, by conducting controlled variable experiments at varying data sizes and data dimensions, and different clustering centers. We find that the algorithm has higher stability and good processing speed on MGPUSim.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zaki, M.J.: Parallel and distributed data mining: an introduction. In: Zaki, M.J., Ho, C.-T. (eds.) LSPDM 1999. LNCS (LNAI), vol. 1759, pp. 1–23. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-46502-2_1
Cuomo, S., et al.: A GPU-accelerated parallel K-means algorithm. Comput. Elect. Eng. 75, 262–274 (2017)
Wang, Z., et al.: The parallelization and optimization of k-means algorithm based on spark. In: 2020 15th International Conference on Computer Science & Education (2020)
Bachem, O., et al.: Scalable k-means clustering via lightweight coresets. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2018)
Yang, L., et al.: High performance data clustering: a comparison analysis of performance for GPU, RASC, MPI, and OpenMP implementation. Parallel Distrib. Process. Techn. App. 70, 284–300 (2010)
Hall, J., et al.: GPU Acceleration of Iterative Clustering. Siggraph Poster (2004)
Li, Y., Zhao, K., Chu, X., Liu, J.: Speeding up the k-means algorithm by GPUs. J. Comput. Syst. Sci. 79(2), 216–229 (2013)
Sun, Y., et al.: MGPUSim: enabling multi-GPU performance modeling and optimization. In: Proceedings of the 46th International Symposium on Computer Architecture (2019)
Ausavarungnirun, R., et al.: Mosaic: a GPU memory manager with application-transparent support for multiple page sizes. In: the 50th Annual IEEE/ACM International Symposium ACM (2017)
Sun, Y., et al.: Daisen: a framework for visualizing detailed GPU execution. Comput. Graph. Forum. 40(3), 1–12 (2021)
Baydoun, M., Dawi, M., Ghaziri, H.: Enhanced parallel implementation of the K-means clustering algorithm. In: Advances in Computational Tools for Engineering Applications (ACTEA), 2016 3rd International Conference on IEEE, pp. 7–11 (2016)
Bhimani, J., Leeser, M., Mi, N.: Accelerating K-means clustering with parallel implementations and GPU computing. In: High Performance Extreme Computing Conference (HPEC), pp. 1–6 (2015)
Nelson, J., and Roberto, P.: Don't forget about synchronization! A case study of K-means on GPU. In: Proceedings of the 10th International Workshop on Programming Models and Applications for Multicores and Manycores- PMAM'192019, pp. 11–20 (2019)
Daoudi, S., et al.: A Comparative study of parallel CPU/GPU implementations of the K-means algorithm. In: 2019 International Conference on Advanced Electrical Engineering (ICAEE). IEEE (2019)
Salvatore, C., De Angelis, V., Gennaro, F., Livia, M., Gerardo, T.: A GPU-accelerated parallel K-means algorithm. Comput. Elect. Eng. 75, 262–274 (2019)
Baydoun, M., Ghaziri, H., Al-Husseini, M.: CPU and GPU parallelized kernel K-means. J. Supercomput. 74(8), 3975–3998 (2018). https://doi.org/10.1007/s11227-018-2405-7
Kruliš, M., Miroslav, K.: Detailed analysis and optimization of CUDA K-means algorithm. In: 49th International Conference on Parallel Processing-ICPP (2020)
Sun, Y., et al.: MGSim+MGMark: a framework for multi-GPU system research. arXiv preprint arXiv:1811.02884 (2018)
Young, V., et al.: Combining HW/SW mechanisms to improve NUMA performance of multi-GPU systems. In: 2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), IEEE (2018)
Lee, S., Won, W.R.: Parallel gpu architecture simulation framework exploiting architectural-level parallelism with timing error prediction. IEEE Trans. Comput. 65(4), 1253–1265 (2015)
Michie, D., Spiegelhalter, D.J., Taylor, C.C.: Machine Learning, Neural and Statistical Classification. Ellis Horwood, Amsterdam (1994)
Blake, C., Merz, C.J.: UCI repository of machine learning databases. University of California, Irvine
Acknowledgment
This work has been supported by a grant from the National Natural Science Foundation of China General Program (61672438) and the Special Project of the China Association of Higher Education (21SZYB16).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mo, Z. et al. (2022). The Parallelization and Optimization of K-means Algorithm Based on MGPUSim. In: Pimenidis, E., Angelov, P., Jayne, C., Papaleonidas, A., Aydin, M. (eds) Artificial Neural Networks and Machine Learning – ICANN 2022. ICANN 2022. Lecture Notes in Computer Science, vol 13532. Springer, Cham. https://doi.org/10.1007/978-3-031-15937-4_26
Download citation
DOI: https://doi.org/10.1007/978-3-031-15937-4_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15936-7
Online ISBN: 978-3-031-15937-4
eBook Packages: Computer ScienceComputer Science (R0)