Abstract
Existing approaches for query processing are insufficient to provide accurate results quickly in a big data environment due to the needing to identify approximation measures and execution time for the queries are increasing. The practical results are obtained through the clustering process. However, how to scale up and speed up clustering algorithms with minimum sacrifice to the clustering quality remain a big problem. The main contribution of this work is to propose a modified dynamic distributed clustering approach applicable in spatial query processing for big data. The dynamic nature of the approach comes from the fact that it does not need to give the number of correct clusters in advance. The suggested model works through two main phases: the first phase builds local clusters based on its portion of the entire dataset; this stage takes full advantage of task parallelism paradigm based on Spark framework; whereas the second phase aggregate the local clusters in such a way that the final clusters are compact and accurate. Experimental results show that the approach is scalable and produces results of high quality for spatial query processing.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zulkarnain, N., Anshari, M.: Big data: concept, applications, & challenges. In: International Conference on Information Management and Technology, Indonesia, pp. 307–310 (2016)
Suganya, R., Pavithra, M., Nandhini, P.: Algorithms and challenges in big data clustering. Int. J. Eng. Tech. 4(4), 40–47 (2018)
Tang, S., He, B., Yu, C., Li, Y., Li, K.: A Survey on Spark Ecosystem for Big Data Processing, pp. 1–21 (2018). arXiv:1811.08834v1
Fahad, A., Alshatri, N., Tari, Z., Alamri, A.: A survey of clustering algorithms for big data: taxonomy and empirical analysis. IEEE Trans. Emerg. Top. Comput. 2(3), 267–279 (2014)
Yue, X., Man, W., Yue, J., Liu, G.: Parallel K-Medoids++ Spatial Clustering Algorithm based on MapReduce, pp. 1–8 (2016). arXiv:1608.06861
Jiang, Y., Zhang, J.: Parallel K-medoids clustering algorithm based on Hadoop. In: IEEE 5th International Conference on Software Engineering and Service Science, China, pp. 649–652 (2014)
Martino, A., Rizzi, A., Mascioli, F.: Efficient approaches for solving the large-scale k-medoids problem. In: 9th International Joint Conference on Computational Intelligence, Portugal, pp. 338–347 (2017)
Martino, A., Rizzi, A., Mascioli, F.: Distance matrix pre-caching and distributed computation of internal validation indices in k-medoids clustering. In: International Joint Conference on Neural Networks, Brazil, pp. 1–8 (2018)
Kusuma, I., Ma’sum, M., Habibie, N., Jatmiko, W., Suhartanto, H.: Design of intelligent K-means based on spark for big data clustering. In: International Workshop on Big Data and Information Security, Indonesia, pp. 89–96 (2016)
Song, H., Lee, J., Han, W.: PAMAE: parallel k-medoids clustering with high accuracy and efficiency. In: 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Canada, pp. 1087–1096 (2017)
Bendechache, M., Le-Khac, N., Kechadi, M.: Efficient large scale clustering based on data partitioning. In: International Conference on Data Science and Advanced Analytics, Canada, pp. 612–621 (2016)
Bendechache, M., Le-Khac, N., Kechadi, M.: Performance evaluation of a distributed clustering approach for spatial datasets. In: 15th Australasian Data Mining Conference, Canada, pp. 38–56 (2018)
Kamaruddin, S., Ravi, V., Mayank, P.: Parallel evolving clustering method for big data analytics using apache spark: applications to banking and physics. In: International Conference on Big Data Analytics, Switzerland, pp. 278–292 (2017)
Shah, J.: News clustering using spark. Int. J. Latest Technol. Eng. Manag. Appl. Sci. 5(9), 58–61 (2016)
Laloux, J.-F., Le-Khac, N.-A., and Kechadi, M.-T.: Efficient distributed approach for density-based clustering. In: IEEE 20th International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises, France, pp. 145–150 (2011)
Bai, J.-W., Wang, J.-Z., Huang, J.-L.: Spatial query processing on distributed databases. Adv. Intell. Syst. Appl. 1, 251–260 (2013)
Shah, P., Chaudhary, S.: Big data analytics framework for spatial data. In: Lecture Notes in Computer Science, vol. 11297, pp. 250–265 (2018)
Narayana, G., Vasumathi, D.: An attributes similarity-based k-medoids clustering technique in data mining. Arab. J. Sci. Eng. 43(8), 3979–3992 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Darwish, S.M., Elsaidy, R.D., Mesbah, S. (2020). A Modified Query Processing Algorithm Based on Dynamic Clustering for Big Data Applications. In: Hassanien, AE., Azar, A., Gaber, T., Oliva, D., Tolba, F. (eds) Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2020). AICV 2020. Advances in Intelligent Systems and Computing, vol 1153. Springer, Cham. https://doi.org/10.1007/978-3-030-44289-7_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-44289-7_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-44288-0
Online ISBN: 978-3-030-44289-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)