Abstract
Skyline computations are a way of finding the best data points based on multiple criteria for location-based decision-making. However, as the input data grows larger, these computations become slower and more challenging. To address this issue, we propose an efficient algorithm that uses Apache Spark, a platform for distributed processing, to perform area skyline computations faster and more salable. Our algorithm consists of three main phases: calculating distances between data points, generating distance tuples, and computing the skyline. In the second phase, we apply a technique called local partial skyline extraction, which reduces the amount of data that needs to be sent from each executor (a parallel processing unit) to the driver (a central processing unit). The driver then computes the final skyline from the received data and creates filters to eliminate irrelevant points. Our experiments show that our algorithm can significantly reduce the data size and the computation time of the area skyline.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bartling, M., et al.: Adapting mobile map application designs to map use context: a review and call for action on potential future research themes. Cartogr. Geogr. Inf. Sci. 49(3), 237–251 (2022)
Bashabsheh, M.Q., Abualigah, L., Alshinwan, M.: Big data analysis using hybrid meta-heuristic optimization algorithm and mapReduce framework. In: Houssein, E.H., Abd Elaziz, M., Oliva, D., Abualigah, L. (eds.) Integrating Meta-Heuristics and Machine Learning for Real-World Optimization Problems. Studies in Computational Intelligence, vol. 1038, pp. 181–223. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-99079-4_8
Bessani, A.N., et al.: Making hadoop mapReduce byzantine fault-tolerant. DSN, Fast abstract (2010)
Borzsony, S., Kossmann, D., Stocker, K.: The skyline operator. In: Proceedings 17th International Conference on Data Engineering, pp. 421–430. IEEE (2001)
Chauhan, A., Fontama, V., Hart, M., Tok, W.H., Woody, B.: Introducing Microsoft Azure HDInsight. Microsoft press (2014)
Chomicki, J., Godfrey, P., Gryz, J., Liang, D.: Skyline with presorting. ICDE 3, 717–719 (2003)
Grasmann, L., Pichler, R., Selzer, A.: Integration of skyline queries into spark sql. arXiv preprint arXiv:2210.03718 (2022)
Neogi, A.G., Eltaher, A., Sargsyan, A.: NGS data analysis with apache spark. In: Dorpinghaus, J., Weil, V., Schaaf, S., Apke, A. (eds.) Computational Life Sciences. Studies in Big Data, vol. 112, pp. 441–467. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-08411-9_16
Kar, M., Sadhukhan, S., Parida, M.: Location planning of park-and-ride facilities around rapid transit systems in cities: a review. J. Urban Plann. Develop. 149(1), 03122004 (2023)
Lapatta, N.T.: Ecotourism recommendations based on sentiments using skyline query and apache-spark. J. Soc. Sci. 3(3), 534–546 (2022)
Li, C., Annisa, A., Zaman, A., Qaosar, M., Ahmed, S., Morimoto, Y.: MapReduce algorithm for location recommendation by using area skyline query. Algorithms 11(12), 191 (2018)
Li, C., Zaman, A., Morimoto, Y., et al.: MapReduce-based computation of area skyline query for selecting good locations in a map. In: 2017 IEEE International Conference on Big Data (Big Data), pp. 4779–4782. IEEE (2017)
Li, H., Yu, L., Cheng, E.W.: A GIS-based site selection system for real estate projects. Constr. Innov. 5(4), 231–241 (2005)
Pallamala, R.K., Rodrigues, P.: An investigative testing of structured and unstructured data formats in big data application using apache spark. Wireless Pers. Commun. 122(1), 603–620 (2022)
Papanikolaou, I.: Distributed algorithms for skyline computation using apache spark (2020)
Park, Y., Min, J.K., Shim, K.: Parallel computation of skyline and reverse skyline queries using mapReduce. Proceed. VLDB Endow. 6(14), 2002–2013 (2013)
Tan, K.L., Eng, P.K., Ooi, B.C., et al.: Efficient progressive skyline computation. VLDB 1, 301–310 (2001)
Xia, T., Zhang, D., Tao, Y.: On skylining with flexible dominance relation. In: 2008 IEEE 24th International Conference on Data Engineering, pp. 1397–1399. IEEE (2008)
Zaman, A., Morimoto, Y., et al.: Area skyline query for selecting good locations in a map. J. Inf. Process. 24(6), 946–955 (2016)
Zhang, B., Zhou, S., Guan, J.: Adapting skyline computation to the mapReduce framework: algorithms and experiments. In: Xu, J., Yu, G., Zhou, S., Unland, R. (eds.) DASFAA 2011. LNCS, vol. 6637, pp. 403–414. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20244-5_39
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, C. et al. (2023). An Enhanced Distributed Algorithm for Area Skyline Computation Based on Apache Spark. In: Jin, Z., Jiang, Y., Buchmann, R.A., Bi, Y., Ghiran, AM., Ma, W. (eds) Knowledge Science, Engineering and Management. KSEM 2023. Lecture Notes in Computer Science(), vol 14120. Springer, Cham. https://doi.org/10.1007/978-3-031-40292-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-40292-0_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40291-3
Online ISBN: 978-3-031-40292-0
eBook Packages: Computer ScienceComputer Science (R0)