Abstract
In this paper, one of the most important aspects of solar energy resources which are solar irradiation is addressed. A fast analysis and calculation of time series solar irradiation using Hadoop integrated programming environment (RHIPE) is performed. The proposed work is evaluated using data provided by London weather centre for the period from (1996–2005) that includes monthly mean daily irradiation on inclined planes in terms of scalability, speedup and accuracy. The speedup of RHIPE in computation is analysed through Gustafson’s law. This law is revised to enhance its capability to analyse the performance gain in computation for parallelizing data in a cluster computing environment. The experimental results showed that RHIPE improves the performance analysis from the aspects of speedup, scalability and accuracy in comparison with R language. After the revision of Gustafson’s law, the speedup of RHIPE was evaluated using Different sizes of data blocks ranging from 8 to 64 MB and showed that the execution time of RHIPE decreases with an increasing size of data blocks while the speedup of RHIPE increases. Furthermore, the results showed that the speedup of RHIPE is 2.2 times faster in computation using 64 than when using 8 MB.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gowri, R., Rathipriya, R.: MR-GABiT: map reduce based genetic algorithm for biclustering time series data. In: IEEE International Conference on Advances in Computer Applications (ICACA), pp. 381–387 (2016)
Reinert, G.: Time Series, Hilar. Term, pp. 1–66. USA (2010). http://www.stats.ox.ac.uk/~reinert/time/notesht10short.pdf
Prajapati, V.: Big Data Analytics with R and Hadoop. Packt Publishing Ltd. (2013)
Wang, X., Yan, Z., Li, L.: A grid computing based approach for the power system dynamic security assessment. Comput. Electr. Eng. 36(3), 553–564 (2010)
Ezhilarasi, G.A., Swarup, K.: Parallel contingency analysis in a high performance computing environment. Int. Conf. Power Syst. 2009, 1–6 (2009)
Gabriel, E., et al. : Open MPI: goals, concept, and design of a next generation MPI implementation. In: 11th European PVM/MPI Users’ Group Meeting, pp. 97–104 (2004)
Gorton, I., et al.: A high-performance hybrid computing approach to massive contingency analysis in the power grid. In: e-Science 2009—5th IEEE International Conference on e-Science, pp. 277–283 (2009)
Gao, W., Chen, X.: Distributed generation placement design and contingency analysis with parallel computing technology. Fuel 4(4), 347–354 (2009)
Yin, J., Liao, Y., Baldi, M., Gao, L., Nucci, A.: GOM-Hadoop: a distributed framework for efficient analytics on ordered datasets. J. Parallel Distrib. Comput. 83, 58–69 (2015)
Li, J., Ma, X., Yoginath, S., Kora, G., Samatova, N.F.: Transparent runtime parallelization of the R scripting language. J. Parallel Distrib. Comput. 71(2), 157–168 (2011)
Agrawal, B., Chakravorty, A., Rong, C., Wlodarczyk, T.W.: R2Time: a framework to analyse open TSDB Time-series data in HBase. In: Proceedings of the International Conference on Cloud Computing Technology and Science, CloudCom (2015)
Di Geronimo, L., Ferrucci, F., Murolo, A., Sarro, F.: A parallel genetic algorithm based on hadoop MapReduce for the automatic generation of junit test suites. In: Proceedings—IEEE 5th International Conference on Software Testing, Verification and Validation, p. 2012. ICST, Limerick (2012)
Li, L., et al.: Rolling window time series prediction using MapReduce. In: 2014 IEEE 15th International Conference on Information Reuse and Integration (IRI), pp. 757–764 (2014)
Liu, Y.Y., Thulasiraman, P., Thulasiram, R.K.: Parallelizing active memory ants with MapReduce for clustering financial time series data. In: 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom) (2016)
Sheng, C., Zhao, J., Leung, H., Wang, W.: Extended Kalman filter based echo state network for time series prediction using MapReduce framework
Bach, F., Çakmak, H.K., Maass, H., Kuehnapfel, U.: Power grid time series data analysis with pig on a Hadoop cluster compared to multi core systems. In: Proceedings of the 2013 International Conference on Parallel, Distributed and Network-Based Processing. PDP 2013, pp. 208–212 (2013)
Brutlag, J.D.: 14th systems administration conference aberrant behavior detection in time series for network monitoring. In: Proceedings of the 14th Systems Administration Conference (LISA 2000), no. 14, pp. 139–146 (2000)
Yin, H., Yang, S., Ma, S., Liu, F., Chen, Z.: A novel parallel scheme for fast similarity search in large time series. China Commun. 12(2), 129–140 (2015)
Laurinec, P., Lóderer, M., Vrablecová, P., Lucká, M., Rozinajová, V., Ezzeddine, A.B.: Adaptive Time Series Forecasting of Energy Consumption using Optimized Cluster Analysis (2016)
Manikandan, S.G.: Big data analysis using apache hadoop
Isard, M., Budiu, M., Yu, Y., Birrell, A., Fetterly, D.: Dryad: distributed data-parallel programs from sequential building blocks. ACM SIGOPS Oper. Syst. Rev. 59–72 (2007)
He, B., Fang, W., Luo, Q., Govindaraju, N.K., Wang, T.: Mars: a MapReduce framework on graphics processors. In: Proceedings of the 17th International Conference on Parallel Architecture Compilation. Techniques—PACT ’08, p. 260 (2008)
Ranger, C., Raghuraman, R., Penmetsa, A., Bradski, G., Kozyrakis, C.: Evaluating MapReduce for multi-core and multiprocessor systems. In: Proceedings—International Symposium on High-Performance Computer Architecture, pp. 13–24 (2007)
“Welcome to ApacheTM Hadoop®!” [Online]. Available: http://hadoop.apache.org/. Accessed: 18 Sept 2017
Patel, A.B., Birla, M., Nair, U.: Addressing big data problem using Hadoop and Map Reduce. In: 3rd Nirma University International Conference on Engineering, NUiCONE (2012)
Sun, X.H., Chen, Y.: Reevaluating Amdahl’s law in the multicore era. J. Parallel Distrib. Comput. 70(2), 183–188 (2010)
Acknowledgements
The authors would like to thank the Iraqi Ministry of Higher Education and Scientific Research to financially sponsor the current research and study. The authors also thank Brunel University London for providing the efficient environment to implement this work experimentally.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Khaleel, A., Al-Raweshidy, H.S. (2019). Solar Radiation Time Series Analytics Based on Hadoop Integrated Programming Language Environment (RHIPE). In: Arai, K., Kapoor, S., Bhatia, R. (eds) Intelligent Systems and Applications. IntelliSys 2018. Advances in Intelligent Systems and Computing, vol 869. Springer, Cham. https://doi.org/10.1007/978-3-030-01057-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-01057-7_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01056-0
Online ISBN: 978-3-030-01057-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)