Abstract
The pervasiveness of computers in everyday life has already increased and keeps increasing the available digital data both in volume and variety/disparity. This large and dynamic availability of digital data is referred to as Big Data and is very promising in bringing forward new insights and knowledge. For obtaining these insights, the proper combination and processing of the data is required. However, the dynamicity and the increasing size of data start making their handling impossible for analysts and raise many concerns on the manner in which data will be processed from now on. Towards this direction, this paper proposes a tool that processes and combines disparate data in order to create insights regarding a future network load. In particular, the tool (based on the unsupervised machine learning technique of Self-Organizing Maps) builds knowledge on the network load that is encountered with respect to the date of interest, the location, the weather, and the features of the day (e.g., weekend, bank holiday, etc.). The obtained results reveal that the tool is capable of learning the traffic pattern and thus predicting the network load that will be encountered in the near or distant future given information for the above presented parameters with small deviations (up to 0.000553 in terms of Mean Square Error). Moreover, the tool maintains only the most representative data instances and thus reduces the data storage requirements with no loss of information.
Similar content being viewed by others
References
Lohr, S.: The Age of Big Data. New York Times. http://www.nytimes.com/2012/02/12/sunday-review/big-datas-impact-in-the-world.html?_r=1 (2012). Accessed 20 February 2012
European Committee for Standardization Website. http://www.cen.eu/cen/News/Newsletter/Pages/default.aspx. Accessed 17 October 2012
Kohonen, T.: Self-Organizing Maps. Series in Information Sciences, 2nd ed., vol. 30. Springer, Heidelberg (1997)
Kohonen, T.: The self-organizing map. Neurocomputing 21, 1–6 (1998)
Gianchandani, E.: Obama Administration Unveils $200 M Big Data R&D Initiative. The Computing Community Consortium Blog. http://www.cccblog.org/2012/03/29/obama-administration-unveils-200m-big-data-rd-initiative/ (2012). Accessed 26 June 26 2012
Weiss, R., Zgorski, L.-J.: Obama Administration Unveils “Big Data” Initiative: Announces $200 Million in New R&D Investments. Office of Science and Technology Policy Executive Office of the President. http://www.whitehouse.gov/sites/default/files/microsites/ostp/big_data_press_release.pdf (2012). Accessed 4 July 2012
Rouse, M.: Definition: In-memory Database. WhatIs.com. http://whatis.techtarget.com/definition/in-memory-database (2012). Accessed 24 June 2013
Vizard, M.: The Rise of In-Memory Databases. shashdot.org. http://slashdot.org/topic/datacenter/the-rise-of-in-memory-databases/ (2012). Accessed 30 June 2013
Gianchandani, E.: NIST’s BIG DATA Workshop: Too Much Data, Not Enough Solutions. The Computing Community Consortium Blog. http://www.cccblog.org/2012/06/21/nists-big-data-workshoptoo-much-data-not-enough-solutions/ (2012). Accessed 26 June 2012
Manyika, J., Chui, M., Bughin, J., Brown, B., Dobbs, R., Roxburgh, C., Hung Byers, A.: Big Data: The Next Frontier for Innovation, Competition, and Productivity. McKinsey Global Institute, McKinsey&Company, Insights & Publications. http://www.mckinsey.com/Insights/MGI/Research/Technology_and_Innovation/Big_data_The_next_frontier_for_innovation (2011). Accessed 2 July 2012
Sharma, A., Pejovic, V.: Towards, Real-Time Prediction of Network Traffic Load. University of Computer Science, Department of Computer Science. http://www.cs.ucsb.edu/~veljko/docs/cs290d_paper.pdf (2009). Accessed 9 July 2012
Papadopouli, M., Raftopoulos, E., Shen, H.: Evaluation of short-term traffic forecasting algorithms in wireless networks. In: Proceedings of the 2nd Conference on Next Generation Internet Design and Engineering, 3–5 April 2006, Valencia, Spain, pp. 102–109. http://www.ics.forth.gr/mobile/publications/ngi06.pdf (2006). Accessed 5 July 2012
Nae, V., Prodan, R., Fahringer, T.: Neural network-based load prediction for highly dynamic, distributed online games. In: Proceedings of the 14th International European-Parallel Conference on Parallel Processing (Euro-Par ‘08). Lecture Notes in Computer Science (LNCS), vol. 5168, pp. 202–211 (2008)
Fillatre, L., Marakov, D., Vaton, S.: Forecasting seasonal traffic flows. In: Proceedings of the Workshop on QoS and Traffic Control, Paris, France (2005)
Bantouna, A., Tsagkaris, K., Stavroulaki, V., Poulios, G., Demestichas, P.: Learning techniques for context diagnosis and prediction in cognitive communications. In: Zhang H., Grace D. (eds.) Cognitive Communications: Distributed Artificial Intelligence (DAI), Regulatory Policy & Economics, Implementation, pp. 231–256. Wiley, Hoboken (2012)
Qian-Mu, L., Man-Wu, X., Feng-Yu, L.: Network congestion prediction based on RFNN. In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics 2005 (2005). doi:10.1109/ICSMC.2005.1571477
Lim, A. O., Ab-Hamid, K.: Kalman prediction method for congestion avoidance in ATM networks. In: Proceedings of the Technology, Education and Networking Conference (TENCon) 2000, vol. 1, pp. 346–351 (2000)
Valero, S., Aparicio, J., Senabre, C., Ortiz, M., Sancho, J.: Comparative analysis of Self Organizing Maps vs. multilayer perceptron neural networks for short-term load forecasting. In: Proceedings of the International Symposium Modern Electric Power Systems (MEPS), pp. 1–5 (2010)
Aquino, I., Perez, C., Chavez, J. K., Oporto, S.: Daily load forecasting using quick propagation neural network with a special holiday encoding. In: International Joint Conference on Neural Networks (IJCNN) 2007, pp. 1935–1940 (2007)
White paper: Pattern Recognition and Learning Based Self Optimizing Networks. Reverb intelligent SON solutions. http://www.reverbnetworks.com/white-papers (2012). Accessed 9 August 2012
Zhou, J., Fu, Y.: Clustering High-Dimensional Data Using Growing SOM. Advances in Neural Networks—ISNN 2005. Lecture Notes in Computer Science (LNCS), vol. 3497, pp. 63–68 (2005)
Berglund, E., Sitte, J.: The parameterless self-organizing map algorithm. IEEE Trans. Neural Netw. 17(2), 305–316 (2006)
Sarasamma, S.T., Zhu, Q.A., Huff, J.: Hierarchical kohonen net for anomaly detection in network security. IEEE Trans. Syst. Man Cybern. Part B Cybern. 35(2), 302–312 (2005)
Dittenbach, M., Merkl, D., Rauber, A.: Growing hierarchical self-organizing map. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN ’00), vol. 6, pp. 15–19 (2000)
Kuremoto, T., Komoto, T., Kobayashi, K., Obayashi, M.: Parameterless-growing-SOM and its application to a voice instruction learning system. J. Robot. (2010). doi:10.1155/2010/307293
CRAWDAD database: http://crawdad.cs.dartmouth.edu/index.php. Accessed 23 July 2012
The Île Sans Fil website: http://www.ilesansfil.org/. Accessed 10 August 2012
CRAWDAD database: Trace ilesansfil/wifidog/session/04_07 (v. 2007-08-27). http://crawdad.cs.dartmouth.edu/meta.php?name=ilesansfil/wifidog#N1006B. Accessed 10 August 2012
Wunderground.com Website: http://www.wunderground.com/history/. Accessed 15 August 2012
Acknowledgments
The research leading to these results has been performed within the UniverSelf project (www.univerself-project.eu) and received funding from the European Community’s Seventh Framework Programme (FP7/2007-2013) under Grant Agreement No. 257513.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bantouna, A., Poulios, G., Tsagkaris, K. et al. Network Load Predictions Based on Big Data and the Utilization of Self-Organizing Maps. J Netw Syst Manage 22, 150–173 (2014). https://doi.org/10.1007/s10922-013-9285-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10922-013-9285-1