Skip to main content
SpringerLink
Log in

Network Load Predictions Based on Big Data and the Utilization of Self-Organizing Maps

  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

The pervasiveness of computers in everyday life has already increased and keeps increasing the available digital data both in volume and variety/disparity. This large and dynamic availability of digital data is referred to as Big Data and is very promising in bringing forward new insights and knowledge. For obtaining these insights, the proper combination and processing of the data is required. However, the dynamicity and the increasing size of data start making their handling impossible for analysts and raise many concerns on the manner in which data will be processed from now on. Towards this direction, this paper proposes a tool that processes and combines disparate data in order to create insights regarding a future network load. In particular, the tool (based on the unsupervised machine learning technique of Self-Organizing Maps) builds knowledge on the network load that is encountered with respect to the date of interest, the location, the weather, and the features of the day (e.g., weekend, bank holiday, etc.). The obtained results reveal that the tool is capable of learning the traffic pattern and thus predicting the network load that will be encountered in the near or distant future given information for the above presented parameters with small deviations (up to 0.000553 in terms of Mean Square Error). Moreover, the tool maintains only the most representative data instances and thus reduces the data storage requirements with no loss of information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Lohr, S.: The Age of Big Data. New York Times. http://www.nytimes.com/2012/02/12/sunday-review/big-datas-impact-in-the-world.html?_r=1 (2012). Accessed 20 February 2012

  2. European Committee for Standardization Website. http://www.cen.eu/cen/News/Newsletter/Pages/default.aspx. Accessed 17 October 2012

  3. Kohonen, T.: Self-Organizing Maps. Series in Information Sciences, 2nd ed., vol. 30. Springer, Heidelberg (1997)

  4. Kohonen, T.: The self-organizing map. Neurocomputing 21, 1–6 (1998)

    Article  MATH  Google Scholar 

  5. Gianchandani, E.: Obama Administration Unveils $200 M Big Data R&D Initiative. The Computing Community Consortium Blog. http://www.cccblog.org/2012/03/29/obama-administration-unveils-200m-big-data-rd-initiative/ (2012). Accessed 26 June 26 2012

  6. Weiss, R., Zgorski, L.-J.: Obama Administration Unveils “Big Data” Initiative: Announces $200 Million in New R&D Investments. Office of Science and Technology Policy Executive Office of the President. http://www.whitehouse.gov/sites/default/files/microsites/ostp/big_data_press_release.pdf (2012). Accessed 4 July 2012

  7. Rouse, M.: Definition: In-memory Database. WhatIs.com. http://whatis.techtarget.com/definition/in-memory-database (2012). Accessed 24 June 2013

  8. Vizard, M.: The Rise of In-Memory Databases. shashdot.org. http://slashdot.org/topic/datacenter/the-rise-of-in-memory-databases/ (2012). Accessed 30 June 2013

  9. Gianchandani, E.: NIST’s BIG DATA Workshop: Too Much Data, Not Enough Solutions. The Computing Community Consortium Blog. http://www.cccblog.org/2012/06/21/nists-big-data-workshoptoo-much-data-not-enough-solutions/ (2012). Accessed 26 June 2012

  10. Manyika, J., Chui, M., Bughin, J., Brown, B., Dobbs, R., Roxburgh, C., Hung Byers, A.: Big Data: The Next Frontier for Innovation, Competition, and Productivity. McKinsey Global Institute, McKinsey&Company, Insights & Publications. http://www.mckinsey.com/Insights/MGI/Research/Technology_and_Innovation/Big_data_The_next_frontier_for_innovation (2011). Accessed 2 July 2012

  11. Sharma, A., Pejovic, V.: Towards, Real-Time Prediction of Network Traffic Load. University of Computer Science, Department of Computer Science. http://www.cs.ucsb.edu/~veljko/docs/cs290d_paper.pdf (2009). Accessed 9 July 2012

  12. Papadopouli, M., Raftopoulos, E., Shen, H.: Evaluation of short-term traffic forecasting algorithms in wireless networks. In: Proceedings of the 2nd Conference on Next Generation Internet Design and Engineering, 3–5 April 2006, Valencia, Spain, pp. 102–109. http://www.ics.forth.gr/mobile/publications/ngi06.pdf (2006). Accessed 5 July 2012

  13. Nae, V., Prodan, R., Fahringer, T.: Neural network-based load prediction for highly dynamic, distributed online games. In: Proceedings of the 14th International European-Parallel Conference on Parallel Processing (Euro-Par ‘08). Lecture Notes in Computer Science (LNCS), vol. 5168, pp. 202–211 (2008)

  14. Fillatre, L., Marakov, D., Vaton, S.: Forecasting seasonal traffic flows. In: Proceedings of the Workshop on QoS and Traffic Control, Paris, France (2005)

  15. Bantouna, A., Tsagkaris, K., Stavroulaki, V., Poulios, G., Demestichas, P.: Learning techniques for context diagnosis and prediction in cognitive communications. In: Zhang H., Grace D. (eds.) Cognitive Communications: Distributed Artificial Intelligence (DAI), Regulatory Policy & Economics, Implementation, pp. 231–256. Wiley, Hoboken (2012)

  16. Qian-Mu, L., Man-Wu, X., Feng-Yu, L.: Network congestion prediction based on RFNN. In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics 2005 (2005). doi:10.1109/ICSMC.2005.1571477

  17. Lim, A. O., Ab-Hamid, K.: Kalman prediction method for congestion avoidance in ATM networks. In: Proceedings of the Technology, Education and Networking Conference (TENCon) 2000, vol. 1, pp. 346–351 (2000)

  18. Valero, S., Aparicio, J., Senabre, C., Ortiz, M., Sancho, J.: Comparative analysis of Self Organizing Maps vs. multilayer perceptron neural networks for short-term load forecasting. In: Proceedings of the International Symposium Modern Electric Power Systems (MEPS), pp. 1–5 (2010)

  19. Aquino, I., Perez, C., Chavez, J. K., Oporto, S.: Daily load forecasting using quick propagation neural network with a special holiday encoding. In: International Joint Conference on Neural Networks (IJCNN) 2007, pp. 1935–1940 (2007)

  20. White paper: Pattern Recognition and Learning Based Self Optimizing Networks. Reverb intelligent SON solutions. http://www.reverbnetworks.com/white-papers (2012). Accessed 9 August 2012

  21. Zhou, J., Fu, Y.: Clustering High-Dimensional Data Using Growing SOM. Advances in Neural Networks—ISNN 2005. Lecture Notes in Computer Science (LNCS), vol. 3497, pp. 63–68 (2005)

  22. Berglund, E., Sitte, J.: The parameterless self-organizing map algorithm. IEEE Trans. Neural Netw. 17(2), 305–316 (2006)

    Article  Google Scholar 

  23. Sarasamma, S.T., Zhu, Q.A., Huff, J.: Hierarchical kohonen net for anomaly detection in network security. IEEE Trans. Syst. Man Cybern. Part B Cybern. 35(2), 302–312 (2005)

    Article  Google Scholar 

  24. Dittenbach, M., Merkl, D., Rauber, A.: Growing hierarchical self-organizing map. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN ’00), vol. 6, pp. 15–19 (2000)

  25. Kuremoto, T., Komoto, T., Kobayashi, K., Obayashi, M.: Parameterless-growing-SOM and its application to a voice instruction learning system. J. Robot. (2010). doi:10.1155/2010/307293

    Google Scholar 

  26. CRAWDAD database: http://crawdad.cs.dartmouth.edu/index.php. Accessed 23 July 2012

  27. The Île Sans Fil website: http://www.ilesansfil.org/. Accessed 10 August 2012

  28. CRAWDAD database: Trace ilesansfil/wifidog/session/04_07 (v. 2007-08-27). http://crawdad.cs.dartmouth.edu/meta.php?name=ilesansfil/wifidog#N1006B. Accessed 10 August 2012

  29. Wunderground.com Website: http://www.wunderground.com/history/. Accessed 15 August 2012

Download references

Acknowledgments

The research leading to these results has been performed within the UniverSelf project (www.univerself-project.eu) and received funding from the European Community’s Seventh Framework Programme (FP7/2007-2013) under Grant Agreement No. 257513.

Author information

Authors and Affiliations

  1. Department of Digital Systems, University of Piraeus (UPRC), Piraeus, Greece

    Aimilia Bantouna, Giorgos Poulios, Kostas Tsagkaris & Panagiotis Demestichas

Authors
  1. Aimilia Bantouna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giorgos Poulios
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kostas Tsagkaris
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Panagiotis Demestichas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aimilia Bantouna.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bantouna, A., Poulios, G., Tsagkaris, K. et al. Network Load Predictions Based on Big Data and the Utilization of Self-Organizing Maps. J Netw Syst Manage 22, 150–173 (2014). https://doi.org/10.1007/s10922-013-9285-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10922-013-9285-1

Keywords

Search

Navigation