Synonyms
Overview
Network monitoring applications (e.g., anomaly detection and traffic classification) are among the first sources of big data. With the advent of algorithms and frameworks able to handle datasets of unprecedented scales, researchers and practitioners have the opportunity to face network monitoring problems with novel data-driven approaches. This section summarizes the state of the art on the use of big data approaches for network monitoring. It describes why network monitoring is a big data problem and how the big data approaches are assisting on network monitoring tasks. Open research directions are then highlighted.
Network Monitoring: Goals and Challenges
Monitoring and managing the Internet is more fundamental than ever, since the critical services that rely on the Internet to operate are growing day by day. Monitoring helps administrators to guarantee that the network is working as expected as well as...
This is a preview of subscription content, log in via an institution to check access.
References
Akidau T, Bradshaw R, Chambers C, Chernyak S, Fernández-Moctezuma RJ, Lax R, McVeety S, Mills D, Perry F, Schmidt E, Whittle S (2015) The dataflow model: a practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing. Proc VLDB Endow 8(12):1792–1803
Apache Beam (2017) Apache Beam: an advanced unified programming model. https://beam.apache.org/
Apache Spot (2017) A community approach to fighting cyber threats. http://spot.incubator.apache.org/
Bär A, Finamore A, Casas P, Golab L, Mellia M (2014) Large-scale network traffic monitoring with DBStream, a system for rolling big data analysis. In: Proceedings of the BigData, pp 165–170
Bhuyan MH, Bhattacharyya DK, Kalita JK (2013) Network anomaly detection: methods, systems and tools. Commun Surv Tutorials 16(1):303–336
Callado A, Kamienski C, Szabó G, Gero BP, Kelner J, Fernandes S, Sadok D (2009) A survey on internet traffic identification. Commun Surv Tutorials 11(3): 37–52
Casas P, D’Alconzo A, Zseby T, Mellia M (2016) Big-DAMA: big data analytics for network traffic monitoring and analysis. In: Proceedings of the LANCOMM, pp 1–3
Čermák M, Jirsík T, Laštovička M (2016) Real-time analysis of NetFlow data for generating network traffic statistics using Apache Spark. In: Proceedings of the NOMS, pp 1019–1020
Dean J, Ghemawat S (2004) MapReduce: simplified data processing on large clusters. In: Proceedings of the OSDI, pp 10–10
Fayyad UM, Piatetsky-Shapiro G, Smyth P (1996) From data mining to knowledge discovery: an overview. AI Mag 17(3):37–54
Fontugne R, Mazel J, Fukuda K (2014) Hashdoop: a mapreduce framework for network anomaly detection. In: Proceedings of the INFOCOM WKSHPS, pp 494–499
García-Teodoro P, Díaz-Verdejo J, Maciá-Fernández G, Vázquez E (2009) Anomaly-based network intrusion detection: techniques, systems and challenges. Comput Secur 28:18–28
Hofstede R, Čeleda P, Trammell B, Drago I, Sadre R, Sperotto A, Pras A (2014) Flow monitoring explained: from packet capture to data analysis with NetFlow and IPFIX. Commun Surv Tutorials 16(4):2037–2064
Lee Y, Lee Y (2013) Toward scalable internet traffic measurement and analysis with hadoop. SIGCOMM Comput Commun Rev 43(1):5–13
Liu J, Liu F, Ansari N (2014) Monitoring and analyzing big traffic data of a large-scale cellular network with hadoop. IEEE Netw 28(4):32–39
Marchal S, Jiang X, State R, Engel T (2014) A big data architecture for large scale security monitoring. In: Proceedings of the BIGDATACONGRESS, pp 56–63
Nguyen TT, Armitage G (2008) A survey of techniques for internet traffic classification using machine learning. Commun Surv Tutorials 10(4):56–76
Orsini C, King A, Giordano D, Giotsas V, Dainotti A (2016) BGPStream: a software framework for live and historical BGP data analysis. In: Proceedings of the IMC, pp 429–444
Sperotto A, Schaffrath G, Sadre R, Morariu C, Pras A, Stiller B (2010) An overview of IP flow-based intrusion detection. Commun Surv Tutorials 12(3):343–356
Trevisan M, Finamore A, Mellia M, Munafo M, Rossi D (2017) Traffic analysis with off-the-shelf hardware: challenges and lessons learned. IEEE Commun Mag 55(3):163–169
Valenti S, Rossi D, Dainotti A, Pescapè A, Finamore A, Mellia M (2013) Reviewing traffic classification. In: Data traffic monitoring and analysis – from measurement, classification, and anomaly detection to quality of experience, 1st edn. Springer, Heidelberg
Vanerio J, Casas P (2017) Ensemble-learning approaches for network security and anomaly detection. In: Proceedings of the Big-DAMA, pp 1–6
Wang Y, Ke W, Tao X (2016) A feature selection method for large-scale network traffic classification based on spark. Information 7(1):6
Wullink M, Moura GCM, Müller M, Hesselman C (2016) ENTRADA: a high-performance network traffic data stream. In: Proceedings of the NOMS, pp 913–918
Zaharia M, Chowdhury M, Franklin MJ, Shenker S, Stoica I (2010) Spark: cluster computing with working sets. In: Proceedings of the HotCloud, pp 10–10
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this entry
Cite this entry
Drago, I., Mellia, M., D’Alconzo, A. (2018). Big Data in Computer Network Monitoring. In: Sakr, S., Zomaya, A. (eds) Encyclopedia of Big Data Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-63962-8_26-1
Download citation
DOI: https://doi.org/10.1007/978-3-319-63962-8_26-1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-63962-8
Online ISBN: 978-3-319-63962-8
eBook Packages: Living Reference MathematicsReference Module Computer Science and Engineering
Publish with us
Chapter history
-
Latest
Big Data in Computer Network Monitoring- Published:
- 17 March 2022
DOI: https://doi.org/10.1007/978-3-319-63962-8_26-2
-
Original
Big Data in Computer Network Monitoring- Published:
- 05 February 2018
DOI: https://doi.org/10.1007/978-3-319-63962-8_26-1