Skip to main content
SpringerLink
Log in

Proposed Solution for Log Collection and Analysis in Kubernetes Environment

  • Conference paper
  • First Online:
Nature of Computation and Communication (ICTCC 2022)

Included in the following conference series:

  • 213 Accesses

Abstract

The aim of the paper is to design and verify a solution for collecting and analysing logs of a distributed application, which is operated as Software as a Service (SaaS) in the cloud environment in Kubernetes technology. Applications running in cloud environment are not monolithic in most cases, but consist of a large number of co-operating microservices. Providing logging for such distributed applications presents a complex issue, where to provide a comprehensive view of the application state, it is necessary to provide logging across all microservices representing the application. This paper first introduces modern approaches for application development using the technical means of virtualization, containerization and orchestration with an emphasis on Kubernetes technology. Next, approaches and analysis of application logging options are presented with the emphasis on the use of ELK and PLG stack technologies. Based on the analysis, a technical solution for logging applications in Kubernetes environment, operated in the form of SaaS, is proposed and verified.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Morley, J., Widdicks, K., Hazas, M.: Digitalisation, energy and data demand: the impact of Internet traffic on overall and peak electricity consumption. Energy Res. Soc. Sci. 38, 128–137 (2018). ISSN 22146296

    Article  Google Scholar 

  2. Abbasi, A.A., Abbasi, A., Shamshirband, S., Chronopoulos, A.T., Persico V., Pescape, A.: Software-defined cloud computing: a systematic review on latest trends and developments (2019). ISSN 2169-3536

    Google Scholar 

  3. Tranos, E., Stich, Ch.: Individual internet usage and the availability of online content of local interest: a multilevel approach. Comput. Environ. Urban Syst. 79, 101371 (2020). ISSN 01989715

    Google Scholar 

  4. Villamizar, M., Garces, O., Castro, H., et al.: Evaluating the monolithic and the microservice architecture pattern to deploy web applications in the cloud. In: Sanchez, M., Gonzalez, O. (eds.) 2015 10th Computing Colombian Conference (10ccc). IEEE, New York (2015). ISBN 978-1-4673-9464-2, iSSN 2378-8216

    Google Scholar 

  5. El Kafhali, S., El Mir, I., Salah, K., Hanini, M.: Dynamic scalability model for containerized cloud services. Arab. J. Sci. Eng. 45(12), 10693–10708 (2020). https://doi.org/10.1007/s13369-020-04847-2. SSN 2193-567X

    Article  Google Scholar 

  6. Piraghaj, S.F., Vahid Dastjerdi, A., Calheiros R.N., Buyya, R.: A survey and taxonomy of energy efficient resource management techniques in platform as a service cloud. In: Handbook of Research on End-to-End Cloud Computing Architecture Design. Advances in Systems Analysis, Software Engineering, and High Performance Computing (2017). ISBN 9781522507598

    Google Scholar 

  7. Senapathi, M., Buchan, J., Osman, H.: DevOps capabilities, practices, and challenges. In: Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering (2018). ISBN 9781450364034

    Google Scholar 

  8. Maroukian, K., Gulliver, S.R.: Leading DevOps practice and principle adoption. In: 9th International Conference on Information Technology Convergence and Services (ITCSE 2020) (2020)

    Google Scholar 

  9. Balalaie, A., Heydarnoori, A., Jamshidi, P.: Microservices architecture enables DevOps migration to a cloud-native architecture. IEEE Computer, Los Alamitos (2016). ISSN 0740-7459

    Google Scholar 

  10. Elbert, C., Gallardo, E., Hernantes, J.; DevOps. IEEE Computer Society, Los Alamitos (2016). ISSN 0740-7459

    Google Scholar 

  11. Pi, A., Chen, W., Zhou, X., Ji, M.: Profiling distributed systems in lightweight virtualized environments with logs and resource metrics. In: Proceedings of the 27th International Symposium on High-Performance Parallel and Distributed Computing (2018). ISBN 9781450357852

    Google Scholar 

  12. Jash, S., Ganesh, R., Rachhadia, T.D., Shah, P.K.: A hierarchical approach to extract application logs with visualization in a containerized environment. In: 2019 International Conference on Computing, Power and Communication Technologies (GUCON) (2019)

    Google Scholar 

  13. Solomon, F.I.: Securing websites web applications using data analytics. In: 2019 International Conference on Computational Intelligence in Data Science (ICCIDS) (2019)

    Google Scholar 

  14. Li, Y., Jiang, Y., Gu, J., et al.: A cloud-based framework for large-scale log mining through apache spark and elasticsearch. Appl. Sci. 9(6) (2019. https://doi.org/10.3390/app9061114). ISSN 2076-3417. Accessed 09 Nov 2021

  15. Lee, B.-H., Yang, D.-M.: A security log analysis system using Logstash based on apache elasticsearch. J. Korea Inst. Inf. Commun. Eng. 22(2), 382–389 (2018)

    Google Scholar 

  16. Shonia, O., Topuria, N., & Kulijanovi, K. Collection and analysis of log data with cloud services. Bull. Georg. Natl. Acad. Sci. (2021)

    Google Scholar 

  17. Mfula, H., Nurminen, J.K.: Self-healing cloud services in private multi-clouds. In: 2018 International Conference on High Performance Computing & Simulation (HPCS), pp. 165–170. IEEE (2018)

    Google Scholar 

  18. Ramos, F., Viegas, E., Santin, A., Horchulhack, P., Dos Santos, R., Espindola, A.: A machine learning model for detection of Docker-based APP overbooking on kubernetes. In: ICC 2021 - IEEE International Conference on Communications (2021). ISBN 978-1-7281-7122-7

    Google Scholar 

  19. Pradhan, R., Dash, A.K.: An Overview of Microservices. Lecture Notes in Electrical Engineering, vol. 601. Springer, Singapore (2020)

    Google Scholar 

  20. Pahl, C., Jamshidi, P., Zimmermann, O.: Microservices and containers. Softw. Eng. 2020 (2020)

    Google Scholar 

  21. Srirama, S.N., Adhikari, M., Paul, S.: Application deployment using containers with auto-scaling for microservices in cloud environment. J. Netw. Comput. Appl. 160, 102629 (2020). ISSN 10848045

    Google Scholar 

  22. Abdullah, M., Iqbal, W., Bukhari, F.: Containers vs virtual machines for auto-scaling multi-tier applications under dynamically increasing workloads. In: Bajwa, I.S., Kamareddine, F., Costa, A. (eds.) INTAP 2018. CCIS, vol. 932, pp. 153–167. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-6052-7_14 ISBN 9789811360527, ISSN 1865-0929

    Chapter  Google Scholar 

  23. Zhang, Q., Liu, L., Pu, C., et al.: A comparative study of containers and virtual machines in big data environment. New York (2018). ISBN 978-1-5386-7235-8

    Google Scholar 

  24. Khan, A.: Key characteristics of a container orchestration platform to enable a modern application. IEEE Cloud Computing (2017). ISSN 2325-6095

    Google Scholar 

  25. Muddinagiri, R., Ambavane, S., Bayas, S.: Self-hosted kubernetes: deploying docker containers locally with minikube. In: 2019 International Conference on Innovative Trends and Advances in Engineering and Technology (ICITAET) (2019)

    Google Scholar 

  26. Dewi, L.P., Noertjahyana, A., Palit, H.N., Yedutun, K.: Server scalability using kubernetes. In: 2019 4th Technology Innovation Management and Engineering Science International Conference (TIMES-iCON) (2019). ISBN 978-1-7281-3755-1

    Google Scholar 

  27. Ferreira, A.P., Sinnott, R.: A performance evaluation of containers running on managed kubernetes services. In 2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom) (2019). ISSN 2330-2186

    Google Scholar 

  28. Vayghan, L.A., Saied, M.A., Toeroe, M., et al.: Deploying microservice based applications with kubernetes: experiments and lessons learned. In: 2018 IEEE 11th International Conference on Cloud Computing (CLOUD) (2018). ISSN 2159-6190

    Google Scholar 

  29. Vayghan, L.A., Saied, M.A., Toeroe, M., et al.: Microservice based architecture: towards high-availability for stateful applications with kubernetes. In: 2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS) (2019)

    Google Scholar 

  30. Pichan, A., Lazarescu, M., Soh, S.T.: Towards a practical cloud forensics logging framework. J. Inf. Secur. Appl. 42, 18–28 (2019). ISSN 22142126

    Google Scholar 

  31. Jayathilaka, H., Krintz, C., Wolski, R.: Performance monitoring and root cause analysis for cloud-hosted web applications. Association for Computing Machinery, New York (2017). ISBN 978-1-4503-4913-0

    Google Scholar 

  32. Lamouchi, N.: Flying All Over the Sky with Quarkus and Kubernetes. In: Lamouchi, N. (ed.) Pro Java Microservices with Quarkus and Kubernetes, pp. 363–395. Apress, Berkeley, (2021). https://doi.org/10.1007/978-1-4842-7170-4_14 ISBN 978-1-4842-7169-8

    Chapter  Google Scholar 

  33. Dooley, R., Brandt, S., Liang, K., Tanner, E. Experiences migrating the agave platform to a kubernetes native system on the jetstream cloud. In: Practice and Experience in Advanced Research Computing, 17 July 2021, pp. 1–4. ACM, New York (2021). https://doi.org/10.1145/3437359.346559. ISBN 9781450382922. Accessed 09 Nov 2021

Download references

Acknowledgment

The research has been supported by the Faculty of Informatics and Management UHK specific research project 2107 Integration of Departmental Research Activities and Students’ Research Activities Support II. We would like to thank Mr. P. Kratochvil, a graduate of Faculty of management and informatics, University of Hradec Kralove, for the practical verification of the proposed solutions and close cooperation in the solution.

Author information

Authors and Affiliations

  1. Faculty of Management and Informatics, University of Hradec Kralove, Hradec Kralove, Czech Republic

    Josef Horalek, Patrik Urbanik, Vladimir Sobeslav & Tomas Svoboda

Authors
  1. Josef Horalek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patrik Urbanik
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vladimir Sobeslav
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tomas Svoboda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladimir Sobeslav .

Editor information

Editors and Affiliations

  1. Nguyen Tat Thanh University, Ho Chi Minh City, Vietnam

    Cong Vinh Phan

  2. Mekong University, Vinh Long Province, Vietnam

    Thanh Dung Nguyen

Rights and permissions

Reprints and permissions

Copyright information

© 2023 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Horalek, J., Urbanik, P., Sobeslav, V., Svoboda, T. (2023). Proposed Solution for Log Collection and Analysis in Kubernetes Environment. In: Phan, C.V., Nguyen, T.D. (eds) Nature of Computation and Communication. ICTCC 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 473. Springer, Cham. https://doi.org/10.1007/978-3-031-28790-9_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-28790-9_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-28789-3

  • Online ISBN: 978-3-031-28790-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation