Skip to main content
Springer Nature Link
Log in

Adaptive Monitoring: Application of Probing to Adapt Passive Monitoring

  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

Availability of good quality monitoring data is a vital need for management of today’s data centers. However, effective use of monitoring tools demands an understanding of the monitoring requirements that system administrators most often lack. Instead of a well-defined process of defining a monitoring strategy, system administrators adopt a manual and intuition-based approach. In this paper, we propose to replace the ad-hoc, manual, intuition-based approach with a more systematic, automated, and analytics-based approach for system monitoring. We propose an adaptive monitoring framework where end-to-end probing-based solutions are used to adapt the at-a-point monitoring tools. We present a systematic framework to use probes for adjusting monitoring levels. We present algorithms to select and analyze probes and to dynamically adapt the monitoring policies based on probe analysis. We demonstrate the effectiveness of the proposed solution using real-world examples as well as simulations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Natu, M., Sethi, A.S.: Application of adaptive probing for fault diagnosis in computer networks. In: Proceedings of NOMS’08, Brazil (2008)

  2. Natu, M., Sethi, A.S.: Probabilistic fault diagnosis using adaptive probing. In: Proceedings of DSOM 2007, San Jose, CA (2007)

  3. Motwani, R., Raghavan, P.: Randomized algorithms. Cambridge University Press, ISBN 978-0-521-47465-8 (1995)

  4. Zhang, T., Ramakrishnan, R., Livny, M.: Birch: an efficient data clustering method for very large databases. In: Proceedings of SIGMOD’96, Montreal, Canada, pp. 103–114 (1996)

  5. Yoon, H., Yang, K., Shahabi, C.: Feature subset selection and feature ranking for multivariate time series. IEEE Trans. Knowl. Data Eng. 17, 1186–1198 (2005)

    Article  Google Scholar 

  6. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  7. Jolliffe, I.: Principal Component Analysis. Springer, Berlin (1986)

    Book  MATH  Google Scholar 

  8. Stewart, G.W.: On the early history of the singular value decomposition. SIAM Rev. 35(4), 551–566 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  9. Ithaka, R., Gentleman, R.: R: a language for data analysis and graphics. J. Comput. Gr. Stat. 5(3), 299–314 (1996)

    Google Scholar 

  10. Medina, A., Lakhina, A., Matta, I., Byers, J.: Brite: an approach to universal topology generation. In: MASCOTS’01. Cincinnati, Ohio (2001)

  11. Schwetman, H.: Csim: a C-based process-oriented simulation language. In: Proceedings of WSC ’86 (1986)

  12. Scherr, A.: An analysis of time shared computer systems. MIT Press, Cambridge (1967)

    Google Scholar 

  13. Brodie, M., Rish, I., Ma, S., Grabarnik, G., Odintsova, N.: Active probing. Technical report, IBM (2002)

  14. Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. In: Proceedings of SIGCOMM’04, Portland, Oregon, USA (2004)

  15. Groenendijk, J., Huang, Y., Fallon, L.: Adaptive terminal reporting for scalable service quality monitoring in large networks. In: Proceedings of CNSM 2011, Paris, France (2011)

  16. Bhatia, S., Kumar, A., Fiuczynski, M.E., Peterson, L.: Lightweight, high-resolution monitoring for troubleshooting production systems. In: Proceedings of the 8th USENIX conference on Operating systems design and implementation, San Diego, California, ser. OSDI’08 (2008)

  17. Gaspary, L.P., Canterle, E.: Assessing transaction-based internet applications performance through a passive network traffic monitoring approach. In: Proceedings of GLOBECOM’04, Dallas, USA (2004)

  18. Han, S.-H., Kim, M.-S., Ju, H.-T., Hong, W.-K.J.: The architecture of ng-mon: a passive network monitoring system for high-speed ip networks. In: Proceedings of DSOM’02, Montreal, Canada (2002)

  19. Yu, L., Cheng, L., Qiao, Y., Yuan, Y., Chen, X.: An efficient active probing approach based on the combination of online and offline strategies. In: Proceedings of CNSM’10, pp. 298–301 (2010)

  20. Quan, L., Heidemann, Z., Pradkin, Y.: Detecting internet outages with precise active probing. Technical Report ISI-TR-2012-678 (2012)

  21. Jaggard, A., Kopparty, S., Ramachandran, V., Wright, R.N.: The design space of probing algorithms for network-performance measurement. SIGMETRICS Perform. Eval. Rev. 41(1), 105–116 (2013)

    Article  MATH  Google Scholar 

  22. Quan, L., Heidemann, J., Pradkin, Y.: Trinocular: understanding internet reliability through adaptive probing. SIGCOMM Comput. Commun. Rev. 43(4), 255–266 (2013)

    Google Scholar 

  23. Zheng, Q., Cao, G.: Minimizing probing cost and achieving identifiability in probe-based network link monitoring. IEEE Trans. Comput. 62(3), 510–523 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  24. Liu, S., Zafer, M., Wong, H., Lee, K.: Gateway selection in hybrid wireless networks through cooperative probing. In: Proceedings of IFIP/IEEE international symposium on integrated network management (IM 2013), Ottawa, Canada (2013)

  25. Tang, Y., Al-Shaer, E.S., Boutaba, R.: Active integrated fault localization in communication networks. In: Proceedings of IM 2005, pp. 543–556 (2005)

  26. Al-Shaer, E., Tang, Y.: Qos path monitoring for multicast networks. J. Netw. Syst. Manag. 10(3), 357–381 (2002)

  27. Yu, M., Greenberg, A., Maltz, D., Rexford, J., Yuan, L., Kandula, S., Kim, C.: Profiling network performance for multi-tier data center applications. In: Proceedings of NSDI’11, Boston, USA (2011)

  28. Huang, L., Nguyen, X., Garofalakis, M., Hellerstein, J.M.: Communication-efficient online detection of network-wide anomalies. In: Proceedings of INFOCOM’07, Anchorage, Alaska (2007)

  29. Gao, K., Kar, G., Kermani, P.: Approaches to building self healing systems using dependency analysis. In: Proceedings of IEEE/IFIP network operations and management symposium (NOMS), pp. 119–132 (2004)

  30. Wolski, R.: Experiences with predicting resource performance on-line in computational grid settings. SIGMETRICS Perform. Eval. Rev. 30(4), 41–49 (2003)

    Article  Google Scholar 

  31. Jeswani, D., Natu, M.., Ghosh, R.K.: Adaptive monitoring: a framework to adapt passive monitoring using probing. In: Proceedings of CNSM 2012, Las Vegas, USA (2012)

Download references

Author information

Authors and Affiliations

  1. IBM Research Lab, New Delhi, India

    Deepak Jeswani

  2. Tata Research Development and Design Centre, Pune, India

    Maitreya Natu

  3. Indian Institute of Technology, Kanpur, India

    R. K. Ghosh

Authors
  1. Deepak Jeswani
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Maitreya Natu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. R. K. Ghosh
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Maitreya Natu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jeswani, D., Natu, M. & Ghosh, R.K. Adaptive Monitoring: Application of Probing to Adapt Passive Monitoring. J Netw Syst Manage 23, 950–977 (2015). https://doi.org/10.1007/s10922-014-9330-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10922-014-9330-8

Keywords