Abstract
Existing transaction monitoring solutions are either platform-specific or rely on instrumentation techniques, which limit their applicability. Consequently, transaction monitoring in enterprise environments often involves the manual collation of information spread across a variety of infrastructure elements and applications, and is a time-consuming and labor-intensive task. To address this problem, we have developed an online, non-intrusive and platform-agnostic solution for transaction monitoring. The solution includes a transaction model discovery component that leverages historical system log files, containing transaction footprints and generates a model of the transaction in terms of valid sequence of steps that a transaction instance may execute and the expected footprint patterns at each step. The online monitoring system, in turn, takes in only (a) online system log files and (b) the transaction model, as inputs and generates a dynamic execution profile of ongoing transaction instances that allows their status to be tracked at individual and aggregate levels, even when transaction footprints do not necessarily carry correlating identifiers as those injected through instrumentation. In this paper, we describe the transaction model discovery and monitoring system including the architecture and algorithms, followed by results from an empirical study, ongoing work on run-time model validation and directions for future research.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
ITCAM for SOA. http://www-306.ibm.com/software/tivoli/products/composite-application-mgr-soa/
Agrawal, R., Gunopulos, D., Leymann, F.: Mining process models from workflow logs. Int’l Conf. on Extending Database Technology. LNCS, vol. 1377 (1998)
Chen, M.Y., Kiciman, E., Fratkin, E., Fox, A., Brewer, E.: Pinpoint: problem determination in large, dynamic internet services. In Proc. of DSN, pp. 595–604 (2002)
Krishnamurthy, B., Sengupta, B., Neogi, A., Singh, R.: Data tagging architecture for system monitoring in dynamic environments. NOMS 2008 (main technical track)
Vaardi, R.: A data clustering algorithm for mining patterns from event logs. In 3rd Workshop on IP Operations and Management (IPOM 2003), Kansas City, MO, USA, Oct. 1–3, 2003
Apache derby. http://db.apache.org/derby/
Google Web Toolkit. code.google.com/webtoolkit
GraphML File Format. http://graphml.graphdrawing.org/
Logfile Clustering Tool. http://kodu.neti.ee/~risto/slct/
Cook, J.E., Wolf, A.L.: Discovering models of software processes from event-based data. ACM Trans. Softw. Eng. Methodol. 7(3), 215–249 (1998)
Herbst, J., Karagiannis, D.: An inductive approach to the acquisition and adaptation of workflow models. IJCAI Workshop on Intelligent Workflow and Process Management, pp. 52–57 (1999)
van der Aalst, W.M.P., van Dongen, B.F., et al.: Workflow mining: a survey of issues and approaches. Data Knowl. Eng. 47(2), 237–267 (2003)
Peng, W., Li, T., Ma, S.: Mining logs files for data-driven system management. SIGKDD Explor. 7, 44–51 (2005)
http://www.ibm.com/developerworks/autonomic/btmpd/, CBE: http://www.ibm.com/developerworks/library/specification/ws-cbe/
Hellerstein, J., Ma, S.: Mining event data for actionable patterns. The Computer Measurement Group, Orlando, December 2000
Aman, J., Eilert, C.K., Emmes, D., Yocom, P., Dillenberger, D.: Adaptive algorithms for managing a distributed data processing workload. IBM Syst. J. 36, 2 (1997)
Schmid, M., Thoss, M., et al.: A generic application-oriented performance instrumentation for multi-tier environments. Integr. Netw. Manag. 304–313 (2007)
Hasselmeyer, P.: Managing dynamic service dependencies. In Proc. of 12th International Workshop on Distributed Systems: Operations and Management, Nancy, France, pp. 141–150 (2001)
E2EProf: Automated end-to-end performance management for enterprise systems. In S. Agarwala, F. Alegre et al. (eds.) 37th Int’l Conf. on Dependable Systems and Networks (DSN), pp. 749–758 (2007)
Agarwal, M., Gupta, M., Kar, G., Neogi, A., Sailer, A.: Mining Activity Data for Dynamic Dependency Discovery in e-Business Systems. IEEE eTrans. Netw. Serv. Manag. J. (eTNSM), Fall 2004
Brown, A., Kar, G., Keller, A.: An active approach to characterizing dynamic dependencies for problem determination in distributed environment. In Proc. of 7th IEEE/IFIP Int’l Symposium on Integrated Network Management, Seattle, pp. 377–390, May 2001
Network Monitoring Tools. http://www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html#app
Application Monitoring. http://www.monitortools.com/application/
HP OpenView. http://h20229.www2.hp.com/
IBM Tivoli. http://www-306.ibm.com/software/tivoli/
Branch, J.W., Bisdikian, C., (Starsky) Wong, H.Y., Agrawal, D.: A semantically agnostic framework supporting model composition for business management solutions. In IFIP/IEEE International Symposium on Integrated Network Management (IM 2009), June 2009
Anandkumar, A., Bisdikian, C., Agrawal, D.: Tracking in a spaghetti bowl: monitoring transactions using footprints. In ACM SIGMETRICS 2008, Annapolis, Maryland, USA, June 2–6, 2008
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sengupta, B., Banerjee, N., Bisdikian, C. et al. Tracking transaction footprints for non-intrusive end-to-end monitoring. Cluster Comput 12, 59–72 (2009). https://doi.org/10.1007/s10586-008-0066-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10586-008-0066-7