Skip to main content
SpringerLink
Log in

Analyzing execution traces: critical-path analysis and distance analysis

  • Regular Paper
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

System designers make trade-offs between metrics of interest such as execution time, functional quality and cost to create a properly balanced system. Execution traces, which are sequences of timestamped start and end events of system tasks, are a general and powerful means to understand the system behavior that gives rise to these trade-offs. Such traces can be produced by, e.g., executable models or prototype systems. Their interpretation, however, often is non-trivial. We present two automated analysis techniques that work on execution traces to help the system designer with interpretation. First, critical-path analysis can be used to answer the typical “what is the bottleneck” question, and we extend earlier work of [16] with a technique that uses application information to refine the analysis. Second, we define a pseudo-metric on execution traces, which is useful for calibration and validation purposes, and which can be used to visualize the differences between traces. Both techniques are based on a common graph representation of execution traces. We have implemented our techniques in the Trace visualization tool [12], and have applied them in a case study from the digital printing domain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. We do not precisely specify the execution semantics of the system as this is out of scope. What matters are the execution traces that are produced by the system.

  2. Sometimes w can be derived by static analysis of the model. For instance, it could be bound by the cardinality of a maximal antichain in the application task graph or by the number of resources in the platform. In general, however, w can only be determined by examination of the execution trace.

  3. This assumption is not limiting because the task graph can have an arbitrary set of additional task executions that can be inserted between the task executions of the application graph. It, therefore, can delay application task executions arbitrarily.

  4. From now on we use task instead of task execution for brevity.

  5. Of course, it is very hard to guarantee beforehand that adding small random numbers to the execution time of tasks does not significantly change the system behavior. Significant changes by very small variations in the model, however, are a sign of limited robustness, which should be subject of analysis anyhow.

  6. We liberally skip the duration part of the approximated task graph definition to avoid more notation.

  7. Note that these execution traces are job dependent. Furthermore, we have ensured that the execution of the individual tasks is free from interference from other tasks to get the nominal execution time.

  8. The fact that sIP3 instances are blocked is not visible in the figure due to the small execution time of sIP3. The console output of the critical-path analysis step in the Trace tool, however, contains this information.

References

  1. van der Aalst, W.M.P., van Dongen, B.F.: Discovering workflow performance models from timed logs. In: 1st International Conference on Engineering and Deployment of Cooperative Information Systems. Springer, New York (2002)

  2. van der Aalst, W.M.P., et al.: Workflow mining: a survey of issues and approaches. Data Knowl. Eng. 47, 237–267 (2003)

    Article  Google Scholar 

  3. Aho, A.V., Garey, M.R., Ullman, J.D.: The transitive reduction of a directed graph. SIAM J. Comput. 1, 131–137 (1972)

    Article  MathSciNet  MATH  Google Scholar 

  4. Arkhangelski, A.V., Fedorchuk, V.V.: The basic concepts and constructions of general topology. General Topology I. volume 17 of Encyclopaedia of Mathematical Sciences, pp. 1–90. Springer, Berlin (1990)

  5. Kluwer, F., et al.: Hardware–Software Co-design of Embedded Systems: The POLIS Approach. Kluwer, Amsterdam (1997)

    Google Scholar 

  6. Barford, P., Crovella, M.: Critical path analysis of TCP transactions. SIGCOMM Comput. Commun. Rev. 30, 127–138 (2000)

    Article  Google Scholar 

  7. Basten, T. et al.: Model-driven design-space exploration for embedded systems: the octopus toolset. In: 4th International Symposium on Leveraging Applications of Formal Methods, Verification, and Validation (ISoLA 2010), volume 6415 of Lecture Notes in Computer Science, pp. 90–105. Springer, New York (2010)

  8. Bjorn-Jorgensen, P., Madsen, J.: Critical path driven cosynthesis for heterogeneous target architectures. In: 5th International Workshop on Hardware/Software Co-Design, IEEE CS (1997)

  9. Bohme, D., Wolf, F., de Supinski, B.R., Schulz, M., Geimer, M.: Scalable critical-path based performance analysis. In: 26th International Parallel Distributed Processing Symposium (IPDPS). IEEE (2012)

  10. Cormen, T.H., Leiserson, C.E., Rivest, R.L.: Introduction to Algorithms. The MIT Press, Cambridge (1990)

    MATH  Google Scholar 

  11. Eclipse Foundation. Eclipse website. http://www.eclipse.org/. Accessed 9 Sept 2016

  12. Embedded Systems Innovation by TNO. TRACE website. http://trace.esi.nl. Accessed 9 Sept 2016

  13. Gao, X., Xiao, B., Tao, D., Li, X.: A survey of graph edit distance. Pattern Anal. Appl. 13(1), 113–129 (2010)

    Article  MathSciNet  Google Scholar 

  14. Goldratt, E.: Critical Chain. North River Press, Barrington (1997)

    Google Scholar 

  15. Hendriks, M., Basten, T., Verriet, J., Brassé, M., Somers, L.: A blueprint for system-level performance modeling of software-intensive embedded systems. Int. J. Softw. Tool Technol.Transf. 18(1), 21–40 (2016)

  16. Hendriks, M., Vaandrager, F.W.: Reconstructing critical paths from execution traces. In: International Conference on Embedded and Ubiquitous Computing (EUC 2012). IEEE Computer Society Press (2012)

  17. Henzinger, T.A., Majumdar, R., Prabhu, V.S.: Quantifying similarities between timed systems. In: Formal Modeling and Analysis of Timed Systems, Vol. 3829. Lecture Notes in Computer Science. Springer, Berlin (2005)

  18. Hollingsworth, J.K.: An online computation of critical path profiling. In: 1st ACM SIGMETRICS Symposium on Parallel and Distributed Tools. ACM (1996)

  19. Huang, J., Voeten, J., Geilen, M.: Real-time property preservation in concurrent real-time systems. In: 10th International Conference on Real-Time and Embedded Computing Systems and Applications (2004)

  20. Intel. Intel ARK website. http://ark.intel.com/. Accessed 9 Sept 2016

  21. Kastor, A., Sirakoulis, K.: The effectiveness of resource levelling tools for resource constraint project scheduling problem. Int. J. Project Manag. 27, 493–500 (2009)

    Article  Google Scholar 

  22. Kelley, J.E., Walker, M.R.: Critical-path planning and scheduling. In: Eastern Joint IRE-AIEE-ACM Computer Conference. ACM (1959)

  23. Kengne, C.K., Ibrahim, N., Rousset, M.-C., Tchuente, M.: Distance-based trace diagnosis for multimedia applications: Help me TED! In: 7th International Conference on Semantic Computing (ICSC). IEEE (2013)

  24. Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions and reversals. Sov. Phys. Doklady 10, 707–710 (1966)

    MathSciNet  MATH  Google Scholar 

  25. Lockyer, K.G.: Introduction to Critical Path Analysis. Pitman Publishing Co., New Jersey (1964)

    Google Scholar 

  26. Luo, J., Jha, N.K.: Static and dynamic variable voltage scheduling algorithms for real-time heterogeneous distributed embedded systems. In: ASP-DAC. IEEE CS (2002)

  27. Mannila, H., Ronkainen, P.: Similarity of event sequences. In: 4th International Workshop on Temporal Representation and Reasoning. IEEE Computer Society Press (1997)

  28. Nakatsu, N., Kambayashi, Y., Yajima, S.: A longest common subsequence algorithm suitable for similar text strings. Acta Inf. 18(2), 171–179 (1982)

    Article  MATH  Google Scholar 

  29. Obweger, H., Suntinger, M., Schiefer, J., Raidl, G.: Similarity searching in sequences of complex events. In: 4th International Conference on Research Challenges in Information Science (RCIS). IEEE (2010)

  30. Pimentel, A.D., Thompson, M., Polstra, S., Erbas, C.: Calibration of abstract performance models for system-level design space exploration. J. Signal Process. Syst. 50(2), 99–114 (2008)

    Article  Google Scholar 

  31. Pinedo, M.: Scheduling: Theory, Algorithms and Systems, 2nd edn. Prentice Hall, New Jersey (2002)

    MATH  Google Scholar 

  32. Schulz, M.: Extracting critical path graphs from MPI applications. In: International Conference on Cluster Computing. IEEE (2005)

  33. Wiest, J.D.: Some properties of schedules for large projects with limited resources. Oper. Res. 12, 395–418 (1964)

    Article  Google Scholar 

  34. Wu, M.Y., Gajski, D.D.: Hypertool: a programming aid for message-passing systems. IEEE Trans. Parall. Distrib. Syst. 1, 330–343 (1990)

    Article  Google Scholar 

  35. Wu, S., Manber, U., Myers, G., Miller, W.: An O(NP) sequence comparison algorithm. Inf. Process. Lett. 35(6), 317–323 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  36. Yang, C.-Q., Miller, B.P.: Critical path analysis for the execution of parallel and distributed programs. In: 8th International Conference on Distributed Computing Systems. IEEE (1988)

  37. Yang, Y., Geilen, M., Basten, T., Stuijk, S., Corporaal, H.: Automated bottleneck-driven design-space exploration of media processing systems. In: Design, Automation Test in Europe Conference Exhibition (DATE), 2010, pp. 1041–1046. IEEE (2010)

  38. Zeng, Z., Tung, A.K.H., Wang, J., Feng, J., Zhou, L.: Comparing stars: on approximating graph edit distance. Proc. VLDB Endow. 2(1), 25–36 (2009)

    Article  Google Scholar 

Download references

Acknowledgments

We thank the anonymous reviewers for their valuable comments that helped us to improve the paper. The research is partially carried out as part of the Octo+ program under the responsibility of Embedded Systems Innovation by TNO (TNO-ESI) with Océ Technologies B.V. as the carrying industrial partner. The Octo+ research is supported by the Netherlands Ministry of Economic Affairs. The research reported in this paper is also partially supported by the ARTEMIS joint undertaking under Grant Agreement No. 621439 (ALMARVI).

Author information

Authors and Affiliations

  1. Embedded Systems Innovation by TNO, Eindhoven, The Netherlands

    Martijn Hendriks, Jacques Verriet, Twan Basten & Bart Theelen

  2. Eindhoven University of Technology, Eindhoven, The Netherlands

    Twan Basten & Lou Somers

  3. Océ Technologies B.V., Venlo, The Netherlands

    Marco Brassé & Lou Somers

Authors
  1. Martijn Hendriks
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacques Verriet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Twan Basten
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bart Theelen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marco Brassé
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lou Somers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martijn Hendriks.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 227 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hendriks, M., Verriet, J., Basten, T. et al. Analyzing execution traces: critical-path analysis and distance analysis. Int J Softw Tools Technol Transfer 19, 487–510 (2017). https://doi.org/10.1007/s10009-016-0436-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-016-0436-z

Keywords

Search

Navigation