Skip to main content
SpringerLink
Log in

FERA: A Framework for Critical Assessment of Execution Monitoring Based Approaches for Finding Concurrency Bugs

  • Conference paper
  • First Online:
Intelligent Computing (SAI 2020)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1228))

Included in the following conference series:

Abstract

The software community has recognized the importance of concurrent multithreaded software and the criticality of concurrency bugs. However, it has still not fully acknowledged the complexity of the process for finding these bugs. When based on execution monitoring, the process includes tracing of the software execution, specification of concurrency bug models, modeling of synchronization mechanism patterns, and several other process steps and components. Existing approaches, however, focus only on the parts of the process for finding concurrency bugs (PFCB) and fail to notice their inter-dependencies, which affect the analysis outcome. Without a proper understanding of these inter-dependencies, the advances in this research area remain partially incremental and limited to specific use cases. In order to better understand this problem, we provide a model of a generic process for finding concurrency bugs (PFCB) and a framework for the critical assessment of execution-monitoring-based approaches for finding concurrency bugs (FERA). Our framework captures the inter-dependencies between the individual PFCB steps and establishes the relation between these inter-dependencies and the final outcome of the monitoring. The FERA framework is suitable for assessing the suitability of approaches for finding concurrency bugs, for specific software systems (e.g., with specific synchronization properties). With this paper, we demonstrate that finding concurrency bugs is not an isolated action consisting of applying an algorithm to an execution trace, but rather a continuous engineering process.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Agarwal, R., Sasturkar, A., Wang, L., Stoller, S.D.: Optimized run-time race detection and atomicity checking using partial discovered types. In: Conference on Automated Software Engineering, ASE 2005, pp. 233–242. ACM, New York (2005)

    Google Scholar 

  2. Arora, V., Bhatia, R., Singh, M.: A systematic review of approaches for testing concurrent programs. Concurr. Comput.: Pract. Exp. 28(5), 1572–1611 (2016)

    Article  Google Scholar 

  3. Balakrishnan, G., Ganai, M.K., Gupta, A., Ivančić, F., Kahlon, V., Li, W., Maeda, N., Papakonstantinou, N., Sankaranarayanan, S., Sinha, N., Wang, C.: Scalable and precise program analysis at NEC. In: Formal Methods in Computer Aided Design, pp. 273–274, October 2010

    Google Scholar 

  4. Ben Khadra, M.A., Stoffel, D., Kunz, W.: Speculative disassembly of binary code. In: CASES 2016, pp. 16:1–16:10. ACM, New York (2016)

    Google Scholar 

  5. Bianchi, F., Margara, A., Pezze, M.: A survey of recent trends in testing concurrent software systems. IEEE Trans. Softw. Eng. PP(99), 1 (2017)

    Google Scholar 

  6. Bonakdarpour, B., Navabpour, S., Fischmeister, S.: Time-triggered runtime verification. Formal Methods Syst. Des. 43(1), 29–60 (2013)

    Article  Google Scholar 

  7. Brat, G., Drusinsky, D., Giannakopoulou, D., Goldberg, A., Havelund, K., Lowry, M., Pasareanu, C., Venet, A., Visser, W., Washington, R.: Experimental evaluation of verification and validation tools on martian rover software. Formal Methods Syst. Des. 25(2), 167–198 (2004)

    Article  Google Scholar 

  8. Chapman, B., Jost, G., van der Pas, R.: Using OpenMP: Portable Shared Memory Parallel Programming (Scientific and Engineering Computation). MIT Press, Cambridge (2007)

    Google Scholar 

  9. Choi, S.E., Lewis, E.C.: A study of common pitfalls in simple multi-threaded programs. SIGCSE Bull. 32(1), 325–329 (2000)

    Article  Google Scholar 

  10. Dijkstra, E.W.: The Origin of Concurrent Programming, pp. 65–138. Springer-Verlag New York, Inc., New York (2002)

    Google Scholar 

  11. Dinning, A., Schonberg, E.: Detecting access anomalies in programs with critical sections. SIGPLAN Not. 26(12), 85–96 (1991)

    Article  Google Scholar 

  12. Durkin, T.: What the media couldn’t tell you about the Mars pathfinder. Robot Sci. Technol. 1(1), 3 (1998)

    Google Scholar 

  13. El-Hokayem, A., Falcone, Y.: Can we monitor all multithreaded programs? In: RV 2018 - Conference on Runtime Verification, pp. 1–24. Limassol, Cyprus (2018)

    Google Scholar 

  14. Fitzgerald, B., Stol, K.J.: Continuous software engineering: a roadmap and agenda. J. Syst. Softw. 123, 176–189 (2017). http://www.sciencedirect.com/science/article/pii/S0164121215001430

  15. Fraser, K.: Practical lock-freedom. Ph.D. thesis, University of Cambridge (2004)

    Google Scholar 

  16. Goldberg, A., Havelund, K.: Instrumentation of Java bytecode for runtime analysis. Technical Reports from ETH Zurich 408, ETH Zurich, Zurich, Switzerland (2003)

    Google Scholar 

  17. Havelund, K., Goldberg, A.: Verify Your Runs. Springer, Heidelberg (2008)

    Book  Google Scholar 

  18. Hoare, C.A.R.: Monitors: an operating system structuring concept. Commun. ACM 17(10), 549–557 (1974)

    Article  Google Scholar 

  19. Hong, S., Ahn, J., Park, S., Kim, M., Harrold, M.J.: Testing concurrent programs to achieve high synchronization coverage. In: International Symposium on Software Testing and Analysis, ISSTA 2012, pp. 210–220. ACM, New York (2012)

    Google Scholar 

  20. Hong, S., Kim, M.: A survey of race bug detection techniques for multithreaded programmes. Softw. Test. Verif. Reliab. 25(3), 191–217 (2015)

    Article  Google Scholar 

  21. Israeli, A., Rappoport, L.: Disjoint-access-parallel implementations of strong shared memory primitives. In: ACM Symposium on Principles of Distributed Computing, PODC 1994, pp. 151–160. ACM, New York (1994)

    Google Scholar 

  22. Jacob, B., Ng, S., Wang, D.: Memory Systems: Cache, DRAM, Disk. Morgan Kaufmann Publishers Inc., San Francisco (2007)

    Google Scholar 

  23. Jahić, J., Kumar, V., Antonino, P.O., Wirrer, G.: Testing the implementation of concurrent AUTOSAR drivers against architecture decisions. In: 2019 IEEE International Conference on Software Architecture (ICSA), pp. 171–180, March 2019

    Google Scholar 

  24. Jahić, J., Jung, M., Kuhn, T., Kestel, C., Wehn, N.: A framework for non-intrusive trace-driven simulation of manycore architectures with dynamic tracing configuration. In: Runtime Verification, pp. 458–468. Springer, Limassol (2018)

    Google Scholar 

  25. Jahić, J., Kuhn, T., Jung, M., Wehn, N.: Supervised testing of concurrent software in embedded systems. In: Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation (SAMOS), pp. 233–238, July 2017

    Google Scholar 

  26. Jahić, J., Kuhn, T., Jung, M., Wehn, N.: BOSMI: a framework for non-intrusive monitoring and testing of embedded multithreaded software on the logical level. In: Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation, SAMOS 2018, pp. 131–138. ACM, New York (2018)

    Google Scholar 

  27. Jayasinghe, D., Xiong, P.: Core: visualization tool for fault localization in concurrent programs (2010)

    Google Scholar 

  28. Kane, A., Chowdhury, O., Datta, A., Koopman, P.: A case study on runtime monitoring of an autonomous research vehicle (ARV) system. In: Runtime Verification, pp. 102–117. Springer, Cham (2015)

    Google Scholar 

  29. Kelly, J.H., Dan, S.V., John, J.C., Leanna, K.R.: A practical tutorial on modified condition/decision coverage. Technical report, NASA (2001)

    Google Scholar 

  30. Kirsch, C.M., Sokolova, A.: The Logical Execution Time Paradigm. Springer (2012)

    Google Scholar 

  31. Kitchenham, B., Charters, S.: Guidelines for performing systematic literature reviews in software engineering (2007)

    Google Scholar 

  32. Krena, B., Letko, Z., Tzoref, R., Ur, S., Vojnar, T.: Healing data races on-the-fly. In: Workshop on Parallel and Distributed Systems: Testing and Debugging, PADTAD 2007, pp. 54–64. ACM (2007)

    Google Scholar 

  33. Lamport, L.: How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE Trans. Comput. C-28(9), 690–691 (1979)

    Google Scholar 

  34. Lee, E.A.: The problem with threads. Computer 39(5), 33–42 (2006)

    Article  Google Scholar 

  35. Lin, S., Wellings, A., Burns, A.: Supporting lock-based multiprocessor resource sharing protocols in real-time programming languages. Concurr. Comput.: Pract. Exp. 25(16), 2227–2251 (2013)

    Article  Google Scholar 

  36. Lu, S., Tucek, J., Qin, F., Zhou, Y.: AVIO: detecting atomicity violations via access-interleaving invariants. IEEE Micro 27(1), 26–35 (2007)

    Article  Google Scholar 

  37. Lu, S., Park, S., Hu, C., Ma, X., Jiang, W., Li, Z., Popa, R.A., Zhou, Y.: MUVI: automatically inferring multi-variable access correlations and detecting related semantic and concurrency bugs. SIGOPS Oper. Syst. Rev. 41(6), 103–116 (2007)

    Article  Google Scholar 

  38. Lu, S., Park, S., Seo, E., Zhou, Y.: Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. SIGOPS Oper. Syst. Rev. 42(2), 329–339 (2008)

    Article  Google Scholar 

  39. Lundqvist, T., Stenström, P.: Timing anomalies in dynamically scheduled microprocessors. In: Real-Time Systems Symposium, RTSS 1999, p. 12. IEEE Computer Society, Washington, DC (1999)

    Google Scholar 

  40. Memon, A., Nguyen, B., Nickell, E., Micco, J., Dhanda, S., Siemborski, R., Gao, Z.: Taming Google-scale continuous testing. In: ICSE 2017: Proceedings of the 39th International Conference on Software Engineering (2017)

    Google Scholar 

  41. Meng, X., Miller, B.P.: Binary code is not easy. In: Symposium on Software Testing and Analysis, ISSTA, pp. 24–35. ACM, New York (2016)

    Google Scholar 

  42. Moir, M.: Transparent support for wait-free transactions. In: Mavronicolas, M., Tsigas, P. (eds.) Distributed Algorithms, pp. 305–319. Springer, Heidelberg (1997)

    Chapter  Google Scholar 

  43. Necula, G.C., McPeak, S., Rahul, S.P., Weimer, W.: CIL: intermediate language and tools for analysis and transformation of C programs. In: Horspool, R.N. (ed.) Compiler Construction, pp. 213–228. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  44. Nethercote, N., Seward, J.: Valgrind: a framework for heavyweight dynamic binary instrumentation. In: Conference on Programming Language Design and Implementation, PLDI 2007, pp. 89–100. ACM, New York (2007)

    Google Scholar 

  45. Ruwase, O., Kozuch, M.A., Gibbons, P.B., Mowry, T.C.: Guardrail: A high fidelity approach to protecting hardware devices from buggy drivers. In: International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2014, pp. 655–670. ACM, New York (2014)

    Google Scholar 

  46. Saff, D., Ernst, M.D.: Reducing wasted development time via continuous testing. In: 14th International Symposium on Software Reliability Engineering, ISSRE 2003, pp. 281–292, November 2003

    Google Scholar 

  47. Savage, S., Burrows, M., Nelson, G., Sobalvarro, P., Anderson, T.: Eraser: a dynamic data race detector for multi-threaded programs. In: Symposium on Operating Systems Principles, SOSP 1997, pp. 27–37. ACM, New York (1997)

    Google Scholar 

  48. Savage, S., Burrows, M., Nelson, G., Sobalvarro, P., Anderson, T.: Eraser: a dynamic data race detector for multithreaded programs. ACM Trans. Comput. Syst. 15(4), 391–411 (1997)

    Article  Google Scholar 

  49. Sheng, T., Vachharajani, N., Eranian, S., Hundt, R., Chen, W., Zheng, W.: RACEZ: a lightweight and non-invasive race detection tool for production applications. In: Conference on Software Engineering, ICSE 2011. ACM, New York (2011)

    Google Scholar 

  50. Song, Y.W., Lee, Y.H.: On the existence of probe effect in multi-threaded embedded programs. In: Conference on Embedded Software (EMSOFT) (2014)

    Google Scholar 

  51. Souza, S.R.S., Brito, M.A.S., Silva, R.A., Souza, P.S.L., Zaluska, E.: Research in concurrent software testing: a systematic review. In: Workshop on Parallel and Distributed Systems: Testing, Analysis, and Debugging, PADTAD 2011, pp. 1–5. ACM, New York (2011)

    Google Scholar 

  52. Stallings, W.: Operating Systems: Internals and Design Principles, 6th edn. Prentice Hall Press, Upper Saddle River (2008)

    Google Scholar 

  53. Wang, C., Said, M., Gupta, A.: Coverage guided systematic concurrency testing. In: Conference on Software Engineering, ICSE 2011. ACM, New York (2011)

    Google Scholar 

  54. Wilhelm, A., Sharmay, B., Malakary, R., Schule, T., Gerndt, M.: Parceive: Interactive parallelization based on dynamic analysis. In: Workshop on Program Comprehension through Dynamic Analysis (PCODA), pp. 1–6 (2015)

    Google Scholar 

  55. Woo, S.C., Ohara, M., Torrie, E., Singh, J.P., Gupta, A.: The SPLASH-2 programs: characterization and methodological considerations. In: International Symposium on Computer Architecture, ISCA 1995, pp. 24–36. ACM, New York (1995)

    Google Scholar 

  56. Xiong, W., Park, S., Zhang, J., Zhou, Y., Ma, Z.: Ad hoc synchronization considered harmful. In: Conference on Operating Systems Design and Implementation, OSDI 2010, pp. 163–176. USENIX Association, Berkeley (2010)

    Google Scholar 

  57. Zhou, P., Teodorescu, R., Zhou, Y.: Hard: hardware-assisted lockset-based race detection. In: International Symposium on High Performance Computer Architecture, HPCA 2007, pp. 121–132. IEEE, Washington, DC (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Fraunhofer IESE, Kaiserslautern, Germany

    Jasmin Jahić, Thomas Bauer, Thomas Kuhn & Pablo Oliveira Antonino

  2. Microelectronic Systems Design Research Group, University of Kaiserslautern, Kaiserslautern, Germany

    Norbert Wehn

Authors
  1. Jasmin Jahić
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Bauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Kuhn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Norbert Wehn
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pablo Oliveira Antonino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jasmin Jahić .

Editor information

Editors and Affiliations

  1. Saga University, Saga, Japan

    Kohei Arai

  2. The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

    Supriya Kapoor

  3. The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

    Rahul Bhatia

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jahić, J., Bauer, T., Kuhn, T., Wehn, N., Antonino, P.O. (2020). FERA: A Framework for Critical Assessment of Execution Monitoring Based Approaches for Finding Concurrency Bugs. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Intelligent Computing. SAI 2020. Advances in Intelligent Systems and Computing, vol 1228. Springer, Cham. https://doi.org/10.1007/978-3-030-52249-0_5

Download citation

Publish with us

Policies and ethics

Search

Navigation