skip to main content
10.1145/3229631.3229641acmotherconferencesArticle/Chapter ViewAbstractPublication PagessamosConference Proceedingsconference-collections
research-article

BOSMI: a framework for non-intrusive monitoring and testing of embedded multithreaded software on the logical level

Published: 15 July 2018 Publication History

Abstract

Traditional software testing methods are inefficient in cases where data inputs alone do not determine the outcome of a program's execution. In order to verify such software, testing is often complemented by analysis of the execution trace. For monitoring the execution trace, most approaches today insert additional instructions at the binary level, making the monitoring intrusive. Binary instrumentation operate on a low level, making it difficult to properly modify a program's states and to quantify its code coverage.
In this paper, we present a framework for testing complex embedded multithreaded software on the logical level. Testing software on this level avoids dependency on concrete compilers and relates the execution to the source code, thus enabling coverage. Our non-intrusive execution monitoring and control is implemented using the LLVM interpreter compiler infrastructure. Instead of forcing thread interleaving, we suggest simulating interleaving effects through non-intrusive changes of shared variables. This makes it possible to test a single thread without executing the full software stack, which is especially useful in situations where the full software stack is not available (e.g., pre-integration testing). We complement existing approaches with new features such as dynamic configuration of monitoring and execution rollback to the checkpoints. Our approach introduces acceptable overhead without any complex setup.

References

[1]
Apache-commons. 2017. The Byte Code Engineering Library (Apache Commons BCEL). (2017). http://jakarta.apache.org/bcel/
[2]
Matthias Blasing. 2017. Java Native Access (JNA). (2017). Retrieved November 05, 2017 from https://github.com/java-native-access/jna
[3]
B. Alpern et al. 2000. The Jalapeño virtual machine. IBM Systems Journal 39, 1 (2000), 211--238.
[4]
C.Luk et al. 2005. Pin: building customized program analysis tools with dynamic instrumentation. In Conference on Programming language design and implementation (PLDI). ACM, Chicago, IL, USA.
[5]
C. Wang et al. 2011. Coverage guided systematic concurrency testing. In International Conference on Software Engineering (ICSE '11). ACM, Waikiki, Honolulu, HI, USA, 221--230. https://dl.acm.org/citation.cfm?id=1985824
[6]
G. C. Necula et al. 2002. CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs. In Compiler Construction (CC '02). Springer-Verlag, London, UK, 213--228.
[7]
J. Jahic et al. 2014. Automatic Test Coverage Measurements to support Design Space Exploration. In Design Space Exploration of Cyber-Physical Systems (IDEAL). Springer-Verlag, Berlin, Germany.
[8]
J. Jahic et al. 2017. Supervised Testing of Concurrent Software in Embedded Systems. In International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation (SAMOS XVII). IEEE, Samos Island, Greece.
[9]
R. Agarwal et al. 2005. Optimized run-time race detection and atomicity checking using partial discovered types. In Automated software engineering (ASE '05). ACM, Long Beach, CA, USA, 233--242.
[10]
R. Vallée-Rai et al. 1999. Soot - a Java bytecode optimization framework. In Centre for Advanced Studies on Collaborative research (CASCON '99). IBM Press, Mississauga, Ontario, Canada, 13.
[11]
S. Hong et al. 2012. Testing concurrent programs to achieve high synchronization coverage. In Software Testing and Analysis (ISSTA 2012). ACM, Minneapolis, MN, USA, 210--220.
[12]
S. Savage et al. 1997. Eraser: a dynamic data race detector for multithreaded programs. ACM Transactions on Computer Systems (TOCS) 15, 4 (July 1997), 391--411.
[13]
T. Kuhn et al. 2013. FERAL - Framework for simulator coupling on requirements and architecture level. In Formal Methods and Models for Codesign (MEMOCODE). IEEE, Portland, OR, USA.
[14]
Z. Zhao et al. 2017. Network/System Co-Simulation for Design Space Exploration of IoT Applications. In International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation (SAMOS XVII). IEEE, Samos Island, Greece.
[15]
A. Goldberg and K. Havelund. 2003. Instrumentation of Java Bytecode for Runtime Analysis. Technical Reports from ETH Zurich 408. ETH Zurich, Zurich, Switzerland.
[16]
Helgrind. 2012. Valgrind-project., Helgrind: a data-race detector. (2012). Retrieved 2007 from http://valgrind.org/docs/manual/hgmanual.html
[17]
Hayhurst K. J. 2001. A Practical Tutorial on Modified Condition/Decision Coverage. Technical Report. NASA, NASA Langley Technical Report Server.
[18]
J. Jahic and T. Kuhn. 2014. Analysis of Functional Software Dependencies through Supervised Execution. In Software Reliability Engineering Workshops (ISSREW). IEEE, Naples, Italy.
[19]
L. Lamport. 1978. Time, clocks, and the ordering of events in a distributed system. Commun. ACM 21, 7 (July 1978), 558--565.
[20]
C. Lattner and V. Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In International symposium on Code generation and optimization: feedback-directed and runtime optimization (CGO '04). IEEE, San Jose, CA, USA, 75.
[21]
W. Kunz M. Ammar Ben Khadra, D. Stoffel. 2016. Speculative disassembly of binary code. In Compilers, Architectures and Synthesis for Embedded Systems (CASES). ACM, Pittsburgh, Pennsylvania.
[22]
Xiaozhu Meng and Barton P. Miller. 2016. Binary Code is Not Easy. In Proceedings of the 25th Symposium on Software Testing and Analysis (ISSTA). ACM, New York, NY, USA, 24--35.
[23]
N. Nethercote and J. Seward. 2007. Valgrind: a framework for heavyweight dynamic binary instrumentation. In Conference on Programming Language Design and Implementation (PLDI). ACM, San Diego, California, USA.
[24]
Y. W. Song and Y.-H. Lee. 2014. On the existence of probe effect in multi-threaded embedded programs. In International Conference on Embedded Software (EMSOFT). IEEE, Jaypee Greens, India.
[25]
S. C. Woo. 1995. The SPLASH-2 programs: characterization and methodological considerations. In Computer architecture (ISCA '95). ACM SIGARCH Computer Architecture News, S. Margherita Ligure, Italy, 24--36.

Cited By

View all
  • (2022)CASA: An Approach for Exposing and Documenting Concurrency-Related Software PropertiesEmbedded Computer Systems: Architectures, Modeling, and Simulation10.1007/978-3-031-15074-6_9(139-154)Online publication date: 14-Aug-2022
  • (2022)Accurate LLVM IR to Binary CFGs Mapping for Simulation of Optimized Embedded SoftwareEmbedded Computer Systems: Architectures, Modeling, and Simulation10.1007/978-3-031-04580-6_1(3-15)Online publication date: 27-Apr-2022
  • (2020)FERA: A Framework for Critical Assessment of Execution Monitoring Based Approaches for Finding Concurrency BugsIntelligent Computing10.1007/978-3-030-52249-0_5(54-74)Online publication date: 4-Jul-2020

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
SAMOS '18: Proceedings of the 18th International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation
July 2018
263 pages
ISBN:9781450364942
DOI:10.1145/3229631
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 July 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. LLVM
  2. concurrency
  3. coverage
  4. execution control
  5. execution monitoring

Qualifiers

  • Research-article

Funding Sources

  • German Federal Ministry of Education and Research (BMBF)

Conference

SAMOS XVIII
SAMOS XVIII: Architectures, Modeling, and Simulation
July 15 - 19, 2018
Pythagorion, Greece

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)2
Reflects downloads up to 25 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2022)CASA: An Approach for Exposing and Documenting Concurrency-Related Software PropertiesEmbedded Computer Systems: Architectures, Modeling, and Simulation10.1007/978-3-031-15074-6_9(139-154)Online publication date: 14-Aug-2022
  • (2022)Accurate LLVM IR to Binary CFGs Mapping for Simulation of Optimized Embedded SoftwareEmbedded Computer Systems: Architectures, Modeling, and Simulation10.1007/978-3-031-04580-6_1(3-15)Online publication date: 27-Apr-2022
  • (2020)FERA: A Framework for Critical Assessment of Execution Monitoring Based Approaches for Finding Concurrency BugsIntelligent Computing10.1007/978-3-030-52249-0_5(54-74)Online publication date: 4-Jul-2020

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media