short-paper

gMutant: A gCov based Mutation Testing Analyser

Authors:

Monika Rani Golla,

Sangharatna GodboleyAuthors Info & Claims

ISEC '23: Proceedings of the 16th Innovations in Software Engineering Conference

Article No.: 22, Pages 1 - 5

https://doi.org/10.1145/3578527.3578546

Published: 23 February 2023 Publication History

Abstract

gMutant is a novel mutation analyser to measure the quality of test cases. It is based on gCov. The existing gCov outputs code coverage (line and branch). Here, we scale it up in order to produce an effective metric i.e., Mutation Score (%) which is useful to report the quality of test cases. The key idea is to generate one Mutant Meta Program that has all the mutants instrumented in it. Hence, for each test input, only the generated Mutant Meta Program gets executed, instead of executing all the mutants separately. This mitigates the computational cost challenge of Mutation Testing in producing Mutation score. The gMutant that has been proposed is a generic tool which can be plugged in with any tester. It needs test cases as well as a program to generate a score (%) and a descriptive report. In our experimentation, we used Bounded Model Checker and Fuzzer to generate test cases. The run time of the gMutant is very advantageous in contrast to the traditional method of computing mutation score. It gives scores, in 4.03 (s) for CBMC and 4.01 (s) for AFL, which proves its efficiency.

References

[1]

Richard Baker and Ibrahim Habli. 2013. An Empirical Evaluation of Mutation Testing for Improving the Test Quality of Safety-Critical Software. IEEE Transactions on Software Engineering 39, 6 (2013), 787–805. https://doi.org/10.1109/TSE.2012.56

Digital Library

[2]

Code-gSC-MCC 2021. Code:https://github.com/Monika-rani12/gMutant/tree/master.

[3]

R.A. DeMilli and A.J. Offutt. 1991. Constraint-based automatic test data generation. IEEE Transactions on Software Engineering 17, 9 (1991), 900–910. https://doi.org/10.1109/32.92910

Digital Library

[4]

R.A. DeMillo, E.W. Krauser, and A.P. Mathur. 1991. Compiler-integrated program mutation. In [1991] Proceedings The Fifteenth Annual International Computer Software & Applications Conference. 351–356. https://doi.org/10.1109/CMPSAC.1991.170202

[5]

Shuitao Gan, Chao Zhang, Xiaojun Qin, Xuwen Tu, Kang Li, Zhongyu Pei, and Zuoning Chen. 2018. Collafl: Path sensitive fuzzing. In IEEESSP. 679–696.

[6]

Sangharatna Godboley and Arpita Dutta. 2021. Dy-COPECA: A Dynamic Version of MC/DC Analyzer for C Program(ENASE). 197–204. https://doi.org/10.5220/0010401501970204

[7]

Sangharatna Godboley, Joxan Jaffar, Rasool Maghareh, and Arpita Dutta. 2021. Toward Optimal Mc/Dc Test Case Generation(ISSTA 2021). Association for Computing Machinery, New York, NY, USA, 505–516. https://doi.org/10.1145/3460319.3464841

Digital Library

[8]

R.G. Hamlet. 1977. Testing Programs with the Aid of a Compiler. IEEE Transactions on Software Engineering SE-3, 4 (1977), 279–290. https://doi.org/10.1109/TSE.1977.231145

Digital Library

[9]

Joxan Jaffar, Sangharatna Godboley, and Rasool Maghareh. 2019. Optimal MC/DC Test Case Generation. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). 288–289. https://doi.org/10.1109/ICSE-Companion.2019.00118

Digital Library

[10]

Yue Jia and Mark Harman. 2011. An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering 37, 5 (2011), 649–678. https://doi.org/10.1109/TSE.2010.62

Digital Library

[11]

[11] Janne Kauttio.2013. MC/DC Based Test Selection for Dynamic Symbolic Execution. G2 Pro gradu, diplomityö. http://urn.fi/URN:NBN:fi:aalto-201310307764

[12]

Daniel Kroening and Michael Tautschnig. 2014. CBMC – C Bounded Model Checker, 8413. 389–391. https://doi.org/10.1007/978-3-642-54862-8_26

[13]

A. Jefferson Offutt, Ammei Lee, Gregg Rothermel, Roland H. Untch, and Christian Zapf. 1996. An Experimental Determination of Sufficient Mutant Operators. ACM Trans. Softw. Eng. Methodol. 5, 2 (apr 1996), 99–118. https://doi.org/10.1145/227607.227610

Digital Library

[14]

Golla Monika Rani and Sangharatna Godboley. 2022. Poster: A gCov based new profiler, gMCov, for MC/DC and SC-MCC. In 2022 IEEE Conference on Software Testing, Verification and Validation (ICST). 469–472. https://doi.org/10.1109/ICST53961.2022.00058

[15]

Screencast 2021. Screencast Media for gMutant: https://figshare.com/s/5ad8d8d9e6b0025aa792.

[16]

Kuo-Chung Tai. 1996. Theory of fault-based predicate testing for computer programs. IEEE Transactions on Software Engineering 22, 8 (1996), 552–562. https://doi.org/10.1109/32.536956

Digital Library

[17]

Roland H. Untch. 1992. Mutation-Based Software Testing Using Program Schemata. In Proceedings of the 30th Annual Southeast Regional Conference (Raleigh, North Carolina) (ACM-SE 30). Association for Computing Machinery, New York, NY, USA, 285–291. https://doi.org/10.1145/503720.503749

Digital Library

[18]

Lars van Hijfte and Ana Oprescu. 2021. MutantBench: an Equivalent Mutant Problem Comparison Framework. In 2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). 7–12. https://doi.org/10.1109/ICSTW52544.2021.00015

[19]

Lingming Zhang, Tao Xie, Lu Zhang, Nikolai Tillmann, Jonathan de Halleux, and Hong Mei. 2010. Test generation via Dynamic Symbolic Execution for mutation testing. In 2010 IEEE International Conference on Software Maintenance. 1–10. https://doi.org/10.1109/ICSM.2010.5609672

Digital Library

Cited By

Godboley SKrishna PJha R(2024)Atomic Condition Coverage Analysis for Structured Text Based Programmable Logic Controller (PLC)Proceedings of the 17th Innovations in Software Engineering Conference10.1145/3641399.3641427(1-5)Online publication date: 22-Feb-2024
https://dl.acm.org/doi/10.1145/3641399.3641427
Peng XJia PFan XLiu J(2024)ENZZ: Effective N-gram coverage assisted fuzzing with nearest neighboring branch estimationInformation and Software Technology10.1016/j.infsof.2024.107582(107582)Online publication date: Sep-2024
https://doi.org/10.1016/j.infsof.2024.107582
Golla MGodboley S(2024)Automated SC-MCC test case generation using coverage-guided fuzzingSoftware Quality Journal10.1007/s11219-024-09667-332:3(849-880)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s11219-024-09667-3
Show More Cited By

Index Terms

gMutant: A gCov based Mutation Testing Analyser
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Prioritizing mutants to guide mutation testing
ICSE '22: Proceedings of the 44th International Conference on Software Engineering

Mutation testing offers concrete test goals (mutants) and a rigorous test efficacy criterion, but it is expensive due to vast numbers of mutants, many of which are neither useful nor actionable. Prior work has focused on selecting representative and ...
Faster mutation testing inspired by test prioritization and reduction
ISSTA 2013: Proceedings of the 2013 International Symposium on Software Testing and Analysis

Mutation testing is a well-known but costly approach for determining test adequacy. The central idea behind the approach is to generate mutants, which are small syntactic transformations of the program under test, and then to measure for a given test ...
On the Improvement of the Mutation Score Using Distinguishing Test Cases
ICST '11: Proceedings of the 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation

Software testing, i.e., discovering software failures through test case execution, plays a crucial role in the software development process. A high quality software must have a strong test suite. Therefore it is of high importance for a software to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ISEC '23: Proceedings of the 16th Innovations in Software Engineering Conference

February 2023

193 pages

ISBN:9798400700644

DOI:10.1145/3578527

Editors:
Sonali Agarwal
IIIT Allahabad, India
,
Abhik Roychoudhury
National University of Singapore
,
Rahul Purandare
University of Nebraska-Lincoln, USA
,
Saurabh Tiwari
DA-IICT Gandhinagar, India
,
Lov Kumar
NIT Kurukshetra, India

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 February 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

ISEC 2023

ISEC 2023: 16th Innovations in Software Engineering Conference

February 23 - 25, 2023

Allahabad, India

Acceptance Rates

Overall Acceptance Rate 76 of 315 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
69
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)6

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Godboley SKrishna PJha R(2024)Atomic Condition Coverage Analysis for Structured Text Based Programmable Logic Controller (PLC)Proceedings of the 17th Innovations in Software Engineering Conference10.1145/3641399.3641427(1-5)Online publication date: 22-Feb-2024
https://dl.acm.org/doi/10.1145/3641399.3641427
Peng XJia PFan XLiu J(2024)ENZZ: Effective N-gram coverage assisted fuzzing with nearest neighboring branch estimationInformation and Software Technology10.1016/j.infsof.2024.107582(107582)Online publication date: Sep-2024
https://doi.org/10.1016/j.infsof.2024.107582
Golla MGodboley S(2024)Automated SC-MCC test case generation using coverage-guided fuzzingSoftware Quality Journal10.1007/s11219-024-09667-332:3(849-880)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s11219-024-09667-3
Godboley SKulshreshtha SAgarwal PRanilbala B(2023)PRCMHFL: A Fault Localization Technique Based on Predicate Ranking and CMH Method2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C60940.2023.00085(148-156)Online publication date: 22-Oct-2023
https://doi.org/10.1109/QRS-C60940.2023.00085
Godboley SKrishna PJoshi AGupta IKhatav R(2023)Smart Contract Test Case Prioritization based on Frequency and Gas Consumption2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C60940.2023.00084(290-299)Online publication date: 22-Oct-2023
https://doi.org/10.1109/QRS-C60940.2023.00084
Godboley SGolla MKrishna P(2023)SC-MCC: A Stronger Code Coverage Criterion2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C60940.2023.00083(62-71)Online publication date: 22-Oct-2023
https://doi.org/10.1109/QRS-C60940.2023.00083
Godboley SRadha Krishna PRaj G(2023)Study on Unknown Bugs for Smart Contracts using Echidna2023 IEEE 20th India Council International Conference (INDICON)10.1109/INDICON59947.2023.10440926(1359-1364)Online publication date: 14-Dec-2023
https://doi.org/10.1109/INDICON59947.2023.10440926
Godboley S(2023)An Extensive Investigation of Condition Reachability using CBMC: Study on Negative Results2023 IEEE 20th India Council International Conference (INDICON)10.1109/INDICON59947.2023.10440808(1305-1310)Online publication date: 14-Dec-2023
https://doi.org/10.1109/INDICON59947.2023.10440808

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten