Efficacy, Efficiency and Effectiveness of SMarty-based Software Product Line Inspection Techniques: a Controlled Quasi-Experiment
Authors: 
Giovanna Bettin, Julio Herculani, Amanda Melo, Luiz C. M. Andrade, Edson OliveiraJrAuthors Info & Claims
Giovanna Bettin, Julio Herculani, Amanda Melo, Luiz C. M. Andrade, Edson OliveiraJrAuthors Info & ClaimsPages 40 - 49
Abstract
Software quality is impacted by several factors, including the quality of the artifacts generated during the development process. Especially for Software Product Lines (SPL), as artifacts are reused for various specific products, a defect can be spread out over an SPL. Our research group has previously created two inspection techniques for UML-based SPLs: a checklist-based, named SMartyCheck, and a perspective-based, named SMartyPerspective. Seeking to understand the efficiency, efficacy, and effectiveness of such techniques, we carried out a controlled quasi-experiment with 16 participants from the Software Engineering area. It aimed at inspecting feature diagrams and use case, class, component, and sequence diagrams designed using the SMarty approach for UML-based variability support. We also considered ad hoc inspections in such a study. The results of this experiment provide incipient evidence of no statistical difference among the compared techniques for efficiency, efficacy, and effectiveness.
References
[1]
V. Basili, S. Green, O. Laitenberger, F. Lanubile, F. Shull, S. Sørumgård, and M. Zelkowitz. 1996. The empirical investigation of perspective-based reading. Empirical Software Engineering 1, 2 (1996), 133–164. https://doi.org/10.1007/BF00368702
[2]
G. Bettin, R. T. Geraldi, and E. OliveiraJr. 2018. Experimental Evaluation of the SMartyCheck Techinique for Inspecting Defects in UML Component Diagrams. In Anais do 17º Simpósio Brasileiro de Qualidade de Software. 101–110. https://doi.org/10.1145/3275245.3275256
[3]
G. Bettin and E. OliveiraJr. 2021. SMartyPerspective: a perspective-based inspection technique for software product lines. In Brazilian Symposium on Software Engineering. 90–94. https://doi.org/10.1145/3474624.3474626
[4]
S. Biffl and M. Halling. 2003. Investigating the defect detection effectiveness and cost benefit of nominal inspection teams. IEEE Transactions on Software Engineering 29, 5 (2003), 385–397. https://doi.org/10.1109/TSE.2003.1199069
[5]
M. Ciolkowski, C. Differding, O. Laitenberger, and J. Münch. 1997. Empirical Investigation of Perspective-based Reading: A Replicated Experiment. Fraunhofer Institute for Experimental Software Engineering, Germany (1997).
[6]
L. Etxeberria, G. Sagardui, and L. Belategi. 2008. Quality aware software product line engineering. J Braz Comp Soc 14(2008), 57–69. https://doi.org/10.1007/BF03192552
[7]
M. E. Fagan. 1979. Design and Code Inspections to Reduce Errors in Program Development. IBM Systems Journal 38, 2/3 (1979), 258. https://doi.org/10.1147/sj.382.0258
[8]
V. Furtado, E. OliveiraJr, and M. Kalinowski. 2021. Guidelines for Promoting Software Product Line Experiments. In 15th Brazilian Symposium on Software Components, Architectures, and Reuse. 31–40. https://doi.org/10.1145/3483899.3483909
[9]
Pierfrancesco Fusaro, Filippo Lanubile, and Giuseppe Visaggio. 1997. A replicated experiment to assess requirements inspection techniques. Empirical Software Engineering 2, 1 (1997), 39–57. https://doi.org/10.1023/A:1009742216007
[10]
R. T. Geraldi, T. U. Conte, I. F. Steinmacher, and E. OliveiraJr. 2015. Checklist-based Inspection of SMarty Variability Models - Proposal and Empirical Feasibility Study. In Int. Conf. on Enterprise Information Systems. ScitePress, 268–275. https://doi.org/10.5220/0005350102680276
[11]
R. T. Geraldi and Edson OliveiraJr. 2017. Towards Initial Evidence of SMartyCheck for Defect Detection on Product-Line Use Case and Class Diagrams. Journal of Software 12(2017), 379–392. https://doi.org/10.17706/jsw.12.5.379-392
[12]
O. Laitenberger and C. Atkinson. 1999. Generalizing perspective-based inspection to handle object-oriented development artifacts. In Proceedings of the 21st International Conference on Software Engineering. 494–503. https://doi.org/10.1145/302405.302680
[13]
O. Laitenberger and J. DeBaud. 2000. An encompassing life cycle centric survey of software inspection. Journal of systems and software 50, 1 (2000), 5–31. https://doi.org/10.1016/S0164-1212(99)00073-4
[14]
E. OliveiraJr, I. M. S. Gimenes, and José C. Maldonado. 2010. Systematic Management of Variability in UML-based Software Product Lines. Journal of Universal Computer Science 16, 17 (2010), 2374–2393.
[15]
A. Porter and L. Votta. 1998. Comparing detection methods for software requirements inspections: A replication using professional subjects. Empirical software engineering 3, 4 (1998), 355–379. https://doi.org/doi.org/10.1023/A:1009776104355
[16]
Adam A Porter, Lawrence G Votta, and Victor R Basili. 1995. Comparing detection methods for software requirements inspections: A replicated experiment. IEEE Transactions on software Engineering 21, 6 (1995), 563–575. https://doi.org/10.1109/32.391380
[17]
I. Reinhartz-berger, A. Sturm, and A. Tsoury. 2011. Comprehension and Utilization of Core Assets Models in Software Product Line Engineering. In CEUR Workshop. 171–178.
[18]
I. Reinhartz-Berger and A. Tsoury. 2011. Specification and Utilization of Core Assets: Feature-Oriented vs. UML-Based Methods. In Advances in Conceptual Modeling. 302–311. https://doi.org/10.1007/978-3-642-24574-9_38
[19]
F. Shull, I. Rus, and V. Basili. 2000. How perspective-based reading can improve requirements inspections. Computer 33, 7 (2000), 73–79. https://doi.org/10.1109/2.869376
[20]
G. Travassos. 2014. Software Defects: Stay Away from Them. Do Inspections!. In 2014 9th International Conference on the Quality of Information and Communications Technology. IEEE, 1–7. https://doi.org/10.1109/QUATIC.2014.8
[21]
G. Travassos, F. Shull, M. Fredericks, and V. Basili. 1999. Detecting defects in object-oriented designs: using reading techniques to increase software quality. In Proceedings of the 14th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications (OOPSLA). ACM Press, 47–56. https://doi.org/10.1145/320384.320389
[22]
F. Van der Linden, K. Schmid, and E. Rommes. 2007. Software product lines in action: the best industrial practice in product line engineering. Springer Science & Business Media. https://doi.org/10.1007/978-3-540-71437-8
[23]
C. Wohlin, P. Runeson, M. Höst, M. C Ohlsson, B. Regnell, and A. Wesslén. 2012. Experimentation in Software Engineering. Springer Science & Business Media.
[24]
Y. Zhu. 2016. Software Reading Techniques: Twenty Techniques for More Effective Software Review and Inspection. Apress. https://doi.org/10.1007/978-1-4842-2346-8
Index Terms
- Efficacy, Efficiency and Effectiveness of SMarty-based Software Product Line Inspection Techniques: a Controlled Quasi-Experiment
Recommendations
SMartyPerspective: a perspective-based inspection technique for software product lines
SBES '21: Proceedings of the XXXV Brazilian Symposium on Software EngineeringSoftware Product Line (SPL) is an approach for reusing software artifacts for a specific domain. To improve products quality, verification and validation activities for SPL artifacts are necessary, thus defects are not propagated to derived products. ...
Checklist-based Inspection of SMarty Variability Models
ICEIS 2015: Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 2Software inspection is a particular type of software review applied to all life-cycle artifacts and follows a
rigorous and well-defined defect detection process. Existing literature defines several inspection techniques
for different domains. However, ...
Evidence-based SMarty Support for Variability Identification and Representation in Component Models
ICEIS 2015: Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 2Variability modeling is an essential activity for the success of software product lines. Although existing literature presents several variability management approaches, there is no empirical evidence of their effectiveness for representing variability ...
Comments
Information & Contributors
Information
Published In
January 2023
101 pages
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 25 January 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- CAPES
Conference
VaMoS 2023
VaMoS 2023: 17th International Working Conference on Variability Modelling of Software-Intensive Systems
January 25 - 27, 2023
Odense, Denmark
Acceptance Rates
Overall Acceptance Rate 66 of 147 submissions, 45%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 35Total Downloads
- Downloads (Last 12 months)5
- Downloads (Last 6 weeks)0
Reflects downloads up to 02 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format