research-article

A Conceptual Model to Support Teaching of Software Engineering Controlled (Quasi-)Experiments

Authors:

Edson Oliveirajr,

Igor SteinmacherAuthors Info & Claims

SBES '23: Proceedings of the XXXVII Brazilian Symposium on Software Engineering

Pages 236 - 245

https://doi.org/10.1145/3613372.3614202

Published: 25 September 2023 Publication History

Abstract

Throughout controlled experimentation, it is possible to provide evidence of the software being developed. In the academic environment, Experimentation in Software Engineering (ESE) is essential to understanding cause-effect relations, enabling a vision of the development process, and taking action on actual events in the software industry. As much as the experimentation processes have been used in industry and academia, there is a lack of formalization of the principles of ESE teaching and artifacts that can be useful to support it in higher education. One of the means to contribute to such a topic would be the design of a conceptual model, which is widely discussed in the literature, thus applying empirical methods for a better understanding of the context and representation of ESE teaching. Thus, in this paper, we developed a conceptual model to support the teaching of controlled experiments and quasi-experiments. To design the conceptual model, we carried out an analysis of metadata from controlled experiments and quasi-experiments in the literature and conducted a survey to collect data from instructors who teach ESE. We evaluated the model with the Technology Acceptance Model (TAM). Results consist of a feasible conceptual model aiming to standardize the basic concepts of ESE and further support the production and reuse of ESE materials.

References

[1]

L. M. Alves, R. J. Machado, and P. Ribeiro. 2012. Experimental Software Engineering in Educational Context. 2012 Eighth International Conference on the Quality of Information and Communications Technology (2012), 336–341.

[2]

Alex Baker, Emily Oh Navarro, and Andre Van Der Hoek. 2005. An experimental card game for teaching software engineering processes. Journal of Systems and Software 75, 1-2 (2005), 3–16.

Digital Library

[3]

Victor R Basili, Forrest Shull, and Filippo Lanubile. 1999. Building knowledge through families of experiments. IEEE Trans. Soft. Eng. 25, 4 (1999), 456–473.

Digital Library

[4]

Nergiz Ercil Cagiltay. 2007. Teaching software engineering by means of computer-game development: Challenges and opportunities. British Journal of Educational Technology 38, 3 (2007), 405–415.

[5]

Deepak Dahiya. 2010. Teaching software engineering: A practical approach. ACM SIGSOFT Software Engineering Notes 35, 2 (2010), 1–5.

Digital Library

[6]

Fred D Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly (1989), 319–340.

Digital Library

[7]

Davide Falessi, Natalia Juristo, Claes Wohlin, Burak Turhan, Jürgen Münch, Andreas Jedlitschka, and Markku Oivo. 2018. Empirical Software Engineering Experts on the Use of Students and Professionals in Experiments. Empirical Software Engineering 23 (02 2018). https://doi.org/10.1007/s10664-017-9523-3

Digital Library

[8]

David Forsyth. 2018. Probability and Statistics for Computer Science. Springer, Cham. https://doi.org/10.1007/978-3-319-64410-3

[9]

Viviane Furtado, Edson OliveiraJr, and Marcos Kalinowski. 2021. Guidelines for Promoting Software Product Line Experiments. In Brazilian Symposium on Software Components, Architectures, and Reuse. 31–40. https://doi.org/10.1145/3483899.3483909

Digital Library

[10]

Michael Gnatz, Leonid Kof, Franz Prilmeier, and Tilman Seifert. 2003. A practical approach of teaching software engineering. In CSEET. IEEE, 120–128.

[11]

Thomas B Hilburn and Massood Towhidnejad. 2007. A case for software engineering. In 20th Conference on Software Engineering Education and Training (CSEET’07). IEEE, 107–114.

Digital Library

[12]

Martin Host. 2002. Introducing empirical software engineering methods in education. In Proceedings 15th Conference on Software Engineering Education and Training (CSEET 2002). IEEE, 170–179.

Digital Library

[13]

Jan Hylén. 2006. Open educational resources: Opportunities and challenges. Open Education 2006: Community, Culture, and Content (2006), 49–63.

[14]

David S. Janzen, Clark S. Turner, and Hossein Saiedian. 2007. Empirical Software Engineering in Industry Short Courses. In Conference on Software Engineering Education and Training. 89–96. https://doi.org/10.1109/CSEET.2007.20

Digital Library

[15]

Natalia Juristo and Ana M Moreno. 2010. Basics of Software Engineering Experimentation. Springer Publishing Company.

Digital Library

[16]

Barbara Ann Kitchenham, David Budgen, and Pearl Brereton. 2015. Evidence-based software engineering and systematic reviews. Vol. 4. CRC press.

[17]

Marco Kuhrmann. 2017. Teaching Empirical Software Engineering Using Expert Teams. In SEUH. 20–31.

[18]

Marco Kuhrmann and Jürgen Münch. 2018. Enhancing software engineering education through experimentation: an experience report. In 2018 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC). IEEE, 1–9.

[19]

Carlos Danilo Luz, Edson OliveiraJr, and Igor F Steinmacher. 2022. How Experimentation in Software Engineering has been taught? Survey and Research Agenda. In International Conference on Evaluation and Assessment in Software Engineering. 299–304.

[20]

Edson OliveiraJr, Viviane Furtado, Henrique Vignando, Carlos Luz, André Cordeiro, Igor Steinmacher, and Avelino Zorzo. 2021. Towards Improving Experimentation in Software Engineering. In SBES. ACM, 335–340. https://doi.org/10.1145/3474624.3477073

Digital Library

[21]

Mary Prentice and Gail Robinson. 2010. Improving student learning outcomes with service learning. American Association of Community Colleges. Washington, DC (2010).

[22]

Rodrigo Pereira Santos, Paulo Ségio Medeiros Santos, Cláudia Maria Lima Werner, and Guilherme Horta Travassos. 2008. Using experimentation to support research and education of software engineering in Brazil. Fórum de Educação em Engenharia de Software (in Portuguese) (2008), 55.

[23]

Graeme Shanks, Elizabeth Tansley, and Ron Weber. 2003. Using ontology to validate conceptual models. Commun. ACM 46, 10 (2003), 85–89.

Digital Library

[24]

Walter F Tichy. 1998. Should Computer Scientists Experiment More?Computer 31 (1998), 32–40.

[25]

G. H. Travassos, P. S. M. d. Santos, P. G. Mian, A. C. D. Neto, and J. Biolchini. 2008. An Environment to Support Large Scale Experimentation in Software Engineering. IEEE International Conference on Engineering of Complex Computer Systems (ICECCS 2008) 13, 6 (2008), 193–202.

[26]

Henrique Vignando, Viviane Furtado, Lucas Teixeira, and Edson OliveiraJr. 2020. OntoExper-SPL: An Ontology for Software Product Line Experiments. In ICEIS. 401–408. https://doi.org/10.5220/0009575404010408

[27]

Claes Wohlin. 2007. Empirical Software Engineering: Teaching Methods and Conducting Studies. In Empirical Software Engineering Issues: Critical Assessment and Future Directions, Victor R. Basili, Dieter Rombach, Kurt Schneider, Barbara Kitchenham, Dietmar Pfahl, and Richard W. Selby (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 135–142.

[28]

Claes Wohlin. 2007. Empirical software engineering: Teaching methods and conducting studies. In Empirical Software Engineering Issues. Critical Assessment and Future Directions. Springer, 135–142.

[29]

Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, and Anders Wesslén. 2012. Experimentation in software engineering. Springer Science and Business Media.

[30]

Claes Wohlin, Per Runeson, Martin Ohlsson Host, Magnus C, Bjoorn Regnell, and Anders Wessl. 2012. An Environment to Support Large Scale Experimentation in Software Engineering. Verlag Berlin Heidelberg (2012).

Cited By

Santana CCamiletti ARand DNascimento EOliveiraJr EAmaral A(2024)Scratch no Desenvolvimento do Pensamento Computacional: um Quasi-Experimento com Alunos 9º anoAnais do XXXII Workshop sobre Educação em Computação (WEI 2024)10.5753/wei.2024.2121(513-524)Online publication date: 21-Jul-2024
https://doi.org/10.5753/wei.2024.2121
Meireles MRocha SMaldonado JConte TRoychoudhury APaiva AAbreu RStorey MGama KSiegmund J(2024)An experience report on the use of Active Learning in Empirical Software Engineering Education: Understanding the pros and cons from the student's perspectiveProceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3639474.3640077(380-390)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639474.3640077

Index Terms

A Conceptual Model to Support Teaching of Software Engineering Controlled (Quasi-)Experiments
1. General and reference
  1. Cross-computing tools and techniques
    1. Empirical studies
    2. Experimentation
2. Software and its engineering

Recommendations

A Survey of Controlled Experiments in Software Engineering

The classical method for identifying cause-effect relationships is to conduct controlled experiments. This paper reports upon the present state of how controlled experiments in software engineering are conducted and the extent to which relevant ...
A web-based support environment for software engineering experiments

The software engineering communities frequently propose new software engineering technologies, such as new development techniques, programming languages and tools, without rigorous scientific evaluation. One way to evaluate software engineering ...
A Controlled Experiment Quantitatively Comparing Software Development Approaches

A software engineering research study has been undertaken to empirically analyze and compare various software development approaches; its fundamental features and initial findings are presented in this paper. An experiment was designed and conducted to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

SBES '23: Proceedings of the XXXVII Brazilian Symposium on Software Engineering

September 2023

570 pages

ISBN:9798400707872

DOI:10.1145/3613372

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 September 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

CNPq/Brazil

Conference

SBES 2023

SBES 2023: XXXVII Brazilian Symposium on Software Engineering

September 25 - 29, 2023

Campo Grande, Brazil

Acceptance Rates

Overall Acceptance Rate 147 of 427 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
41
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)3

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Santana CCamiletti ARand DNascimento EOliveiraJr EAmaral A(2024)Scratch no Desenvolvimento do Pensamento Computacional: um Quasi-Experimento com Alunos 9º anoAnais do XXXII Workshop sobre Educação em Computação (WEI 2024)10.5753/wei.2024.2121(513-524)Online publication date: 21-Jul-2024
https://doi.org/10.5753/wei.2024.2121
Meireles MRocha SMaldonado JConte TRoychoudhury APaiva AAbreu RStorey MGama KSiegmund J(2024)An experience report on the use of Active Learning in Empirical Software Engineering Education: Understanding the pros and cons from the student's perspectiveProceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3639474.3640077(380-390)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639474.3640077

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten