skip to main content
10.1145/3422392.3422458acmotherconferencesArticle/Chapter ViewAbstractPublication PagessbesConference Proceedingsconference-collections
research-article

Software testing ecosystems insights and research opportunities

Published: 21 December 2020 Publication History

Abstract

[Context] Software testing research is a robust field of study with a lot of research that aims to improve testing activities. There are several related elements in the testing activity that can be further investigated through the concepts defined in the software ecosystem (SECO) research. The study of testing and joint SECO addresses challenges and opportunities that go forward application development in a project, through an entire development network from suppliers to stakeholders. [Objective] In this paper, we propose a definition of software testing ecosystem (STECO) and report research opportunities, to bring close these two different research fields fostering the conduction of new research at the intersection of these two topics to investigate the elements of testing activity that constitute an ecosystem and how it can be tested to improve software quality. [Method] The concept of STECO is defined and we perform a literature search. Hence, to illustrate the idea of a STECO, we designed a model to describe the elements involved around the JUnit SECO and describe some research opportunities considering the knowledge of the testing and SECO fields. [Results and Conclusions] The following research opportunities are highlighted to foster new research works: (i) the conduction of functional and nonfunctional requirements testing, as well as the challenges imposed from this activity; (ii) the need for testing products developed separated from the central SECO platform; (iii) conduction of studies on testing automation on SECO platforms; (iv) testing between the different technologies that composed a SECO; and (v) testing the mobile software ecosystem (MSECO) environment, that has been presenting a fast growth in the number of applications.

References

[1]
Paul Ammann and Jeff Offutt. 2016. Introduction to software testing. Cambridge University Press.
[2]
Jakob Axelsson and Mats Skoglund. 2016. Quality assurance in software ecosystems: A systematic literature mapping and research agenda. Journal of Systems and Software 114 (2016), 69--81. https://doi.org/10.1016/j.jss.2015.12.020
[3]
Alfred Baars and Slinger Jansen. 2012. A framework for software ecosystem governance. In International conference of software business. Springer, 168--180.
[4]
Stefan Bechtold, Sam Brannen, Johannes Link, Matthias Merdes, Marc Philipp, and Christian Stein. 2019. JUnit 5 User Guide. https://junit.org/junit5/docs/current/user-guide/
[5]
Stefan Berner, Roland Weber, and Rudolf K Keller. 2005. Observations and lessons learned from automated testing. In Proceedings of the 27th international conference on Software engineering. ACM, 571--579.
[6]
Antonia Bertolino. 2007. Software testing research: Achievements, challenges, dreams. In 2007 Future of Software Engineering. IEEE Computer Society, 85--103.
[7]
Jan Bosch and Petra Bosch-Sijtsema. 2010. From integration to composition: On the impact of software product lines, global development and ecosystems. Journal of Systems and Software 83, 1 (2010), 67--76.
[8]
Vasilis Boucharas, Slinger Jansen, and Sjaak Brinkkemper. 2009. Formalizing Software Ecosystem Modeling. In Proceedings of the 1st International Workshop on Open Component Ecosystems (IWOCE '09). ACM, New York, NY, USA, 41--50. https://doi.org/10.1145/1595800.1595807
[9]
Pierre Bourque, Richard E Fairley, et al. 2014. Guide to the software engineering body of knowledge (SWEBOK (R)): Version 3.0. IEEE Computer Society Press.
[10]
Piers RJ Campbell and Faheem Ahmed. 2010. A three-dimensional view of software ecosystems. In Proceedings of the Fourth European Conference on Software Architecture: Companion Volume (ECSA '10). 81--84.
[11]
Lee Copeland. 2004. A practitioner's guide to software test design. Artech House.
[12]
Emanuel Ferreira Coutinho, Italo Santos, and Carla Ilane Moreira Bezerra. 2017. A software ecosystem for a virtual learning environment: Solar seco. In 2017 IEEE/ACM Joint 5th International Workshop on Software Engineering for Systems-of-Systems and 11th Workshop on Distributed Software Development, Software Ecosystems and Systems-of-Systems (JSOS). IEEE, 41--47.
[13]
Emanuel F. Coutinho, Italo Santos, Leonardo O. Moreira, and Carla I. M. Bezerra. 2019. A Report on the Teaching of Software Ecosystems in Software Engineering Discipline. In Proceedings of the XXXIII Brazilian Symposium on Software Engineering (SBES 2019). 130--139.
[14]
Emanuel Ferreira Coutinho, Davi Viana, and Rodrigo Pereira dos Santos. 2017. An Exploratory Study on the Need for Modeling Software Ecosystems: The Case of SOLAR SECO. In 9th International Workshop on Modelling in Software Engineering (MISE) (MISE '17). IEEE Press, Piscataway, NJ, USA, 47--53. https://doi.org/10.1109/MiSE.2017.3
[15]
Simone da Silva Amorim, Félix Simas S. Neto, John D. McGregor, Eduardo Santana de Almeida, and Christina von Flach G. Chavez. 2017. How Has the Health of Software Ecosystems Been Evaluated?: A Systematic Review. In Proceedings of the 31st Brazilian Symposium on Software Engineering (SBES' 17). 14--23.
[16]
Awdren de Lima Fontão, Rodrigo Pereira dos Santos, and Arilo Claudio Dias-Neto. 2015. Mobile software ecosystem (mseco): a systematic mapping study. In 2015 IEEE 39th Annual Computer Software and Applications Conference, Vol. 2. IEEE.
[17]
Marcio Delamaro, Mario Jino, and José Maldonado. 2017. Introdução ao teste de software. Elsevier Brasil.
[18]
Awdren Fontão, Bernardo Estácio, Juliana Fernandes, Rodrigo Pereira dos Santos, and Arilo Claudio Dias-Neto. 2018. Which Factors Affect the Evangelist's Support During Training Sessions in Mobile Software Ecosystems?. In Proceedings of the 12th European Conference on Software Architecture: Companion Proceedings (ECSA '18). Article 22, 7 pages.
[19]
Oscar Franco-Bedoya, David Ameller, Dolors Costal, and Xavier Franch. 2017. Open source software ecosystems: A Systematic mapping. Information and Software Technology 91 (2017), 160--185.
[20]
Brent Hailpern and Padmanabhan Santhanam. 2002. Software debugging, testing, and verification. IBM Systems Journal 41, 1 (2002), 4--12.
[21]
Slinger Jansen, Anthony Finkelstein, and Sjaak Brinkkemper. 2009. A sense of community: A research agenda for software ecosystems. In 2009 31st International Conference on Software Engineering-Companion Volume. IEEE, 187--190.
[22]
Konstantinos Manikas. 2016. Revisiting software ecosystems Research: A longitudinal literature study. Journal of Systems and Software 117 (2016), 84--103. https://doi.org/10.1016/j.jss.2016.02.003
[23]
Konstantinos Manikas and Klaus Marius Hansen. 2013. Software Ecosystems - A Systematic Literature Review. J. Syst. Softw. 86, 5 (May 2013), 1294--1306.
[24]
Glenford J Myers, Corey Sandler, and Tom Badgett. 2011. The art of software testing. John Wiley & Sons.
[25]
Valdemar Vicente Graciano Neto, Fabio Basso, Rodrigo Pereira dos Santos, Noor Hasrina Bakar, Mohamad Kassab, Claudia Werner, Toacy Oliveira, and Elisa Yumi Nakagawa. 2019. Model-driven engineering ecosystems. In Proceedings of the 7th International Workshop on Software Engineering for Systems-of-Systems and 13th Workshop on Distributed Software Development, Software Ecosystems and Systems-of-Systems. IEEE Press, 58--61.
[26]
Srinivas Nidhra and Jagruthi Dondeti. 2012. Black box and white box testing techniques-a literature review. International Journal of Embedded Systems and Applications (IJESA) 2, 2 (2012), 29--50.
[27]
Marcelo Iury S Oliveira, Glória de Fátima Barros Lima, and Bernadette Farias Lóscio. 2019. Investigations into Data Ecosystems: a systematic mapping study. Knowledge and Information Systems (2019), 1--42.
[28]
Italo Santos, Silvana Morita Melo, Paulo Sergio Lopes de Souza, and Simone RS Souza. 2020. Towards a unified catalog of attributes to guide industry in software testing technique selection. In 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 398--407.
[29]
Martín Soto and Marcus Ciolkowski. [n. d.]. The QualOSS open source assessment model measuring the performance of open source communities. In Proceedings of the Third International Symposium on Empirical Software Engineering and Measurement, ESEM 2009, October 15-16, 2009, Lake Buena Vista, Florida, USA.
[30]
Caio Steglich, Sabrina Marczak, Cleidson RB de Souza, Luiz Pedro Guerra, Luiz Henrique Mosmann, Marcelo Perin, et al. 2019. Social aspects and how they influence MSECO developers. In Proceedings of the 12th International Workshop on Cooperative and Human Aspects of Software Engineering. IEEE Press, 99--106.
[31]
Caio Steglich, Sabrina Marczak, Luiz Pedro Guerra, Luiz Henrique Mosmann, Marcelo Perin, Fernando Figueira Filho, and Cleidson de Souza. 2019. Revisiting the Mobile Software Ecosystems Literature. In Proceedings of the 7th International Workshop on Software Engineering for Systems-of-Systems and 13th Workshop on Distributed Software Development, Software Ecosystems and Systems-of-Systems (SESoS-WDES '19). 50--57.
[32]
Joost te Molder, Ben van Lier, and Slinger Jansen. 2011. Clopenness of Systems: The Interwoven Nature of Ecosystems. In IWSECO@ ICSOB. 52--64.
[33]
Dolores R. Wallace and Roger U. Fujii. 1989. Software verification and validation: an overview. Ieee Software 6, 3 (1989), 10--17.
[34]
Liguo Yu. 2012. The market-driven software ecosystem. IT Professional 15, 5 (2012), 46--50.

Cited By

View all
  • (2024)From Literature to Practice: Exploring Fairness Testing Tools for the Software Industry AdoptionProceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3695404(549-555)Online publication date: 24-Oct-2024
  • (2023)Comparing Mobile Testing Tools Using Documentary Analysis2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)10.1109/ESEM56168.2023.10304798(1-6)Online publication date: 26-Oct-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
SBES '20: Proceedings of the XXXIV Brazilian Symposium on Software Engineering
October 2020
901 pages
ISBN:9781450387538
DOI:10.1145/3422392
© 2020 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

In-Cooperation

  • SBC: Brazilian Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 December 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Software testing
  2. research opportunities
  3. software ecosystems

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • Conselho Nacional de Desenvolvimento Científico e Tecnológico
  • Fundação de Amparo à Pesquisa do Estado de São Paulo

Conference

SBES '20

Acceptance Rates

Overall Acceptance Rate 147 of 427 submissions, 34%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)22
  • Downloads (Last 6 weeks)2
Reflects downloads up to 27 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)From Literature to Practice: Exploring Fairness Testing Tools for the Software Industry AdoptionProceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3695404(549-555)Online publication date: 24-Oct-2024
  • (2023)Comparing Mobile Testing Tools Using Documentary Analysis2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)10.1109/ESEM56168.2023.10304798(1-6)Online publication date: 26-Oct-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media