Abstract
Establishing a research strategy that is suitable for undertaking research on software engineering is vital if we are to guarantee that research products are developed and validated following a systematic and coherent method. We took this into account as we carried out the COMPETISOFT research project, which investigated software process improvement (SPI) in the context of Latin American small companies. That experience has enabled us to develop a research strategy based on the integrated use of action research and case study methods. This paper introduces the proposed research strategy and provides extensive discussion of its application for: (1) developing the Methodological framework of COMPETISOFT for SPI, (2) putting this framework into practice in eight small software companies, and (3) refine the Methodological framework due to the practice feedback. The use of this research strategy allowed us to observe that it was suitable for developing, refining, improving, applying, and validating COMPETISOFT’s Methodology framework. Furthermore, having seen it applied, we believe that this strategy offers a successful integration of action research and case study, which can be useful for conducting research in other software engineering areas which address needs of small software companies.
Similar content being viewed by others
References
Aaen, I., Arent, J., Mathiassen, L., & Ngwenyama, O. (2001). A conceptual MAP of software process improvement. Scandinavian Journal of Information Systems, 13, 81–101.
Avison, D., Lan, F., Myers, M., & Nielsen, A. (1999). Action research. Communications of the ACM, 42(1), 94–97.
Basili, V. (2000). Keynote on “Experimental software engineering”. In Proceedings 7th European workshop on software process technology (EWSPT 2000), Kaprun (Austria). Springer LNCS 1780, pp. 150.
Baskerville, R. (1997). Distinguishing action research from participative case studies. Journal of Systems and Information Technology, 1(1), 25–45.
Baskerville, R. (1999). Investigating information systems with action research. Communications of the Association for Information Systems, 2(19), 1–32.
Baskerville, R., & Wood-Harper, A. T. (1998). Diversity in information systems action research methods. European Journal of Information Systems, 7(2), 90–107.
Benbasat, I., Goldstein, D., & Mead, M. (1987). The case research strategy in studies of information systems. MIS Quarterly, 11(3), 369–386.
Braa, K., & Vidgen, R. (1999). Interpretation, intervention, and reduction in the organizational laboratory: A framework for in-context information system research. Accounting Management and Information Technology, 9, 25–47.
Brereton, P., Kitchenham, B., Budgen, D., & Li, Z. (2008). Using a protocol template for case study planning. In G. Visaggio, M. T. Baldassarre, S. Linkman, & M. Turner (Eds.), Evaluation and assessment in software engineering (pp. 1–8). Bari, Italy: British Computer Society.
Casey, V., & Richardson, I. (2004). A practical application of the IDEAL model. Software Process: Improvement and Practice, 9(3), 123–132.
Checkland, P. (1991). From framework through experience to learning: The essential nature of action research. In H. Nissen, H. Klein, & R. Hirscheim (Eds.), Information systems research: Contemporary approaches and emergent traditions (pp. 397–403). The Netherlands: North Holland.
Chiasson, M., Germonprez, M., & Mathiassen, L. (2009). Pluralist action research: A review of the information systems literature. Information Systems Journal, 19(1), 31–54.
Coghlan, D. (2001). Insider action research projects: Implications for practicing managers. Management Learning, 32, 49–60.
Cruz Mendoza, R., Morales Trujillo, M., Morgado, C. M., Oktaba, H., Ibarguengoitia, G. E., Pino, F. J., et al. (2009). Supporting the software process improvement in very small entities through e-learning: The HEPALE! Project. In 2009 Mexican international conference on computer science (ENC), September 21–25, 2009, pp. 221–231. doi:10.1109/enc.2009.33
CYTED (2015). COMPETISOFT methodological framework on EPF composer (in Spanish). http://alarcos.esi.uclm.es/competisoft/web/completo/index.htm
Davison, R. (1998). An action research perspective of group support systems: How to improve meetings in Hong Kong. Ph.D. Thesis, Hong Kong: City University of Hong Kong.
Davison, R., Martinsons, M. G., & Kock, N. (2004). Principles of canonical action research. Information Systems Journal, 14(1), 65–86.
Dittrich, Y., Rönkkö, K., Eriksson, J., Hansson, C., & Lindeberg, O. (2008). Cooperative method development. Combining qualitative empirical research with method, technique and process improvement. Empirical Software Engineering, 13(3), 231–260.
French, W. L., & Bell, C. H. (1999). Organization development: Behavioral science interventions for organization improvement. London: Prentice-Hall.
Garzás, J., Pino, F., Piattini, M., & Fernández, C. (2013). A maturity model for the Spanish software industry based on ISO standards. Computer Standards & Interfaces, 35(6), 616–628.
Glass, R. L., Vessey, I., & Ramesh, V. (2002). Research in software engineering: An analysis of the literature. Information and Software Technology, 44(8), 491–506.
Gustavsen, B. (1993). Action research and the generation of knowledge. Human Relations, 46, 1361–1365.
Hareton, L., & Terence, Y. (2001). A process framework for small projects. Software Process: Improvement and Practice, 6(2), 67–83.
Hernández, M., Florez, A., Pino, F., Garcia, F., Piattini, M., & Ibargüengoitia, G., et al. (2008). Supporting the improvement process for small software enterprises through a software tool. In Software engineering symposium during ninth Mexican international conference on computer science (ENC’08). October, 2008. SES Proceedings, Mexicali, México, pp. 1–8.
Höst, M., & Runeson, P. (2007) Checklists for software engineering case study research. In Proceedings of 1st international symposium on empirical software engineering & measurement (ESEM). IEEE Computer Society Press, pp. 479–482.
Howard, M., Vidgen, R., & Powell, P. (2004). Exploring industry dynamics in e-procurement: Sense making by collaborative investigation. In 13th European conference on information systems, Turku, Finland, pp. 1–12.
ISO (2011). ISO/IEC 29110. Software engineering—lifecycle profiles for very small entities (VSEs). Genova: International Organization for Standardization.
Iversen, J., & Mathiassen, L. (2003). Cultivation and engineering of a software metrics program. Information Systems Journal, 13(1), 3–20.
Iversen, J., Mathiassen, L., & Nielsen, P. (2004). Managing risk in software process improvement: An action research approach. MIS Quarterly, 28(3), 395–433.
Johnson, D. L., & Brodman, J. G. (1999). Tailoring the CMM for small businesses, small organizations, and small projects. In K. El Emam & N. H. Madhavji (Eds.), Elements of software process assessment and improvement (pp. 239–259). Silver Spring, MD: IEEE CS Press.
Kauppinen, M., Vartiainen, M., Kontio, J., Kujala, S., & Sulonen, R. (2004). Implementing requirements engineering processes throughout organizations: Success factors and challenges. Information and Software Technology, 46(14), 937–953.
Kock, N., McQueen, R., & Scott, J. (1997). Can action research be made more rigorous in a positivistic sense? The contribution of an interpretive approach. Journal of Systems and Information Technology, 1, 1–24.
Laporte, C., Alexandre, S., & O’Connor, R. (2008a). A software engineering lifecycle standard for very small enterprises. In EuroSPI 2008, Dublin, Ireland. Springer CCIS, pp. 129–141.
Laporte, C., Alexandre, S., Renault, A., & Crowder, K. V. (2008b). The development of international standards for very small enterprises. In 18th annual international symposium of INCOSE (INCOSE 2008), Amsterdam, The Netherlands, pp. 1–12.
Levin, M. (1993). Creating networks for rural economic development in Norway. Human Relations, 46, 193–218.
Luzuriaga, J., Martínez, R., & Cechich, A. (2008). Setting SPI practices in Latin America: An exploratory case study in the justice area. In 2nd international conference on theory and practice of electronic governance (ICEGOV 2008), pp. 172–177.
Martínez-Ruiz, T., Pino, F., León-Pavón, E., García, F., & Piattini, M. (2009). Supporting the process assessment through a flexible software environment. In Software and data technologies (third international conference, ICSOFT 2008). Springer CCIS 47, pp. 187–199.
Mathiassen, L. (2002). Collaborative practice research. Information Technology & People, 15(4), 321–345.
Mathiassen, L., Pries-Heje, J., & Ngwenyama, O. (2002). Improving software organizations: From principles to practice. Upper Saddle River, NJ: Addison-Wesley.
McKay, J., & Marshall, P. (2001). The dual imperatives of action research. Information Technology & People (Special Issue on Action Research in Information Systems), 14(1), 46–59.
McTaggart, R. (1991). Principles for participatory action research. Adult Education Quarterly, 41(3), 168–187.
Medeiros, P. S., & Travassos, G. (2011). Action research can swing the balance in experimental software engineering. In M. Zelkowitz (Ed.), Advances in computers (Vol. 86, pp. 205–276). Burlington: Academia Press.
Mejía, R., López, A., & Molina, A. (2007). Experiences in developing collaborative engineering environments: An action research approach. Computers in Industry, 58(4), 329–346.
Montesi, M., & Lago, P. (2008). Software engineering article types: An analysis of the literature. Journal of Systems and Software, 81(10), 1694–1714.
Moody, D. (2000). Building links between IS research and professional practice: Improving the relevance and impact of IS research. In Proceedings of the 21st international conference on information systems, Brisbane (Australia), pp. 351–360.
Oktaba, H. (2006). MoProSoft®: A software process model for small enterprises. In S. Garcia (Ed.), Proceedings of the first international research workshop for process improvement in small settings (pp. 93–101). Pittsburgh: Carnegie Mellon University.
Oktaba, H., Garcia, F., Piattini, M., Pino, F., Alquicira, C., & Ruiz, F. (2007). Software process improvement: The COMPETISOFT project. IEEE Computer, 40(10), 21–28.
Philips, P. A. (1998). Disseminating and applying the best evidence. Medical Journal of Australia (MJA), 168, 260–261.
Pino, F., Garcia, F., & Piattini, M. (2008). Software process improvement in small and medium software enterprises: A systematic review. Software Quality Journal, 16(2), 237–261.
Pino, F., García, F., & Piattini, M. (2009). An integrated framework to guide software process improvement in small organizations. In European systems & software process improvement and innovation (EuroSPI 2009), Madrid, Spain. CCIS 42, Springer, pp. 213–224.
Pino, F., Ruiz, F., García, F., & Piattini, M. (2012). A software maintenance methodology for small organizations: Agile_MANTEMA. Journal of Software Maintenance and Evolution: Research and Practice, 24(8), 851–876.
Polo, M., Piattini, M., & Ruiz, F. (2002). Using a qualitative research method for building a software maintenance methodology. Software Practice and Experience, 32(13), 1239–1260.
Richardson, I. (2001). Software process matrix: A small company SPI model. Software Process: Improvement and Practice, 6(3), 157–165.
Robson, C. (2002). Real world research (2nd ed.). Oxford: Blackwell.
Runeson, P., & Höst, M. (2009). Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering, 14(2), 131–164.
Saiedian, H., & Carr, N. (1997). Characterizing a software process maturity model for small organizations. ACM SIGICE Bulletin, 23(1), 2–11.
Seaman, C. B. (1999). Qualitative methods in empirical studies of software engineering. IEEE Transaction on Software Engineering, 25(4), 557–572.
Staples, M., Niazi, M., Jeffery, R., Abrahams, A., Byatt, P., & Murphy, R. (2007). An exploratory study of why organizations do not adopt CMMI. Journal of Systems and Software, 80(6), 883–895.
Travassos, G., Medeiros, P. S., Gomes, P., Dias, A. C., & Biolchini, J. (2008). An environment to support large scale experimentation in software engineering. In Proceedings of the 13th IEEE international conference on engineering of complex computer systems (ICECCS 2008), Belfast, Northern Ireland. CS Digital Library, pp. 193–202.
Wadsworth, Y. (1998). What is participatory Action Research? Action Research International (Paper 2).
Walsham, G. (1993). Interpreting information systems in organizations. Chichester, UK: Wiley.
Wohlin, C. (2005). An analysis of the most cited articles in software engineering journals. Information and Software Technology, 47(15), 957–964.
Wood-Harper, T. (1985). Research methods in information systems: Using action research. In E. Mumford, R. Hirschheim, G. Fitzgerald, & A. T. Wood-Harper (Eds.), Research methods in information systems (pp. 169–191). Amsterdam, North-Holland: Elsevier Science Publishers B.V.
Wouters, M. (2009). A developmental approach to performance measures—results from a longitudinal case study. European Management Journal, 27(1), 64–78.
Yin, R. K. (2003). Case study research: Design and methods. Newbury Park: Sage Publications.
Zannier, C., Melnik, G., & Maurer, F. (2006). On the success of empirical studies in the international conference on software engineering. In International conference on software engineering (ICSE 2006), Shanghai, China, pp. 341–350.
Zelkowitz, M. (2009). An update to experimental models for validating computer technology. Journal of Systems and Software, 82(3), 373–376.
Acknowledgments
This work has been funded by: (1) the GEODAS-BC project (Ministerio de Economía y Competitividad and Fondo Europeo de Desarrollo Regional FEDER, TIN2012-37493-C03-01), (2) the GLOBALIA project (Consejería de Educación, Ciencia y Cultura y Fondo Europeo de Desarrollo Regional FEDER, PEII11-0291-5274), and (3) LATIn Project (Universidad del Cauca, VRI3930). Francisco J. Pino acknowledges the contribution of the University of Cauca, where he works as a Full Professor.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pino, F.J., García, F., Piattini, M. et al. A research framework for building SPI proposals in small organizations: the COMPETISOFT experience. Software Qual J 24, 489–518 (2016). https://doi.org/10.1007/s11219-015-9278-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11219-015-9278-2