Abstract
Medium-sized, open-participation Open Source Software (OSS) projects do not usually perform explicit software process improvement on any routine basis. It would be useful to understand how to get such a project to accept a process improvement proposal and hence to perform process innovation. We want to determine an effective and feasible qualitative research method for studying the above question. We present (narratively) a case study of how we worked towards and eventually found such a research method. The case involves four attempts at collecting suitable data about innovation episodes (direct participation (twice), polling developers for episodes, manually finding episodes in mailing list archives) and the adaptation of the Grounded Theory data analysis methodology. Direct participation allows gathering rather rich data, but does not allow for observing a sufficiently large number of innovation episodes. Polling developers for episodes did not prove to be useful. Using mailing list archives to find data to be analyzed is both feasible and effective. We also describe how the data thus found can be analyzed based on the Grounded Theory Method with suitable adjustments. By-and-large, our findings ought to apply to studying various phenomena in OSS development processes that are similarly heavyweight and infrequent. However, specific details may block this possibility and we cannot predict which details that might be. The amount of effort involved in direct participation approaches to qualitative research can easily be underestimated. Also, survey approaches are not well-suited for many process issues in OSS, because too few developers are sufficiently process-conscious. An approach based on passive observation is a viable alternative in the OSS context due to the availability of large amounts of fairly complete archival data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
This is opposed to the few large high-profile projects such as Apache, Linux, OpenOffice, or Mozilla, many of which are carried to a large degree by a formal organization (O’Mahony 2005).
Bergquist and Ljungberg (2001) call such behavior towards OSS projects a code gift and consider it an important means for gaining influence.
In particular user lists of tools and APIs commonly used for Open Source development such as http://lists.boost.org/mailman/listinfo.cgi/boost (a C+ + library for common application development tasks), http://lists.gnu.org/mailman/listinfo/autoconf (a package for generating platform-specific scripts to be used in the build-process of applications), http://subversion.tigris.org/ds/viewForumSummary.do?dsForumId=1065 (a software for source code version management), http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org (a multimedia library for accessing input and graphics devices), and http://lists.mozilla.org/listinfo/support-bugzilla (a software for bug-tracking).
We also received a small number of critical comments by email, for instance “1) This is lame. 2) You get paid for this and we don’t”.
The Debian project collects many packages of OSS software, performs package management (in particular dependency modeling) on them, and develops a package management software. Krafft (2009, p.xiii) calls Debian “arguably the largest OSS project with over 1000 developers”.
For instance, the threads titled “Changes to U-Boot Development Process” and “Making it Easier to Contribute to Bugzilla (2007 Edition)” turned out to represent innovation episodes, while “Talking about regressions...” and “Quietly promoting ArgoUML” did not.
In order to avoid distorting the observations, it is important that exactly appropriate concepts be used. This is easiest to achieve when new concepts are being invented specifically for this analysis. When previously existing concepts are used, they have to be “re-invented”.
References
Arthur WB (1989) Competing technologies, increasing returns, and lock-in by historical events. Econ J 99(394):116–131. http://www.jstor.org/stable/2234208
Avison DE, Lau F, Myers MD, Nielsen PA (1999) Action research. Commun ACM 42(1):94–97. doi:10.1145/291469.291479
Barcellini F, Détienne F, Burkhardt JM, Sack W (2008) A socio-cognitive analysis of online design discussions in an Open Source Software community. Interact Comput 20(1):141–165. doi:10.1016/j.intcom.2007.10.004
Berdou E (2007) Managing the bazaar: commercialization and peripheral participation in mature, community-led F/OS software projects. Doctoral dissertation, London School of Economics and Political Science, Department of Media and Communications
Bergquist M, Ljungberg J (2001) The power of gifts: organizing social relationships in Open Source communities. Inf Syst J 11(4):305–320. doi:10.1046/j.1365-2575.2001.00111.x
Buzan T, Buzan B (1993) The Mind Map book. BBC Books, London
Charmaz K (2006) Constructing grounded theory: a practical guide through qualitative analysis, 1st edn. Sage Publications Ltd. http://www.amazon.ca/exec/obidos/redirect?tag=citeulike09-20%&path=ASIN/0761973532
CMMI Product Team (2006) CMMI for development, version 1.2. Tech. Rep. CMU/SEI-2006-TR-008, Software Engineering Institute
Cohen MD, March JG, Olsen JP (1972) A garbage can model of organizational choice. Adm Sci Q 17(1):1–25. http://www.jstor.org/stable/2392088
Corbin JM, Strauss AL (2008) Basics of qualitative research: techniques and procedures for developing grounded theory, 3rd edn. SAGE
David PA (1985) Clio and the economics of QWERTY. Am Econ Rev 75(2):332–337. http://www.jstor.org/stable/1805621
Davison R, Martinsons MG, Kock N (2004) Principles of canonical action research. Inf Syst J 14(1):65–86. doi:10.1111/j.1365-2575.2004.00162.x
Denning PJ, Dunham R (2006) Innovation as language action. Commun ACM 49(5):47–52. doi:10.1145/1125944.1125974
Dietze S (2004) Modell und optimierungsansatz für Open Source Softwareentwicklungsprozesse. Doktorarbeit, Universität Potsdam. http://opus.kobv.de/ubp/volltexte/2005/168/pdf/DIETZE.PDF
Farrand P, Hussain F, Hennessy E (2002) The efficacy of the ‘mind map’ study technique. Med Educ 36(5):426–431. doi:10.1046/j.1365-2923.2002.01205.x
Giddens A (1984) The constitution of society: outline of the theory of structuration. University of California Press, Berkeley
Glaser BG, Strauss AL (1967) The discovery of grounded theory: strategies for qualitative research. Aldine de Gruyter, New York
Hedlund G (1986) The hypermodern MNC—a heterarchy? Hum Resour Manag 25(1):9–35. doi:10.1002/hrm.3930250103
Hertel G, Niedner S, Herrmann S (2003) Motivation of software developers in Open Source projects: an internet-based survey of contributors to the Linux kernel. Research Policy 32(7):1159–1177. doi:10.1016/S0048-7333(03)00047-7, http://www.sciencedirect.com/science/article/B6V77-48BV04S-2/%2/5f8c87b7acd7f8541a97820b22805248 (Open Source Software development)
Highsmith J, Cockburn A (2001) Agile software development: the business of innovation. IEEE Softw 18(5):120–122
Jeffries R, Melnik G (2007) TDD—the art of fearless programming. IEEE Softw 24(3):24–30. doi:10.1109/MS.2007.75
Jorgensen DL (1989) Participant observation: a methodology for human studies, applied social research methods series, vol 15. Sage, Newbury Park, CA
Krafft MF (2009) A Delphi study of the influences on innovation adoption and process evolution in a large open-source project—the case of Debian. PhD thesis, University of Limerick, Ireland, 2009.10.01—version submitted to examiners
Langley A (1999) Strategies for theorizing from process data. Acad Manage Rev 24(4):691–710. http://search.ebscohost.com/login.aspx?direct=true&db=buh&AN=%2553248&site=ehost-live
Lave J, Wenger E (1991) Situated learning: legitimate peripheral participation. Cambridge University Press. http://www.amazon.ca/exec/obidos/redirect?tag=citeulike09-20%&path=ASIN/0521423740
Law J (1992) Notes on the theory of the actor-network: ordering, strategy and heterogeneity. Syst Pract 5(4):379–393
Lessig L (2000) Code and other laws of cyberspace. Basic Books, New York
Li Q, Heckman R, Crowston K, Howison J, Allen E, Eseryel UY (2008) Decision making paths in self-organizing technology-mediated distributed teams. In: Proccedings of the international conference on information systems (ICIS) 2008, Association for Information Systems
Luhmann N (1984) Soziale Systeme. Grundriß einer allgemeinen Theorie. Suhrkamp, Frankfurt am Main, http://www.amazon.ca/exec/obidos/redirect?tag=citeulike09-20%&path=ASIN/3518282662
Manns ML, Rising L (2004) Fearless change: patterns for introducing new ideas. Addison-Wesley. http://www.amazon.ca/exec/obidos/redirect?tag=citeulike09-20%&path=ASIN/0201741571
Mayring P (2002) Einführung in die qualitative Sozialforschung. Beltz, Weinheim. http://hdl.handle.net/2003/23120
Muhr T (1991) Atlas/ti — a prototype for the support of text interpretation. Qual Sociol 14(4):349–371. doi:10.1007/BF00989645
Novak JD, Cañas AJ (2006) The theory underlying concept maps and how to construct them. IHMC CmapTools 2006-01, Florida Institute for Human and Machine Cognition
Novak JD, Gowin DB (1984) Learning how to learn. Cambridge University Press, New York
Oezbek C (2010) Introducing automated regression testing in Open Source projects. Tech. Rep. TR-B-10-01, Freie Universität Berlin, Institut für Informatik, Berlin, Germany. http://www.inf.fu-berlin.de/inst/ag-se/pubs/OSSmediateTR-2008%.pdf
Oezbek C (2010) Introducing innovations into Open Source projects. Doctoral thesis, Freie Universität Berlin (to appear)
Oezbek C, Prechelt L (2007) On understanding how to introduce an innovation to an open source project. In: Proceedings of the 29th international conference on software engineering workshops (ICSEW ’07), IEEE computer society, Washington, DC, USA, reprinted in UPGRADE. The European Journal for the Informatics Professional vol 8(6), pp 40–44
Oezbek C, Schuster R, Prechelt L (2008) Information management as an explicit role in OSS projects: a case study. Tech. Rep. TR-B-08-05, Freie Universität Berlin, Institut für Informatik, Berlin, Germany. http://www.inf.fu-berlin.de/inst/ag-se/pubs/OSSmediateTR-2008%.pdf
O’Mahony S (2003) Guarding the commons: how community managed software projects protect their work. Res Policy 32(7):1179–1198. doi:10.1016/S0048-7333(03)00048-9, http://www.sciencedirect.com/science/article/B6V77-48PM412-1/%2/06fb5992706ae69b561640f2a93a6d4e (Open Source Software development)
O’Mahony S (2005) Nonprofit foundations and their role in community-firm software collaboration. In: Feller J, Fitzgerald B, Hissam SA, Lakhani KR (eds) Perspectives on free and open source software. The MIT Press Ltd., Cambridge, MA, chap 20, pp 393–414
Peirce CS (1883) Studies in Logic, Little, Brown, and Company, Boston, MA, chap A Theory of Probable Inference, pp 126–181
Quintela García L (2006) Die Kontaktaufnahme mit Open Source Software-Projekten. Eine Fallstudie. Bachelor thesis, Freie Universität Berlin. http://projects.mi.fu-berlin.de/w/bin/view/SE/ThesisCommandLi%ne
Raymond ES (1998) Homesteading the Noosphere. First Monday 3(10):n/a. http://www.firstmonday.org/issues/issue3_10/raymond/index.htm%l
Rogers EM (2003) Diffusion of innovations, 5th edn. Free Press, New York
Salinger S, Prechelt L (2008) What happens during pair programming? In: Proceedings of the 20th annual workshop of the psychology of programming interest group (PPIG ’08), Lancaster, England. http://www.ppig.org/workshops/20th-programme.html
Sarma A, Maccherone L, Wagstrom P, Herbsleb J (2009) Tesseract: interactive visual exploration of socio-technical relationships in software development. In: ICSE ’09: Proceedings of the 2009 IEEE 31st international conference on software engineering. IEEE Computer Society, Washington, DC, USA, pp 23–33. doi:10.1109/ICSE.2009.5070505
Schuster R (2005) Effizienzsteigerung freier softwareprojekte durch informationsmanagement. Studienarbeit, Freie Universität Berlin. https://www.inf.fu-berlin.de/w/SE/ThesisFOSSIM
Scott J (1988) Social network analysis. Sociology 22(1):109–127. doi:10.1177/0038038588022001007, http://soc.sagepub.com/cgi/content/abstract/22/1/109, http://soc.sagepub.com/cgi/reprint/22/1/109.pdf
Scotto M, Succi G (eds) (2005) The first international conference on Open Source Systems, Genova
Shaikh M, Cornford T (2003) Version management tools: CVS to BK in the linux kernel. In: Feller J, Fitzgerald B, Hissam S, Lakhani K (eds) Taking stock of the bazaar: the 3rd Workshop on open source software engineering. IEEE Computer Society, Portland, Oregon, pp 127–132
Siefkes C (2007) An incrementally trainable statistical approach to information extraction based on token classification and rich context models. PhD thesis, Freie Universität Berlin, Berlin
Strauss AL, Corbin JM (1998) Basics of qualitative research: techniques and procedures for developing grounded theory, 2nd edn. SAGE. http://www.amazon.co.uk/exec/obidos/ASIN/0803959400/citeulike%-21
Suddaby R (2006a) From the editors: what grounded theory is not. Acad Manage J 49(4):633–642
Suddaby R (2006B) From the editors: what grounded theory is not. Acad Manage J 49(4):633–642
Torvalds L, Diamond D (2001) Just for fun: the story of an accidental revolutionary. HarperCollins
Wenger E (1999) Communities of practice: learning, meaning, and identity. Cambridge University Press. http://www.amazon.ca/exec/obidos/redirect?tag=citeulike09-20%&path=ASIN/0521663636
West J, O’Mahony S (2005) Contrasting community building in sponsored and community founded open source projects. In: 38th annual Hawaii international conference on system sciences. IEEE Computer Society, Los Alamitos, CA, USA, vol 7, p 196c. doi:10.1109/HICSS.2005.166
West J, O’Mahony S (2008) The role of participation architecture in growing sponsored open source communities. Ind Innov 15(2):145–168, 10.1080/13662710801970142
Whittaker JA (2000) What is software testing? and why is it so hard? IEEE Softw 17(1):70–79. doi:10.1109/52.819971
Windeler A (2001) Unternehmungsnetzwerke: konstitution und strukturation. VS Verlag
Yamauchi Y, Yokozawa M, Shinohara T, Ishida T (2000) Collaboration with lean media: how open-source software succeeds. In: CSCW ’00: proceedings of the 2000 ACM conference on computer supported cooperative work, ACM, New York, NY, USA, pp 329–338. doi:10.1145/358916.359004
Yin RK (2003) Case study research: design and methods. Sage. http://www.amazon.de/Case-Study-Research-Design-Methods/dp/07%61925538/
Acknowledgements
The authors would like to thank Robert Schuster, Luis Quintela-Garcia, and Alexander Roßner, who conducted three of the studies as part of their theses, and the participants of the Open Source projects without whom this work would not have been possible. We thank the reviewers for insisting on some changes that make this article a less complicated read.
Author information
Authors and Affiliations
Corresponding author
Additional information
Editors: Carolyn Seaman, Jonathan Sillito, Rafael Prikladnicki, Tore Dybå, and Kari Rönkkö
Rights and permissions
About this article
Cite this article
Prechelt, L., Oezbek, C. The search for a research method for studying OSS process innovation. Empir Software Eng 16, 514–537 (2011). https://doi.org/10.1007/s10664-011-9160-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-011-9160-1