skip to main content
research-article

Impact of classes of development coordination tools on software development performance: A multinational empirical study

Published: 05 May 2008 Publication History

Abstract

Although a diverse variety of software development coordination tools are widely used in practice, considerable debate surrounds their impact on software development performance. No large-scale field research has systematically examined their impact on software development performance. This paper reports the results of a multinational field study of software projects in 209 software development organizations to empirically examine the influence of six key classes of development coordination tools on the efficiency (reduction of development rework, budget compliance) and effectiveness (defect reduction) of software development performance.
Based on an in-depth field study, the article conceptualizes six holistic classes of development coordination tools. The results provide nuanced insights—some counter to prevailing beliefs—into the relationships between the use of various classes of development coordination tools and software development performance. The overarching finding is that the performance benefits of development coordination tools are contingent on the salient types of novelty in a project. The dimension of development performance—efficiency or effectiveness—that each class of tools is associated with varies systematically with whether a project involves conceptual novelty, process novelty, multidimensional novelty (both process and conceptual novelty), or neither. Another noteworthy insight is that the use of some classes of tools introduces an efficiency-effectiveness tradeoff. Collectively, the findings are among the first to offer empirical support for the varied performance impacts of various classes of development coordination tools and have important implications for software development practice. The paper also identifies several promising areas for future research.

References

[1]
Abdel-Hamid, T., Sengupta, K., and Ronan, D. 1993. Software project control: An experimental investigation of judgment with fallible information. IEEE Trans. Soft. Engin. 19, 6, 603--612.
[2]
Adelson, B. and Soloway, E. 1985. The role of domain experience in software design. IEEE Trans. Soft. Engin. 11, 11, 1351--1360.
[3]
Adler, P. S. 1995. Interdepartmental interdependence and coordination---the case of the design/manufacturing interface. Organiz. Sci. 6, 2, 147--167.
[4]
Aiken, L. and West, S. 1991. Multiple Regression: Testing and Interpreting Interactions. Sage, Newbury, CA.
[5]
Andres, H. P. and Zmud, R. W. 2002. A contingency approach to software project coordination. J. Manag. Info. Sys. 18, 3, 41--70.
[6]
Babbie, E. R. 1973. Survey Research Methods. Wadsworth, Belmont, CA.
[7]
Bagozzi, R. P., Yi, Y., and Phillips, L.W. 1991. Assessing construct validity in organizational research. Admin. Sci. Quarterly. 36, 3, 421--458.
[8]
Banker, R., Davis, G., and Slaughter, S. 1998. Software development practices, software complexity, and software maintenance performance. Manag. Sci. 44, 4, 433--450.
[9]
Barki, H., Rivard, S., and Talbot, J. 2001. An integrative contingency model of software project risk management. J. Manag. Info. Syst. 17, 4, 37--69.
[10]
Baron, R. and Kenny, D. 1986. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. J. Person. Soc. Psychol. 51, 1173--1182.
[11]
Baskerville, R. and Pries-Heje, J. 1999. Grounded action research: A method for understanding IT in practice. Account. Manag. Info. Techno. 9, 1, 1--23.
[12]
Belsley, D.A., Kuh, E., and Welsch, R.E. 1980. Regression Diagnostics: Identifying Influential Data and Forces of Collinearity. Wiley, New York.
[13]
Brown, D. 2003. The developer's art today: Aikido or sumo? ACM Queue 1, 6.
[14]
Cohen, J. and Cohen, P. 1983. Applied Multiple Regression Analysis for the Behavioral Sciences. Lawrence Erlbaum, Hillsdale.
[15]
Creswell, J. 1994. Research Design: Qualitative and Quantitative Approaches. Sage Publications, Newbury, CA.
[16]
Cronbach, L.J. and Meehl, P.E. 1955. Construct validity in psychological tests. Psychol. Bull. 52, 4, 281--302.
[17]
Duncan, O. 1985. Path analysis: Sociological examples. In Causal Models in Social Sciences, H. Blalock (Ed.). Aldine Publishing, Hawthorne, NY,
[18]
Faraj, S. and Sproull, L. 2000. Coordinating expertise in software development teams. Manag. Sci. 46, 12, 1554--1568.
[19]
Glaser, B. and Strauss, A. 1967. The discovery of Grounded Theory: Strategies for Qualitative Research. Aldine Publishing Company, Chicago.
[20]
Gopal, A., Sivaramakrishnan, K. Krishnan, M., and Mukhopadhyay, T. 2003. Contracts in offshore software development: An empirical analysis. Manag. Sci. 49, 12, 1671--1683.
[21]
Gottesdiener, E. 2003. Requirements by collaboration. IEEE Soft. March--April, 52--55.
[22]
Groth, R. 2004. Is the software industry's productivity declining? IEEE Soft. 21, 6, 92--94.
[23]
Guinan, P., Cooprider, J., and Sawyer, S. 1997. The effective use of automated application development tools. IBM Syst. J. 36, 1, 124--139.
[24]
Hair, J.F., JR., Anderson, R.E. Tatham, R.L., and Black, W.C. 1995. Multivariate Data Analysis. Prentice Hall, Englewood Cliffs, NJ.
[25]
Ho, V., Ang, S., and Straub, D. 2003. When subordinates become IT contractors: Persistent managerial expectations in IT outsourcing. Inform. Syst. Resear. 14, 1, 66--86.
[26]
King, S. and Galliers, R. 1994. Modelling the case process: Empirical issues and future directions. Info. Softw. Tech. 36, 10, 587--596.
[27]
Kirsch, L., Sambamurthy, V. Ko, D., and Purvis, R. 2002. Controlling information systems development projects: The view from the client. Manag. Sci. 48, 4, 484--498.
[28]
Kokol, P. 1989. Formalization of the information system development process using metamodels. ACM SIGSOFT Softw. Engin. Notes 14, 5, 118--122.
[29]
Kumar, N., Stern, L.W., and Anderson, J.C. 1993. Conducting interorganizational research using key informants. Academy Manag. J. 36, 6, 1633--1651.
[30]
Lawther, W.C. 1986. Content validation: Conceptual and methodological issues. Rev. Public Person. Admini 6, 3, 37--49.
[31]
Lee, A.S. A scientific methodology for MIS case studies, MIS Quart., 1989, 33--50.
[32]
Maccormack, A., Verganti, R., and Iansiti, M. 2001. Developing products on internet time: The anatomy of a flexible development process. Manag. Sci. 47, 1, 133--150.
[33]
Mcafee, A. 2003. When too much IT knowledge is a dangerous thing. MIT Sloan Manag. Rev. 44, 2, 83--89.
[34]
Messerschmitt, D. and Szyperski, C. 2003. Software Ecosystem. MIT Press, Cambridge, MA.
[35]
Mingers, J., 2001. Combining IS research methods: Towards a pluralist methodology. Info. Sys. Resea. 12, 3, 240--259.
[36]
Mookerjee, V.S. and Chiang, R. 2002. A dynamic coordination policy for software system construction. IEEE Trans. Softw. Engin. 28, 6, 684--694.
[37]
Morgan, G. and Smircich, L., Sawyer, S. 1980. The case for qualitative research. Acad. Manag. Rev. 5, 4, 491--500.
[38]
Mumford, M., Costanza, D., and Connelly, M. 1996. Item generation procedures and background data scales: Implications for construct and criterion-related validity. Person. Psychol. 49, 2, 361--398.
[39]
Orlikowski, W. 1989. Division among the ranks: The social implications of case tools for system developers,. In Proceedings of the Tenth International Conference on Information Systems, J. Degross, J. Henderson and B. Konsynski Eds. ACM Press.
[40]
Orlikowski, W. J. 1993. Case tools as organizational change: Investigating incremental and radical changes in systems development. MIS Quart. 17, 309--340.
[41]
Paese, P.W. and Switzer, F.S., III. 1988. Validity generalization and hypothetical reliability distributions: A test of the schmidt-hunter procedure. J. Appl. Psychol. 73, 2, 267--274.
[42]
Pedhazur, E.J. 1982. Multiple Regression in Behavioral Research: Explanation and Prediction. Dryden Press, New York, NY.
[43]
Ramesh, B. 1998. Factors influencing requirements traceability practice. Comm. ACM 41, 12, 37--44.
[44]
Ramesh, B. and Dhar, V. 1992. Supporting systems development by capturing deliberations during requirements engineering. IEEE Trans. Softw. Engin. 18, 498--510.
[45]
Ramesh, B. and Jarke, M. 2001. Towards reference models for requirements traceability. IEEE Trans. Softw. Engin. 27, 1, 58--93.
[46]
Robillard, P. 1999. The role of knowledge in software development. Comm. ACM 42, 1, 87--92.
[47]
Rowen, R. 1990. Software project management under incomplete and ambiguous specifications. IEEE Trans. Engin. Manag. 37, 1, 10--21.
[48]
Rus, I. and Lindvall, M. 2002. Knowledge management in software engineering. IEEE Softw. 19, 3, 26--38.
[49]
Schmidt, F. and Hunter, J. 1989. Interrater reliability coefficients cannot be computed when only one stimulus is rated. J. Appl. Psychol. 74, 368--370.
[50]
Schwab, D.P. 1980. Construct validity in organizational behavior. Research in Organizational Behavior. 2, 3--43.
[51]
Seaman, C. and Basili, V. 1998. Communication and organization: An empirical study of discussion in inspection meetings. IEEE Trans. Softw. Engin. 24, 7, 559--572.
[52]
Seeley, D. 2003. Coding smart: People vs. Tools. ACM Queue 1, 6.
[53]
Shore, T.H., Shore, L.M., and Thoronton, G.C., III. 1992. Construct validity of self- and peer evaluations of performance dimensions in an assessment center. J. Appl. Psychol. 77, 1, 42--54.
[54]
Spinellis, D. 2005. The tools at hand. IEEE Softw. 26, 1, 10--12.
[55]
Straub, D.W. 1989. Validating instruments in MIS research. MIS Quart. 13, 2, 147--166.
[56]
Strauss, A. and Corbin, J. 1998. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Sage Publications, Inc., Newbury, CA.
[57]
Takeishi, A. 2002. Knowledge partitioning in the interfirm division of labor: The case of automotive product development. Organiz. Sci. 13, 3, 321--338.
[58]
Tiwana, A. and Keil, M. 2004. The one minute risk assessment tool. Comm. ACM 47, 11, 73--77.
[59]
Tiwana, A. and Mclean, E. 2003. Managing the unexpected: The tightrope to e-business project success. Comm. ACM 46, 12, 345--350.
[60]
Tiwana, A. and Mclean, E. R. 2005. Expertise integration and creativity in information systems development. J. Manag. Info. Sys. 22, 1, 13--43.
[61]
Todd, J.D. 1979. Mixing qualitative and quantitative methods: Triangulation in action. Administrative Sciences Quart. 24, 4, 602--611.
[62]
Walz, D., Elam, J., and Curtis, B. 1993. Inside a software design team: Knowledge, sharing, and integration. Comm. ACM 36, 10, 63--77.
[63]
Weekley, J.A. and Gier, J.A. 1989. Ceilings in the reliability and validity of performance ratings: The case of expert raters. Acad. Manag. J. 32, 1, 213--222.
[64]
Yin, R.K. 1994. Case Study Research: Design and Methods. Sage Publications, Newbury, CA.

Cited By

View all
  • (2023)Information Technology Project Management Research: A Review of Works by Influential PioneersProject Management Journal10.1177/8756972823117105654:4(366-391)Online publication date: 29-May-2023
  • (2021)Software Development Process Ambidexterity and Project Performance: A Coordination Cost-Effectiveness ViewIEEE Transactions on Software Engineering10.1109/TSE.2019.290457147:4(836-849)Online publication date: 1-Apr-2021
  • (2012)Selective availability: coordinating interaction initiation in distributed software developmentIET Software10.1049/iet-sen.2011.00776:3(185)Online publication date: 2012

Index Terms

  1. Impact of classes of development coordination tools on software development performance: A multinational empirical study

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Software Engineering and Methodology
      ACM Transactions on Software Engineering and Methodology  Volume 17, Issue 2
      April 2008
      207 pages
      ISSN:1049-331X
      EISSN:1557-7392
      DOI:10.1145/1348250
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 05 May 2008
      Accepted: 01 March 2007
      Revised: 01 March 2006
      Received: 01 August 2005
      Published in TOSEM Volume 17, Issue 2

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Software development
      2. collaborative software engineering. Software outsourcing
      3. coordination
      4. development coordination tools
      5. development tools
      6. efficiency effectiveness tradeoff
      7. empirical study
      8. field study
      9. knowledge integration
      10. knowledge management
      11. outsourcing
      12. project management
      13. regression analysis

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)6
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 07 Mar 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2023)Information Technology Project Management Research: A Review of Works by Influential PioneersProject Management Journal10.1177/8756972823117105654:4(366-391)Online publication date: 29-May-2023
      • (2021)Software Development Process Ambidexterity and Project Performance: A Coordination Cost-Effectiveness ViewIEEE Transactions on Software Engineering10.1109/TSE.2019.290457147:4(836-849)Online publication date: 1-Apr-2021
      • (2012)Selective availability: coordinating interaction initiation in distributed software developmentIET Software10.1049/iet-sen.2011.00776:3(185)Online publication date: 2012

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media