How project description length and expected duration affect bidding and project success in crowdsourcing software development
Introduction
Crowdsourcing holds considerable economic promise to companies (Lacity and Reynolds, 2014). Crowdsourcing is a method of letting out work to many potential providers on the Internet by publishing the request for proposals (RFP) through an online marketplace.1 Indeed, crowdsourcing software development markets (CSMs) are increasingly becoming a viable alternative to more traditional software outsourcing markets. Sites specializing in managing such contracting, especially of small projects, have indeed seen enormous growth in recent years (Carmel and Abbott, 2007, Gefen and Carmel, 2013, Zhao and Zhu, 2014). Despite their increasing popularity, little has been published about such markets relative to more traditional IT software and related outsourcing service configurations in comprehensive literature review papers (e.g. McLeod and MacDonell, 2011, Lacity et al., 2010). This is an important gap because drawing conclusions from typical software outsourcing projects to CSMs may not be as straightforward considering that CSM projects are orders of magnitude smaller. CSM projects typically last only several days, rather than months or years in more traditional software development and services outsourcing contexts. Moreover, CSMs are only outsourced on a fixed price (FP) basis which makes risk allocation and control in such markets different than in traditional outsourcing contracting.2
A software development market is an online site where potential buyers post RFPs with the details of the software project or related services they need developed or provided and its estimated delivery date, and where potential coders then bid on those RFPs. The CSM site manages and oversees the process by creating the trading environment and related escrow services. Typically in traditional outsourcing contracts, the more risky the project is, the higher the chances are that it will be outsourced as a Time and Materials (T&M) contract rather than as a FP one. The allocation of some projects, or parts of projects, as T&M rather than FP allows for a more even balance of risks between buyer and provider in traditional software outsourcing contracts (Fink and Lichtenstein, 2014, Gopal and Sivaramakrishnan, 2008, Gopal et al., 2003). That option does not exist in CSM where all the risk is borne entirely by the provider. Indeed, in a CSM the buyer can refuse to accept and pay for the project if it is not to its satisfaction without recourse. All these combine to make CSM rather unique compared to traditional software outsourcing.
In this study we examine how providers respond in their bid amounts to the buyer's description length of the project and estimation of its duration. As might be expected, providers bid on average higher on longer description and longer duration projects. However, the providers actually bid higher than they should have done based on description and duration length alone in projects that eventually proved to be unsuccessful. The reason for that may be that, as our post-hoc analysis shows, unsuccessful projects were mostly shorter than successful ones, and buyers were less accurate in their estimation of the duration of shorter projects. The providers were apparently responding quite rationally in bidding relatively higher to cover their higher duration risks in shorter duration, and therefore more risky, projects. Providers bid higher on short duration projects because those are more likely to fail.
CSMs have been studied in the past, but the little that has been published about CSM is centered on the buyers, their risks, and their decision making (Carmel and Tjia, 2005, Gefen and Carmel, 2008, Gefen and Carmel, 2013, Snir and Hitt, 2003, Snir and Hitt, 2004). In this article we examine CSM as viewed by the providers. Studying provider behavior in a CSM thus presents a mostly new context of a little studied but important software development and service context. The theory base applied is based on Agency Theory (Eisenhardt, 1989), but reversed so that it deals with the risks the agents face rather than the risks the principal has.
Section snippets
Theory
Agency Theory (Eisenhardt, 1989) visions business transactions as contracts between principals who let out work and agents who perform it. The terms principal and agent are how this theory names the buyer and the provider, respectively. In relation to a CSM, the buyer, i.e. the person who posts the RFP, is the principal and the providers bidding on the RFP to provide its requested services are the agents. A central issue Agency Theory deals with is information asymmetry as viewed from the
Hypotheses
As a reversed agency problem we look at CSM in view of the adverse selection risks the agent faces. Moral hazard is not a concern in this case because the principal is not creating anything for the agent after the contract has been signed so the agent need not be concerned that it cannot control the quality of what the principal is doing after the contract has been signed. Likewise, the projects are too short for unforeseen contingency risks such as changing specifications to be a major concern.
The data
The data for this study came from one of the leading CSMs. The data contained all the RFP and all the bids on them in 2005 and 2006. The owners gave the authors the data and permission to publish papers based on it in exchange for previewing the early results of the analyses. The company that provided the data operated in a manner that is equivalent of the other leading CSM companies. It was a privately owned company located in the US. There were 10,568 principals in the CSM during that period.
Data analysis
The data analyses were done in several stages. In the first stage a GLM was run to test H1 and H2, and in the process create the best fitting line that predicts the Amount Bid based on Duration and Project description length. Duration and Project description length were entered as covariates. The GLM also included the RFP number as a class variable to account for the fact that there would be shared variance among all the amounts bid on the same RFP. The unit of analysis at this stage was the
Post-hoc analysis
At the core of Agency Theory is the assumption that information asymmetry adds risks to the principals, who, in turn, take actions to reduce that risk. Applied in reverse to the agents, the theory implies that information asymmetry created by the principals, because it increases uncertainty to the agents, should increase the risks that the agents face, and that the agents will accordingly do what they can to reduce that risk. The objective of the post-hoc analysis was to verify that this
Summary of results
Analyzing a leading crowdsourcing software development site over a 2 year period shows some expected and some unexpected yet informative results. Based on a direct application of Contract Theory (Bolton and Dewatripomt, 2005), longer duration projects and projects that are described in greater length, and therefore can be presumably assumed to be larger projects, are bid at higher amounts (H1 and H2). That is a rational response of the agents to the signals the principals are giving them: if
David Gefen is a Professor of MIS at Drexel University, and the Provost Distinguished Research Professor. David teaches applied statistics and data-mining with SAS, IS Outsourcing Management, Strategic Management of Information Systems, Database Analysis and Design, and VB.NET programming. David has authored some of the most cited papers in MIS about trust and about gender in the context of IS adoption and management. His research focuses on information systems implementation as well as
References (37)
- et al.
An investigation of factors that influence the duration of IT outsourcing relationships
Decision Support Syst.
(2007) - et al.
A study in machine learning for imbalanced data for sentence boundary detection in speech
Comput. Speech Lang.
(2006) - et al.
The information technology outsourcing risk: a transaction cost and agency theory-based perspective
J. Inf. Technol.
(2003) Software Systems Techniques
(1990)- et al.
Managing IT Outsourcing: Governance in Global Partnerships
(2006) - et al.
Contract Theory
(2005) Principles of Statistics
(1979)- et al.
Why nearshore means that distance matters
Commun. ACM
(2007) - et al.
Offshoring Information Technology Sourcing and Outsourcing to a Global Workforce
(2005) - et al.
Outsourced information systems failures in SMEs: a multiple case study
Electron. J. Inf. Syst. Eval.
(2008)
Information systems outsourcing: a survey and analysis of the literature
Database Adv. Inf. Syst.
Agency theory: an assessment and review
Acad. Manage. Rev.
A method for managing re-identification risk from small geographic areas in Canada
BMC Med. Inf. Decis. Making
Where do capabilities come from and how do they matter? A study in the software services industry
Strategic Manage. J.
Why project size matters for contract choice in software development outsourcing
Database Adv. Inf. Syst.
The Art of Successful Information Systems Outsourcing
Is the world really flat? A look at offshoring in an online programming marketplace
MIS Q.
Why the first provider takes it all: the consequences of a low trust culture on pricing and ratings in online sourcing markets
Eur. J. Inf. Syst.
Cited by (28)
Seekers’ complaint behavior in crowdsourcing: An uncertainty perspective
2022, Journal of Retailing and Consumer ServicesCitation Excerpt :First, our study explores an important yet under-studied phenomenon: seekers’ complaint behavior. Existing research primarily focuses on optimization of task design in crowdsourcing (Zheng et al., 2011; Ye and Kankanhalli, 2015; Allen et al., 2018; Jiang et al., 2020; Moayedikia et al., 2020), solvers’ participation behavior (Terwiesch and Xu, 2008; Bockstedt et al., 2016; Gefen et al., 2016; Ye et al., 2017; Dissanayake et al., 2018, 2019; Zheng et al., 2019; Dargahi et al., 2021), and seekers’ retention (Yang et al., 2020). This research, on the other hand, focuses on a meaningful topic, seekers’ complaint behavior, which despite its apparent practical values has to date received little attention in crowdsourcing research.
Treating the crowd fairly: Increasing the solvers’ self-selection in idea innovation contests
2020, Industrial Marketing ManagementCost and time project management success factors for information systems development projects
2017, International Journal of Project ManagementCitation Excerpt :These findings corroborate previous studies (Benaroch et al., 2007; Fichman et al., 2005; Lewis et al., 2004). Although some previous studies affirm that outsourcing IS projects has a positive effect on project cost and time (Gefen et al., 2016; Gorla and Somers, 2014; Schwarz, 2014; Srivastava and Teo, 2012), our study did not show the same result. Our hypothesis that a higher level of project outsourcing has a positive effect on PMS was not supported.
Exploring the trends, characteristic antecedents, and performance consequences of crowdsourcing project risks
2016, International Journal of Project ManagementCitation Excerpt :Finally, this study determines that project characteristics differentially influence the risk dimensions in crowdsourcing projects, thus filling the void of project characteristic–risk relationship in the context of crowdsourcing. This result provides empirical evidence for the perspective of Gefen et al. (2016), who contended that the change in several key characteristics of crowdsourcing projects led to different risk levels. Although no single project characteristic significantly affects all risk dimensions simultaneously, each risk dimension will vary if all project characteristics change.
David Gefen is a Professor of MIS at Drexel University, and the Provost Distinguished Research Professor. David teaches applied statistics and data-mining with SAS, IS Outsourcing Management, Strategic Management of Information Systems, Database Analysis and Design, and VB.NET programming. David has authored some of the most cited papers in MIS about trust and about gender in the context of IS adoption and management. His research focuses on information systems implementation as well as informatics analyses of transactional data as these apply to the psychological and rational processes involved in the management of many types of information systems. Before becoming an academic David was a programmer and systems analyst, and then senior manager of a large logistics information system. His research findings have been published in some of the leading journals, including MISQ, ISR, IEEE TEM, JMIS, Omega, and Journal of the Association for Information Systems. David is an author of a textbook on Advanced VB.NET Programming Web and Desktop Applications in ADO.NET and ASP.NET in 2004 and a book on the Art of IS Outsourcing in 2011. David is on the Editorial Board of MISQ and JMIS.
Gavriel Gefen is a graduate student in the Business Analytics program at the LeBow College of Business at Drexel University. He is also an E.I.T. civil engineer certified by the state of PA.
Erran Carmel is Dean of the Kogod School of Business, Management of Global Information Technology, American University, Washington, DC. Professor Erran Carmel studies various kinds of sourcing in the context of the globalization of technology work. He recently completed his third book about the special issues that time zone separation imposes on global coordination of work. His interest in the topic of this paper has now taken him to research crowdsourcing and the “human cloud.” He is a Professor of the Information Technology department, Kogod School of Business at American University in Washington, DC.