skip to main content
10.1145/2788993.2789844acmotherconferencesArticle/Chapter ViewAbstractPublication PagesopencollabConference Proceedingsconference-collections
research-article

The rise and fall of an online project: is bureaucracy killing efficiency in open knowledge production?

Published: 19 August 2015 Publication History

Abstract

We evaluate the efficiency of an online knowledge production project and identify factors that affect efficiency. To assess efficiency, we used the Data Envelopment Analysis (DEA) modelling methodology. We apply DEA to data from more than 30 Wikipedia language projects over three years. We show that the main Wikipedia projects were indeed less efficient that smaller ones, an effect that can be attributed in part to decreasing returns to scale.

References

[1]
M. Akrich, M. Callon, and B. Latour. Sociologie de la traduction: textes fondateurs. Les presses des Mines de Paris, Paris, 2006.
[2]
R. D. Banker, A. Charnes, and W. W. Cooper. Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis. Management Science, 30(9):1078--1092, 1984.
[3]
J. E. Blumenstock. Size matters: Word count as a measure of quality on wikipedia. In Proceeding of the 17th International Conference on World Wide Web 2008, WWW'08, pages 1095--1096, 2008.
[4]
E. Brynjolfsson and L. M. Hitt. Computing productivity: Firm-level evidence. Review of Economics and Statistics, 85 (4):793--808, 2003.
[5]
B. Butler, L. Sproull, S. Kiesler, and R. Kraut. Community effort in online groups: Who does the work and why? In S. Weisband, editor, Leadership at a distance: Research in Technologically Supported Work. Lawrence Erlbaum, Mahwah, NJ, 2007.
[6]
B. Butler, E. Joyce, and J. Pike. Don't look now, but we've created a bureaucracy: The nature and roles of policies and rules in Wikipedia. In Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, CHI '08, pages 1101--1110, New York, NY, USA, 2008. ACM.
[7]
K. Carillo and C. Okoli. Generating quality open content: A functional group perspective based on the time, interaction, and performance theory. Information & Management, 48(6): 208--219, 2011.
[8]
A. Charnes, W. Cooper, and R. E. Measuring the efficiency of decision making units. European Journal of Operational Research, 2: 429--444, 1978.
[9]
S. Y. Choi, H. Lee, and Y. Yoo. The impact of information technology and transactive memory systems on knowledge sharing, application, and team performance: A field study. MIS quarterly, 34(4):855--870, 2010.
[10]
P. Cohendet, F. Créplet, and O. Dupouet. Interactions between epistemic communities and communities of practice as a mechanism of creation and diffusion of knowledge. In J.-B. Zimmermann and A. Kirman, editors, Interaction and Market Structure. Springer, Londres, 2001.
[11]
B. Collier and J. Bear. Conflict, criticism, or confidence: An empirical examination of the gender gap in Wikipedia contributions. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, CSCW '12, pages 383--392, New York, NY, USA, 2012. ACM.
[12]
W. W. Cooper, L. M. Seiford, and K. Tone. Introduction to data envelopment analysis and its uses: with DEA-solver software and references. Birkhquser, 2006.
[13]
W. W. Cooper, L. M. Seiford, and J. Zhu, editors. Handbook on Data Envelopment Analysis. Springer, 2nd edition, 2011.
[14]
K. Crowston and M. E. Treacy. Assessing the impact of information technology on enterprise level performance. Citeseer, 1986.
[15]
K. Crowston, J. Howison, and H. Annabi. Information system Success in Free and Open Source Software Development: Theory and Measures. Software Process Improvement and Practice, 11: 123--148, 2006.
[16]
K. Crowston, N. Jullien, and F. Ortega. Is Wikipedia Inefficient? Modelling Effort and Participation in Wikipedia. In Forty-sixth Hawai'i International Conference on System Sciences (HICSS-46). IEEE Computer society, January 2013. URL http://ssrn.com/abstract=1960696.
[17]
D. H. Dalip, M. A. Gonçalves, M. Cristo, and P. Calado. Automatic quality assessment of content created collaboratively by web communities: A case study of wikipedia. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries, pages 295--304, 2009.
[18]
M. Douglas. Cultural Bias. Royal Anthropological Institute, London, 1978.
[19]
P. F. Drucker. Knowledge-Worker Productivity: The Biggest Challenge. California Management Review, 41(2):79--94, 1999.
[20]
K. Ehmann, A. Large, and J. Beheshti. Collaboration in context: Comparing article evolution among subject disciplines in wikipedia. First Monday, 13(10), 2008. URL http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2217/2034.
[21]
Y. Fang and D. Neufeld. Understanding Sustained Participation in Open Source Software Projects. Journal on Management Information Systems, 25(4):9--50, Apr. 2009.
[22]
M. J. Farell. The measurement of productive efficiency. Journal of the Royal Statistical Society, Series A, 120(3):250--290, 1957.
[23]
A. Forte, V. Larco, and A. Bruckman. Decentralization in wikipedia governance. Journal of Management Information Systems, 26(1):49--72, 2009.
[24]
A. Halfaker, R. S. Geiger, J. T. Morgan, and J. Riedl. The rise and decline of an open collaboration system how Wikipedia's reaction to popularity is causing its decline. American Behavioral Scientist, 57(5):664--688, 2013.
[25]
R. Hammwöhner. Interlingual aspects of wikipedia's quality. In Proceedings of the 12th International Conference on Information Quality, MIT, Cambridge, MA, USA, pages 477--488. MIT, 2007.
[26]
M. T. Hannan and J. Freeman. Structural inertia and organizational change. American sociological review, pages 149--164, 1984.
[27]
T. J. Hargrave and A. H. Van De Ven. A collective action model of institutional innovation. Academy of Management Review, 31(4):864--888, 2006.
[28]
H. Hasan and C. Pfaff. The wiki: an environment to revolutionise employees' interaction with corporate knowledge. In Proceedings of the Australasian Computer-Human Interaction Conference, Sydney, 2006. OZCHI 2006.
[29]
D. D. Heckathorn. The dynamics and dilemmas of collective action. American Sociological Review, 61: 250--277, Apr. 1996.
[30]
M. Helfen and J. Sydow. Negotiating as Institutional Work: The Case of Labour Standards and International Framework Agreements. Organization Studies, 34(8):1073--1098, 2013.
[31]
C. Hess and E. Ostrom, editors. Understanding Knowledge as a Commons. From Theory to Practice. MIT Press, december 2006.
[32]
C. Hess and E. Ostrom. Introduction: An Overview of the Knowledge Commons. In {31}, editor, Understanding Knowledge as a Commons. From Theory to Practice, pages 3--26. 2006.
[33]
P. Hofmann and D. Riehle. Estimating Commit Sizes Efficiently. In Open Source Ecosystems: Diverse Communities Interacting (IFIP 2.13), volume 299/2009 of IFIP Advances in Information and Communication Technology, pages 105--115. Springer, Springer, 2009. URL http://flosshub.org/sites/flosshub.org/files/EstimatingCommitSizesEffciently.pdf.
[34]
G. Hofstede. Cultures and organizations: Software of the mind. McGraw-Hill, London, 1991.
[35]
S. Javanmardi, Y. Ganjisaffar, C. Lopes, and P. Baldi. User contribution and trust in Wikipedia. In 5th International Conference on Collaborative Computing: Networking, Applications and Worksharing, 2009.
[36]
D. Jemielniak. Common Knowledge?: An Ethnography of Wikipedia. Stanford University Press, 2014.
[37]
W. A. Katz. Introduction to reference work, vol. 1: Basic Information Services. McGraw-Hill, Boston, MA, 8th edition, 2002.
[38]
B. A. Kitchenham. The question of scale economies in software - why cannot researchers agree? Information and Software Technology, 44(1): 13--24, 2002.
[39]
S. Koch. Exploring the effects of sourceforge.net coordination and communication tools on the efficiency of open source projects using data envelopment analysis. Empirical Software Engineering, 14: 397--417, 2009.
[40]
S. Koch. Organisation of work in open source projects: expended effort and efficiency. Revue d'économie industrielle, 136: 17--38, 2011.
[41]
S. K. Lam, A. Uduwage, Z. Dong, S. Sen, D. R. Musicant, L. Terveen, and J. Riedl. WP:Clubhouse? An Exploration of Wikipedia's Gender Imbalance. In Proceedings of the 7th International Symposium on Wikis and Open Collaboration, WikiSym '11, New York, NY, USA, Oct. 2011. ACM.
[42]
D. Lewandowski and U. Spree. Ranking of Wikipedia articles in search engines revisited: Fair ranking for reasonable quality? Journal of the American Society for Information Science and Technology, 62(1):117--132, 2011.
[43]
H.-T. Liao. Conflict and consensus in the chinese version of wikipedia. IEEE Technology and Society Magazine, 28, 2009.
[44]
D. Lindsey. Evaluating quality control of wikipedia's feature articles. First Monday, 15(4), 2010. URL http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2721/2482.
[45]
G. Marwell and P. Oliver. The Critical Mass in Collective Action: A Micro-Social Theory. Cambridge University Press, Cambridge, 1993.
[46]
A. P. McAfee. Enterprise 2.0: The Dawn of Emergent Collaboration. Management of Technology and Innovation, 47 (3), 2006.
[47]
B. L. Myers, L. A. Kappelman, and V. R. Prybutok. A comprehensive model for assessing the quality and productivity of the information systems function: toward a theory for information systems assessment. Information Resources Management Journal (IRMJ), 10(1):6--26, 1997.
[48]
P. V. Norden. On the anatomy of development projects. IRE Transactions on Engineering Management, 7(1):34--42, 1960.
[49]
M. Olson. The logic of Collective Action. Harvard University Press, Cambridge Mass., 1965.
[50]
F. Ortega, D. Izquierdo-Cortazar, J. M. Gonzalez-Barahona, and G. Robles. On the analysis of contributions from privileged users in virtual open communities. In 42th Hawaii International Conference on System Sciences (HICSS), page 110, Waikoloa, Big Island, Hawaii, USA, Jan. 2009. IEEE Computer Society.
[51]
P. Otto and M. Simon. Dynamic perspectives on social characteristics and sustainability in online community networks. System Dynamics Review, 24(3):321--347, 2008.
[52]
G. Poderi. Comparing featured article groups and revision patterns correlations in Wikipedia. First Monday, 14(5), 2009. URL http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2365/2182.
[53]
S. Ransbotham and G. C. Kane. Membership Turnover and Collaboration Success in Online Communities: Explaining Rises and Falls from Grace in Wikipedia. MIS Quarterly, 14 (9):541--545, September 2011.
[54]
J. Reagle. "free as in sexist?" free culture and the gender gap. First Monday, 18(1), 2012.
[55]
F. Rullani and S. Haefliger. The periphery on stage: The intra-organizational dynamics in online communities of creation. Research Policy, 42(4):941--953, 2013.
[56]
S. Sadiq. The Final Frontier: A SAS Approach to Data Envelopment Analysis. In SAS Global Forum 2011 Operations Research, 2011. URL http://support.sas.com/resources/papers/proceedings11/198-2011.pdf. Paper 198--2011.
[57]
B. Simpson, R. Willer, and C. L. Ridgeway. Status Hierarchies and the Organization of Collective Action. Sociological Theory, 30(3):149--166, 2012.
[58]
J. Struben and B. Lee. Market Formation: Examining the Coordination of Heterogeneous Contributions. Technical report, DRUID, Copenhagen Business School, Department of Industrial Economics and Strategy/Aalborg University, Department of Business Studies, 2012.
[59]
B. Stvilia, M. B. Twidale, L. C. Smith, and L. Gasser. Information Quality Work Organization in Wikipedia. Journal of the American Society for Information Science and Technology, 59(6):983--1001, 2008.
[60]
B. Stvilia, A. Al-Faraj, and Y. Yi. Issues of cross-contextual information quality evaluation--The case of Arabic, English, and Korean Wikipedias. Library & Information Science Research, 31(4):232--239, 2009.
[61]
B. Suh, G. Convertino, E. H. Chi, and P. Pirolli. The singularity is not near: Slowing growth of wikipedia. In Proceedings of the 5th International Symposium on Wikis and Open Collaboration, WiKiSym 2009, 2009.
[62]
B. Uzzi. A social network's changing statistical properties and the quality of human innovation. Journal of Physics A: Mathematical and Theoretical, 41(22):224023, 12pgs, June 2008.
[63]
H. R. Varian. Intermediate microeconomics: a modern approach. W. W. Norton, New York, NY, 2nd edition, 2005.
[64]
G. Von Krogh, S. Spaeth, and K. R. Lakhani. Community, joining, and specialization in open source software innovation: A case study. Research Policy, 32(7):1217--1241, 2003.
[65]
D. Wallace and C. Van Fleet. The democratization of information? wikipedia as a reference resource. Reference and User Services Quarterly, 45: 100--103, 2005.
[66]
T. Wöhner and R. Peters. Assessing the quality of Wikipedia articles with lifecycle based metrics. In WikiSym '09: Proceedings of the 5th International Symposium on Wikis and Open Collaboration, pages 1--10, New York, NY, USA, 2009. ACM.
[67]
S.-H. Yu, Y.-G. Kim, and M.-Y. Kim. Do we know what really drives km performance? Journal of Knowledge Management, 11(6):39--53, 2007.
[68]
V. Zlatić and H. Stefancić. Model of Wikipedia growth based on information exchange via reciprocal arcs. EPL, 93(5): 58005, 2011.

Cited By

View all
  • (2022)Governing Online Goods: Maturity and Formalization in Minecraft, Reddit, and World of Warcraft CommunitiesProceedings of the ACM on Human-Computer Interaction10.1145/35551916:CSCW2(1-23)Online publication date: 11-Nov-2022
  • (2022)Online community development in the early stages: the life cycle model application to Medical Sciences Stack ExchangeAslib Journal of Information Management10.1108/AJIM-11-2021-034574:6(1214-1232)Online publication date: 13-May-2022
  • (2019)PracticeThe Theory and Practice of Social Machines10.1007/978-3-030-10889-2_3(103-147)Online publication date: 15-Feb-2019
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
OpenSym '15: Proceedings of the 11th International Symposium on Open Collaboration
August 2015
144 pages
ISBN:9781450336666
DOI:10.1145/2788993
  • General Chair:
  • Dirk Riehle
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 August 2015

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Funding Sources

  • French science foundation, Agence Nationale pour la Recherche (ANR)

Conference

OpenSym '15
OpenSym '15: The 11th International Symposium on Open Collaboration
August 19 - 21, 2015
California, San Francisco

Acceptance Rates

Overall Acceptance Rate 108 of 195 submissions, 55%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Governing Online Goods: Maturity and Formalization in Minecraft, Reddit, and World of Warcraft CommunitiesProceedings of the ACM on Human-Computer Interaction10.1145/35551916:CSCW2(1-23)Online publication date: 11-Nov-2022
  • (2022)Online community development in the early stages: the life cycle model application to Medical Sciences Stack ExchangeAslib Journal of Information Management10.1108/AJIM-11-2021-034574:6(1214-1232)Online publication date: 13-May-2022
  • (2019)PracticeThe Theory and Practice of Social Machines10.1007/978-3-030-10889-2_3(103-147)Online publication date: 15-Feb-2019
  • (2019)Characterising Social MachinesThe Theory and Practice of Social Machines10.1007/978-3-030-10889-2_1(1-41)Online publication date: 15-Feb-2019
  • (2019)Topics of example questions and discussion at the early stage of community building for health Q&A: A case study of health stack exchangeProceedings of the Association for Information Science and Technology10.1002/pra2.3756:1(399-403)Online publication date: 18-Oct-2019
  • (2017)Wikipedia: a complex social machine by Ramine Tinati and Markus Luczak-Roesch with Martin Vesely as coordinatorACM SIGWEB Newsletter10.1145/3027141.30271472017:Winter(1-10)Online publication date: 3-Feb-2017
  • (2016)Mining team characteristics to predict Wikipedia article qualityProceedings of the 12th International Symposium on Open Collaboration10.1145/2957792.2971802(1-9)Online publication date: 17-Aug-2016
  • (2016)Surviving an "Eternal September"Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems10.1145/2858036.2858356(1152-1156)Online publication date: 7-May-2016

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media