Skip to main content
SpringerLink
Log in

The Crowdsourcing Delphi: A Method for Combing Expert Judgements and Wisdom of Crowds

  • Conference paper
  • First Online:
Knowledge Management in Organisations (KMO 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1593))

Included in the following conference series:

Abstract

How to combine expert judgment and mass intelligence? This study can be characterized as methodological concept development. We combine the relevant theories and methodological approaches from expert-dominated Delphi methods and layman-dominated crowdsourcing approaches to introduce a novel Crowdsourcing Delphi research approach. The key contributions of this study consist: (1) the definition of Crowdsourcing Delphi term, (2) identification of differences between crowdsourcing and Delphi methodologies, (3) developing a framework for selecting and evaluating Crowdsourcing Delphi research strategies and (4) identifying pros and cons for different research strategy combinations. One notable advantage for knowledge management need is that the Crowdsourcing methodology allows a large variety of different Delphi variants and different Crowdsourcing techniques. The Crowdsourcing Delphi allows also combination of Data pools/Big Data flows of crowds to the methodological practices of various Delphi expert-panel methods (Delphi variants). The Crowdsourcing Delphi is a new knowledge management methodology tool package.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Linstone, H.A., Turoff, M.: The Delphi Method: Techniques and Applications. Wesley Publishing Company, Reading (1975)

    MATH  Google Scholar 

  2. Expert in Cambridge Dictionary. https://dictionary.cambridge.org/. Accessed 10 Mar 2022

  3. Baker, J., Lovell, K., Harris, N.: How expert are the experts? An exploration of the concept of ‘expert’ within Delphi panel techniques. Nurse Res. 14(1), 59–70 (2006)

    Article  Google Scholar 

  4. Keeney, S., Hasson, F., McKenna, H.P.: A critical review of the Delphi technique as a research methodology for nursing. Int. J. Nurs. Stud. 38(2), 195–200 (2001)

    Article  Google Scholar 

  5. Okoli, C., Pawlowski, S.D.: The Delphi method as a research tool: an example, design considerations and applications. Inf. Manag. 42(1), 15–29 (2004)

    Article  Google Scholar 

  6. Van Zolingen, S.J., Klaassen, C.A.: Selection processes in a Delphi study about key qualifications in senior secondary vocational education. Technol. Forecast. Soc. Chang. 70(4), 317–340 (2003)

    Article  Google Scholar 

  7. Goodman, C.M.: The Delphi technique: a critique. J. Adv. Nurs. 12(6), 729–734 (1987)

    Article  Google Scholar 

  8. Sackman, H.: Delphi Critique, Expert Opinion, Forecasting and the Group Process. Lexington Books (1975)

    Google Scholar 

  9. McMillan, S.S., King, M., Tully, M.P.: How to use the nominal group and Delphi techniques. Int. J. Clin. Pharm. 38(3), 655–662 (2016). https://doi.org/10.1007/s11096-016-0257-x

    Article  Google Scholar 

  10. Hussler, C., Muller, P., Rondé, P.: Is diversity in Delphi panelist groups useful? Evidence from a French forecasting exercise on the future of nuclear energy. Technol. Forecast. Soc. Change 78(9), 1642–1653 (2011)

    Article  Google Scholar 

  11. Estellés-Arolas, E., González-Ladrón-De-Guevara, F.: Towards an integrated crowdsourcing definition. J. Inf. Sci. 38(2), 189–200 (2012)

    Article  Google Scholar 

  12. Kaivo-oja, J., Santonen, T., Myllylä, Y.: The crowdsourcing Delphi: combining the Delphi methodology and crowdsourcing techniques. In: International Society for Professional Innovation Management. ISPIM Conference Proceedings (2013). URN:NBN:fi:amk-2016101015024

    Google Scholar 

  13. Halicka, K.: Innovative classification of methods of the Future-oriented Technology Analysis. Technol. Econ. Dev. Econ. 22(4), 574–597 (2016)

    Article  Google Scholar 

  14. Strasser, A.: Delphi method variants in information systems research: taxonomy development and application. Electron. J. Bus. Res. Methods 15(2), 120–133 (2017)

    Google Scholar 

  15. Howe, J.: Crowdsourcing: How the Power of the Crowd is Driving the Future of Business. Business Books, Great Britain (2008)

    Google Scholar 

  16. Ghezzi, A., Gabelloni, D., Martini, A., Natalicchio, A.: Crowdsourcing: a review and suggestions for future research. Int. J. Manag. Rev. 20(2), 343–363 (2018)

    Article  Google Scholar 

  17. Geiger, D., Rosemann, M., Fielt, E., Schader, M.: Crowdsourcing information systems-definition typology, and design. In: George, J.F. (ed.) Proceedings of the 33rd Annual International Conference on Information Systems, pp. 1–11. Association for Information Systems/AIS Electronic Library (AISeL) (2012). http://aisel.aisnet.org/

  18. Prpić, J., Shukla, P.P., Kietzmann, J.H., McCarthy, I.P.: How to work a crowd: developing crowd capital through crowdsourcing. Bus. Horiz. 58(1), 77–85 (2015)

    Article  Google Scholar 

  19. Dalkey, N., Helmer, O.: An experimental application of the Delphi method to use of experts. Manag. Sci. 9(3), 458–467 (1963)

    Article  Google Scholar 

  20. Turoff, M., Linstone, H.A.: The Delphi method-techniques and applications (2002). https://web.njit.edu/~turoff/pubs/delphibook/delphibook.pdf. Accessed 10 Mar 2022

  21. McKenna, H.P.: The Delphi technique: a worthwhile research approach for nursing? J. Adv. Nurs. 19(6), 1221–1225 (1994)

    Article  Google Scholar 

  22. Mullen, P.M.: Delphi: myths and reality. J. Health Organ. Manag. 17(1), 37–52 (2003)

    Article  Google Scholar 

  23. Keeney, S.: The Delphi technique. In: Gerrish, K., Lacey, A. (eds.) The Research Process in Nursing, 6th edn., pp. 227–236. Wiley-Blackwell, London (2010)

    Google Scholar 

  24. Turoff, M.: The design of a policy Delphi. Technol. Forecast. Soc. Change 2(2), 149–171 (1970)

    Article  Google Scholar 

  25. Rauch, W.: The decision Delphi. Technol. Forecast. Soc. Change 15(3), 159–169 (1979)

    Article  Google Scholar 

  26. Schmidt, R.C.: Managing Delphi surveys using nonparametric statistical techniques. Decis. Sci. 28(3), 763–774 (1997)

    Article  Google Scholar 

  27. Kuusi, O.: Expertise in the future use of generic technologies. VATT research reports 59. VATT, Helsinki (1999)

    Google Scholar 

  28. Nowack, M., Endrikat, J., Guenther, E.: Review of Delphi-based scenario studies: quality and design considerations. Technol. Forecast. Soc. Change 78(9), 1603–1615 (2011)

    Article  Google Scholar 

  29. Tapio, P.: Disaggregative policy Delphi: using cluster analysis as a tool for systematic scenario formation. Technol. Forecast. Soc. Change 70(1), 83–101 (2003)

    Article  Google Scholar 

  30. Flostrand, A.: Finding the future: crowdsourcing versus the Delphi technique. Bus. Horiz. 60(2), 229–236 (2017)

    Article  Google Scholar 

  31. Popper, K.: The Logic of Scientific Discovery. Routledge, London (2005)

    Book  Google Scholar 

  32. Marchau, V., van de Linde, E.: The Delphi method. In: van der Duin, P. (ed.) Foresight in Organizations. Methods and Tools, pp. 59–79. Routledge, New York (2016)

    Google Scholar 

  33. Brabham, D.C.: Crowdsourcing as a model for problem solving: an introduction and cases. Convergence 14(1), 75–90 (2008)

    Article  Google Scholar 

  34. Ranard, B.L., et al.: Crowdsourcing—harnessing the masses to advance health and medicine, a systematic review. J. Gen. Internal Med. 29(1), 187–203 (2014). https://doi.org/10.1007/s11606-013-2536-8

    Article  Google Scholar 

  35. Leech, N.L., Onwuegbuzie, A.J.: A typology of mixed methods research designs. Qual. Quant. 43, 265–275 (2009). https://doi.org/10.1007/s11135-007-9105-3

    Article  Google Scholar 

  36. Kaivo-oja, J.: Towards better participatory processes in technology foresight: how to link participatory foresight research to the methodological machinery of qualitative research and phenomenology? Futures 86, 94–106 (2017)

    Article  Google Scholar 

  37. Jari, K.-O., Theresa, L.: Knowledge management and triangulation logic in the foresight research and analyses in business process management. In: Uden, L., Lu, W., Ting, I.-H. (eds.) KMO 2017. CCIS, vol. 731, pp. 228–238. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-62698-7_20

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laurea University of Applied Sciences, Espoo, Finland

    Teemu Santonen

  2. University of Turku, Turku, Finland

    Jari Kaivo-oja

Authors
  1. Teemu Santonen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jari Kaivo-oja
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Teemu Santonen .

Editor information

Editors and Affiliations

  1. Staffordshire University, Stoke-on-Trent, UK

    Lorna Uden

  2. University of Kaohsiung, Kaohsiung, Taiwan

    I-Hsien Ting

  3. FernUniversität in Hagen, Hagen, Germany

    Birgit Feldmann

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Santonen, T., Kaivo-oja, J. (2022). The Crowdsourcing Delphi: A Method for Combing Expert Judgements and Wisdom of Crowds. In: Uden, L., Ting, IH., Feldmann, B. (eds) Knowledge Management in Organisations. KMO 2022. Communications in Computer and Information Science, vol 1593. Springer, Cham. https://doi.org/10.1007/978-3-031-07920-7_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-07920-7_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-07919-1

  • Online ISBN: 978-3-031-07920-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation