Abstract
Big data is increasingly employed in predictive social analyses, yet there are many visible instances of unreliable models or failure, raising questions about methodological validity in data driven approaches. From meta-analysis of methodological institutionalization across three scholarly disciplines, there is evidence that traditional statistical quantitative methods, which are more institutionalized and consistent, are important to develop, structure, and institutionalize data scientific approaches for new and large n quantitative methods, indicating that data driven research approaches may be limited in reliability, validity, generalizability, and interpretability. Results also indicate that interdisciplinary collaborations describe methods in significantly greater detail on projects employing big data, with the effect that institutionalization makes data science approaches more transparent.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, H., Chiang, R.H., Storey, V.C.: Business intelligence and analytics: from big data to big impact. MIS Q. 36(4), 1165–1188 (2012)
Mayer-Schönberger, V., Cukier, K.: Big Data: A Revolution that will Transform How We Live, Work, and Think. Houghton Mifflin Harcourt, Boston (2013)
Borgman, C.L.: Big Data, Little Data, no Data: Scholarship in the Networked World. MIT Press, Cambridge (2015)
Driscoll, K., Walker, S.: Big data, big questions| working within a black box: transparency in the collection and production of big Twitter data. Int. J. Commun. 8, 20 (2014)
Jick, T.D.: Mixing qualitative and quantitative methods: triangulation in action. Adm. Sci. Q. 24(4), 602–611 (1979)
Thompson, B.: What future quantitative social science research could look like: confidence intervals for effect sizes. Educ. Res. 31(3), 25–32 (2002)
Weidlich, W.: Quantitative social science. Phys. Scr. 35(3), 380 (1987)
Livne, A., Simmons, M.P., Adar, E., Adamic, L.A.: The party is over here: structure and content in the 2010 election. In: ICWSM 2011, pp. 17–21 (2011)
Lazer, D., Kennedy, R., King, G., Vespignani, A.: The parable of Google flu: traps in big data analysis. Science 343(6176), 1203–1205 (2014)
Wagner, K.: “Sports Analytics” is bullshit now. Deadspin (2015). http://deadspin.com/sports-analytics-is-bullshit-now-1688293396
Enten, H.: Live polls and online polls tell different stories about the election. FiveThirtyEight (2016). http://fivethirtyeight.com/features/live-polls-and-online-polls-tell-different-stories-about-the-election/
Bruer, T.: US elections: How could predictions be so wrong? J. Mark. Anal. 4(4), 125–134 (2016)
Stevens, J.P.: Applied Multivariate Statistics for the Social Sciences. Routledge, New York (2012)
Goodman, S.N., Fanelli, D., Ioannidis, J.P.: What does research reproducibility mean? Sci. Transl. Med. 8(341), 1–6 (2016)
Agresti, A., Finlay, B.: Statistical Methods for the Social Sciences. Dellen Publishers, CA (2009)
Runyon, R.P., Coleman, K.A., Pittenger, D.J.: Fundamentals of Behavioral Statistics. McGraw-Hill, New York (2000)
Miles, M.B., Huberman, A.M.: Qualitative Data Analysis: A Sourcebook of New Methods. Sage, California (1984)
Aggarwal, C.C., Philip, S.Y.: A survey of uncertain data algorithms and applications. IEEE Trans. Knowl. Data Eng. 21(5), 609–623 (2009)
Sayer, A.: Method in Social Science: Revised 2nd edn. Routledge, London (2010)
Watson, G.: Resistance to change. In: Bennis, W., Benne, K., Chin, R., (eds.) Washington (1967)
LaValle, S., Lesser, E., Shockley, R., Hopkins, M.S., Kruschwitz, N.: Big data, analytics and the path from insights to value. MIT Sloan Manage. Rev. 52(2), 21 (2011)
Gandomi, A., Haider, M.: Beyond the hype: Big data concepts, methods, and analytics. Int. J. Inf. Manage. 35(2), 137–144 (2015)
Japec, L., et al.: Big data in survey research AAPOR task force report. Public Opin. Q. 79(4), 839–880 (2015)
Kaisler, S., Armour, F., Espinosa, J.A., Money, W.: Big data: issues and challenges moving forward. In: 2013 46th Hawaii International Conference on System Sciences (HICSS), pp. 995–1004. IEEE (2013)
Kitchin, R.: The Data Revolution: Big Data, Open Data, Data Infrastructures and their Consequences. Sage Publications, London (2014)
Kitchin, R.: Big Data, new epistemologies and paradigm shifts. Big Data Soc. 1(1), 1–12 (2014)
Ekbia, H., et al.: Big data, bigger dilemmas: a critical review. J. Assoc. Inform. Sci. Technol. 66(8), 1523–1545 (2015)
González-Bailón, S.: Social science in the era of big data. Policy Internet 5(2), 147–160 (2013)
Frické, M.: Big data and its epistemology. J. Assoc. Inform. Sci. Technol. 66(4), 651–661 (2015)
Sivarajah, U., Kamal, M.M., Irani, Z., Weerakkody, V.: Critical analysis of Big Data challenges and analytical methods. J. Bus. Res. 70, 263–286 (2017)
Boyd, D., Crawford, K.: Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon. Inform. Commun. Soc. 15(5), 662–679 (2012)
Manovich, L.: Trending: the promises and the challenges of big social data. Debates Digital Humanit. 2, 460–475 (2011)
Ruppert, E.: Doing the Transparent State: open government data as performance indicators. In: Mugler, J., Park, S.-J. (eds.) A World of Indicators: The Production of Knowledge and Justice in an Interconnected World, pp. 51–78. Cambridge University Press, Cambridge (2013)
McNeely, C.L., Hahm, J.O.: The big (data) bang: policy, prospects, and challenges. Rev. Policy Res. 31(4), 304–310 (2014)
Carmines, E.G., Zeller, R.A.: Reliability and Validity Assessment, vol. 17. Sage Publications, Thousand Oaks (1979)
Furr, R.M., Bacharach, V.R.: Psychometrics: An Introduction. Sage, Thousand Oaks (2013)
Johnson, J.B., Reynolds, H.T., Mycoff, J.D.: Political Science Research Methods. CQ Press, Washington, D.C (2015)
Crawford, S.E., Ostrom, E.: A grammar of institutions. Am. Polit. Sci. Rev. 89(03), 582–600 (1995)
Borgman, C.L.: Scholarly communication and bibliometrics (1990)
Feng, G., Guo, J., Jing, B.Y., Hao, L.: A Bayesian feature selection paradigm for text classification. Inf. Process. Manage. 48(2), 283–302 (2012)
Vakkari, P., Serola, S.: Perceived outcomes of public libraries. Libr. Inform. Sci. Res. 34(1), 37–44 (2012)
Herrera, G.: Google Scholar users and user behaviors: An exploratory study. College & Research Libraries 72(4), 316–330 (2011)
Egghe, L., Rousseau, R.: Introduction to informetrics: Quantitative methods in library, documentation and information science (1990)
Fidel, R.: Are we there yet?: Mixed methods research in library and information science. Libr. Inform. Sci. Res. 30(4), 265–272 (2008)
Richards, N.M., King, J.H.: Three paradoxes of big data. Stanford Law Review, September 2013
Sjoberg, G., Nett, R.: A methodology for social research, pp. 213–214. Harper & Row, New York (1968)
Seba, I., Rowley, J., Lambert, S.: Factors affecting attitudes and intentions towards knowledge sharing in the Dubai Police Force. Int. J. Inf. Manage. 32(4), 372–380 (2012)
Romero-Zaldivar, V.A., Pardo, A., Burgos, D., Kloos, C.D.: Monitoring student progress using virtual appliances: a case study. Comput. Educ. 58(4), 1058–1067 (2012)
Inanc, O., Tuncer, O.: The effect of academic inbreeding on scientific effectiveness. Scientometrics 88(3), 885–898 (2011)
Manatschal, A., Stadelmann-Steffen, I.: Cantonal variations of integration policy and their impact on immigrant educational inequality. Comp. Eur. Politics 11(5), 671–695 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Sanfilippo, M.R., McCoy, C. (2019). Methodological Transparency and Big Data: A Critical Comparative Analysis of Institutionalization. In: Taylor, N., Christian-Lamb, C., Martin, M., Nardi, B. (eds) Information in Contemporary Society. iConference 2019. Lecture Notes in Computer Science(), vol 11420. Springer, Cham. https://doi.org/10.1007/978-3-030-15742-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-15742-5_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-15741-8
Online ISBN: 978-3-030-15742-5
eBook Packages: Computer ScienceComputer Science (R0)