Skip to main content
SpringerLink
Log in

A topic model approach to measuring interdisciplinarity at the National Science Foundation

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

As the National Science Foundation (NSF) implements new cross-cutting initiatives and programs, interest in assessing the success of these experiments in fostering interdisciplinarity grows. A primary challenge in measuring interdisciplinarity is identifying and bounding the discrete disciplines that comprise interdisciplinary work. Using statistical text-mining techniques to extract topic bins, the NSF recently developed a topic map of all of their awards issued between 2000 and 2011. These new data provide a novel means for measuring interdisciplinarity by assessing the language or content of award proposals. Using the Directorate for Social, Behavioral, and Economic Sciences as a case study and drawing on the new topic model of the NSF’s awards, this paper explores new methods for quantifying interdisciplinarity in the NSF portfolio.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Since the topic model accounts for every word used in the proposals, 77 of the original 1000 topic bins in the model contained only non-substantive words (e.g. “the”, “and”, “percent”). These bins were excluded from the analysis.

  2. The NSF portfolio explorer was recently archived when funds to sustain it indefinitely could not be obtained.

  3. Office of the Director, National Science Board, Office of the Inspector General, Directorate for Education and Human Resources, and the Office of Polar Programs.

  4. Each topic bin is labeled with a topic number and a short descriptive title which summarizes the full text of the bin.

  5. Anthropology; Archaeology; Cognitive Science; Decision Sciences; Economics; Geography; Law; Developmental Sciences; Linguistics; Methodology, Measurement, and Statistics; Political Science; Science and Technology Studies; Social Psychology; and Sociology.

  6. If the original topic model data had been available, the vector distances between topic bins could have been empirically calculated. However, since this analysis relies on the simplified data available through the NSF portfolio explorer, these distance values had to be estimated.

References

  • AAAS Intersociety Working Group. (2011). AAAS Report XXXVI: Research and Development FY2012. Washington DC: American Association for the Advancement of Science.

    Google Scholar 

  • Blei, D. M., Ng, A., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of Machine Learning Research., 3, 993–1022.

    MATH  Google Scholar 

  • Gerrish, S. M. & Blei, D. M. (2010). A language-based approach to measuring scholarly impact. Princeton University. Retrieved on April 26, 2013 from http://www.cs.princeton.edu/~blei/papers/GerrishBlei2010.pdf.

  • Hu, D. (2009). Latent dirichlet allocation for text, images, and music. University of California, San Diego. Retrieved April 26, 2013 from http://www.cseweb.ucsd.edu/~dhu/docs/research_exam09.pdf.

  • Leydesdorff, L. (2007). Betweenness centrality as an indicator of the interdisciplinarity of scientific journals. Journal for the American Society for Information Science and Technology., 58(9), 1303–1319.

    Article  Google Scholar 

  • Lu, K., & Wolfram, D. (2012). Measuring author research relatedness: A comparison of word-based, topic-based, and author cocitation approaches. Journal of the American Society for Information Science and Technology, 63(10), 1973–1986.

    Article  Google Scholar 

  • Masse, L. C., Moser, R. P., Stokols, D., Taylor, B. K., Marcus, S. E., Morgan, G. D., et al. (2008). Measuring collaboration and transdisciplinary integration in team science. American Journal of Preventive Medicine, 35, S151–S160.

    Article  Google Scholar 

  • Newman, D., Bonilla, E., & Buntine, W. (2011). Improving topic coherence with regularized topic models. Advances in Neural Information Processing Systems., 24, 496–504.

    Google Scholar 

  • Porter, A., & Rafols, I. (2009). Is science becoming more interdisciplinary? Measuring and mapping six research fields over time. Scientometrics., 81, 719–745.

    Article  Google Scholar 

  • Rafols, I., & Meyer, M. (2008). Diversity and network centralities as indicators of interdisciplinarity: Case studies in bionanoscience. Scientometrics., 82, 263–287.

    Article  Google Scholar 

  • Rosen-Zvi, M., Griffiths, T., Steyvers, M., & Smith, P. (2004). The Author-topic model for authors and documents (pp. 487–494). Banff: AUAI Press.

    Google Scholar 

  • Sci2 Team. (2009). Science of science (Sci2) tool. Indiana University and SciTech Strategies. http://sci2.cns.iu.edu. Accessed 14 Oct 2011.

  • Sterling, A. (2007). A general framework for analysing diversity in science, technology, and society. Journal of the Royal Society Interface, 4, 707–719.

    Article  Google Scholar 

  • Stokols, D., Fuqua, J., Gress, J., Harvey, R., Phillips, K., Baezconde-Garbanati, L., et al. (2003). Evaluating transdisciplinary science. Nicotine & Tobacco Research, 5, S21–S39.

    Article  Google Scholar 

  • Wagner, C., Roessner, J., Bobb, K., Klein, J., Boyack, K., Keyton, J., et al. (2011). Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature. Journal of Informetrics., 165, 14–26.

    Article  Google Scholar 

  • Wang, H., Ding, Y., Tang, J., Dong, X., He, B., Qiu, J., et al. (2011). Finding complex biological relationships in recent PubMed articles using Bio-LDA. PLoS One, 6(3), 1–14.

    Google Scholar 

Download references

Acknowledgments

The author would like to thank Dave Newman from the University of California, Irvine for his assistance with the topic model data; Myron Gutmann, Amber Story, and many others at the NSF for their perspectives and advice on this paper; the Sci2 Team for the use of their Science of Science Tool; Julia Lane who was until recently the Program Director for the Science of Science Policy Program for her vision and work in developing the NSF Portfolio Explorer; and the anonymous reviewers of this paper for their constructive comments and helpful suggestions.

Disclaimer

This work was completed as part of a AAAS Science and Technology Policy Fellowship and supported by the National Science Foundation (Award 1035631). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.

Author information

Authors and Affiliations

  1. National Science Foundation, 4201 Wilson Blvd., Arlington, VA, 22230, USA

    Leah G. Nichols

Authors
  1. Leah G. Nichols
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leah G. Nichols.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nichols, L.G. A topic model approach to measuring interdisciplinarity at the National Science Foundation. Scientometrics 100, 741–754 (2014). https://doi.org/10.1007/s11192-014-1319-2

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-014-1319-2

Keywords

Search

Navigation