Skip to main content
SpringerLink
Log in

Deciphering the US News and World Report Ranking of US Chemistry Graduate Programs

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

The US News & World Reports (US News) regularly publishes highly influential rankings of graduate programs in the sciences. These rankings are exclusively based on reputational surveys sent to a small subset of faculty experts in a given discipline, namely Directors of Graduate Studies and Department Chairs. No other quantitative metric is used to establish a graduate program’s rank. If reputation alone establishes US News rank, what quantifiable metrics underlie it? The question is an important one when considering that these rankings are widely consulted within higher education circles. These can impact a particular program’s ability to attract top faculty, graduate students, and other researchers who directly contribute to the program’s collective publication, citation, and funding profiles. In this study, we focus on US News’ most recently published peer assessment scores for chemistry graduate programs and establish seven departmental and institutional metrics that correlate with these scores. We find that central to US News rank is a chemistry program’s research visibility and impact as quantified by the median career h-index of its tenured and tenure track (T/TT) faculty, departmental T/TT size, and per capita research expenditures. These three predictor variables account for approximately 84% of the total variability in reported average peer assessment scores. When prestige indicators such as institutional membership in the American Association of Universities and percentage T/TT faculty membership in the US National Academy of Sciences are included, over 88% of the variability in average peer assessment score is accounted for. In whole, a seven-variable statistical model we develop explains nearly 91% of the variability in US News’ average peer assessment scores, which form the basis for its ranking of graduate US chemistry programs. We also explore the possibility that an anchoring effect influences reputational scores by analyzing how rank change complementary cumulative distribution functions evolve with time, following release of the initial 1994 US News and World Reports chemistry graduate program rankings. We find that the likelihood of rank changes increase with time with a t\(_\text {1/2}\) of \(\sim\)20 years.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. Chairs and Directors of Graduate Study are nominally faculty within given departments that serve in these capacities on a rotating basis.

  2. Per capita is defined on a per T/TT faculty member basis.

  3. N is nominally 50. However, due to ties in US News rank, the actual number of programs can be larger than 50.

  4. The complete set of 1994 rankings was reported by Webster, 1994. Except for the most recent rankings, which are available online, all other rankings were obtained from print editions of US News. Of note is that the 1996 and 1999 surveys only rank the top 20 programs. The 2002, 2007, and 2010 surveys only list the top 50 programs; the 2014 ranking lists the top 60 programs.

  5. Note that implicit to the use of a median h-index is that a department’s distribution of faculty by rank (assistant, associate, or full) is not skewed towards either assistant or full professors.

  6. Prior to 2019, there were 62 AAU institutions. We have therefore considered only Universities that were AAU members in 2018.

  7. This is clearly hypothetical as the predictors may be interrelated. Changing the value of one likely changes the values of others.

References

  • AAU. (2020). The Association of American Universities. Retrieved June 05, 2020, from https://www.aau.edu/. Accessed 05 June 2020.

  • Abbott, W. F. (1972). University and departmental determinants of the prestige of sociology departments. American Sociological Review, 7, 14–15.

    Google Scholar 

  • Alsmadi, I., Taylor, Z., & Childs, J. (2020). US news & World Report Best Colleges rankings: Which institutional metrics contribute to sustained stratification? Scientometrics, 124(3), 1851–1869.

    Article  Google Scholar 

  • Baldi, S. (1997). Departmental quality ratings and visibility: The advantages of size and age. The American Sociologist, 28, 89–101.

    Article  Google Scholar 

  • Bowman, N. A., & Bastedo, M. N. (2011). Anchoring effects in world university rankings: Exploring biases in reputation scores. Higher Education, 61, 431–444.

    Article  Google Scholar 

  • Brooks, E., & Morse, R. (2020). Methodology: Best online engineering programs rankings. Retrieved June 5, 2020, from https://www.usnews.com/education/online-education/articles/engineering-methodology.

  • Cole, S., & Cole, J. R. (1967). Scientific output and recognition: A study in the operation of the reward system in science. American Sociological Review, 32, 377–390.

    Article  Google Scholar 

  • Crane, D. (1965). Scientists at major and minor universities: A study of productivity and recognition. American Sociological Review, 30, 699–714.

    Article  Google Scholar 

  • Diamond, N., & Graham, H. G. (2000). How should we rate research universities. Change: The Magazine of Higher Learning, 32, 20–33.

    Article  Google Scholar 

  • Ehrenberg, R. G., & Hurst, P. J. (1996). The 1995 NRC ratings of doctoral programs: A hedonic model. Change, 28, 46–50.

    Article  Google Scholar 

  • Fairweather, J. S. (1988). Reputational quality of academic programs: The institutional halo. Research in Higher Education, 28, 345–355.

    Article  Google Scholar 

  • Gallant, J. A., & Prothero, J. W. (1972). Weight-watching at the university: The consequences of growth. Science, 175, 381–388.

    Article  Google Scholar 

  • Gerrity, D. M., & McKenzie, R. B. (1978). The ranking of southern economics departments: New criterion and further evidence. Southern Economic Journal, 45, 608–614.

    Article  Google Scholar 

  • Ghaisi, A., Fountas, G., Anastasopoulos, P. C., & Mannering, F. (2019). Statistical assessment of peer opinions in higher education rankings: The case of U.S. engineering graduate programs. Journal of Applied Research in Higher Education, 11, 1–13.

    Google Scholar 

  • Green, R. G., Baskind, F. R., & Bellin, M. H. (2002). Results of the doctoral faculty publication project: Journal article productivity and its correlates in the 1990s. Journal of Social Work Education, 38(1), 135–152.

    Article  Google Scholar 

  • Green, R. G., Baskind, F. R., Fassler, A., & Jordan, A. (2006). The validity of the 2004 US news & world report’s rankings of schools of social work. Social Work, 51(2), 135–145.

    Article  Google Scholar 

  • Hagstrom, W. O. (1971). Inputs, outputs, and the prestige of university science departments. Sociological Education, 44, 375–397.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102, 16569–16572.

    Article  MATH  Google Scholar 

  • Jacobs, D. (1999). Ascription or productivity? The determinants of departmental success in the NRC quality ratings. Social Science Research, 28, 228–239.

    Article  Google Scholar 

  • James, G.,Witten, D., Hastie, T., & Tibshirani, R. (2021). Linear model selection and regularization. In An introduction to statistical learning (pp. 225–288). Springer.

  • Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux.

    Google Scholar 

  • Kang, K. (2019). Survey of earned doctorates. Retrieved June 05, 2020, from http://www.nsf.gov/statistics/srvydoctorates/#tabs-1. Accessed 05 June 2020.

  • Kieth, B. (1999). The institutional context of departmental prestige in American higher education. American Educational Research Journal, 36, 409–445.

    Article  Google Scholar 

  • Kieth, B., & Babchuk, N. (1994). A longitudinal assessment of productivity in prominent sociology journals and departmental prestige. The American Sociologist, 25, 4–27.

    Article  Google Scholar 

  • Kieth, B., & Babchuk, N. (1998). The quest for institutional recognition: A longitudinal analysis of scholarly productivity and academic prestige among sociology departments. Social Forces, 76, 1495–1533.

    Article  Google Scholar 

  • King, S., & Wolfle, L. M. (1987). Alatent-variable causalmodel of faculty reputational ratings. Research in Higher Education, 27, 99–106.

    Article  Google Scholar 

  • Kirk, S. A., & Corcoran, K. (1995). School rankings: Mindless narcissism or do they tell us something? Journal of Social Work Education, 31(3), 408–414.

    Article  Google Scholar 

  • Kreiner, G. (2016). The slavery of the h-index—Measuring the unmeasurable. Frontiers in Human Neuroscience, 10, 556.

    Article  Google Scholar 

  • Kumar, M. J. (2009). Evaluating scientists: Citations, impact factor, h-index, online page hits and what else? IETE Technical Review, 26, 165–168.

    Article  Google Scholar 

  • Lazaridis, T. (2010). Ranking university departments using the mean h-index. Scientometrics, 82, 211–216.

    Article  Google Scholar 

  • Lee, G., Sanford, T., & Lee, J. (2014). Variables that explain changes in institutional rank in US news & world report rankings. KEDI Journal of Educational Policy, 11(1), 27–47.

    Google Scholar 

  • Lindsey, D. (1989). Using citation counts as a measure of quality in science measuring what’s measurable rather than what’s valid. Scientometrics, 15, 189–203.

    Article  Google Scholar 

  • Liu, H. C. (1978). Faculty citation and quality of graduate engineering departments. Engineering Education, 68, 739–741.

    Google Scholar 

  • Machung, A. (1998). Playing the ranking game. Change: The Magazine of Higher Learning, 30, 12–16.

    Article  Google Scholar 

  • Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of lis faculty: Web of science versus scopus and google scholar. Journal of the Association for Information Science and Technology, 58, 2105–2125.

    Google Scholar 

  • Mendenhall, W., & Sincich, T. (2012). A second course in statistics: Regression analysis (8th ed.). Pearson, Education Inc.

    MATH  Google Scholar 

  • Morey, L. C. (2010). Leading North American programs in clinical assessment research: An assessment of productivity and impact. Journal of Personality Assessment, 92(3), 207–211.

    Article  Google Scholar 

  • Morse, R., Vega-Rodriguez, J., Castonguay, A., Brooks, E., & K., H. (2021). Methodology: 2022 best engineering schools rankings. Retrieved from June 17, 2022, from https://www.usnews.com/education/best-graduate -schools/articles/engineering-schools-methodology.

  • NAS. (2020). National academy of sciences member profile search. Retrieved June 05, 2020, from http://www.nasonline.org/member-directory/. Accessed 05 June 2020.

  • NSF. (2020). National center for science and engineering statistics. Retrieved December 20, 2019, from https://www.nsf.gov/statistics/. Accessed 05 June 2020.

  • Ostriker, J. P., Hinshaw, V. S., Aberle, E. D., Bradburn, N. M., Brauman, J. I., Cole, J. R., & Wheeler, R. P. (2011). An assessment of research-doctorate programs. Retrieved June 05, 2020, from https://www.nationalacademies.org/ our-work/an-assessment-of-research-doctorate-programs. Accessed 05 June 2020.

  • Paxton, P., & Bollen, K. A. (2003). Perceived quality and methodology in graduate department ratings: Sociology, political science, and economics. Sociology of Education, 76, 71–88.

    Article  Google Scholar 

  • Ried, L. D., & Ried, D. B. (2020). Elucidation of a resource reputation model to explain US news & world report ranking using path analysis. Currents in Pharmacy Teaching and Learning, 12(12), 1399–1409.

    Article  Google Scholar 

  • Ried, L. D., & Ried, D. B. (2021). Predicting the 2016 US news & world report rankings using a reputation and prestige model. Currents in Pharmacy Teaching and Learning, 13(2), 91–101.

    Article  MathSciNet  Google Scholar 

  • Savage, P. E. (2003). Productivity and quality indicators for highly ranked ChE graduate programs. Chemical Engineering Education, 37, 94–99.

    Google Scholar 

  • Schlesselman, L., & Coleman, C. I. (2013). College and school of pharmacy characteristics associated with US news and world report rankings. American Journal of Pharmaceutical Education, 77(3), 58.

    Article  Google Scholar 

  • Sornette, D. (2006). Critical phenomena in natural sciences. chaos, fractals, selforganization and disorder: Concepts and tools. Springer.

    MATH  Google Scholar 

  • Stewart, P. K., Roberts, M. C., & Roy, K. M. (2007). Scholarly productivity in clinical psychology PhD programs: A normative assessment of publication rates. Clinical Psychology: Science and Practice, 14(2), 157–171.

    Google Scholar 

  • Tan, D. L. (1986). The assessment of quality in higher education: A critical review of the literature and research. Research in Higher Education, 24, 223–265.

    Article  Google Scholar 

  • Taylor, J. (1995). Research in UK departments of sociology: An analysis based upon the 1992 research assessment exercise database. Sociology, 29, 513–529.

    Article  Google Scholar 

  • Taylor, Z. W., Childs, J., Bicak, I., & Alsmadi, I. (2019). Is bigger, better? Exploring US news graduate education program rankings and internet characteristics. Interchange, 50(2), 205–219.

    Article  Google Scholar 

  • Tomkins, A., Zhang, M., & Heavlin, W. D. (2017). Reviewer bias in singleversus doubleblind peer review. Proceedings of the National Academy of Sciences of the United States of America, 114, 12708–12713.

    Article  Google Scholar 

  • Toutkoushian, R. K., Dundar, H., & Becker, W. E. (1998). The national research council graduate program ratings: What are they measuring? The Review of Higher Education, 21, 427–443.

    Article  Google Scholar 

  • Tsakalis, K. S., & Palais, J. C. (2004). Improving a school’s U.S. News and World report ranking. Journal of Engineering Education, 93, 259–263.

    Article  Google Scholar 

  • U.S. News. (2020). 2020 best national university rankings. Retrieved June 05, 2020, from https://www.usnews.com/best-colleges/rankings/national-universities. Accessed 05 June 2020.

  • Van Noorden, R., & Chawla, D. S. (2019). Policing self-citations. Nature, 572, 578–579.

    Article  Google Scholar 

  • Vojak, B. A., Price, R., & Carnahan, J. V. (2003). The relationship between department rank and college rank in engineering graduate program rankings conducted by U.S. News and World Report. Journal of Engineering Education, 92, 65–72.

    Article  Google Scholar 

  • Volkwein, J. F., & Sweitzer, K. V. (2006). Institutional prestige and reputation among research universities and liberal arts colleges. Research in Higher Education, 47(2), 129–148.

    Article  Google Scholar 

  • Vucetic, S., Chanda, A. K., Zhang, S., Bai, T., & Maiti, A. (2018). Peer assessment of CS doctoral programs shows strong correlation with faculty citations. Communication of ACM, 61, 70–76.

    Article  Google Scholar 

  • Williamson, J. R. (2009). My h-index turns 40: My midlife crisis of impact. ACS Chemical Biology, 4, 311–313.

    Article  MathSciNet  Google Scholar 

  • Zhang, S. (2020). Chemrank-scopus. Retrieved June 05, 2020 from https://github.com/ShubinZhang/ChemRank-Scopus. Accessed 05 June 2020.

Download references

Acknowledgements

We thank Yurii Morozov for creating the initial Python/Scopus API program. We also thank Molly Walsh for assistance with the statistical model of average peer assessment scores and Pavel Frantsuzov for helpful discussions on the eCCDF. We also thank the Department of Chemistry and Biochemistry as well as the College of Science at Notre Dame for partial financial support of this study.

Author information

Authors and Affiliations

  1. Department of Chemistry and Biochemistry, University of Notre Dame, Notre Dame, IN, USA

    Masaru Kuno & Mary Prorok

  2. Department of Physics, University of Notre Dame, Notre Dame, IN, USA

    Masaru Kuno & Shubin Zhang

  3. Department of Information Technology, Analytics and Operations, University of Notre Dame, Notre Dame, IN, USA

    Huy Huynh

  4. Hesburgh Library, University of Notre Dame, Notre Dame, IN, USA

    Thurston Miller

Authors
  1. Masaru Kuno
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mary Prorok
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shubin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huy Huynh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Thurston Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Masaru Kuno or Mary Prorok.

Supplementary Information

Below is the link to the electronic supplementary material.

(PDF 473 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kuno, M., Prorok, M., Zhang, S. et al. Deciphering the US News and World Report Ranking of US Chemistry Graduate Programs. Scientometrics 127, 2131–2150 (2022). https://doi.org/10.1007/s11192-022-04317-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-022-04317-6

Keywords

Search

Navigation