Personalizing papers using Altmetrics: comparing paper ‘Quality’ or ‘Impact’ to person ‘Intelligence’ or ‘Personality’

Abstract

Despite their important position in the research environment, there is a growing theoretical uncertainty concerning what research metrics indicate (e.g., quality, impact, attention). Here we utilize the same tools used to study latent traits like Intelligence and Personality to get a quantitative understanding of what over 20 common research metrics indicate about the papers they represent. The sample is all of the 32,962 papers PLoS published in 2014, with results suggesting that there are at least two important underlying factors, which could generally be described as Scientific Attention/Discussion (citations), General Attention/Discussion (views, tweets), and potentially Media Attention/Discussion (media mentions). The General Attention metric is correlated about .50 with both the Academic and Media factors, though the Academic and Media attention are only correlated with each other below .05. The overall best indicator of the dataset was the total lifetime views on the paper, which is also probably the easiest to game. The results indicate the need for funding bodies to decide what they value and how to measure it (e.g., types of attention, quality).