Evaluating publication trends in clinical research: How reliable are medical databases?

Abstract

The aim of this study was to draw attention to the possible existence of "quirk", inbibliographic databases and to discuss their implications. We analysed the time-trends of"publication types" (PTs) relating to clinical medicine in the most frequently searched medicaldatabase, MEDLINE. We counted the number of entries corresponding to 10 PTs indexed inMEDLINE (1963-1998) and drew up a matrix of [10 PTs × 36 years] which we analysed bycorrespondence factor analysis (CFA). The analysis showed that, although the "internal clock" ofthe database was broadly consistent, there were periods of erratic activity. Thus, observed trendsmight not always reflect true publication trends in clinical medicine but quirks in MEDLINEindexing of PTs. There may be, for instance, different limits for retrospective tagging of entriesrelating to different PTs. The time-trend for Reviews of Reported Cases differed substantiallyfrom that of other publication types.

Despite the quirks, quite rational explanations could be provided for the strongest correlationsamong PTs. The main factorial map revealed how the advent of the Randomised Controlled Trial(RCT) and the accumulation of a critical mass of literature may have increased the rate ofpublication of research syntheses (meta-analyses, practice guidelines...). The RCT is now theiogold standardls in clinical investigation and is often a key component of formal "systematicreviews" of the literature. Medical journal editors have largely contributed to this situation andthus helped to foster the birth and development of a new paradigm, "evidence based medicine"which assumes that expert opinion is biased and therefore relies heavily — virtually exclusively —on critical analysis of the peer-reviewed literature. Our exploratory factor analysis, however, leadsus to question the consistency of MEDLINEs indexing procedures and also the rationale forMEDLINE's choice of descriptors. Databases have biases of their own, some of which are notindependent of expert opinion. User-friendliness should not make us forget that outputs depend onhow the databases are constructed and structured.