Clinical trial search: Using biomedical language understanding models for re-ranking

Highlights

• We quantify the effectiveness of transformer-based models for clinical trial search. This is the first to have compared these methods under the same retrieval framework for a fair comparison.

• We evaluate a broad selection of re-ranking models to uncover the role of pre-training corpus in evidence search for precision medicine.

• Our evaluation points to a neural re-ranking system which achieves state-of-the-art results.

• Our search system is fully automatic; its effectiveness does not rely on manual query reformulation strategies, nor problem-specific heuristic approaches.

• With limited training data, leveraging the transfer learning inherent to transformer-based models makes the proposed model competitive to heuristic, manually optimised systems.

Abstract

Bidirectional Encoder Representations from Transformers (BERT) have achieved state-of-the-art effectiveness in some of the biomedical information processing applications. We investigate the effectiveness of these techniques for clinical trial search systems. In precision medicine, matching patients to relevant experimental evidence or prospective treatments is a complex task which requires both clinical and biological knowledge. To assist in this complex decision making, we investigate the effectiveness of different ranking models based on the BERT models under the same retrieval platform to ensure fair comparisons. An evaluation on the TREC Precision Medicine benchmarks indicates that our approach using the BERT model pre-trained on scientific abstracts and clinical notes achieves state-of-the-art results, on par with highly specialised, manually optimised heuristic models. We also report the best results to date on the TREC Precision Medicine 2017 ad hoc retrieval task for clinical trial search.