Linguistic Uncertainty in Clinical NLP: A Taxonomy, Dataset and Approach

Abstract

Linguistic uncertainty is prevalent in electronic health records (EHRs). The ability to handle and preserve uncertainty in natural language is an essential skill for clinicians, facilitating decidability and effective clinical reasoning processes despite incomplete knowledge in some situations. This has been addressed by previous research in clinical NLP by the development of algorithms that detect uncertainty expressions. However, existing rule-based algorithms have limited uncertainty detection capabilities. Therefore, we seek to reformulate uncertainty detection as a supervised machine learning problem by (i) reevaluating the concept of uncertainty, (ii) embedding this understanding in an improved linguistic uncertainty taxonomy and (iii) introducing a new dataset of EHRs annotated for nine types of uncertainty – the first publicly available dataset of its kind. Many of our classes are novel and emphasise implicit uncertainties – a form of uncertainty that is ignored by existing algorithms, yet has crucial functions in clinical settings. Through an evaluation of our dataset, we demonstrate the scalability of our approach and its utility in relation to research on clinical information extraction.

A Model Implementation

Following the work of Kim [7], a state-of-the-art single channel Convolutional Neural Network (CNN) for sentence classification was used as a binary classifier.

For our experiments (see Sect. 5.2 and 5.3), the majority of hyperparameters were kept constant: the learning rate was set at 0.3; the dropout probability in the dropout layer was 0.1; BioWordVec embeddings were scaled by a factor of 0.65. The window sizes for our two convolutional layers were either 1 and 3 or 3 and 5. The number of training epochs ranged from 30 to 70. These hyperparameters were determined by monitoring the training loss. Random classifiers used as a baseline were drawn from the Scikitlearn library [9].