A self-supervised method for treatment recommendation in sepsis

Abstract

Sepsis treatment is a highly challenging effort to reduce mortality in hospital intensive care units since the treatment response may vary for each patient. Tailored treatment recommendations are desired to assist doctors in making decisions efficiently and accurately. In this work, we apply a self-supervised method based on reinforcement learning (RL) for treatment recommendation on individuals. An uncertainty evaluation method is proposed to separate patient samples into two domains according to their responses to treatments and the state value of the chosen policy. Examples of two domains are then reconstructed with an auxiliary transfer learning task. A distillation method of privilege learning is tied to a variational auto-encoder framework for the transfer learning task between the low- and high-quality domains. Combined with the self-supervised way for better state and action representations, we propose a deep RL method called high-risk uncertainty (HRU) control to provide flexibility on the trade-off between the effectiveness and accuracy of ambiguous samples and to reduce the expected mortality. Experiments on the large-scale publicly available real-world dataset MIMIC-III demonstrate that our model reduces the estimated mortality rate by up to 2.3% in total, and that the estimated mortality rate in the majority of cases is reduced to 9.5%.

摘要

由于每个脓毒症患者治疗反应可能不同,为病人提供量身定制的治疗建议来帮助医生有效、准确地做出决定,并采取有效治疗方案,是降低医院重症监护病房死亡率的一项极具挑战性的工作。本文将强化学习应用于个人治疗推荐,采用对样本不确定性进行建模并评估的方法,根据患者对治疗的反应和状态,将患者样本分为两个域,然后使用辅助迁移学习任务重建两个域的样本,使用特权学习的蒸馏方法与用于迁移学习的变分自动编码器框架关联低质量域和高质量域间的任务。通过结合自监督方式获得更好的状态和动作表示,本文提出一种针对引起较高风险的不确定性进行控制的深度强化学习方法;模型提供一定的灵活性使之可以在不同场景对模糊样本做出保守预测或明确判断,并降低预期死亡率。在大规模公开可用的真实医疗数据集MIMIC-III上的实验表明,所提模型将总体估计死亡率降低了2.3%,并将主要估计死亡率降低到9.5%。