Patients and Slides are Equal: A Multi-level Multi-instance Learning Framework for Pathological Image Analysis

Abstract

In current pathology image classification, methods mostly rely on patch-based multi-instance learning (MIL), which only considers the relationship between patches and slides. However, in clinical medicine, doctors use slide-level labels to summarize patient-level labels as a diagnostic result, indicating the involvement of three levels of patch, slide, and patient in actual pathology image analysis, which we refer to as the multi-level multi-instance learning (ML-MIL) problem. To address this issue, we propose a novel and general framework called Patients and Slides are Equal (P &SrE), inspired by the doctor’s diagnostic process of repeatedly confirming labels at the patient and slide level. In this framework, we treat patients and slides as instances at the same level and use transformers and attention mechanisms to build connections between them. This allows for interaction between patient-level and slide-level information and the correction of their respective features to achieve better classification performance. We evaluate our method on two datasets using two state-of-the-art MIL methods as baselines. The results show that our method improves the performance of the baselines on both slide and patient levels. Our method provides a simple and effective solution to the common problem of ML-MIL in medical clinical scenarios and has broad potential applications.

F. Li, M. Wang and B. Huang—Contribute equally to this work.