Development of an automated combined positive score prediction pipeline using artificial intelligence on multiplexed immunofluorescence images
Graphical abstract
Introduction
Clinical outcomes of immunotherapy by blocking the PD-1/PD-L1 checkpoint pathway are remarkable for patients with recurrent or unresectable late-stage cancer, however, only 20%–40% of patients achieve long-term durable responses [[1], [2], [3]]. Identifying patients who are likely to respond to this treatment, and further understanding mechanisms of resistance to immune checkpoint inhibitors (ICI) are crucial for directing new therapies and reducing medical cost. PD-L1 quantification using immunohistochemistry (IHC) is the most widely validated biomarker analysis to stratify patients who would benefit from ICI, and several PD-L1 IHC based companion diagnostic tests are already approved by the Food and Drug Administration (FDA) [4]. The Combined Positive Score (CPS) on the tumour biopsies stained with Agilent PD-L1 IHC 22C3 PharmDx assay [5,6], has been used to identify patients with an increased likelihood of responsiveness to anti-PD-1 therapy in cancers such as head and neck squamous cell carcinoma (HNSCC) [7], urothelial carcinoma (UC) [8,9], gastric or gastroesophageal junction (GEJ) adenocarcinoma [[10], [11], [12]], and advanced cervical cancer [13,14]. CPS is calculated as the ratio of the number of PD-L1 positive (+) cells to the number of viable tumour cells and multiplied by 100. Current scoring practice requires the pathologist to score PD-L1 expression on at least 100 tumour cells with inclusion and exclusion rules depending on the cancer types [15]. The manual scoring is laborious, time consuming, and dependent on human interpretation. Thus, developing an automated CPS algorithm is beneficial in easing the burden on the pathologists and facilitates the search for reliable immune biomarkers for other ICI [4].
Few automatic CPS scoring on two-colour chromogenic PD-L1 IHC images have been reported [16,17], however, as far as we know we cannot find any publication on automatic CPS scoring using immunofluorescence (IF) images. In comparison to IHC, more immune markers can be labelled using multiplexed immunofluorescence staining method on the same biopsies with high intensity amplification [18,19]. This advantage provides the researchers with the access to understand the complex spatial contexture and cell interactions in tumour microenvironment which has shown to be highly relevant to tumour progression and cancer therapeutic outcomes [20,21]. Thus, developing an automatic CPS scoring using mIF images can expand the power of companion diagnostic using clinical biopsies to benefit cancer patients.
In this study, we propose an automated end-to-end CPS prediction pipeline using mIF images. We employed three main AI-based modules: quality check (QC), nuclear segmentation, and cell phenotyping to design our CPS prediction pipeline. The QC module identifies and excludes the common artefacts in the mIF images, which are different from the IHC image artefacts [18]. We evaluated this pipeline using the biopsies from recurrent HNSCC patients, who had enrolled in a clinical study sponsored by Rakuten Medical Inc. ([22,23]). The IHC images using PD-L1 IHC 22C3 pharmDx and the corresponding adjacent paired mIF images using GE-MultiOmyx platform were prepared for all biopsies. We verified the performance of our AI-based CPS prediction by comparing with the gold-standard CPS of the same biopsies manually scored by a certified pathologist.
Section snippets
Patients and tumour specimens
Tumour biopsies were collected from the patients with recurrent HNSCC (rHNSCC) who had enrolled in a Rakuten Medical sponsored study (NCT02422979). Haematoxylin and eosin (H&E) staining, PD-L1 IHC staining using PD-L1 IHC 22C3 pharmDx kit (Agilent, Santa Clara, CA, USA) [5], the mIF staining with PD-L1 antibody (SP142, Abcam, Cat: ab236238) and PanCK antibody (PCK-26, Sigma, Cat: C5992), and the mIF imaging acquisition using GE MultiOmyx platform (GE Research, Niskayuna, NY, USA) were completed
Performance in detecting artefacts, nuclear segmentation, and cell phenotyping in mIF images
We evaluated the detection of blur on CPS-test dataset images, air-bubble artefacts from 28 images among the Akoya-set, and tissue-fold artefacts using 95 images also from the Akoya-set. For air-bubble detection, 18 images contained air-bubbles and 10 images did not. For tissue-fold and blur detection [50], images contained different sizes of tissue-fold or blur regions whereas 30 did not. We obtained image classification F1-scores of 74%, 88.8%, and 87.4% for blur, tissue-fold, and air-bubble,
Discussion
Immunotherapy targeting the PD-1/PD-L1 checkpoint and deep learning for image analysis are two breakthroughs in the last decade. Determining PD-L1 expression by IHC is the most widely studied predictive biomarker to stratify patients for cancer immunotherapy targeting the PD-L1 checkpoint pathway. Many ongoing and future clinical trials of immunotherapy also contain biomarker testing, including PD-L1 IHC, in enriching patient responses [4]. Deep Learning has shown great promise in clinical
Conclusions
Optimizing the cancer treatment efficacy for individual cancer patients while reducing the cost is the key goal for the health care industry. We addressed those challenges by developing an automatic CPS scoring pipeline to stratify cancer patients who are likely benefit from immune checkpoint therapy. This proof-of-concept study shows that our automatic CPS scoring using artificial intelligence on multiplexed immunofluorescence (mIF) images is concordant with the CPS gold standard by the
Statement of author contributions
MD, SM, CWL conceived and designed the study; AV, SS, DM implemented the algorithm and analysed data; MGG and CWL provided the data; AV, SS, DM, JJ, SM, CWL reviewed and interpreted the analysis. All authors were involved in writing the paper and had final approval of the submitted and published versions.
Data availability statement
Please contact the corresponding author for data availability. All requests for data and algorithms will be reviewed by Rakuten Medical Inc. To verify if the request is subject to any intellectual property or confidentiality obligations. Any data and materials will be released via a Material Transfer Agreement.
Funding statement
The studies described herein were funded by Rakuten Medical, Inc. Abhishek Vahadane (AV), Shreya Sharma (SS), Devraj Mandal (DM), Madan Dabbeeru (MD), and Shantanu Majumdar (SM) are employed by Rakuten India Enterprise; Josephine Jakthong (JJ) is a consulting pathologist for Rakuten Medical, Inc.; Miguel Garcia-Guzman (MGG) and Chung-Wein Lee (CWL) are employed by Rakuten Medical, Inc.
Conflict of interest disclosure
The authors have no conflicts of interest to disclose.
Ethics approval statement
The original study in which patient biopsies were collected was performed in accordance with the principles of the Declaration of Helsinki (1964), Good Clinical Practice guidelines, and applicable US Code of Federal Regulations (CFR), 21 CFR Part 50 & 312. The study protocol, informed consent form, and any other appropriate documents were approved by the Institutional Review Board/Independent Ethics Committee for each participating center.
Patient consent statement
The patients signed informed consent forms prior to enrolment in the Rakuten Medical, Inc. Sponsored trial.
Permission to reproduce material from other sources
The article does not include data from other resources.
Clinical trial registration
NCT02422979
Declaration of competing interest
None Declared.
Acknowledgements
The authors wish to thank the patients and families who participated in the Rakuten Medical sponsored parent clinical study for their participation. The last author thanks the language editing and manuscript proofreading by Dr. Laura Spuhler and Ms. Cindy Chen.
References (64)
- et al.
Four-Year survival with durvalumab after chemoradiotherapy in stage III NSCLC-an update from the PACIFIC trial
J. Thorac. Oncol.
(2021) - et al.
Five-year survival outcomes for patients with advanced melanoma treated with pembrolizumab in KEYNOTE-001
Ann. Oncol. : off. J. Europ. Soc. Med. Oncol.
(2019) - et al.
Pembrolizumab alone or with chemotherapy versus cetuximab with chemotherapy for recurrent or metastatic squamous cell carcinoma of the head and neck (KEYNOTE-048): a randomised, open-label, phase 3 study
Lancet
(2019) - et al.
First-line pembrolizumab in cisplatin-ineligible patients with locally advanced and unresectable or metastatic urothelial cancer (KEYNOTE-052): a multicentre, single-arm, phase 2 study
Lancet Oncol.
(2017) - et al.
Multiplex immunofluorescence staining and image analysis assay for diffuse large B cell lymphoma
J. Immunol. Methods
(2020) - et al.
Workflow and metrics for image quality control in large-scale high-content screens
J. Biomol. Screen
(2012) - et al.
Towards machine learned quality control: a benchmark for sharpness quantification in digital pathology
Comput. Med. Imag. Graph.
(2018) - et al.
Analysis of focus measure operators for shape-from-focus
Pattern Recogn.
(2013) - et al.
COVID-19 diagnosis via chest X-ray image classification based on multiscale class residual attention
Comput. Biol. Med.
(2022) - et al.
Multilevel threshold image segmentation for COVID-19 chest radiography: a framework using horizontal and vertical multiverse optimization
Comput. Biol. Med.
(2022)