Bleeding contour detection for craniotomy

Highlights

• The first result on the bleeding contour detection during craniotomy for the images obtained by a visible-light CCD.

• All types of texture of the surgical area in all three neurosurgical phases are covered and the algorithm can detect the bleeding contour successfully.

• The resultant algorithmic pipeline forms a technique solution for autonomous bleeding removal operation by a medical robot.

Abstract

Objective

Bleeding impairs observation during neurosurgery, and excessive bleeding endangers the life of a patient. Thus, hemostasis is important during neurosurgery. The detection of bleeding areas is a prerequisite for hemostasis.

Methods

To the best of our knowledge, this paper is the first to present results on the detection of neurosurgical craniotomy bleeding scenarios, i.e., scalp incision bleeding, skull incision bleeding, and dura matter-incision bleeding. This is realized via a workflow that combines craniotomy image data preparation and a Mask R-CNN framework. Bleeding images on a porcine skin tissue with a simulated blood injected by a syringe are taken by a visible light camera, and the video frames of the scalp incision, skull incision, and dura matter-incision bleeding are extracted from neurosurgical videos.

Results

The precision of bleeding areas detection for the simulated bleeding scenario and the three craniotomy phase scenarios were 94.40%, 84.44%, 89.48%, and 90.46%.

Conclusion

The contours of the neurosurgical craniotomy bleeding regions can be detected along with the bleeding areas.

Significance

It is beneficial for neurosurgeons to identify the bleeding areas by sending prioritized alerts for bleeding events. Furthermore, it is valuable for a task-level medical robot designed for a neurosurgical procedure, such as craniotomy, or a high-level robot designed for an entire neurosurgery procedure.

Introduction

Blood is an essential matter for humans. When it bleeds, bleeding must be stopped to ensure their safety [1], [2], [3]. In neurosurgery, hemostasis is a key and tedious task accompanying the continuous dissection of brain tissue to reach a tumor. Before hemostasis, an aspirator is used to suck and remove the flowed-out blood that conceals the incision. Thereafter, the bleeding areas are exposed, and an electric coagulation knife is directed toward the bleeding area to coagulate the incised tissue. However, manual hemostasis is neurosurgical time-consuming, and its precision depends on the experience of neurosurgeon as well as physical conditions. In addition, long hours of neurosurgical workload decrease the detection precision of bleeding areas. Furthermore, many brain tissues are being handled and may result in injuries owing to the decreased precision. This may be resolved using a bleeding detection method with a high detection precision. Thus, detection of bleeding areas by an image processing approach can be an ideal alternative for neurosurgeon observation. In addition, for future autonomous neurosurgical robot based on the early works [4], [5], [6], [7], [8], [9], [10], hemostasis is an indispensable task and the method for bleeding contour detection is also the basis of its autonomy.

During a neurosurgery procedure, multiple steps are involved, including craniotomy, dissection of healthy tissue for building a tunnel to tumor, tumor dissection, refilling the built tunnel with tissues of patient, and finally, scalp suture. All steps are accompanied by bleeding and hemostasis. However, owing to the differences between the bleeding scenarios of these steps, it is challenging to design a general bleeding contour detection algorithm that is effective for all these steps. Thus, in this study, we addressed the bleeding area detection during craniotomy. Craniotomy entails the following three consecutive sub-steps: scalp incision, skull flap removal, and dura mater incision. Three typical bleeding scenarios for each sub-step of craniotomy are illustrated in Fig. 1B, Fig. 1C and Fig. 1D, and their corresponding anatomical positions are denoted by arrows in Fig. 1A. After craniotomy, the brain is exposed and ready for the next step of building a connecting tunnel to the tumor.

Neurosurgical hemorrhage detection is studied using multiple methods. The accuracy of the hemorrhage detection method employed by a radiologist is developed by designing a single-stage, end-to-end, fully convolutional neural network, which could both identify and localize abnormalities often missed by human doctors [11]. A deep convolutional neural network for hemorrhage detection has been proposed to concurrently learn features and classify them, as well as to remove multiple hand-tuned workloads, including multiple stages of alignment, image processing, image corrections, manual feature extraction, and classification [12]. A rule-based hemorrhage segmentation approach has been used to detect hemorrhage areas with high accuracy by utilizing pelvic anatomical information [13]. For detection and classification of intracranial hemorrhage, a cascade of deep convolutional neural networks, which are derived from SE-ResNeXt50 and EfficientNet-B3, can achieve both automatic feature extraction of head computed tomography images and the classification of five subtypes of intracranial hemorrhage; its detection accuracies can reach the neurosurgeon level [14]. For the detection of intracranial hemorrhage and its five subtypes in non-contrast head CT (NCCT), similar to the objective of [14], a joint convolutional and recurrent neural network (CNN-RNN) classification framework has been provided and trained with both subject-level or slice-level labels [15]. Furthermore, for the detection of intracranial hemorrhage, a customized deep-learning tool has been proven to be clinically useful for the detection and quantification of hemorrhage using non-contrast head CT [16]. For the detection of acute intracranial hemorrhage (ICH) and classification of 5 intracranial hemorrhage subtypes from unenhanced head computed tomography scans, an understandable deep learning system has been developed [17]. This is another artificial intelligence algorithm that can optimize workflow and reduce time to detect intracranial hemorrhage by 96%, and it may also recognize subtle intracranial hemorrhage missed by human doctors [18]. However, the existing methods of neurosurgical hemorrhage detection have been designed for intracranial hemorrhage using the CT imaging modality, and not for bleeding contour detection using images obtained by a white light camera. Thus, these methods are not suitable for bleeding contour detection during craniotomy consecutive sub-steps, i.e., the scalp, skull, and dura mater incisions.

In addition to neurosurgery, bleeding contour detection is a key task in the diagnosis and therapy of diseases in multiple organs, including gastrointestinal, epistaxis, and abdominal bleeding with multiple imaging modalities. For gastrointestinal computer-assisted diagnosis (CAD), a classifier fusion approach is presented by exploiting a couple of optimized support vector machine classifiers to automatically recognize bleeding areas existing in wireless capsule endoscopy (WCE) images [19]. Similarly, another supervised method for automatic recognition of bleeding regions in wireless capsule endoscopy frames has been developed, and the presented algorithm distinguishes the image regions by extracting statistical features obtained from the first-order histogram probability of the three planes of RGB color space [20]. Moreover, an automatic multiple bleeding spot detection method for wireless capsule endoscopy images consists of two core tools, namely feature extraction and supervised and unsupervised learning techniques, which is used to determine the visual properties of the multiple bleeding areas, and then supervised and unsupervised learning procedures are employed to precisely identify multiple bleeding spots [21]. Further, the bleeding contour detection of wireless capsule endoscopy images is tackled by the normalized RGB color space that is aided by different shades of red [22]. For multiple detection of both bleeding and ulcer in wireless capsule endoscopy images, a method has been designed using a chromaticity moment as the key features to differentiate healthy regions and abnormal areas [23]. For cerebral bleeding diagnosis, a densely connected neural network has been employed to identify the cerebral micro-bleedings area through the transfer learning technique [24]. Another technique for bleeding detection is exploiting the color information, which is a critical feature for the primary detection of bleeding images, and the texture, which is important in extracting more disease information [25]. For nasal bleeding, a method has been proposed to demonstrate its effectiveness in identifying bleeding areas in the nasal cavity and nasopharynx that could not be detected via routine anterior rhinoscopy [26]. For a minimally invasive surgery, an image processing algorithm based on local entropy is proposed to identify spurts of blood and locate their sites in real time [27]. However, these methods are not for the bleeding detection of scalp incision bleeding, skull incision bleeding, and dura mater incision bleeding. Limited studies or results can be found on the bleeding contour detection during neurosurgical craniotomy.

Recent studies on deep learning to detect intraoperative bleeding contour are as follows. There is an improved wireless capsule endoscope (WCE) image bleeding detection method based on Alexnet [28], but Alexnet method is a classical CNN network and cannot segment at the pixel level. A study used deep learning technology to automatically classify the blood loss of transurethral resection of prostate surgery (TURP) [29]. According to the surgical characteristics of TURP, an integrated Res-Unet model was proposed to eliminate the interference of red light generated by surgical cutting ring on bleeding area recognition. This method has strong pertinence, however, its IOU of 0.66 is far lower than our results, so it is not suitable for the identification of intraoperative bleeding in general surgery. Therefore, the work of using neural network to detect intraoperative bleeding proposed in this paper is groundbreaking. Mask R-CNN, as an example segmentation network, can distinguish different blood examples and output high-quality masks. Therefore, it can be used as the visual part of subsequent robot work.

In previous work, there are few methods to automatically detect bleeding in brain surgery by neural network. Hemorrhagic spots are identified by neural network in the brain on computed tomography images [30]. Bleeding in wireless capsule endoscopy Images is detected using a combination of superpixel segmentation and SVM [31]. Therefore, the work of using neural network to detect intraoperative bleeding proposed in this paper is groundbreaking. Mask R-CNN, as an example segmentation network, can distinguish different blood examples and output high-quality masks. Therefore, it can be used as the visual part of subsequent robot work.

Therefore, in this study, we detect the bleeding contours during neurosurgical craniotomy by employing the Mask R-CNN framework. It is beneficial for neurosurgeons to identify the bleeding areas by sending prioritized alerts for bleeding events. Furthermore, it is valuable for a task-level medical robot designed for a neurosurgical procedure, such as craniotomy, or a high-level robot designed for an entire neurosurgery procedure. It should be noted that the autonomous surgical robot is only used to assist doctors in better surgery, and will not control the procedure related bleeding.