Keywords

1 Introduction

Liver disorders correspond to the variety of diseases, infections and symptoms influencing the morphology, histology and physiological functions of the liver. Hepatic abnormalities have been reported to be the major cause of around 20,000 deaths every year [1]. The susceptibility of the liver to different fatal diseases especially jaundice, hepatitis, liver cirrhosis etc. is provoked by the excessive assimilation of various drugs as well as chemicals in the liver hepatocytes [2, 12]. Phytomedicines are routinely being used for the treatment of such hepatic disorders and have been found to be much more efficient than other drugs. Hence exploration and development of more and more herbal drugs from ethnomedicinally important medicinal plants for fast and effective therapy of liver diseases is the demand of time. Pteris semipinnata L. is a potential medicinal pteridophyte that is used for the treatment of hepatitis, enteritis and snake bite [5]. Paste of fronds of this plant is used externally for the treatment of carbuncle by reang tribe of Tripura [15]. This plant is widely used in curing diarrhoea as well as inflammatory disorders also [6]. The liver is an essential organ for survival and no other organ can compensate for it if the liver is damaged. Necrosis is a widely recognized form of hepatocyte damage. Microscopic imaging is an important tool in medical imaging and it provides rich and complex information about the structure of tissues and the current state of the cells. These microscopic images help pathologists study the cells and derive information from it. Due to complexity of the microscopic images, it takes time to make the diagnosis and sometimes leads to disagreement between pathologists which suggests a rising demand for stable, reliable and robust computational methods to increase the diagnosis efficiency. In the recent years, deep learning has contributed greatly to the analysis of images in the medical field. Convolutional neural networks (CNNs) has given state of the art performance on image classification [16]. CNNs are widely used for object detection and other macro landscapes. Researchers all across the world have applied CNNs to medical images like X-rays, microscopic images etc. CNN classifiers are used to segment and classify histopathological images. Classification assigns a label to the image i.e. normal/abnormal. CNNs are used in segmentation where the contours and boundaries are detected. Pre-Processing the microscopic images improve the performance of CNNs. In this experiment we present a CNN model based to classify healthy or damage liver tissues. We have combined a binary classification strategy with image splitting to calculate the amount of damage in the given sample. We also use data processing techniques and data augmentation as we lack in sufficient amount of data.

2 Related Works

CNN’s are widely used in biomedical imagery applications as they overpower traditional methods which require handcoded feature extraction. A comprehensive tutorial with selected use cases that uses deep learning for digital pathology image analysis [7]. A deep CNN is used for endotracheal tube position and X-ray image classification [9]. Cell detection in pathology and microscopy images with multi-scale CNN [13]. A categorization for liver cirrhosis has been proposed using CNNs for health informatics [17]. A customized CNN uses shallow convolution layer to classify lung image patches from interstitial lung disease [10]. A deep CNN network classifies mice hepatic granuloma microscopic images [18]. There has been a requirement for applying computational approaches to get better diagnosis efficiency. A search algorithm known as Modified cuckoo segments microscopic images hippocampus [4]. Another CNN based method uses clustering and manifold learning for diabetic plantar pressure imaging dataset [11]. An optimized LPA-ICI filter denoises light microscopy image [3]. A deep CNN is used to analyse the histopathological snapshots by segmentation-based classification of epithelial and stromal regions. [8]. An arrowhead detection technique for biomedical images [14]. A deep CNN is used for segmenting and classifying epithelial and stromal regions in histopathological images [19].

3 Pre-processing

Several Pre-processing techniques were used to make our model learn features easily Fig. 1. Grayscaling is done so that the image sharpness increases also the light and dark areas are easily identifiable. Dataset Normalisation is done so that all the values of the data lie at zero centered mean and a variance of one. To increase the overall contrast and making the highlights look brighter CLAHE is used. Gamma Correction is again used for brightness control and to make sure the images don’t become too dark or bleached out.

Fig. 1.
figure 1

Pre-processed output generated

Full size image

3.1 RGB to Grayscale Conversion

The usual method for grayscale conversion is the average method which is the most simple one but it does not give a dynamic grayscale image. In this project, we have chosen to go with the original ITU-R recommendation (BT.601, specifically) which gives a more dynamic grayscale image. The formula is:

$$\begin{aligned} Gray = (B\times 0.114 + + G \times 0.587 +R\times 0.299) \end{aligned}$$
(1)

where R, G, B is in the range 0–255. After this the image has one channel.

3.2 Dataset Normalisation

The images in the dataset is normalized by subtracting the mean of the dataset which serves to ‘center’ the data. Next it is divided by the standard deviation of the dataset. This process is done because in while training our network, we’re going to be multiplying (weights) and adding to (biases) these initial inputs in order to cause activations of the neurons that we then backpropagate with the calculated gradients to train and update the model. Normalisation is done for each feature to have a similar range so that our gradients don’t go out of control. The pseudo code for normalisation is:

$$\begin{aligned} (z-z.mean())/z.std()) \end{aligned}$$
(2)

For standardizing the values between 0 and 1 the following was done where x represents each image in the dataset.

$$\begin{aligned} (z - z.min()) / (z.max() - z.min()) \end{aligned}$$
(3)

3.3 Contrast-Limited Adaptive Histogram Equalization (CLAHE)

Histogram equalization is used to give a linear trend to the cumulative probability function associated to the image and improve it’s quality. This process images to adjust the contrast of an image by modifying the intensity distribution of the histogram. Histogram equalization uses the cumulative probability function (cdf). The cdf is a cumulative sum of all the probabilities lying in its domain and defined by:

$$\begin{aligned} cdf(x)=\sum _{k=-\infty }^{x} P(k) \end{aligned}$$
(4)

These images are encoded with 8 bits will only have \(2^8=256\) possible intensity values going from 0 representing black to L−1 = 255 representing white. The following formula gives the new pdf:

$$\begin{aligned} S_k=(L-1)cdf(x) \end{aligned}$$
(5)

In CLAHE, the image is divided into small blocks called ‘tiles’ (\(8\times 8\) pixels). Then every tile is histogram equalized. So in a tile, the histogram would be localized to a small region. If noise is present, it will be amplified and so to avoid this, contrast limiting is used. If any histogram bin is greater than the contrast limit that is specified then those pixels are clipped and distributed uniformly to other bins before applying histogram equalization.

3.4 Gamma Correction

Gamma, which is denoted by the Greek letter \(\gamma \), is described as the relationship between an input that will be the RGB intensity values of an image and the output. The equation for computing the resulting output image is as follows:

$$\begin{aligned} I^{'}=255\times (\frac{I}{255})^{\gamma } \end{aligned}$$
(6)

When the gamma is 1.0 the input equals the output producing a straight line. For calculating gamma correction the formula is as follows:

$$\begin{aligned} I^{'}=255\times (\frac{I}{255})^{\frac{1}{\gamma }} \end{aligned}$$
(7)

4 Proposed Methodology

4.1 Preparation of Test

Male Swiss albino mice were randomly classified into four groups (n = 6). A single water dose (25 ml/kg) was orally administered to the animals of group 1 (control), daily for 6 consecutive days. The same group of animals was also administered olive oil (10 ml/kg, i.p) on 1st and 2nd day. The animals of group 2 (CCl4) were given water (25 ml/kg) for 6 days and CCl4 (0.2%) in olive oil (10 ml/kg, i.p.) was administered on 1st and 2nd day. Group 3 animals were orally administered with the standard drug silymarin (100 ml/kg) once in a day for 4 days from 3rd to 6th day. They also received CCL4 (0.2%) in olive oil (10 ml/kg, i.p) on 1st and 2nd day. On day 7, the mice (n = 6 per group) were anaesthetised by chloral hydrate (350 mg/kg b.w.; I.P.), and then mice were perfused transcardially with 50 mL each of ice-cold 0.1M phosphate buffered saline (PBS; pH 7.4) and 4% w/v paraformaldehyde (in PBS). After perfusion, liver of the mice were dissected out, and stored in 4% paraformaldehyde, cryoprotected overnight in 30% w/v sucrose solution, 5 \(\upmu \)m thick liver sections were taken on poly-L-lysine coated slides and Haematoxylin-Eosin staining was performed. This dataset has been prepared by the technicians of Biotech Hub, Assam University using foldscope.

4.2 Model

After the images were pre-processed with shape \(240\times 320\times 1\) they were then fit to the convolution model. Our architecture is made up of repeating convolution block Fig. 2.

Fig. 2.
figure 2

Convolution block which was repeatedly used in our architecture

Full size image

Each convolution block consists of three parts mainly first a convolution layer then a Max Pooling layer and finally Batch Normalization. The Convolution layer has a kernel size 3, stride of 1 and uses same padding. After the convolution ReLU activation was applied on it. Then Max Pooling was performed with a pool size of 2 and a stride of 2. Finally Batch Normalization was performed to stabilize the training. The first convolution block had 32 filters then the second had 64 filters and the third had 128 filters. The fourth and the fifth block had 256 filters. The sixth and seventh consists of 512 filters after that the output was flattened to a dense layer with hidden units of 512 and activation of ReLU. Final layer consists of one hidden unit and an activation of sigmoid was used. Last layer predicts whether the image is normal or damaged Fig. 3.

Fig. 3.
figure 3

Our proposed model architecture

Full size image

5 Training

The training data in our hand was very low, we had 4 images for normal mice liver tissue and 8 images for damaged mice liver tissue. For validation purposes we hold out 1 image from both normal and damaged liver tissue to test our network. As deep learning model requires a lot of data for training the model, we used several techniques mentioned below to augment the data and make it feasible for our network to learn.

  1. 1.

    As the number of damaged images in our dataset is almost twice of the normal images, to balanced the dataset we took 500 random images from damaged tissue and 1000 random cropped images from the normal tissue.

  2. 2.

    The images were random cropped with a dimension of \(240\times 320\) which is 16 times smaller from the original image of dimension \(960\times 1280\).

  3. 3.

    If the image taken is damaged then set the label of the 1000 random cropped image as 1 else if the image taken is normal then set the label to 0.

  4. 4.

    The images are then converted from RGB to grayscale.

  5. 5.

    Then they are normalized by subtracting the mean and dividing by variance and 255.

  6. 6.

    After normalization Clahe Equalization was performed with a clip size of 2 and tile grid size of \(8\times 8\).

  7. 7.

    Gamma Adjustment was done with Gamma value of 1.

  8. 8.

    The prepared data was further augmented by taking vertical flip, horizontal flip and rotated by a value of 20.

6 Result

Fig. 4.
figure 4

(a) Our model loss for 100 epochs (b) Our model accuracy for 100 epochs

Full size image

The loss used in our experiment to train the model was Binary Cross Entropy function and for optimizing our model Adam Optimizer was used. The weights of our model were saved whenever the validation loss was less than the previous validation loss. The model was trained for 100 epochs with a batch size of 150. Learning rate decay was used with a factor of 0.8 which monitored the validation loss for 10 epochs. The Graphs for losses and accuracy have been plotted Fig. 4 (Tables 1 and 2).

Table 1. Our experiments results
Full size table
Table 2. Confusion matrix on test dataset
Full size table

7 Conclusion

Liver dysfunction is one of the major reasons for increasing death rate. Accurate and fast diagnosis with safe medication can control the aggravated situation. Presently, the analysis of microscopic liver images are required to find out various anomalies and using Artificial Intelligence to assist pathologists is the future of medical development. In this experiment, we propose a novel methodology to detect Necrosis in mice liver tissues using a combination of pre-processing and a deep convolutional neural network. In future, we are required to deal with the problem of less number of training images. To assist pathologists more comprehensively, there is room for detection of the damaged areas more precisely.