Naive Bayesian fusion based deep learning networks for multisegmented classification of fishes in aquaculture industries

Highlights

• Novel fusion-based deep learning architecture is proposed for fish classification.

• Automatic head detection and orientation correction technique is developed.

• Naive Bayesian layer is introduced to fuse the deep learning networks of the fish body, head, and scale image segments.

• Classification accuracy of 98.64% and 98.94% for the ‘Fish-Pak’ and BYU fish dataset respectively is achieved.

Abstract

Fish classification is an essential requirement for biomass estimation, disease identification, and quality analysis. In aquaculture industries, fish classification is carried out in the processing unit. The fishes are out of water, subjecting them to structural deformation and orientation misalignments, makes classification challenging. A multisegmented fish classification technique using deep learning networks with naive Bayesian type fusion is proposed in this work to address these challenges. Fish images are acquired using an overhead camera. The fish head is identified by observing a minimal convexity deficiency region to facilitate segmentation. A multi-stage exhaustive enumerative optimization method is used to adjust the orientation, which can minimize unwanted background region in the image segment. Fish head, scales, and body are segmented from the fish image. For each fish segment, AlexNet is trained by using the transfer learning approach. A naive Bayesian fusion layer is introduced to fuse these trained deep learning networks and enhance classification accuracy. Experimental results illustrate a classification accuracy of 98.64% for ‘Fish-Pak’ image dataset with six different fish species and 98.94% for BYU fish dataset with four species. Comparative analysis with standard networks and ablation study demonstrates the accuracy and robustness of the proposed fusion architecture, respectively. Various fusion layers have also been analyzed, and observations illustrate the accuracy of the proposed NBC layer. Significant improvements in other classification performance metrics were also observed.

Introduction

Fish species classification (FC) is a challenging problem in aquaculture industries (Alsmadi and Almarashdeh, 2020). It is an ongoing research problem that receives significant attention over the past two decades. Unlike, ocean environment, aquaculture is a controlled environment, and often different species are bred in separate units. Hence, FC is trivial during the growing stages and underwater. However, once the fishes reach processing units, FC is needed to estimate commercial value, quality, and type of processing. (Dos and Goncalves, 2019). Each variety of fish species has customized rates and health parameters (Hu et al., 2012; Taheri et al., 2020), which need to be evaluated based on FC. Hence, it is vital to have FC technique in the processing unit using a camera, which is least investigated. In processing units, fishes are out of water and possess diversity in shape factor compared with underwater images. Also, fishes are placed in different orientations due to manual handling. It makes the FC a challenging task, which has been attempted in this work.

Predominant of the FC techniques found in literature uses surface (Rauf et al., 2019) or underwater images (Jalal et al., 2020; Zion et al., 2008). An overhead camera is used to acquire fish images at the processing unit or surface swimming, whereas underwater cameras classify live fishes. Apart from vision systems, hyperspectral imaging systems (Pettersen et al., 2019) are also employed for FC, which overcomes turbidity and blurred vision problems in underwater images. FC demands segmentation of region corresponding to fishes from its background. Fish occlusion and missing areas are some of the challenges at this stage (Funkur et al., 2020). Fish segmentation is a challenging operation in underwater images with a complex and dynamically changing background (Salman et al., 2019).

After segmentation, feature required for FC is extracted. A maximum of 133 features is reported for FC in the literature (Kutlu et al., 2017). Geometrical, statistical, color, and textual features (Hu et al., 2012; Larsen et al., 2009) are widely used for FC. Transform techniques like HSV (hue, saturation, value), wavelet, and Fourier transforms are also attempted to have features from other domain spaces (Dutta et al., 2016; Kartika and Herumurti, 2016). These extracted features are used to classify the fish types by machine learning techniques (Fouad et al., 2013). Artificial neural network (ANN) with a supervised learning algorithm has been a preferred classifier for FC (Andayani et al., 2019; Funkur et al., 2020; Lalabadi et al., 2020; Pornpanomchai et al., 2013). SVM (Support Vector Machine) ((Islam et al., 2019; Robotham et al., 2011), Naive Bayesian classifier (NBC) (Iscimen et al., 2014), KNN (Badawi and Alsmadi, 2014), decision tree (Robotham et al., 2011), and backpropagation classifier (Pornpanomchai et al., 2013) are also designed using various features for FC.

Recent literature illustrates a paradigm shift in image-based classification algorithms, and more research attention is observed towards a feature-free or automatic feature extraction method (Banan et al., 2020). Deep learning networks (DLNs) having an integrated feature extraction layer and learning layers are used for this purpose. It enables them to automatically extract optimal features required for learning and provides an end-to-end architecture with fewer or no manual interventions (Lopez et al., 2020). CNN (Convolutional neural network) is a popular DLN that uses image convolution to extract features (Ding et al., 2017). CNN found to be widely employed for FC over recent years (Dos and Goncalves, 2019; Miyazono and Saitoh, 2018; Rekha et al., 2019; Taheri et al., 2020). DLNs can provide accurate FC even in underwater environments, where images suffer intensity variations and water turbulence (Jalal et al., 2020; Sun et al., 2018). The transfer learning approach has also been attempted to use pre-trained networks for FC (Liawatimena et al., 2018). It customizes the DLN trained for image-based classification to fit a new FC problem, which is computationally less intensive than training a new CNN model. Integration of traditional classifiers like random forest tree and SVM as a layer is reported to provide improved classification performance compared with other standard deep learning networks. (Liu et al., 2019).

Predominant of the DLN based FC algorithms use a complete fish image, making the classification rely on shape factor majorly. However, shape variations are inevitable and depend on the camera's field of view, fish orientation, fish age, and other ecological factors, which degrades the classifier performance. Also, DLNs have no control over the choice of features extracted for classification. However, it is possible to make the DLN learn specific features by providing segmented images rich in those features. Compression of the complete fish image to fit DLN's input layer makes the loss of features inevitable (Zheng et al., 2016). However, segmentation breaks the entire fish image into smaller regions, lowers the compression ratio, and retains features. Motivated by these factors, the proposed work aims to train three DLNs for various fish images: fish head, scales, and body. These DLNs are fused to bring out a reliable classification accuracy.

Significant contribution of this work is as follows, (i) Automatic segmentation of fish images into head, scales, and body, (ii) Optimal orientation correction to enhance the fish-centric region of interest, (iii) Training of various DLNs for fish head, scales, and body by transfer learning, (iv) Design of naive Bayesian-based fusion layer for integration of DLNs, and (v) Experimental validation of proposed classifier using ‘Fish-Pak’ database.

Rest of this article is organized as follows. Section-2 describes the overall methodology of the proposed fusion based classifier. Section-3 demonstrates image preparation techniques used for automatic segmentation and feature enhancement by orientation correction. Training of DLNs using the segmented images and design of fusion layer using naive Bayesian classifier is explained in Section-4. Experimental results and performance assessment of the proposed technique are discussed in Section 5. Section 6 concludes with insights on future extension of the proposed work.