Keywords

1 Introduction

The bony fishes have calcified structures called otoliths. They are located in the inner ear of fish species, and are composed of calcium carbonate crystals and organic materials (Fig. 1). Among fish species, otoliths have various shapes and are characteristic features for fish species classification [1–4]. Since otoliths are resistant to degradation they are often the only identifiable structures that can be recovered from the stomach and feces of fish specimen. In this context, the shape analysis of otolith is largely used to identify food web of fish species, but also in taxonomy, phylogenetic, paleontology and stock discrimination [2, 5]. The classical approach for identification of fish species using otoliths is based on natural observation from experts using a naked eye approach. However, this technique is quite expensive and time consuming [6, 7]. Accordingly, looking for a more accurate, automatic, inexpensive and faster methodology to recognize fish species using otoliths is relevant for marine biologists [8]. In this field, a variety of results have been reported [9–12], usually by applying classical complex Fourier descriptors (CFD) or elliptical Fourier descriptors (EFD) [13].

The EFD method decomposes in harmonic series the contour of an otolith, otherwise EFD outperforms CFD in the classification of otoliths when otoliths have complex shapes. In this paper, we present an automatic classification method of otoliths based on a representation by the normal angles of the counterclockwise 8-connected contour [14] of the otolith. Finally, the Fourier descriptors of the curve of the normal angles along the contour are used in a neural network classification method.

Fig. 1.
figure 1

Otolith fish species of Engraulis encrasicolus.

Full size image

The paper is structured as follows: the Sect. 2 describes the architecture of the proposed approach. In this section, we present the algorithm to approximate the contour by a discrete polygonal in order to avoid discontinuities along the contour obtained from the acquisition of images. Then, we describe the extraction of a counterclockwise 8-connected otoliths contour and the calculation of normal angles of the contour. At the end of this section, we present the extraction of the characteristic parameters (Fourier descriptors) and the classification method. Section 3 is devoted to experimental tests. Finally, a conclusion and future works are discussed in Sect. 4.

2 System Design

Fig. 2.
figure 2

Design of proposed system.

Full size image

The aim of the proposed approach is to design an automatic recognition and classification system of otolith images. This approach is based on shape analysis of otolith in order to manage marine resources. The system design is showed in Fig. 2.

2.1 Image Acquisition

The first step is to collect the sample images for building the database in order to train and test the classification system. The otoliths dataset (DB) used in this study derived from the Moroccan Atlantic area between Larache and Dakhla. The otoliths were collected by scientists of the National Institute of Fisheries Research (INRH) during sampling campaigns, spread over year 2002 to 2014, on research vessels and trawlers commercial landing. After the collection operation, we proceed to otolith acquisition using a stereo microscope Leica S8 APO, a Leica camera EC3 connected to a PC and Leica LAS EZ software (Version 3.0.0 for windows). The database contains 450 images from 15 different species, with 30 images by species. In Fig. 3, we illustrate examples of otolith images in the database.

Fig. 3.
figure 3

(a) Micromesitius poutassou(C2), (b) Merluccius merluccis(C3), (c) Merluccius polli(C12), (d) Merluccius senegalensis(C13), (e) Helicolenus dactylopterus(C8), (f) Trachurus trachurus(C9), (g) Engraulis encrasicolus(C10), (h) Sardina pilchardus(C11), (i) Trachyscorpia cristulata(C14), (j) Argentina sphyraena(C1), (k) Mullus surmuletus(C4), (l) Pagellus acarne(C5), (m) Pagellus erythrinus(C6), (n) Dicologoglossa cuneata(C7), (p) Plectorhynchus mediterraneus(C15).

Full size image

2.2 Polygonal Contour Approximation

After the capture of otoliths images, we observe many short discrete segments of pixels followed by sharp fluctuations in the pattern of normal directions along the frontier (Freeman directions) and consequently these discontinuities will introduce noise in the sequence of the normal directions to the contour. It follows that the curve of variations of the normal angles seems to be inaccurate in view to have a good discrimination if the contour is noisy. The classical smoothing methods such as median filter, average filter and operators of mathematical morphology are not satisfactory to smooth the contours of our images. In order to overcome this difficulty, we first approximate the contour by a polygonal contour to obtain a more compact description of the contour suitable for further processing and shape classification [15]. Several polygonal approximation methods have been proposed in recent years. We used the algorithm proposed by HUANG and WANG [16]. In general, the pixels on the 8-connected initial contour (see Sect. 2.3) are successively examined in order to determine a sequence of longest segments that satisfy a predefined threshold. Basically, let P0 and P1 the two pixels on the frontier of the otolith such that the segment [P0, P1] is the longest secant on the otolith itself. This secant divides the otolith into two regions R0 and R1 with contours C0 and C1 respectively. In each region, we look for the pixel P2 on the contour, one pixel P2 in each region, such that the distance to its projection P on the segment [P0, P1] is maximal. If the length of the segment [P, P2] is less than \(\epsilon >0\) (threshold) we retain P2 as a summit of the polygon. This process is repeated on each successive sub region obtained as long as the criterion is respected. Figure 4 below illustrates the result of this method.

Fig. 4.
figure 4

Original image (left) and the polygonal approximation (right).

Full size image

2.3 Counterclockwise 8-Connected Contour and Normal Angles Contour

In this section we present a method to extract the counterclockwise 8-connected contour of an otolith image. This contour has a minimum length and is characterized by an ordered sequence of Freeman chain codes [17] associated to the successive displacements between pixels along the contour.

Extraction of the Counterclockwise 8-Connected Contour: After the polygonal approximation, which is not necessarily 8-connected, we consider the binary image of the otolith (white pixels inside and black pixels outside). Each black pixel is indexed by the value 0 and each white pixel by the value 255 (Fig. 5). Our objective is to construct a counterclockwise 8-connected contour and from this contour we calculate the normal angles contour. For this, we applied an algorithm proposed by Chalifour et al. [14]. As a result, the contour of an otolith will be a closed, simple or weakly simple path, positively oriented (counterclockwise) with a thickness equal to one pixel. The contour of black pixels \(C = [P_{1}, P_{2}, ,P_{i}, ,P_{L}, P_{L+1}=P_{1}]\) have a minimum chain length L, with regard to the 8-connectivity. The pixel \(P_{1}\) is called the starting pixel of C and this unique pixel is located at the upper left corner with a Freeman direction \(\alpha =5\) between \(P_{1}\) and \(P_{2}\).

Fig. 5.
figure 5

Binary images of two different species.

Full size image

From that first displacement, we construct successively, and unequivocally, the arcs between pixels of the otolith contour, such that the arc \(a_{k} = [P_{k}, P_{k+1}]\), for \(k\ge 2\) is constructed from the previous arc \(a_{k-1}\), respecting hypotheses [14, 18] : the choice of \(P_{k+1}\) minimizes the contour length with regard to the 8-connectivity, the chain is counterclockwise, the extremity of each arc is a black pixel, and the pixels at the left of the arc, compared to the oriented displacement, are white pixels inside the otolith. With these hypotheses and given an arc \(a_{k-1}\), there is the only oriented arc \(a_{k}\), that simultaneously verifies the hypotheses above. The choice of the pixel \(P_{k+2}\) is performed according to the pixel \(P_{k+1}\). We use the Freeman indices to indicate the direction of each arc. The only admissible pairs (i, j) of successive displacements (Freeman directions) are presented in the Table 1 below, where the first index i (line) indicates the direction of an arc \(a_{k}\) and the index j (column) indicates the admissible direction of the following arc \(a_{k+1}\). The configurations can have two opposite successive directions in the case where the contour growths (thin frontier) inside the region with white pixels.

Table 1. Pairs of admissible successive directions.
Full size table

We associate to the contour C, the oriented chain of Freeman directions \([\alpha _{1}, \alpha _{2}, ..., \alpha _{k}, ..., \alpha _{L}]\) where ak is the direction of the arc \([P_{k}, P_{k+1}],\) for \(k = 1, ..., L.\) These sequences of pixels and directions allow us to compute the normal angles on the contour. We associate a normal angle \(\theta _{k}\) to each pixel \(P_{k} ,1 \le k \le L,\) along the contour. For this, we consider two successive displacements between the pixels \([P_{k-1}, P_{k} ,P_{k+1}]\) for \( 2 \le k \le L \) and the pixels \([P_{L}, P_{1} ,P_{2}]\) for \(k=1\). The normal angle associated to \(P_{k}\) is the average value (in radians), of the perpendicular directions of the directions \(\alpha _{k-1}\) and \(\alpha _{k}\). For the allowable displacements presented in Table 1, the only possible angles are in the set \(\{0, arctan (0.5), \frac{\pi }{4} , arctan (2)\}\) and their appropriate rotations. We note \(C_{a} = [\theta _{1}, \theta _{2}, ... , \theta _{i-1}, \theta _{i}, ..., \theta _{L}, \theta _{L+1} = \theta _{1}]\) the set of normal directions associated to the pixels of the contour. We obtain a discrete function of the normal angles along the contour, each angle being dependant of the position of pixels in the chain of the contour (Fig. 6). In order to get a standardized representation of the functions of normal angles, independent of the length of the otolith contour, we brought back by contraction, the definition interval of these functions to the interval [0, 1]. We approximate \(C_{a}\) by a periodical function using the classical discrete Fourier transform (Fig. 7). We only kept the first fifteen coefficients that multiply each cosine or sine base function in the Fourier development (\(a_{k}\), \(b_{k}\) \(k=1,....,15\)). Among the properties of the Fourier descriptors we cite its invariance to translation, rotation, and scale change.

Fig. 6.
figure 6

Patterns of normal angles along the contour of three otoliths.

Full size image
Fig. 7.
figure 7

Patterns of normal angles after Fourier approximation of the otoliths in Fig. 6.

Full size image

2.4 Classification Method

In this section we describe the classification system based on the extracted features from the otolith images as input in view to associate each image to a class. This system is based on a neural networks classifier. This classifier consists of three layers of processing nodes (neurons) [19], an input layer, a hidden layer and an output layer. Only one hidden layer is used in order to restrict the calculation time. Figure 8 illustrates the architecture of the neural network classifier. The input layer consists of the Fourier coefficients (n = 30). The hidden layer contains thirty two neurons (H = 32), and the output layer contains the 15 classes (species) tested.

Fig. 8.
figure 8

ANN Architecture with layers.

Full size image

3 Results and Discussion

The goal of this section is to evaluate and test our approach based on the normal angles contour (NAC method). We compare the quality of the classification of the specimen obtained with our approach and the methods using complex Fourier descriptors (CFD) and Elliptic Fourier descriptors (EFD). Table 2 presents the classification rates obtained with the three methods applied over all otoliths (450 specimens) in the database. With the NAC method, we obtained the better classification rate (94.7 %) with 426 otoliths well identified against 320 otoliths using CFD and 361 otoliths using EFD.

Table 2. Classification rate: CFD, EFD, NAC.
Full size table

The improvement of the classification provided by the NAC method, in comparison with other approaches, is statistically significant and is confirmed by statistical tests (T-test, McNemar test, Odds ratio). These tests demonstrated that the proposed method correctly identify otolith specimens which are not recognized by the other methods (Table 3). This shows that the classification features proposed offer a significant improvement of the recognition performance.

Table 3. Classification with CFD and EFD methods revisited by NAC method.
Full size table

The details of the classification are reported in the table (confusion matrix) in Fig. 9 (15 classes or species). The confusion matrix summarizes the reclassification of the specimens using NAC approach and a total of 426 otolith images are correctly recognized, 24 are misclassified. We used 70 % of the images for the training of the neural network, 15 % for testing and 15 % for the validation. The validation data serves to avoid the overlearning (to determine a stopping point of the neural network learning). Figure 10 shows an efficient validation results.

Fig. 9.
figure 9

Confusion matrix.

Full size image
Fig. 10.
figure 10

Validation after training.

Full size image

The classification errors are mainly caused by the similarity of the shapes of some classes. For example, 3 otoliths images of Merluccius senegalensis (C13) are classified as Merluccius merluccius (C3), and 2 otolith images of Pagellus acarne (C5) are classified as Pagellus erythrinus (C6). In fact, this resemblance of otolith shapes, for species in that case lying in the same genus, occurs at a specific age for these species. To solve this problem we need to increase the number of otoliths in the training phase of the multi-layer neural networks.

4 Conclusion and Future Work

Recognition of otolith images is the aim of the current study. The approach aimed to design an automatic classification system based on the Fourier descriptors of the normal angles of the contour. The developed system was tested successfully on a national otoliths database collected in collaboration with INRH (Morocco). The experimental results indicate that the proposed approach is promising. Future works will focus on the rising of the number of images in the training phase and the addition of other features in order to improve the discriminating power and the recognition rate. In addition, our approach will be to the stocks discrimination of fishes.