Estimating pig weights from images without constraint on posture and illumination

Highlights

• An image based pig weight estimation without constraint on pig posture.

• 2D images are used and no manual image selection is required.

• A new feature, curvature, is introduced to measure non-straightness of pig posture.

• A fully connected dense network predicts weights from images.

Abstract

This paper proposes an image based pig weight estimation method different from the previous works in three ways. The first difference is no constraint on pig posture and image capture environment, reducing the stress of the pigs. The second one is that the features obtained from 2D images are used without depending on 3D depth information. And the third is that our estimation model is constructed by exploiting the recent advances in machine learning. Besides the pig area size which had been a major feature parameter for the estimation, two new features, curvature and deviation, are introduced because those are related with the postures, thus being able to quantify the weight adjustment. A set of experiments are conducted to investigate how the performance is affected by the combination of the features and the neural network configurations. By using 477 training and 103 test images, the average estimation error of 3.15 kg was achieved, and the coefficient of determination of the model was $R^2=0.79$.

Introduction

Monitoring pig weights in a regular basis is one of the important tasks in pig raising farms. The changes in weights provide direct means to assess the health and growth state of the pigs. The weight is also a key factor to determine whether the pigs reach a state of maturity for market. However, the weight measuring work is labor-intensive and stressful for not only farm workers but also the pigs. In a worse case, when the farm workers drive the pigs to a weighing equipment, both the workers and the pigs can be exposed to injury. To facilitate this job, a narrow corridor is commonly used; the pigs are herded to walk through a path which is one-pig wide and has an underneath scaling device. However, this weighing corridor approach does not completely remove the pig stress.

Computer vision algorithms using 2D or 3D cameras have been widely applied to automate farming jobs such as monitoring animal feeding, location, condition, and aggressive or reproductive behaviors (Marchant et al., 1999, Azzaro et al., 2011, Tasdemir et al., 2011, Yilmaz et al., 2013, Kulikov et al., 2014, Lee et al., 2016, Spoliansky et al., 2016, Nasirahmadi et al., 2015, Nasirahmadi et al., 2016, Nasirahmadi et al., 2017). Even fish raising farms have introduced optical sensors and machine vision algorithms to manage the quality of fish products (Saberioon et al., 2017). Besides normal images, the use of ultrasound and thermal images have been examined to overcome the drawbacks of the cameras which are affected by ambient lighting (Stajnko et al., 2008, Duff et al., 2010, Halachmi et al., 2013). Typically, 3D image based methods have been studied for various applications because of the advantages from depth information (Menesatti et al., 2014, Salau et al., 2014, Weber et al., 2014, Kuzuhara et al., 2015, Vázquez-Arellano et al., 2016). To this end, the adoption of low cost 3D depth cameras such as Microsoft Kinect has been increasing in agricultural and livestock applications (Kawasue et al., 2013, Viazzi et al., 2014, Rosell-Polo et al., 2015).

To overcome the problems caused by the manual weighing, digital image based approaches were attempted to estimate or measure body features of various livestock (Costa et al., 2013, Pallottino et al., 2015, Porto et al., 2015, Mortensen et al., 2016, Oczak et al., 2014, Guo et al., 2017, Pezzuolo et al., 2018a). In the case of the pigs, images were captured from the top and sometimes from the side at the same time by using 2D or 3D depth cameras (Brandl and Jørgensen, 1996, Schofield et al., 1999, Wu et al., 2004, McFarlane et al., 2005, Kollis et al., 2007, Parsons et al., 2007, Wang et al., 2008, Kashiha et al., 2014, Shi et al., 2016, Pezzuolo et al., 2018b). The images were then processed by image processing algorithms in order to estimate body measures from extracted features. Depth images could produce point clouds which improved the accuracy of estimation and measurement.

Commercial automatic weighing systems such as Weight-Detect by PLF Agritech, eYeScan by Fancom, Pigwei by Ymaging, optiSCAN by Holscher Leuschner, Growth Sensor by GroStat, qscan by Innovent and WUGGL by WUGGL have been introduced (Vranken and Berckmans, 2017). The proprietary algorithms for the image processing separate pig areas from background, and determine shape and volume features, which are then used to calculate various body measures. The regression models correlate these measures with weight estimation.

The image based weight estimation consists of a set of sequential steps. The pig segmentation, which separates pig region from image background is, the first step. Hough transform (Hough, 1962) and a-priori knowledge about the pig shape were exploited to improve the segmentation results (Kashiha et al., 2014). It still had the limitation that it did not work on non-straight postures and was suitable for estimating the average weight rather than individual ones.

The accuracy of the weight estimation largely depends on the features that are used as input to the prediction models, which have been mostly the linear or non-linear regression models in previous works. For instance, the use of the boundary length of the pig’s shape extracted from 2D images and the average distance of the pig pixels to the edges was examined (Wongsriworaphon et al., 2015). Despite the introduction of the new features, it still required the straight posture, the consistency of the illumination, and human intervention to mark the pig areas manually.

In addition to the optical sensors and the computer vision algorithms, a neural network approach was introduced to estimate pig weights (Wang et al., 2008). The network was trained to correlate pig image features with corresponding weights. Obtaining the training images was restrictive to the pigs. Since the images should be captured only when the pigs had straight posture, the pigs were forced to be kept in a narrow corridor. Also, the corridor was installed with a cover to prevent ambient sunlight from affecting the images. The network model was relatively simple compared with the recent complex and advanced deep learning models.

Rather than 2D images, 3D images were exploited to improve the estimation accuracy because they could provide depth information (Kongsro, 2014, Banhazi and Dunn, 2016, Shi et al., 2016, Guo et al., 2017, Pezzuolo et al., 2018b). Pig heights were estimated from the depth information of a stereo vision camera (Shi et al., 2016). The heights along with the pig shape information were then used in a linear regression model for the weight estimation. The pig heights could be estimated from the Kinect camera images and were used to supplement 2D area information (Kongsro, 2014). The pig volume was calculated from the depth information and used as one of the search keys in a database that lists the volume with the corresponding weights (Banhazi and Dunn, 2016). Point clouds, which were generated from 3D images, were used to measure body features (Guo et al., 2017, Pezzuolo et al., 2018b). The point clouds are more versatile than 2D images because the body curvature such as heart girth can be approximated.

Despite the introduction of the additional 3D information, the straight posture of pigs in the images was still required. And, the lighting condition has become more important because some infrared based depth cameras such as Kinect are susceptible to sunlight, limiting its application to indoor use. Hence, it is still necessary to manually select those images that satisfies the posture and the lighting requirements in order to estimate the weights.

The posture constraint that these image based methods have sometimes requires the use of stressful environment such as narrow space in order to keep the pig bodies straightened. However, the need for such narrow space is ironic considering that one of the motivations of the image based works is to remove the narrow corridor that gives the stress to the pigs. Considering the limitations from the posture and the indoor use of the depth cameras, the advantages of the image based works over the manual scaling diminish, although non-contact weighing is a significant benefit.

This paper proposes a novel image based method that has no posture constraint and depends only on the 2D features for the weight estimation. Pigs are kept in a roomy space and are free to have non-straight postures during the image capture. Our method can work by using 2D cameras, only if the image segmentation of the pigs from the background is successful. It makes our work more applicable to outdoor farm environments in which infrared 3D depth cameras have difficulties due to sunlight. These advantages enable us to weigh pigs at pens where they grow, reducing the stress and required efforts.

Those improvements are achieved by developing a novel set of image processing and feature retrieval methods along with the help of recent advances in machine learning. One of our contributions is the definition of two features that represent the shape and locality of pig segmentations in the images; both are vital information to the weight estimation. Those features along with sizes of the pig segmentations are given as inputs to our estimation model which are implemented as a fully connected neural network.

The paper is organized as follows. Section 2 describes our proposed work in detail. Involved image processing steps are explained and feature extraction is discussed, which is then followed by the design of the fully connected network. Section 3 presents the results from experiments which are different in the number of used features and the structure of the neural network, giving a few hints about how the different combinations affect the accuracy. Section 4 concludes the paper.