A new deep-learning strawberry instance segmentation methodology based on a fully convolutional neural network

Abstract

Instance segmentation is one of the image processing problems where deep learning techniques are beginning to show potential. In agriculture, one of its main application is automatic fruit harvesting. This study focuses on its application on strawberry crops, where the development of automatic harvesting machines is of particular interest. At present, the reference methodology to deal with instance segmentation is Mask R-CNN. However, Mask R-CNN requires a large processing power which limits its implementation in real-time systems. This work proposes a new methodology to carry out instance segmentation of strawberries based on the use of a fully convolutional neural network. Instance segmentation is achieved by adding two new channels to the network output so that each strawberry pixel predicts the centroid of its strawberry. The final segmentation of each strawberry is obtained by applying a grouping and filtering algorithm. The methodology was tested using the publicly available StrawDI_Db1 database. The evaluation results show values of mean average precision (mAP) and mean instance intersection over union (I$^{2}$oU) of 52.61 and 93.38, respectively, with a processing speed of 30 fps. These figures mean an increase in precision higher than 15% and a fps rate six times higher than those obtained in the reference methodologies based on Mask R-CNN. Therefore, the methodology presented in this paper can be considered as the latest reference methodology for strawberry segmentation, meeting the precision and speed requirements needed for it to be used in the automatic strawberry harvesting systems that work in real time.