Skip to main content
SpringerLink
Log in

Detection of tomato organs based on convolutional neural network under the overlap and occlusion backgrounds

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Traditional detection methods are not sensitive to small-sized tomato organs (flowers and fruits), because the immature green tomatoes are highly similar to the background color. The overlap among fruits and the occlusion of stems and leaves on tomato organs can lead to false and missing detection, which decreases the accuracy and generalization ability of the model. Therefore, a tomato organ recognition method based on improved Feature Pyramid Network was proposed in this paper. To begin with, multi-scale feature fusion was used to fuse the detailed bottom features and high-level semantic features to detect small-sized tomato organs to improve recognition rate. And then repulsion loss was used to take place of the original smooth L1 loss function. Besides, Soft-NMS (Soft non-maximum suppression) was adopted to replace non-maximum suppression to screen the bounding boxes of tomato organs to construct a recognition model of tomato key organ. Finally, the network was trained and verified on the collected image data set. The results showed that compared with the traditional Faster R-CNN model, the performance was greatly improved (mean average precision was improved from 90.7 to 99.5%). Subsequently, the training model can be compressed so that it can be embedded into the microcontroller to develop further precise pesticide targeting application system of tomato organs and the automatic picking device.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Paran, E., Engelhard, Y.: Effect of tomato’s lycopene on blood pressure, serum lipoproteins, plasma homocysteine and oxidative sress markers in grade I hypertensive patients. Am. J. Hypertens. 14(4), A141–A141 (2001)

    Article  Google Scholar 

  2. He, S., He, D., Xu, C., et al.: Effects of nutrient solution on growth and quality of short-term cultivation tomatoes grown in rockwool. Trans. CSAE. 33(18), 188–195 (2017)

    Google Scholar 

  3. Li, H., Zhang, M., Gao, Y., et al.: Green ripe tomato detection method based on machine vision in greenhouse. Trans. CSAE. 33(Supp. 1), 328–334 (2017)

    Google Scholar 

  4. Jiang, H., Peng, Y., Shen, H., et al.: Recognizing and locating ripe tomatoes based on binocular stereo vision technology. Trans. CSAE. 24(8), 279–283 (2008)

    Google Scholar 

  5. Zhao, J., Yang, G., Liu, M., et al.: Discrimination of mature tomato based on HIS color space in natural outdoor scenes. Trans. CSAM. 35(5), 101–120 (2004)

    Google Scholar 

  6. Zhang, R., Ji, C., Shen, M., et al.: Application of computer vision to tomato harvesting. Trans. CSAM. 32(5), 50–52 (2001)

    Google Scholar 

  7. Wang, L., Wei, S., Zhao, B., et al.: Target extraction method of ripe tomato in greenhouse based on Niblack self-adaptive adjustment parameter. Trans. CSAE. 33(Supp. 1), 322–327 (2017)

    Google Scholar 

  8. Yamamoto, K., Guo, W., Yoshioka, Y., et al.: On plant detection of intact tomato fruits using image analysis and machine learning methods. Sensors 14(7), 12191–12206 (2014)

    Article  Google Scholar 

  9. Farabet, C., Couprie, C., Najman, L., et al.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1915–1929 (2013)

    Article  Google Scholar 

  10. Zhang, X., Cheng, L., Li, B., et al.: Too far to see? Not really! —Pedestrian detection with scale-aware localization policy. IEEE Trans. Image Process. 27(8), 3703–3715 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  11. Liu, W., Anguelov, D., Erhan, D., et al.: SSD: Single shot multibox detector. In: Proceedings of the European Conference on Computer Vision, pp. 21–37. Springer (2016)

  12. Zheng, L., San, Z., Hong, S., et al.: Scene text recognition using residual convolutional recurrent neural network. Mach. Vis. Appl. 29(5), 861–871 (2018)

    Article  Google Scholar 

  13. Sho, K., Kazuhiro, H., Takio, K.: Mixture of counting CNNs. Mach. Vis. Appl. 29(7), 1119–1126 (2018)

    Article  Google Scholar 

  14. Jang, C., Sunwoo, M.: Semantic segmentation-based parking space detection with standalone around view monitoring system. Mach. Vis. Appl. 30(2), 1–11 (2018)

    Google Scholar 

  15. Zhou, Y., Xu, T., Zhen, W., et al.: Classification and recognition approaches of tomato main organs based on DCNN. Trans. CSAE. 33(15), 219–226 (2017)

    Google Scholar 

  16. Inkyu, S., Zong, G., Feras, D., et al.: Deep fruits: a fruit detection system using deep neural networks. Sensors 16(8), 1222–1230 (2016)

    Article  Google Scholar 

  17. Peng, H., Huang, B., Shao, Y., et al.: General improved SSD model for picking object recognition of multiple fruits in natural environment. Trans. CSAE. 34(16), 155–162 (2018)

    Google Scholar 

  18. Lin, Y., Dollár, Piotr, et al.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 936–944 (2017)

  19. Han, J., Zhang, D., et al.: Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning. IEEE Trans. Geosci. Remote Sens. 53(6), 3325–3337 (2015)

    Article  Google Scholar 

  20. Yuan, F., Zhang, L., Wan, B., et al. Convolutional neural networks based on multi-scale additive merging layers for visual smoke recognition. Mach. Vis. Appl. pp. 1–14 (2018)

  21. Hu, Y., Lu, M., Lu, X.: Driving behaviour recognition from still images by using multi-stream fusion CNN. Mach. Vis. Appl. 30(5), 851–865 (2019)

    Article  Google Scholar 

  22. Zhang, L., Zhang, Q., et al.: Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding[J]. Pattern Recogn. 48(10), 3102–3112 (2015)

    Article  Google Scholar 

  23. Tang, H., Xiao, B., et al.: Pixel convolutional neural network for multi-focus image fusion. Inf. Sci. 433–434, 125–141 (2018)

    Article  MathSciNet  Google Scholar 

  24. Wang, X., Xiao, T., Jiang, Y., et al.: Repulsion loss: detecting pedestrians in a crowd. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 7774–7783 (2018)

  25. Qiu, S., Wen, G., Deng, Z., et al.: Accurate non-maximum suppression for object detection in high-resolution remote sensing images. Remote Sens. Lett. 9(3), 238–247 (2018)

    Article  Google Scholar 

  26. Bodla, N., Singh, B., Chellappa, R., et al.: Soft-NMS-Improving object detection with one line of code. In Proceedings of the IEEE International Conference on Computer Vision. pp.5562–5570 (2017)

  27. Sun, J., He, X., Tan, W., et al.: Recognition of crop seedling and weed recognition based on dilated convolution and global pooling in CNN. Trans. CSAE. 34(11), 159–165 (2018)

    Google Scholar 

  28. Barter, R., Yu, B.: Superheat: an R package for creating beautiful and extendable heatmaps for visualizing complex data. Statistics 27(4), 1–30 (2017)

    MathSciNet  Google Scholar 

  29. Zhou, B., Khosla, A., Lapedriza, A., et al.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2921–2929 (2016)

  30. Davis, J., Goadrich, M.: The relationship between precision-recall and ROC curves. In: Proceedings of the International Conference on Machine Learning. pp. 233–240 (2006)

  31. Renmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 7263–7271 (2017)

  32. Renmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv:1804.02767, (2018)

  33. Ren, S., He, K., Girshick, R., et al.: Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv:1506.01497, (2015)

  34. Dai, J., Li, Y., He, K., et al. R-fcn: Object detection via region-based fully convolutional networks. Advances in Neural Information Processing Systems. pp. 379–387 (2017)

  35. Everingham, M., Winn, J.: The PASCAL visual object classes challenge 2007 (VOC2007) development kit. Int. J. Comput. Vis. 111(1), 98–136 (2006)

    Article  Google Scholar 

  36. Zhang, D., Meng, D., et al.: Co-saliency detection via a self-paced multiple-instance learning framework. IEEE Trans. Pattern Anal. Mach. Intell. 39(5), 865–878 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD), Six Talent Peaks Project in Jiangsu Province (ZBZZ-019) and Project of Agricultural Equipment Department of Jiangsu University (4121680001).

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering of Jiangsu University, Zhenjiang, 212013, China

    Jun Sun, Xiaofei He, Minmin Wu, Xiaohong Wu, Jifeng Shen & Bing Lu

Authors
  1. Jun Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaofei He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minmin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaohong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jifeng Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bing Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Sun.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, J., He, X., Wu, M. et al. Detection of tomato organs based on convolutional neural network under the overlap and occlusion backgrounds. Machine Vision and Applications 31, 31 (2020). https://doi.org/10.1007/s00138-020-01081-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-020-01081-6

Keywords

Search

Navigation