Skip to main content
SpringerLink
Log in

Semantic Segmentation of Vineyard Images Using Convolutional Neural Networks

  • Conference paper
  • First Online:
Proceedings of the 21st EANN (Engineering Applications of Neural Networks) 2020 Conference (EANN 2020)

Part of the book series: Proceedings of the International Neural Networks Society ((INNS,volume 2))

Abstract

This paper aims to study the segmentation demands of vineyard images using Convolutional Neural Networks (CNNs). To this end, eleven CNN models able to provide semantic segmented images are examined as part of the sensing subsystem of an autonomous agricultural robot. The task is challenging due to the similar color between grapes, leaves and image’s background. Moreover, the lack of controlled lighting conditions results in varying color representation of grapes and leaves. The studied CNN model architectures combine three different feature learning sub-networks, with five meta-architectures for segmentation purposes. Investigation on three different datasets consisting of vineyard images of grape clusters and leaves, provided segmentation results, by mean pixel intersection over union (IU) performance index, of up to 87.89% for grape clusters and 83.45% for leaves, for the case of ResNet50_FRRN and MobileNetV2_PSPNet model, respectively. Comparative results reveal the efficacy of CNNs to separate grape clusters and leaves from image’s background. Thus, the proposed models can be used for in-field applications for real-time localization of grapes and leaves, towards automation of harvest, green harvest and defoliation agricultural activities by an autonomous robot.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Xue, J., Zhang, L., Grift, T.E.: Variable field-of-view machine vision based row guidance of an agricultural robot. Comput. Electron. Agric. 84, 85–91 (2012). https://doi.org/10.1016/j.compag.2012.02.009

    Article  Google Scholar 

  2. Søgaard, H.T., Lund, I.: Application accuracy of a machine vision-controlled robotic micro-dosing system. Biosyst. Eng. 96, 315–322 (2007). https://doi.org/10.1016/j.biosystemseng.2006.11.009

    Article  Google Scholar 

  3. Mavridou, E., Vrochidou, E., Papakostas, G.A., Pachidis, T., Kaburlasos, V.G.: Machine vision systems in precision agriculture for crop farming. J. Imaging 5, 89 (2019). https://doi.org/10.3390/jimaging5120089

    Article  Google Scholar 

  4. LeCun, Y., Kavukcuoglu, K., Farabet, C.: Convolutional networks and applications in vision. In: Proceedings of 2010 IEEE International Symposium on Circuits and Systems, pp. 253–256. IEEE (2010). https://doi.org/10.1109/ISCAS.2010.5537907

  5. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440. IEEE (2015). https://doi.org/10.1109/CVPR.2015.7298965

  6. Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. In: Proceedings of the British Machine Vision Conference 2014, pp. 6.1–6.12. British Machine Vision Association (2014). https://doi.org/10.5244/C.28.6

  7. Szegedy, C., Wei Liu, Yangqing Jia, Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9. IEEE (2015). https://doi.org/10.1109/CVPR.2015.7298594

  8. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 84–90 (2017). https://doi.org/10.1145/3065386

    Article  Google Scholar 

  9. Badeka, E., Kalabokas, T., Tziridis, K., Nicolaou, A., Vrochidou, E., Mavridou, E., Papakostas, G.A., Pachidis, T.: Grapes visual segmentation for harvesting robots using local texture descriptors. In: 12th International Conference on Computer Vision Systems (ICVS 2019), pp. 98–109 (2019). https://doi.org/10.1007/978-3-030-34995-0_9

    Google Scholar 

  10. Rudolph, R., Herzog, K., Töpfer, R., Steinhage, V.: Efficient identification, localization and quantification of grapevine inflorescences and flowers in unprepared field images using fully convolutional networks. Vitis J. Grapevine Res. 58(3), 95–104 (2019). https://doi.org/10.5073/vitis.2019.58.95-104

    Article  Google Scholar 

  11. Mei, S., Ji, J., Bi, Q., Hou, J., Du, Q., Li, W.: Integrating spectral and spatial information into deep convolutional neural networks for hyperspectral classification. In: 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 5067–5070. IEEE (2016). https://doi.org/10.1109/IGARSS.2016.7730321

  12. Kangune, K., Kulkarni, V., Kosamkar, P.: Grapes ripeness estimation using convolutional neural network and support vector machine. In: 2019 Global Conference for Advancement in Technology (GCAT), pp. 1–5. IEEE (2019). https://doi.org/10.1109/GCAT47503.2019.8978341

  13. . Li, N., Wang, C., Zhao, H., Gong, X., Wang, D.: A novel deep convolutional neural network for spectral-spatial classification of hyperspectral data. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. ISPRS Arch. 42, 897–900 (2018). https://doi.org/10.5194/isprs-archives-XLII-3-897-2018

    Article  Google Scholar 

  14. Zhao, L., Li, Q., Zhang, Y., Wang, H., Du, X.: Integrating the continuous wavelet transform and a convolutional neural network to identify vineyard using time series satellite images. Remote Sens. 11, 2641 (2019). https://doi.org/10.3390/rs11222641

    Article  Google Scholar 

  15. Monga, T.: Estimating vineyard grape yield from images. In: Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 339–343 (2018). https://doi.org/10.1007/978-3-319-89656-4_37

    Chapter  Google Scholar 

  16. Yu, S., Jia, S., Xu, C.: Convolutional neural networks for hyperspectral image classification. Neurocomputing 219, 88–98 (2017). https://doi.org/10.1016/j.neucom.2016.09.010

    Article  Google Scholar 

  17. Personalized Optimal Grape Harvest by Autonomous Robot (POGHAR). http://evtar.eu/

  18. Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: Labelme: a database and web-based tool for image annotation. Int. J. Comput. Vis. 77, 157–173 (2008). https://doi.org/10.1007/s11263-007-0090-8

    Article  Google Scholar 

  19. Wong, S.C., Gatt, A., Stamatescu, V., McDonnell, M.D.: Understanding data augmentation for classification: when to warp? In: 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–6. IEEE (2016). https://doi.org/10.1109/DICTA.2016.7797091

  20. Mikolajczyk, A., Grochowski, M.: Data augmentation for improving deep learning in image classification problem. In: 2018 International Interdisciplinary Ph.D. Workshop (IIPhDW), pp. 117–122. IEEE (2018). https://doi.org/10.1109/IIPHDW.2018.8388338

  21. Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2414–2423. IEEE (2016). https://doi.org/10.1109/CVPR.2016.265

  22. Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition. In: 3rd International Conference Learning Representations, ICLR 2015, Conference Track Proceedings (2014)

    Google Scholar 

  23. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778. IEEE (2016). https://doi.org/10.1109/CVPR.2016.90

  24. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C.: Mobilenetv2: inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4510–4520. IEEE (2018). https://doi.org/10.1109/CVPR.2018.00474

  25. Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944. IEEE (2017). https://doi.org/10.1109/CVPR.2017.106

  26. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 234–241 (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Google Scholar 

  27. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6230–6239. IEEE (2017). https://doi.org/10.1109/CVPR.2017.660

  28. Pohlen, T., Hermans, A., Mathias, M., Leibe, B.: Full-resolution residual networks for semantic segmentation in street scenes. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3309–3318. IEEE (2017). https://doi.org/10.1109/CVPR.2017.353

  29. Siam, M., Gamal, M., Abdel-Razek, M., Yogamani, S., Jagersand, M.: RTseg: real-time semantic segmentation comparative study. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 1603–1607. IEEE (2018). https://doi.org/10.1109/ICIP.2018.8451495

  30. Zeiler, M.D.: ADADELTA: an adaptive learning rate method (2012)

    Google Scholar 

Download references

Acknowledgment

This research has been co-financed by the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH – CREATE – INNOVATE (project code: T1EDK-00300).

Author information

Authors and Affiliations

  1. Human-Machines Interaction Laboratory (HUMAIN-Lab), International Hellenic University, Agios Loukas, 65404, Kavala, Greece

    Theofanis Kalampokas, Konstantinos Tziridis, Alexandros Nikolaou, Eleni Vrochidou, George A. Papakostas, Theodore Pachidis & Vassilis G. Kaburlasos

Authors
  1. Theofanis Kalampokas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Konstantinos Tziridis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexandros Nikolaou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eleni Vrochidou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. George A. Papakostas
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Theodore Pachidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Vassilis G. Kaburlasos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George A. Papakostas .

Editor information

Editors and Affiliations

  1. School of Engineering, Department of Civil Engineering, Democritus University of Thrace, Xanthi, Greece

    Lazaros Iliadis

  2. Lancaster University, Lancaster, UK

    Plamen Parvanov Angelov

  3. School of Computing and Digital Technologies, Teesside University, Middlesbrough, UK

    Chrisina Jayne

  4. University of the West of England, Bristol, UK

    Elias Pimenidis

Rights and permissions

Reprints and permissions

Copyright information

© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kalampokas, T. et al. (2020). Semantic Segmentation of Vineyard Images Using Convolutional Neural Networks. In: Iliadis, L., Angelov, P., Jayne, C., Pimenidis, E. (eds) Proceedings of the 21st EANN (Engineering Applications of Neural Networks) 2020 Conference. EANN 2020. Proceedings of the International Neural Networks Society, vol 2. Springer, Cham. https://doi.org/10.1007/978-3-030-48791-1_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-48791-1_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-48790-4

  • Online ISBN: 978-3-030-48791-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation