Abstract
Camera-based vision systems are becoming increasingly influential in the advancing automation of agriculture. Smart Farming technologies such as selective mechanical or chemical weeding are already firmly implemented processes in practice utilizing intelligent camera technology. The capabilities of such technologies recently advanced with the implementation of Deep Learning-based Computer Vision algorithms which proved their applicability in the agricultural domain by successfully solving classification, object detection and segmentation tasks. Due to the demanding environment of agricultural fields and the increasing dependence of farmers on the correct and reliable functioning of such systems, we propose to utilize agronomic context of a field to obtain a self-improving system for camera-based detection and segmentation of cabbage plants. For this purpose, we trained and tested a neural network for instance segmentation (Mask R-CNN) with different datasets of white cabbage (brassica oleracea). In our work, the relevant context parameters are the expected height of the plants as well as the color of the plant pixels. A cost-efficient camera setup utilizing a Structure from Motion (SfM) approach was used to gain complementary depth images. Knowledge gaps in our system appearing in form of missed or poorly detected and segmented plants can be closed by means of an Active Learning approach. This leads to an improvement in our experiments by up to 27.2% in terms of mean average precision.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abdulla, W.: Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow (2022). https://github.com/matterport/Mask_RCNN. Accessed 03 Sep 2022
Beck, M.A., Liu, C.-Y., Bidinosti, C.P., Henry, C.J., Godee, C.M., Ajmani, M.: An embedded system for the automated generation of labeled plant images to enable machine learning applications in agriculture. PLoS One 15(12), 1–23 (2020)
Boysen, J., Stein, A.: AI-supported data annotation in the context of UAV-based weed detection in sugar beet fields using deep neural networks. In: Gandorfer, M., Hoffmann, C., El Benni, N., Cockburn, M., Anken, T., Floto, H. (eds.) 42. GIL-Jahrestagung, Künstliche Intelligenz in der Agrar- und Ernährungswirtschaft 2022, pp. 63–68. Gesellschaft für Informatik e.V., Bonn (2022)
Chandra, A.L., Desai, S.V., Balasubramanian, V.N., Ninomiya, S., Guo, W.: Active learning with point supervision for cost-effective panicle detection in cereal crops. Plant Methods 16(34), 1–16 (2020)
COCO. Detection evaluation (2022). http://cocodataset.org/#detection-eval. Accessed 03 Sep 2022
dos Santos Ferreira, A., Freitas, D.M., Da Silva, G.G., Pistori, H., Folhes, M.T.: Unsupervised deep learning and semi-automatic data labeling in weed discrimination. Comput. Electron. Agric. 165, 104963 (2019)
Ducket, T., Pearson, S., Blackmore, S., Grieve, B., Wilson, P., Gill, H. et al.: Agricultural robotics: the future of robotic agriculture. arXiv e-prints. https://arxiv.org/abs/1806.06762 (2018)
Farooq, A., Hu, J., Jia, X.: Weed classification in hyperspectral remote sensing images via deep convolutional neural network. In: IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium 2018, pp. 3816–3819. IEEE, Valencia (2018)
Gai, J., Tang, L., Steward, B.L.: Automated crop plant detection based on the fusion of color and depth images for robotic weed control. J. Field Rob. 37(1), 35–52 (2020)
Gene-Mola, J., Sainz-Cortiella, R., Rosell-Polo, J.R., Morros, J.R., Ruiz-Hidalgo, J., Vilaplana, V., et al.: Fuji-SfM dataset: A collection of annotated images and point clouds for Fuji apple detection and location using structure-from-motion photogrammetry. Data Brief 30, 105591 (2020)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. arXiv e-prints. https://arxiv.org/abs/1703.06870 (2018)
Jiang, Y., Li, C., Paterson, A.H., Robertson, J.S.: DeepSeedling: deep convolutional network and Kalman filter for plant seedling detection and counting in the field. Plant Methods 15(141), 1–19 (2019)
Kautzmann, T., Wuensche, M., Geimer, M., Mostaghim, S., Schmeck, H.: Holistic optimization of tractor management. In: Solutions for Intelligent and Sustainable Farming: Land-Technik AgEng 2011, pp. 275–281. VDI-Verlag, Hannover (2011)
Keras (2022). https://keras.io/getting_started/intro_to_keras_for_engineers/. Accessed 03 Sep 2022
Lottes, P., Behley, J., Chebrolu, N., Milioto, A., Stachniss, C.: Robust joint stem detection and crop-weed classification using image sequences for plant-specific treatment in precision farming. J. Field Rob. 37(1), 20–34 (2020)
Lottes, P., Hoeferlin, M., Sander, S., Muter, M., Schulze, P., Stachniss, L.C.: An effective classification system for separating sugar beets and weeds for precision farming applications. In: 2016 IEEE International Conference on Robotics and Automation (ICRA) 2016, pp. 5157–5163. IEEE, Stockholm, Sweden (2016)
Louargant, M., Jones, G., Faroux, R., Paoli, J.-N., Maillot, T., Gée, C., et al.: Unsupervised classification algorithm for early weed detection in row-crops by combining spatial and spectral information. Remote Sens. 10(5), 761–779 (2018)
Lüling, N., Reiser, D., Griepentrog, H.W.: Volume and leaf area calculation of cabbage with a neural network-based instance segmentation. In: Stafford, J.V. (eds.) Precision Agriculture 2021: Proceedings of the 14th European Conference on Precision Agriculture, pp. 719–726. Wageningen Academic Publishers, Wageningen (2021)
Lüling, N., Reiser, D., Stana, A., Griepentrog, H.W.: Using depth information and color space variations for improving outdoor robustness for instance segmentation of cabbage, In: 2021 IEEE International Conference on Robotics and Automation (ICRA) 2021, pp. 2331–2336. IEEE, Xi’an, China (2021)
Madec, S., Jin, X., Lu, H., De Solan, B., Liu, S., Duyme, F., et al.: Ear density estimation from high resolution RGB imagery using deep learning technique. Agric. For. Meteorol. 264, 225–234 (2019)
Moshou, D., Kateris, D., Pantazi, X.E., Gravalos, I.: Crop and weed species recognition based on hyperspectral sensing and active learning. In: Stafford, J.V. (ed.) Precision agriculture ’13, pp. 555–561. Wageningen Academic Publishers, Wageningen (2013)
Müller-Schloer, C., Tomforde, S.: Organic Computing – Technical Systems for Survival in the Real World, 5th edn. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68477-2
Python 3.6.0 (2022). https://www.python.org/downloads/release/python-360/. Accessed 03 Sep 2022
Reiser, D., Kamman, A., Vázquez Arellano, M., Griepentrog, H.W.: Using terrestrial photogrammetry for leaf area estimation in maize under different plant growth stages. In: Stafford, J.V (eds.), Precision Agriculture 2019: Proceedings of the 12th European Conference on Precision Agriculture 2019, pp. 331–337. Wageningen Academic Publishers, Wageningen (2019)
Reiser, D., Sehsah, E.-S., Bumann, O., Morhard, J., Griepentrog, H.W.: Development of an autonomous electric robot implement for intra-row weeding in vineyards. Agriculture 9(1), 18–30 (2019)
Settles, B.: Active learning literature survey. Technical report 1648, University of Wisconsin Madison (2009)
Smith, P., Gregory, P.: Climate change and sustainable food production. Proc. Nutr. Soc. 72(1), 21–28 (2013)
Stein, A., Tomforde, S., Diaconescu, A., Hähner, J., Müller-Schloer, C.: A concept for proactive knowledge construction in self-learning autonomous systems. In: 2018 IEEE 3rd International Workshops on Foundations and Applications of Self* Systems (FAS*W) 2018, pp. 204–213. IEEE, Trento, Italy (2018)
Szeliski, R.: Computer Vision: Algorithms and Applications, Chapter 5: 2nd edn. Springer, Cham (2022). https://doi.org/10.1007/978-1-84882-935-0
Tensorflow (2021). https://www.tensorflow.org/install/pip. Accessed 22 Nov 2021
Tomforde, S., Prothmann, H., Branke, J., Hähner, J., Mnif, M., Mueller-Schloer, C.: Observation and control of organic systems. In: Müller-Schloer, C., Schmeck, H., Ungerer, T. (eds.) Organic Computing—A Paradigm Shift for Complex Systems. Autonomic Systems, vol. 1, pp. 325–338. Springer, Basel (2011). https://doi.org/10.1007/978-3-0348-0130-0_21
Wspanialy, P., Brooks, J., Moussa, M.: An image labeling tool and agricultural dataset for deep learning. arXiv e-prints. https://arxiv.org/abs/2004.03351 (2020)
Yang, Y., Li, Y., Yang, J., Wen, J.: Dissimilarity-based active learning for embedded weed identification. Turk. J. Agric. For. 46(3), 390–401 (2022)
Acknowledgments
The project DiWenkLa (Digital Value Chains for a Sustainable Small-Scale Agriculture) is supported by funds of the Federal Ministry of Food and Agriculture (BMEL) based on a decision of the Parliament of the Federal Republic of Germany via the Federal Office for Agriculture and Food (BLE) under the innovation support program (grant reference 28DE106A18). DiWenkLa is also supported by the Ministry for Food, Rural Areas and Consumer Protection Baden-Württemberg.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lüling, N., Boysen, J., Kuper, H., Stein, A. (2022). A Context Aware and Self-improving Monitoring System for Field Vegetables. In: Schulz, M., Trinitis, C., Papadopoulou, N., Pionteck, T. (eds) Architecture of Computing Systems. ARCS 2022. Lecture Notes in Computer Science, vol 13642. Springer, Cham. https://doi.org/10.1007/978-3-031-21867-5_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-21867-5_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21866-8
Online ISBN: 978-3-031-21867-5
eBook Packages: Computer ScienceComputer Science (R0)