Skip to main content

MS-Net: A CNN Architecture for Agriculture Pattern Segmentation in Aerial Images

  • Conference paper
  • First Online:
Computer Vision and Image Processing (CVIP 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1567))

Included in the following conference series:

Abstract

Computer vision for crop sciences is overgrowing with the advancement in pattern recognition and deep learning. Agriculture pattern segmentation is an important application such as segmentation of cloud shadow, waterway, standing water, weed cluster, planter skip, double plant, etc. However, the segmentation of agriculture patterns is challenging due to multi-scale variations of patterns and considerable overlap between classes. Furthermore, size and shape variation, unclear boundaries, and missing edges make this task more complex. To address this problem, we proposed the encoder-decoder architecture with input as a multi-scale image pyramid to improve the multi-scale feature extraction of agriculture patterns. EfficientNetB7 is utilized as an encoder for efficient feature extraction. In addition, the proposed Lateral-output layer shows rich semantic features by aggregating low level and high level features from the decoder, shows improvement in dice score. The proposed research uses the Agriculture Vision dataset with aerial farmland images. The proposed method outperformed different state-of-the-art methods in the literature for six agricultural patterns segmentation with a mean dice score of 74.78, 68.11, and 84.23 for RGB, NIR, and RGB \(+\) NIR images, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481ā€“2495 (2017)

    Article  Google Scholar 

  2. Bhagat, S., Kokare, M., Haswani, V., Hambarde, P., Kamble, R.: WheatNet-Lite: a novel light weight network for wheat head detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 1332ā€“1341, October 2021

    Google Scholar 

  3. Bhagat, S., Kokare, M., Haswani, V., Hambarde, P., Kamble, R.: Eff-UNet++: a novel architecture for plant leaf segmentation and counting. Eco. Inform. 68, 101583 (2022)

    Article  Google Scholar 

  4. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 801ā€“818 (2018)

    Google Scholar 

  5. Cheng, B., et al.: SPGNet: semantic prediction guidance for scene parsing. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5218ā€“5228 (2019)

    Google Scholar 

  6. Chiu, M.T., et al.: Agriculture-vision: a large aerial image database for agricultural pattern analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2828ā€“2838 (2020)

    Google Scholar 

  7. Christiansen, P., Nielsen, L.N., Steen, K.A., JĆørgensen, R.N., Karstoft, H.: DeepAnomaly: combining background subtraction and deep learning for detecting obstacles and anomalies in an agricultural field. Sensors 16(11), 1904 (2016)

    Article  Google Scholar 

  8. Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3213ā€“3223 (2016)

    Google Scholar 

  9. Dudhane, A., Biradar, K.M., Patil, P.W., Hambarde, P., Murala, S.: Varicolored image de-hazing. In: proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4564ā€“4573 (2020)

    Google Scholar 

  10. Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303ā€“338 (2010)

    Article  Google Scholar 

  11. Fu, H., Cheng, J., Xu, Y., Wong, D.W.K., Liu, J., Cao, X.: Joint optic disc and cup segmentation based on multi-label deep network and polar transformation. IEEE Trans. Med. Imaging 37(7), 1597ā€“1605 (2018)

    Article  Google Scholar 

  12. Hambarde, P., Dudhane, A., Murala, S.: Single image depth estimation using deep adversarial training. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 989ā€“993. IEEE (2019)

    Google Scholar 

  13. Hambarde, P., Dudhane, A., Patil, P.W., Murala, S., Dhall, A.: Depth estimation from single image and semantic prior. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1441ā€“1445. IEEE (2020)

    Google Scholar 

  14. Hambarde, P., Murala, S.: S2DNet: depth estimation from single image and sparse samples. IEEE Trans. Computat. Imaging 6, 806ā€“817 (2020)

    Article  Google Scholar 

  15. Haug, S., Michaels, A., Biber, P., Ostermann, J.: Plant classification system for crop/weed discrimination without segmentation. In: IEEE Winter Conference on Applications of Computer Vision, pp. 1142ā€“1149. IEEE (2014)

    Google Scholar 

  16. Haug, S., Ostermann, J.: A crop/weed field image dataset for the evaluation of computer vision based precision agriculture tasks. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8928, pp. 105ā€“116. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16220-1_8

    Chapter  Google Scholar 

  17. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770ā€“778 (2016)

    Google Scholar 

  18. Howard, A., Zhmoginov, A., Chen, L.C., Sandler, M., Zhu, M.: Inverted residuals and linear bottlenecks: mobile networks for classification, detection and segmentation (2018)

    Google Scholar 

  19. Innani, S., Dutande, P., Baheti, B., Talbar, S., Baid, U.: Fuse-PN: a novel architecture for anomaly pattern segmentation in aerial agricultural images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2960ā€“2968 (2021)

    Google Scholar 

  20. Jin, S., et al.: Deep learning: individual maize segmentation from terrestrial lidar data using faster R-CNN and regional growth algorithms. Front. Plant Sci. 9, 866 (2018)

    Google Scholar 

  21. Kamilaris, A., Prenafeta-BoldĆŗ, F.X.: Deep learning in agriculture: a survey. Comput. Electron. Agric. 147, 70ā€“90 (2018)

    Article  Google Scholar 

  22. Lin, T.Y., DollĆ”r, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117ā€“2125 (2017)

    Google Scholar 

  23. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740ā€“755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

  24. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431ā€“3440 (2015)

    Google Scholar 

  25. Lottes, P., Hƶrferlin, M., Sander, S., Stachniss, C.: Effective vision-based classification for separating sugar beets and weeds for precision farming. J. Field Robot. 34(6), 1160ā€“1178 (2017)

    Article  Google Scholar 

  26. Lu, H., Fu, X., Liu, C., Li, L.G., He, Y.X., Li, N.W.: Cultivated land information extraction in UAV imagery based on deep convolutional neural network and transfer learning. J. Mountain Sci. 14(4), 731ā€“741 (2017)

    Google Scholar 

  27. Lu, Y., Young, S.: A survey of public datasets for computer vision tasks in precision agriculture. Comput. Electron. Agric. 178, 105760 (2020)

    Article  Google Scholar 

  28. Madhusudhan, L.: Agriculture role on Indian economy. Bus. Econ. J. 6(4), 1 (2015)

    Google Scholar 

  29. McCool, C., Perez, T., Upcroft, B.: Mixtures of lightweight deep convolutional neural networks: applied to agricultural robotics. IEEE Robot. Autom. Lett. 2(3), 1344ā€“1351 (2017)

    Article  Google Scholar 

  30. Milioto, A., Lottes, P., Stachniss, C.: Real-time blob-wise sugar beets vs weeds classification for monitoring fields using convolutional neural networks. ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci. 4 (2017)

    Google Scholar 

  31. Mouret, F., Albughdadi, M., Duthoit, S., KouamƩ, D., Rieu, G., Tourneret, J.Y.: Detecting anomalous crop development with multispectral and SAR time series using unsupervised outlier detection at the parcel-level: application to wheat and rapeseed crops (2020)

    Google Scholar 

  32. Olsen, A., et al.: DeepWeeds: a multiclass weed species image dataset for deep learning. Sci. Rep. 9(1), 1ā€“12 (2019)

    Google Scholar 

  33. Patil, P.W., Biradar, K.M., Dudhane, A., Murala, S.: An end-to-end edge aggregation network for moving object segmentation. In: proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8149ā€“8158 (2020)

    Google Scholar 

  34. Patil, P.W., Dudhane, A., Kulkarni, A., Murala, S., Gonde, A.B., Gupta, S.: An unified recurrent video object segmentation framework for various surveillance environments. IEEE Trans. Image Process. 30, 7889ā€“7902 (2021)

    Article  Google Scholar 

  35. Phutke, S.S., Murala, S.: Diverse receptive field based adversarial concurrent encoder network for image inpainting. IEEE Signal Process. Lett. 28, 1873ā€“1877 (2021)

    Article  Google Scholar 

  36. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234ā€“241. Springer (2015). https://doi.org/10.1007/978-3-319-24574-4_28

  37. Shah, J.P., Prajapati, H.B., Dabhi, V.K.: A survey on detection and classification of rice plant diseases. In: 2016 IEEE International Conference on Current Trends in Advanced Computing (ICCTAC), pp. 1ā€“8. IEEE (2016)

    Google Scholar 

  38. Singh, A., Ganapathysubramanian, B., Singh, A.K., Sarkar, S.: Machine learning for high-throughput stress phenotyping in plants. Trends Plant Sci. 21(2), 110ā€“124 (2016)

    Article  Google Scholar 

  39. Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105ā€“6114. PMLR (2019)

    Google Scholar 

  40. Chiu, M.T., et al.: The 1st agriculture-vision challenge: methods and results. arXiv e-prints pp. arXiv-2004 (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sandesh Bhagat .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

Ā© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bhagat, S., Kokare, M., Haswani, V., Hambarde, P., Kamble, R. (2022). MS-Net: A CNN Architecture for Agriculture Pattern Segmentation in Aerial Images. In: Raman, B., Murala, S., Chowdhury, A., Dhall, A., Goyal, P. (eds) Computer Vision and Image Processing. CVIP 2021. Communications in Computer and Information Science, vol 1567. Springer, Cham. https://doi.org/10.1007/978-3-031-11346-8_42

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-11346-8_42

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-11345-1

  • Online ISBN: 978-3-031-11346-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics