Skip to main content
SpringerLink
Log in
Book cover

DAGM German Conference on Pattern Recognition

DAGM GCPR 2021: Pattern Recognition pp 589–603Cite as

Weakly Supervised Segmentation Pretraining for Plant Cover Prediction

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 13024))

Abstract

Automated plant cover prediction can be a valuable tool for botanists, as plant cover estimations are a laborious and recurring task in environmental research. Upon examination of the images usually encompassed in this task, it becomes apparent that the task is ill-posed and successful training on such images alone without external data is nearly impossible. While a previous approach includes pretraining on a domain-related dataset containing plants in natural settings, we argue that regular classification training on such data is insufficient. To solve this problem, we propose a novel pretraining pipeline utilizing weakly supervised object localization on images with only class annotations to generate segmentation maps that can be exploited for a second pretraining step. We utilize different pooling methods during classification pretraining, and evaluate and compare their effects on the plant cover prediction. For this evaluation, we focus primarily on the visible parts of the plants. To this end, contrary to previous works, we created a small dataset containing segmentations of plant cover images to be able to evaluate the benefit of our method numerically. We find that our segmentation pretraining approach outperforms classification pretraining and especially aids in the recognition of less prevalent plants in the plant cover dataset.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://gbif.org.

  2. 2.

    https://www.idiv.de/en/research/platforms_and_networks/idiv_ecotron/experiments/insect_armageddon.html.

References

  1. Aggemyr, E., Cousins, S.A.: Landscape structure and land use history influence changes in island plant composition after 100 years. J. Biogeogr. 39(9), 1645–1656 (2012)

    Article  Google Scholar 

  2. Barré, P., Stöver, B.C., Müller, K.F., Steinhage, V.: LeafNet: a computer vision system for automatic plant species identification. Eco. Inform. 40, 50–56 (2017)

    Article  Google Scholar 

  3. Chen, J., Chen, J., Zhang, D., Sun, Y., Nanehkaran, Y.: Using deep transfer learning for image-based plant disease identification. Comput. Electron. Agric. 173, 105393 (2020)

    Article  Google Scholar 

  4. Choe, J., Oh, S.J., Lee, S., Chun, S., Akata, Z., Shim, H.: Evaluating weakly supervised object localization methods right. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3133–3142 (2020)

    Google Scholar 

  5. Choe, J., Shim, H.: Attention-based dropout layer for weakly supervised object localization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2219–2228 (2019)

    Google Scholar 

  6. Eisenhauer, N., Türke, M.: From climate chambers to biodiversity chambers. Front. Ecol. Environ. 16(3), 136–137 (2018)

    Article  Google Scholar 

  7. Ganesh, P., Volle, K., Burks, T., Mehta, S.: Deep orange: mask R-CNN based orange detection and segmentation. IFAC-PapersOnLine 52(30), 70–75 (2019)

    Article  MathSciNet  Google Scholar 

  8. GBIF.org: Gbif occurrence downloads, 13 May 2020. https://doi.org/10.15468/dl.xg9y85, https://doi.org/10.15468/dl.zgbmn2, https://doi.org/10.15468/dl.cm6hqj, https://doi.org/10.15468/dl.fez33g, https://doi.org/10.15468/dl.f8pqjw, https://doi.org/10.15468/dl.qbmyb2, https://doi.org/10.15468/dl.fc2hqk, https://doi.org/10.15468/dl.sq5d6f

  9. Gerstner, K., Dormann, C.F., Stein, A., Manceur, A.M., Seppelt, R.: Editor’s choice: review: effects of land use on plant diversity-a global meta-analysis. J. Appl. Ecol. 51(6), 1690–1700 (2014)

    Article  Google Scholar 

  10. Ghazi, M.M., Yanikoglu, B., Aptoula, E.: Plant identification using deep neural networks via optimization of transfer learning parameters. Neurocomputing 235, 228–235 (2017)

    Article  Google Scholar 

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  12. Huang, Z., Wang, X., Wang, J., Liu, W., Wang, J.: Weakly-supervised semantic segmentation network with deep seeded region growing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7014–7023 (2018)

    Google Scholar 

  13. Iandola, F., Moskewicz, M., Karayev, S., Girshick, R., Darrell, T., Keutzer, K.: DenseNet: implementing efficient convnet descriptor pyramids. arXiv preprint arXiv:1404.1869 (2014)

  14. Kattenborn, T., Eichel, J., Wiser, S., Burrows, L., Fassnacht, F.E., Schmidtlein, S.: Convolutional neural networks accurately predict cover fractions of plant species and communities in unmanned aerial vehicle imagery. Remote Sens. Ecol. Conserv. 6, 472–486 (2020)

    Article  Google Scholar 

  15. Kim, B., Han, S., Kim, J.: Discriminative region suppression for weakly-supervised semantic segmentation. arXiv preprint arXiv:2103.07246 (2021)

  16. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  17. Körschens, M., Bodesheim, P., Römermann, C., Bucher, S.F., Ulrich, J., Denzler, J.: Towards confirmable automated plant cover determination. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020. LNCS, vol. 12540, pp. 312–329. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-65414-6_22

    Chapter  Google Scholar 

  18. Körschens, M., Bodesheim, P., Römermann, C., Bucher, S.F., Ulrich, J., Denzler, J.: Automatic plant cover estimation with convolutional neural networks. arXiv preprint arXiv:2106.11154 (2021)

  19. Lee, S.H., Chan, C.S., Wilkin, P., Remagnino, P.: Deep-plant: plant identification with convolutional neural networks. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 452–456. IEEE (2015)

    Google Scholar 

  20. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)

    Google Scholar 

  21. Ling, H., Acuna, D., Kreis, K., Kim, S.W., Fidler, S.: Variational amodal object completion. In: Advances in Neural Information Processing Systems 33 (2020)

    Google Scholar 

  22. Liu, H., et al.: Shifting plant species composition in response to climate change stabilizes grassland primary production. Proc. Natl. Acad. Sci. 115(16), 4051–4056 (2018)

    Article  Google Scholar 

  23. Lloret, F., Peñuelas, J., Prieto, P., Llorens, L., Estiarte, M.: Plant community changes induced by experimental climate change: seedling and adult species composition. Perspect. Plant Ecol. Evol. Syst. 11(1), 53–63 (2009)

    Article  Google Scholar 

  24. Lu, H., Cao, Z., Xiao, Y., Zhuang, B., Shen, C.: TasselNet: counting maize tassels in the wild via local counts regression network. Plant Methods 13(1), 79 (2017)

    Article  Google Scholar 

  25. Milletari, F., Navab, N., Ahmadi, S., V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571

    Google Scholar 

  26. Muhammad, M.B., Yeasin, M.: Eigen-CAM: class activation map using principal components. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2020)

    Google Scholar 

  27. Pfadenhauer, J.: Vegetationsökologie - ein Skriptum. IHW-Verlag, Eching, 2. verbesserte und erweiterte auflage edn. (1997)

    Google Scholar 

  28. Pinheiro, P.O., Collobert, R.: From image-level to pixel-level labeling with convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1713–1721 (2015)

    Google Scholar 

  29. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  30. Rosenzweig, C., et al.: Assessment of observed changes and responses in natural and managed systems. In: Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, pp. 79–131 (2007)

    Google Scholar 

  31. Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  32. Singh, K.K., Lee, Y.J.: Hide-and-seek: forcing a network to be meticulous for weakly-supervised object and action localization. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3544–3553. IEEE (2017)

    Google Scholar 

  33. Souza, L., Zelikova, T.J., Sanders, N.J.: Bottom-up and top-down effects on plant communities: nutrients limit productivity, but insects determine diversity and composition. Oikos 125(4), 566–575 (2016)

    Article  Google Scholar 

  34. Stammes, E., Runia, T.F., Hofmann, M., Ghafoorian, M.: Find it if you can: end-to-end adversarial erasing for weakly-supervised semantic segmentation. arXiv preprint arXiv:2011.04626 (2020)

  35. Türke, M., et al.: Multitrophische biodiversitätsmanipulation unter kontrollierten umweltbedingungen im iDiv ecotron. In: Lysimetertagung, pp. 107–114 (2017)

    Google Scholar 

  36. Ulrich, J., et al.: Invertebrate decline leads to shifts in plant species abundance and phenology. Front. Plant Sci. 11, 1410 (2020)

    Article  Google Scholar 

  37. Wei, Y., Feng, J., Liang, X., Cheng, M.M., Zhao, Y., Yan, S.: Object region mining with adversarial erasing: a simple classification to semantic segmentation approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1568–1576 (2017)

    Google Scholar 

  38. Xiong, H., Cao, Z., Lu, H., Madec, S., Liu, L., Shen, C.: TasselNetv2: in-field counting of wheat spikes with context-augmented local regression networks. Plant Methods 15(1), 150 (2019)

    Article  Google Scholar 

  39. Yalcin, H., Razavi, S.: Plant classification using convolutional neural networks. In: 2016 Fifth International Conference on Agro-Geoinformatics (Agro-Geoinformatics), pp. 1–5. IEEE (2016)

    Google Scholar 

  40. Yao, Q., Gong, X.: Saliency guided self-attention network for weakly and semi-supervised semantic segmentation. IEEE Access 8, 14413–14423 (2020)

    Article  Google Scholar 

  41. Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., Yoo, Y.: CutMix: regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6023–6032 (2019)

    Google Scholar 

  42. Zhan, X., Pan, X., Dai, B., Liu, Z., Lin, D., Loy, C.C.: Self-supervised scene de-occlusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3784–3792 (2020)

    Google Scholar 

  43. Zhang, B., Xiao, J., Wei, Y., Sun, M., Huang, K.: Reliability does matter: an end-to-end weakly supervised semantic segmentation approach. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12765–12772 (2020)

    Google Scholar 

  44. Zhang, B., Zhao, Q., Feng, W., Lyu, S.: AlphaMEX: a smarter global pooling method for convolutional neural networks. Neurocomputing 321, 36–48 (2018)

    Article  Google Scholar 

  45. Zhang, X., Wei, Y., Feng, J., Yang, Y., Huang, T.S.: Adversarial complementary learning for weakly supervised object localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1325–1334 (2018)

    Google Scholar 

  46. Zhang, X., Wei, Y., Kang, G., Yang, Y., Huang, T.: Self-produced guidance for weakly-supervised object localization. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11216, pp. 610–625. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01258-8_37

    Chapter  Google Scholar 

  47. Zhang, X., Wei, Y., Yang, Y., Wu, F.: Rethinking localization map: towards accurate object perception with self-enhancement maps. arXiv preprint arXiv:2006.05220 (2020)

  48. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)

    Google Scholar 

Download references

Acknowledgement

Matthias Körschens thanks the Carl Zeiss Foundation for the financial support. We would also like to thank Alban Gebler and the iDiv for providing the data for our investigations.

Author information

Authors and Affiliations

  1. Friedrich Schiller University Jena, Jena, Germany

    Matthias Körschens, Paul Bodesheim, Christine Römermann, Solveig Franziska Bucher, Josephine Ulrich & Joachim Denzler

  2. Department Biogeochemical Integration, Max Planck Institute for Biogeochemistry, Jena, Germany

    Mirco Migliavacca

Authors
  1. Matthias Körschens
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul Bodesheim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christine Römermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Solveig Franziska Bucher
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mirco Migliavacca
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Josephine Ulrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Joachim Denzler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthias Körschens .

Editor information

Editors and Affiliations

  1. Fraunhofer IAIS, Sankt Augustin, Germany

    Christian Bauckhage

  2. University of Bonn, Bonn, Germany

    Juergen Gall

  3. University of Illinois at Urbana-Champaign, Urbana, IL, USA

    Alexander Schwing

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 2567 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Körschens, M. et al. (2021). Weakly Supervised Segmentation Pretraining for Plant Cover Prediction. In: Bauckhage, C., Gall, J., Schwing, A. (eds) Pattern Recognition. DAGM GCPR 2021. Lecture Notes in Computer Science(), vol 13024. Springer, Cham. https://doi.org/10.1007/978-3-030-92659-5_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-92659-5_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-92658-8

  • Online ISBN: 978-3-030-92659-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation