Abstract
Efficient and accurate estimation of homographies among images is the first step in mosaicking crop fields for phenotyping. The current strategy uses sophisticated vehicles that have excellent telemetry to hover over a grid of waypoints, imaging each one. This approach simplifies homography estimation, but precludes more flexible, adaptive protocols that can collect richer information. It also makes aerial phenotyping impractical for many researchers and farmers. We are developing an alternative strategy that uses consumer-grade vehicles, freely flown over a variety of trajectories, to collect video. We have developed an unsupervised deep learning network that estimates the sequence of planar homography matrices of our corn fields from imagery, without using any metadata to correct estimation errors. The vehicle was freely flown using a variety of trajectories and camera views. Our system, CorNet, performed faster than and with comparable accuracy to the gold standard ASIFT algorithm in many challenging cases.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, M., et al.: TensorFlow: Large-scale machine learning on heterogeneous distributed systems (2016). https://arxiv.org/abs/1805.09662
Agarwal, S., et al.: Building Rome in a day. Commun. ACM 54(10), 105–112 (2011)
Aktar, R., Aliakbarpour, H., Bunyak, F., Seetharaman, G., Palaniappan, K.: Performance evaluation of feature descriptors for aerial imagery mosaicking. In: Applied Imagery Pattern Recognition Workshop (AIPR), pp. 1–7. IEEE (2018)
Aktar, R., Prasath, V.S., Aliakbarpour, H., Sampathkumar, U., Seetharaman, G., Palaniappan, K.: Video haze removal and Poisson blending based mini-mosaics for wide area motion imagery. In: Applied Imagery Pattern Recognition Workshop (AIPR), pp. 1–7. IEEE (2016)
Aktar, R., et al.: Robust mosaicking of maize fields from aerial imagery. Appl. Plant Sci. 8, e11387 (2020). https://doi.org/10.1002/aps3.11387, https://bsapubs.onlinelibrary.wiley.com/doi/full/10.1002/aps3.11387
Aliakbarpour, H., Palaniappan, K., Seetharaman, G.: Stabilization of airborne video using sensor exterior orientation with analytical homography modeling. In: Sergiyenko, O., Flores-Fuentes, W., Mercorelli, P. (eds.) Machine Vision and Navigation, pp. 579–595. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-22587-2_17
Avola, D., Cinque, L., Foresti, G.L., Martinel, N., Pannone, D., Piciarelli, C.: A UAV video dataset for mosaicking and change detection from low-altitude flights. IEEE Trans. Syst. Man Cybern. Syst. PP, 1–11 (2018)
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006). https://doi.org/10.1007/11744023_32
Bentoutou, Y., Taleb, N., Kpalma, K., Ronsin, J.: An automatic image registration for applications in remote sensing. IEEE Trans. Geosci. Remote Sens. 43(9), 2127–2137 (2005)
Blancon, J., et al.: A high-throughput model-assisted method for phenotyping maize green leaf area index dynamics using unmanned aerial vehicle imagery. Fron. Pl. Sci. 10, 685 (2019). https://doi.org/10.3389/fpls.2019.00685
Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. (CSUR) 24(4), 325–376 (1992)
Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2006). https://doi.org/10.1007/s11263-006-0002-3
Chen, H.M., Arora, M.K., Varshney, P.K.: Mutual information-based image registration for remote sensing data. Int. J. Remote Sens. 24(18), 3701–3706 (2003)
Condorelli, G.E., et al.: Comparative aerial and ground based high throughput phenotyping for the genetic dissection of NDVI as a proxy for drought adaptive traits in durum wheat. Frontiers Plant Sci. 9 (2018). https://doi.org/10.3389/fpls.2018.00893
DeTone, D., Malisiewicz, T., Rabinovich, A.: Deep image homography estimation. arXiv preprint arXiv:1606.03798 (2016)
Enciso, J., et al.: Validation of agronomic UAV and field measurements for tomato varieties. Comput. Electron. Agric. 158, 278–283 (2019). https://doi.org/10.1016/j.compag.2019.02.011
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. In: Readings in Computer Vision, pp. 726–740. Elsevier (1987)
Gao, K., AliAkbarpour, H., Palaniappan, K., Seetharaman, G.: Evaluation of feature matching in aerial imagery for structure-from motion and bundle adjustment. In: Geospatial Informatics, Motion Imagery, and Network Analytics VIII, vol. 10645, p. 106450J. International Society for Optics and Photonics (2018)
Gnädinger, F., Schmidhalter, U.: Digital counts of maize plants by unmanned aerial vehicles (UAVs). Remote Sens. 9(6), 544 (2017)
Gracia-Romero, A., Kefauver, S.C., Fernandez-Gallego, J.A., Vergara-Díaz, O., Teresa Nieto-Taladriz, M., Araus, J.L.: UAV and ground image-based phenotyping: a proof of concept with durum wheat. Remote Sens. 11(10), 1244 (2019). https://doi.org/10.3390/rs11101244
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2004)
Huang, Y., Thomson, S.J., Hoffmann, W.C., Lan, Y., Fritz, B.K.: Development and prospect of unmanned aerial vehicle technologies for agricultural production management. Int. J. Agric. Biol. Eng. 6(3), 1–10 (2013)
Jain, P.M., Shandliya, V.: A review paper on various approaches for image mosaicing. Int. J. Comput. Eng. Res. 3(4), 106–109 (2013)
Johansen, K., Morton, M.J.L., Malbeteau, Y.M., Aragon, B., Al-Mashharawi, S.K., Ziliani, M.G., Angel, Y., Fiene, G.M., Negrão, S.S.C., Mousa, M.A.A., Tester, M.A., McCabe, M.F.: Unmanned Aerial Vehicle-based phenotyping using morphometric and spectral analysis can quantify responses of wild tomato plants to salinity stress. Frontiers Plant Sci. 10, 370 (2019). https://doi.org/10.3389/fpls.2019.00370
Kaess, M., Johannsson, H., Roberts, R., Ila, V., Leonard, J.J., Dellaert, F.: iSAM2: incremental smoothing and mapping using the Bayes tree (2011). https://doi.org/10.1177/0278364911430419, https://arxiv.org/abs/1908.02002v1
Kang, L., Wei, Y., Xie, Y., Jiang, J., Guo, Y.: Combining convolutional neural network and photometric refinement for accurate homography estimation. IEEE Access, pp. 109460–109473 (2019)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Kriegman, D.: Homography estimation. Lecture Computer Vision I, CSE a 252 (2007)
Le Moigne, J., Netanyahu, N.S., Eastman, R.D.: Image Registration for Remote Sensing. Cambridge University Press, Cambridge (2011)
Li, Q., Wang, G., Liu, J., Chen, S.: Robust scale-invariant feature matching for remote sensing image registration. IEEE Geosci. Remote Sens. Lett. 6(2), 287–291 (2009)
Lin, Y., Medioni, G.: Map-enhanced UAV image sequence registration and synchronization of multiple image sequences. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR 2007, pp. 1–7. IEEE (2007)
López-Granados, F., Torres-Sánchez, J., De Castro, A.I., Serrano-Pérez, A., Mesas-Carrascosa, F.J., Peña, J.M.: Object-based early monitoring of a grass weed in a grass crop using high resolution UAV imagery. Agron. Sustain. Dev. 36(4), 67 (2016)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)
Molina, E., Zhu, Z.: Persistent aerial video registration and fast multi-view mosaicing. IEEE Trans. Image Process. 23(5), 2184–2192 (2014)
Morel, J.M., Yu, G.: ASIFT: a new framework for fully affine invariant image comparison. SIAM J. Imaging Sci. 2(2), 438–469 (2009)
Nasir, A.K., Tharani, M.: Use of Greendrone UAS system for maize crop monitoring. In: ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, pp. 263–268 (2017)
Nguyen, T., Chen, S.W., Shivakumar, S.S., Taylor, C.J., Kumar, V.: Unsupervised deep homography: a fast and robust homography estimation model. IEEE Rob. Autom. Lett. 3(3), 2346–2353 (2018)
Ono, Y., Trulls, E., Fua, P., Yi, K.M.: LF-Net: learning local features from images. arXiv preprint arXiv:1805.09662v2 (2018)
Pohl, C., Van Genderen, J.L.: Review article multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. Remote Sens. 19(5), 823–854 (1998)
Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to sift or surf. In: 2011 International Conference on Computer Vision, pp. 2564–2571. IEEE (2011). https://doi.org/10.1109/ICCV.2011.6126544
Schonberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4104–4113 (2016)
Seetharaman, G., Palaniappan, K., Akbarpour, H.A.: Method for fast camera pose refinement for wide area motion imagery (2019). U.S. Patent 9,959,625
Shi, Y., et al.: Unmanned Aerial Vehicles for high-throughput phenotyping and sgronomic research. PLoS ONE 11(7), e0159781 (2016). https://doi.org/10.1371/journal.pone.0159781
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Teters, E., AliAkbarpour, H., Palaniappan, K., Seetharaman, G.: Real-time geoprojection and stabilization on airborne GPU-enabled embedded systems. In: Geospatial Informatics, Motion Imagery, and Network Analytics VIII, vol. 10645, p. 106450H. International Society for Optics and Photonics (2018)
Wang, X., et al.: Dynamic plant height QTL revealed in maize through remote sensing phenotyping using a high-throughput unmanned aerial vehicle (UAV). Sci. Rep. 9(1) (2019). https://doi.org/10.1038/s41598-019-39448-z
Wirth, M.A.: Shape Analysis & Measurement. University of Guelph (2004). http://www.cyto.purdue.edu/cdroms/micro2/content/education/wirth10.pdf
Woodward Crossings: Jang TD-1 Push Planter. Woodward Crossings (2019)
Wu, C.: Towards linear-time incremental structure from motion. In: 2013 International Conference on 3D Vision, pp. 127–134 (2013)
Zhu, Z., Riseman, E.M., Hanson, A.R., Schultz, H.: An efficient method for geo-referenced video mosaicing for environmental monitoring. Mach. Vis. Appl. 16(4), 203–216 (2005)
Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)
Acknowledgments
We are grateful for the support of the NSF Midwest Big Data Hub Digital Agriculture Spoke, the Missouri Maize Center, the National Corn Growers Association, the U.S. Army Research Laboratory (cooperative agreement W911NF1820285), the Army Research Office (DURIP W911NF1910181), an Executive Women’s Forum doctoral fellowship through the University of Missouri College of Engineering (to R.A.), and an anonymous gift. We thank Shizeng Zhou, Behirah Hartranft, Steven Suddarth, and Vinny for helpful discussions. We are especially grateful to the referees for their very helpful comments. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors’ and do not necessarily reflect the views of the U. S. Government or any agency thereof. The authors declare no conflict of interest.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Kharismawati, D.E., Akbarpour, H.A., Aktar, R., Bunyak, F., Palaniappan, K., Kazic, T. (2020). CorNet: Unsupervised Deep Homography Estimation for Agricultural Aerial Imagery. In: Bartoli, A., Fusiello, A. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science(), vol 12540. Springer, Cham. https://doi.org/10.1007/978-3-030-65414-6_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-65414-6_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65413-9
Online ISBN: 978-3-030-65414-6
eBook Packages: Computer ScienceComputer Science (R0)